The submodule update was doing a full clone. Also increase clone depth a
bit, since it can apparently cause issues checking out the source if
there are too many concurrent PRs trying to build (causing the commit
for the build to not be the top one on the repository anymore).