Studio Pro uses a mix of Libgit2 and the separately installed Git CLI. The latter is used for performance reasons, but as of today is not (yet) strictly required for Mendix to function.
I can confirm we currently do not support Git LFS. As LFS is designed for files that don't change often, it's important to note this wouldn't be a viable solution for the often-changing .mpr file.
To control the growth of the Git repository we're automatically doing ‘git gc’ (garbage collection) every now and then. This still leaves the issue of an .mpr being to large for Github for example (>100 MB), which is why we're investigating if we want to get rid of the single large binary in favor of another solution.
Any luck finding an answer?