Repository navigation
GitPython corrupts git index #265
Description
Activity
- changed the title
[-]GitPython corrupts git repository[/-][+]GitPython corrupts git index[/+]on Mar 2, 2015 I have seen this too, but in conjunction with submodules. GitPython writes the index itself, and apparently it doesn't write back the expected format.
A workaround, as described there, is rewriting the index using something like
git reset HEAD. The problem certainly occours when usingrepo.index.add(...), as this one will rewrite the index. As a quick fix, one could forcefully use the git command for this in the downstream source, such as inrepo.git.add(...).Can you point me to code (e.g. bash) that helps me reproducing this, and the original source code of
fedpkg? If I can reproduce it, I can certainly fix it.Thank you
Thank you very much.
This should be the steps:
$ fedpkg co GitPython --anonymous # --anonymous in case you are not Fedora contributor ;) $ cd GitPython $ vim GitPython.spec # and do some editing. $ fedpkg srpm $ fedpkg import GitPython-0.3.2-0.7.RC1.fc23.src.rpmfedpkg [1] is just thin wrapper abouve pyrpkg, so this [2] should be the place where the index is modified after
fedpkg import[1] https://git.fedorahosted.org/cgit/fedpkg.git/tree/
[2] https://git.fedorahosted.org/cgit/rpkg.git/tree/src/pyrpkg/__init__.py#n1425Thank you - it's about time that bug gets squashed. I will dedicate my time tomorrow, in favour of finishing other work today.
Thanks to the workarounds we have, there are options in case this is not fast enough.
Besides: My apologies - after all, that one could have been fixed earlier.Thanks a lot. pyrpkg maintainer is now aware about the
repo.git.addworkaround, so he might apply it as well.It turned out that the index is not actually corrupted, which is good news. What happens is that
gitwritesTREEextension data into the index, which causes it to write out the given tree as is next time agit commitis executed. When usinggit add, this extension data is maintained automatically. However, GitPython doesn't do that ... . Usually this is no problem at all, as you are supposed to useIndexFile.commit(...)along withIndexFile.add(...).Thanks to a shortcoming in the GitPython API, the index was automatically written out whenever files have been added, without providing control over whether or not extension data will be written along with it.
My fix consists of an additional flag in
IndexFile.add(...), which causes extension data not to be written by default, so commits can be safely done viagit commitorIndexFile.commit(...).However, this might introduce new subtle bugs in case someone is relying on extension data to be written. As this can be controlled through the said flag though, a fix is easily done in that case.
How to apply the fix to your codebase
- Use GitPython v0.3.7 (due for release this week)
- use
git addinstead ofIndexFile.add(...) - use
IndexFile.commit(...)instead ofgit commit
Unfortunately, there is no other way to make it work with older versions of GitPython, as there is yet another bug in
IndexFile.write(...)which would cause extension data to written even if that was explicitly disabled. It is fixed in v0.3.7 .YouTube
The development stream can be watched on youtube
Wow, thanks for fixing this. Hope it will be fixed in Fedora soon (right @pbabinca ? 😉 )
And thanks especially for the video with analysis. Shame I don't have time to watch it (I watched just first 5 minutes). It would be definitely enlightening. BTW the "Session 2" video is private ;)
Hehe, it's only the most-hardcore people who have the time and are willing to watch me do my thing for two hours ... no one will do that, yet it's good to archive the process of fixing issues just in case I will be wondering why the heck I fixed it the way I did :).
The second video just wasn't ready yet - it works now, and youtube probably said it's still being processed when you tried it.
Fedora's fedpkg [1] is using GitPython in background, but unfortunately, this usage corrupts the Git repository in a way, that added files [2] always stays in staging and they never get commit. Is there any chance you could help fix this issue?
Please see more in Red Hat's Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=822055
Also note that I tried the recent version of GitPython and pythong-gitdb without luck:
$ rpm -q GitPython
GitPython-0.3.6-0.1.fc23.noarch
$ rpm -q python-gitdb
python-gitdb-0.6.4-0.1.fc23.x86_64
[1] https://fedorahosted.org/fedpkg/
[2] https://git.fedorahosted.org/cgit/rpkg.git/tree/src/pyrpkg/__init__.py#n1425