Hi,
we discovered a serious problem with git, the source code versioning system that is used for OLPC work:
I have been using it like I used to with subversion, that is, check in an updated etoys.image.gz every day:
http://dev.laptop.org/git.do?p=projects/etoys
Now Marco, who is on an ISDN connection, tried to check that out, and it downloaded like 30 MB. Turns out checking out from a git repository does download the *whole* history since the dawn of time. Eek.
I hear git does use delta-compression for binary files, but I looks like that does not work for an image:
52 .git git add OLPCPlugin.image 5764 .git git commit 5776 .git run image, save, commit 11492 .git run image, save, commit 17208 .git
The increase for each image check-in is about the gzipped size of the whole image.
So. We might have to put the images somewhere else. Or only store the original image plus changesets in a folder, and have the makefile update the image from those changesets.
A similar problem will arise once we store projects in git.
Any ideas?
- Bert -
I'd ask someone who knows about Git. I can't imagine that in this day and age a content management system isn't capable of dealing with binary files effectively. This reminds me too much of CVS ;-) So before looking for anything else we should check the git docs - there should be something that describes how to deal with binary files effectively.
Cheers, - Andreas
Bert Freudenberg wrote:
Hi,
we discovered a serious problem with git, the source code versioning system that is used for OLPC work:
I have been using it like I used to with subversion, that is, check in an updated etoys.image.gz every day:
http://dev.laptop.org/git.do?p=projects/etoys
Now Marco, who is on an ISDN connection, tried to check that out, and it downloaded like 30 MB. Turns out checking out from a git repository does download the *whole* history since the dawn of time. Eek.
I hear git does use delta-compression for binary files, but I looks like that does not work for an image:
52 .git git add OLPCPlugin.image 5764 .git git commit 5776 .git run image, save, commit 11492 .git run image, save, commit 17208 .git
The increase for each image check-in is about the gzipped size of the whole image.
So. We might have to put the images somewhere else. Or only store the original image plus changesets in a folder, and have the makefile update the image from those changesets.
A similar problem will arise once we store projects in git.
Any ideas?
- Bert -
Etoys mailing list Etoys@laptop.org http://mailman.laptop.org/mailman/listinfo/etoys
I did, and they said it is not possible. They actually *like* having the whole history, although there was discussion about a partial check-out:
http://www.gelato.unsw.edu.au/archives/git/0601/15659.html
It's not even a real SCM but something Linus T. wrote for his use. I guess with today's bandwidth and disk space that's even reasonable - for source code.
- Bert -
Am 15.10.2006 um 19:40 schrieb Andreas Raab:
I'd ask someone who knows about Git. I can't imagine that in this day and age a content management system isn't capable of dealing with binary files effectively. This reminds me too much of CVS ;-) So before looking for anything else we should check the git docs - there should be something that describes how to deal with binary files effectively.
Cheers,
- Andreas
Bert Freudenberg wrote:
Hi, we discovered a serious problem with git, the source code versioning system that is used for OLPC work: I have been using it like I used to with subversion, that is, check in an updated etoys.image.gz every day: http://dev.laptop.org/git.do?p=projects/etoys Now Marco, who is on an ISDN connection, tried to check that out, and it downloaded like 30 MB. Turns out checking out from a git repository does download the *whole* history since the dawn of time. Eek. I hear git does use delta-compression for binary files, but I looks like that does not work for an image: 52 .git git add OLPCPlugin.image 5764 .git git commit 5776 .git run image, save, commit 11492 .git run image, save, commit 17208 .git The increase for each image check-in is about the gzipped size of the whole image. So. We might have to put the images somewhere else. Or only store the original image plus changesets in a folder, and have the makefile update the image from those changesets. A similar problem will arise once we store projects in git. Any ideas?
- Bert -
Etoys mailing list Etoys@laptop.org http://mailman.laptop.org/mailman/listinfo/etoys
In this case I'd point the shortcoming out to OLPC and ask them to consider an alternative (perhaps a partial one for certain projects) to Git. Otherwise they may just have to live with the consequences.
Cheers, - Andreas
Bert Freudenberg wrote:
I did, and they said it is not possible. They actually *like* having the whole history, although there was discussion about a partial check-out:
http://www.gelato.unsw.edu.au/archives/git/0601/15659.html
It's not even a real SCM but something Linus T. wrote for his use. I guess with today's bandwidth and disk space that's even reasonable - for source code.
- Bert -
Am 15.10.2006 um 19:40 schrieb Andreas Raab:
I'd ask someone who knows about Git. I can't imagine that in this day and age a content management system isn't capable of dealing with binary files effectively. This reminds me too much of CVS ;-) So before looking for anything else we should check the git docs - there should be something that describes how to deal with binary files effectively.
Cheers,
- Andreas
Bert Freudenberg wrote:
Hi, we discovered a serious problem with git, the source code versioning system that is used for OLPC work: I have been using it like I used to with subversion, that is, check in an updated etoys.image.gz every day: http://dev.laptop.org/git.do?p=projects/etoys Now Marco, who is on an ISDN connection, tried to check that out, and it downloaded like 30 MB. Turns out checking out from a git repository does download the *whole* history since the dawn of time. Eek. I hear git does use delta-compression for binary files, but I looks like that does not work for an image: 52 .git git add OLPCPlugin.image 5764 .git git commit 5776 .git run image, save, commit 11492 .git run image, save, commit 17208 .git The increase for each image check-in is about the gzipped size of the whole image. So. We might have to put the images somewhere else. Or only store the original image plus changesets in a folder, and have the makefile update the image from those changesets. A similar problem will arise once we store projects in git. Any ideas?
- Bert -
Etoys mailing list Etoys@laptop.org http://mailman.laptop.org/mailman/listinfo/etoys
Etoys mailing list Etoys@laptop.org http://mailman.laptop.org/mailman/listinfo/etoys
Bert,
It's not even a real SCM but something Linus T. wrote for his use. I guess with today's bandwidth and disk space that's even reasonable - for source code.
Linus wrote very early version but now the principal maintainer is different. (he happens to be my colleague at my company, actually). Asking it at the git mailing list would be good. Having more than one SCM for a project would cause more problems...
-- Yoshiki
Hi Yoshiki -
If you know the maintainer can you ask him how we should be dealing with this problem? It seems to me that having to download a hundred versions of a couple of megabytes each can't be the final solution. Even for source code - at some point things get too large to do that.
Cheers, - Andreas
Yoshiki Ohshima wrote:
Bert,
It's not even a real SCM but something Linus T. wrote for his use. I guess with today's bandwidth and disk space that's even reasonable - for source code.
Linus wrote very early version but now the principal maintainer is different. (he happens to be my colleague at my company, actually). Asking it at the git mailing list would be good. Having more than one SCM for a project would cause more problems...
-- Yoshiki _______________________________________________ Etoys mailing list Etoys@laptop.org http://mailman.laptop.org/mailman/listinfo/etoys
etoys-dev@lists.squeakfoundation.org