Configure Git LFS

[Jooms]

Since we're storing large files (like pictures as part of #57), we need to set up Git LFS or else our repo will dramatically balloon in size.

https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage

[briswells]

Since we're storing large files (like pictures as part of #57), we need to set up Git LFS or else our repo will dramatically balloon in size.

https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage

While I'm all about avoiding the repo becoming to large I'm not sure if Git LFS is the right direction here, especially if the ER diagram is your concern. It's less than 100 kb which doesn't seem appropriate for for Git LFS. I think looking into some free image hosting would be more applicable to this project as we'll likely need it for other images that the website will end up using. Any thoughts on moving that direction instead?

[AbhinavReddy-Dev]

Since it is going to a generic App, anyone who wants to run it should be able to upload the images and see those specific ones, even the logo. So, if this concern is about the media files then it needs to be completely on cloud. Please correct me if I have overlooked anything.

[Jooms]

some free image hosting would be more applicable to this project

I think there are 2 needs for images

1. Images that are shown to customers / clients
2. Documentation images

As you both are alluding to, the first one is often optimized to have larger static files hosted on some image hosting service that can host them closer to the "edges" our customers are using. That could be another issue, but perhaps should start as a discussion and brought up during class?

The second one is what we should use Git LFS for long term, but Git LFS also allows us to handle the first one for the short term.

The reason I made this ticket was because of how Git handles large files. Not that you both don't know, but for the others that don't follow the inner workings of git... The .git folders we have locally contain every object ever with a hash pointer. It's why we can go forwards and backwards in history while being offline.
So when a text document goes in to the repo, there's a copy of the entire document stored. When the contents change, another full copy of that file is stored. This isn't a problem for < 1KB files, but when a 100kb file is added, it starts to become a problem.

Now that this image has been uploaded to our repo, every time anyone ever clones our repo, they will need to download this image. If we modify the image slightly, perhaps even the metadata in the file, then another full copy will be stored. Which means yes, anyone who clones the repo after that will always have to download both copies of the image. Even if we delete it from the main branch later.
The reality is Git as a base tool was not made for large files, it was meant for code and text. Which is why Git LFS exists and is so widely used everywhere.

@briswells and @AbhinavReddy-Dev, I agree that there's discussion to be had about long-term solutions for 1) and even 2) above, but there's no reason we should hold off on enabling Git LFS for now. Especially when it doesn't cost anything, and will give us a chance to learn how to do things the right way without having to destroy our git history in the future.

[AbhinavReddy-Dev]

some free image hosting would be more applicable to this project

I think there are 2 needs for images

1. Images that are shown to customers / clients
2. Documentation images

As you both are alluding to, the first one is often optimized to have larger static files hosted on some image hosting service that can host them closer to the "edges" our customers are using. That could be another issue, but perhaps should start as a discussion and brought up during class?

The second one is what we should use Git LFS for long term, but Git LFS also allows us to handle the first one for the short term.

The reason I made this ticket was because of how Git handles large files. Not that you both don't know, but for the others that don't follow the inner workings of git... The .git folders we have locally contain every object ever with a hash pointer. It's why we can go forwards and backwards in history while being offline. So when a text document goes in to the repo, there's a copy of the entire document stored. When the contents change, another full copy of that file is stored. This isn't a problem for < 1KB files, but when a 100kb file is added, it starts to become a problem.

Now that this image has been uploaded to our repo, every time anyone ever clones our repo, they will need to download this image. If we modify the image slightly, perhaps even the metadata in the file, then another full copy will be stored. Which means yes, anyone who clones the repo after that will always have to download both copies of the image. Even if we delete it from the main branch later. The reality is Git as a base tool was not made for large files, it was meant for code and text. Which is why Git LFS exists and is so widely used everywhere.

@briswells and @AbhinavReddy-Dev, I agree that there's discussion to be had about long-term solutions for 1) and even 2) above, but there's no reason we should hold off on enabling Git LFS for now. Especially when it doesn't cost anything, and will give us a chance to learn how to do things the right way without having to destroy our git history in the future.

Couldn't agree more, need to discuss in class.

[briswells]

@Jooms I didn't realized clone pulled the full history for the repo so that makes a lot more sense now. Especially since git pretty much considers any update to a PNG a full rewrite and can't just track the delta. We should move forward that direction for sure.

I remember having issues last time I added LFS to an existing repo that ended up with me nuking and recreating the entire repo so we'll just need to be careful when enabling it. Although maybe I was just doing something stupid ;)