Version Control, gamedev and git lfs

Tulenber 16 December, 2019 ⸱ Beginner ⸱ 7 min ⸱

This post started as a note about git and Unity, and then the stream of thoughts got out of hand =)

Let’s skip topics like “What is version control?” or “Why do you need version control in the 21st century?”. Instead of that, let’s describe the current situation with vcs, say why game dev is slightly different and how we can handle it.

Post contains mention of third-party services and their price policies. Author of this post not related to these services. Also, the author does not bear any responsibility for your expenses.

Version Control System - VCS

For those who didn’t know what it is - remember one simple thought: “Always use VCS!". For the beginning, go google “Github for beginners” and do all in these tutorials. After that, read a book named Pro Git.
If you know what vcs is, I recommend git and GitHub because they cover all that needed for the beginning. Also, I recommend starting all you’ r project with two words: “git init”. I am just reminding you, no offense.

Average company situation

  • Good old company and(or) project - you are using svn or something worse. Don’t think that you can change something if the project still uses it, but you can try. Need one more migration post you do not. But you always can find something positive in this situation - you already in the worst case, you know pain, future will be better.

  • Mildly paranoid - if you have paranoia, this does not mean that no one is following you. Your own private server in the local network. It’s ok maybe the state obliged you. If we get git, for example, then divided by prise, it may be Gitea < GitLab < GitHub Enterprise. All these tools from this century, so it is ok.

  • Hipsters - you can find a company that stores all their source codes in GitHub, and they are fine.

Game dev

Game dev has its specifics in version control, and it consists in the presence of a large number of resources in the form of pictures, videos and similar content, which weighs a lot (this is called BLOB, Binary Large Object). One of my projects was 10GB in total weight, 7GB of which was art. Git not designed to store blobs, and as a result, there is a separation for storage for source code and resources. In any self-respecting project, there will be some tools to simplify life, and a considerable part of it will be the solution to this trouble. People somehow live with it, so do not overestimate its size. But I’ve been in situations when team change blob store rules, sometimes for the sake of processes, sometimes for a price cut. So why not exchange them for something new and fashionable?

Gamedev solutions

We can say that git is the de facto standard, so the first solution which can come to mind are:

  • Custom tools - why not? It always works like: a small project - small tools, a big project - great tools. As said earlier, the blob’s stored separately from code. You have a dedicated server for it if you work inside an office or put it to something like dropbox for a distributed team. As a result, for every split to stabilize the new production version, you got a message in your Slack like “For arts for version 1.6.0 folder 1.6.0 created, all artists must save their changes THERE!!!”. Or maybe your tool grows a little bit more.

  • Perforce - sometimes you can hear that it is popular in the game dev industry and other media fields, that it can store blobs easily and that you can use git with it. When you go to their price page, you can read "Start Free and Pay as You Grow", so you can try it for free, like on most other services, so why not.

  • Git - as a developer, I am satisfied with git. And to resolve the trouble with blob’s some big players on git market write a plugin for git which named git lfs. Those are the same custom tools but integrated into you’ r git client. There is no special magic in it. In the local repository, you store all files as-is. And in the remote repository, all blob files store a reference to some file storage. Nothing complicated. For all that you need to know and get, you can go to the project site.

Git lfs

When you hear “some file storage”, - you immediately feel the catch somewhere. What server, where I can get access to it, how much I will spend? It’s surprisingly hard to consolidate information about this feature, given that all the grandees create it and how much money they take for its use, it’s very astoundingly that they forgot about marketing. So there are few server implementations if you didn’t count grand services, all of them open-source and not so supported, but maybe they are not so bad? I want to understand the situation with git lfs, so what I learned for today(December 2019):

  1. Basic case - clouds, you need to do nothing, all just work from the beginning:

    • GitHub - 1Gb of space and 1Gb of bandwidth for all your projects. Additional packs by 5$/month per 50Gb of space and 50Gb of bandwidth/month (You can find price inside your account settings under Billing button)

    • Bitbucket - 1Gb of space for all your projects, no word about bandwidth. Additional space by 10$ per 100Gb price.

    • Azure DevOps Service - 2Gb of space and different prices for additional space, started from 2$ per 1Gb (price).

    • GitLab - as always, have some troubles and say something like: “Currently, we have a 10GB/repository limit with no option for users to increase their storage, and we will add this feature later.” (link to those post). If you like to read something useless, there are some comments about game dev.

  2. Hybrid - you will need to configure something by yourself, but all your data still be in cloud:

    • AWS S3 - there is some implementation of git lfs storage, which route your files to S3, like git-lfs-s3. Price looks not so bad, but you must be VERY careful with Amazon; it is tough to calculate expenses forehead (Author does not bear any responsibility for your expenses).

  3. Half-hybrid - blob files go to dedicated server:

    • Nexus Repository Manager - it's widespread to meet it in company with maven or python local repositories, and it has a massive variety of storages like docker and almost all that you can imagine so why not use it for lfs also?

  4. Midcore - all store at dedicated server:

    • GitHub Enterprise, Bitbucket Server and Microsoft DevOps Server - if you have such funds and requirements, then why you have not bought them yet? Oo

    • GitLab - instead of cloud version, files storage space limited by HDD and the kindness of a person who configures this server.

    • Gitea - also has lfs implementation, but you need to configure it first.

  5. Harcore:

    • Instead of writing custom tools for your blob pipeline, you can write your lfs server implementation. It could store files to S3 or whenever you like. Are you really need to create your own tools from scratch instead of the expansion of current well known?

As a result, git lfs look not so bad (it will be used for this blog also). Additional infrastructure tools look like overhead for pet projects. And the lowest cost solution always wins for company usage. For myself, I choose… I think you already know by the image at the head of this post. See you next time! =)



Privacy policyCookie policyTerms of service
Tulenber 2020