Snowtrack – a new version control software for graphic designers and 3D artists(snowtrack.io)
snowtrack.io
Snowtrack – a new version control software for graphic designers and 3D artists
http://snowtrack.io/
72 comments
How is this an improvement over a myriad of confusing and recursively-named nested folders with repeated copies of existing content in ever-growing names like "final_final-FINAL-05-CLIENTEXPORT"?
You have a point there :-D
Why don't graphic artist just use hard links and ZFS dedup.
/s
/s
Some comments here about delta compression on image assets and compressed formats - this page from the Fossil SCM has a really interesting discussion of the issues involved: https://fossil-scm.org/home/doc/trunk/www/image-format-vs-re...
Yeah this is cool analysis!
Looking at Fossil's delta code it uses 16 byte blocks and a sliding window when finding deltas:
https://fossil-scm.org/home/file?ci=trunk&name=src/delta.c&l...
Looking at Fossil's delta code it uses 16 byte blocks and a sliding window when finding deltas:
https://fossil-scm.org/home/file?ci=trunk&name=src/delta.c&l...
This is incredibly helpful as I was looking for something like this for weeks. Thanks a lot for sharing! This looks super useful
This looks really interesting.
But Typescript and "Git uses a restrictive license" give me pause. Maybe it's a cultural thing. I'll be hesitantly optimistic I guess.
In this case it's a comparison between MIT and GPL. GPL software is more restrictive than licenses like MIT/Apache/.. and therefore not as much as present in the VFX or CG industry and that's key here. But you are absolutely right! The wording is potentially misleading and should be improved. I'll fix it
Depends on how you look at it. GPL is more permissive in that it allows you to contribute to and benefit from evolutions of the software. While MIT doesn't permit you to reuse code that was expanded on by huge corporations.
Of course, you can frame the debate the other way around, i'm just making the point that "permissive/restrictive" means very little in this context and what matters really as a question is "do you wish huge corporations to benefit from your free-software work and expand on it to develop their own business, without EVER contributing anything back"? If yes, choose MIT and you'll probably regret it like so many open-source maintainers. Otherwise, choose (a)GPL and you probably won't regret it (at least i've never met someone so far who regretted choosing copyleft for their works, but it may exist of course).
Of course, you can frame the debate the other way around, i'm just making the point that "permissive/restrictive" means very little in this context and what matters really as a question is "do you wish huge corporations to benefit from your free-software work and expand on it to develop their own business, without EVER contributing anything back"? If yes, choose MIT and you'll probably regret it like so many open-source maintainers. Otherwise, choose (a)GPL and you probably won't regret it (at least i've never met someone so far who regretted choosing copyleft for their works, but it may exist of course).
"Permissive" at least is an established term of art, meaning a license which does not impose many conditions on redistribution:
https://fossa.com/blog/all-about-permissive-licenses/
Even the GNU project uses it that way:
https://www.gnu.org/licenses/license-list.en.html
You're right that "restrictive" is an unusual term in this context. The antonym of "permissive" is usually "copyleft".
https://fossa.com/blog/all-about-permissive-licenses/
Even the GNU project uses it that way:
https://www.gnu.org/licenses/license-list.en.html
You're right that "restrictive" is an unusual term in this context. The antonym of "permissive" is usually "copyleft".
Just for clarification, there are two projects involved here. SnowFS is the underlying open-source VCS, and Snowtrack is the commercial UI build around it.
But for your statement about licenses, there is actually not much I can say to counter your arguments. I've followed the fight between Amazon and Elastic so you have very valid points. For the moment I don't think SnowFS will face the same issues given its current scope and target audience/market
But for your statement about licenses, there is actually not much I can say to counter your arguments. I've followed the fight between Amazon and Elastic so you have very valid points. For the moment I don't think SnowFS will face the same issues given its current scope and target audience/market
>> Just for clarification, there are two projects involved here. SnowFS is the underlying open-source VCS, and Snowtrack is the commercial UI build around it.
This discussion sounds like they were afraid to build Snowtrack on top of git.
I'm not a fan of the names of git commands and some of the other details about it, but functionally it works well and is the industry standard. To avoid it because of the GPL doesn't seem right. There are many commercial tools built of git.
This discussion sounds like they were afraid to build Snowtrack on top of git.
I'm not a fan of the names of git commands and some of the other details about it, but functionally it works well and is the industry standard. To avoid it because of the GPL doesn't seem right. There are many commercial tools built of git.
Git doesn't work well for binary files, in particular large binary files.
This is the reason so few VFX, graphics, and game devs use Git for anything that needs to version large binary files.
Git LFS can help, but it's not always the right solution.
This is the reason so few VFX, graphics, and game devs use Git for anything that needs to version large binary files.
Git LFS can help, but it's not always the right solution.
You're right. Git is widely used but I wouldn't call it an industry standard (yet), especially not not for repositories that exclusively contain binary data. And the GPL works well with commercial products. On top there is libgit2 which is a fantastic alternative as well. But Git doesn't fulfill the technical requirements for Snowtrack
Makes me happy to see system tools being written in typescript!
I absolutely agree. When I started evaluating the project I was positively surprised about the performance of Node for this field. It was a perfect fit so I continued with it
From a quick read of the SnowFS source code, it looks like it splits large files into 100Mb blocks and builds up a zip of blocks over time. A version of a file is an ordered list of hashes for the blocks in that version.
I like the simplicity of this! But is it at all problematic if something changes early in the file and all the subsequent blocks boundaries shift causing many new blocks to be created?
rsync uses a sliding window to handle this situation. The implementation would be more complicated, but have you considered using librsync internally?
I like the simplicity of this! But is it at all problematic if something changes early in the file and all the subsequent blocks boundaries shift causing many new blocks to be created?
rsync uses a sliding window to handle this situation. The implementation would be more complicated, but have you considered using librsync internally?
I am currently working on the compression, as it is not complete yet. The 100 MB is indeed excessive but the window is dynamic and can differ from file to file since it is written to a `*.hblock` file which is stored next to the object in the object database https://github.com/Snowtrack/SnowFS/blob/03e5f839326e666c891...
Let me explain where the 100 MB window comes from as its not only related to the upcoming compression implementation. Some graphic applications touch the timestamps of their files for no reason, making it harder to detect if a file changed. But some file formats always change their 'header' or 'footer'. Means, comparing the hash of the first or last 100 MB of a file that is 8GB in size gives a great performance boost to detect if a file got modified.
Let me explain where the 100 MB window comes from as its not only related to the upcoming compression implementation. Some graphic applications touch the timestamps of their files for no reason, making it harder to detect if a file changed. But some file formats always change their 'header' or 'footer'. Means, comparing the hash of the first or last 100 MB of a file that is 8GB in size gives a great performance boost to detect if a file got modified.
There's a large set of different algorithms with a sliding window. Another interesting one is the Rabin fingerprint. This kind of chunking is often used in storage file systems w/ deduplication and snapshot features.
https://en.wikipedia.org/wiki/Rabin_fingerprint
https://en.wikipedia.org/wiki/Rabin_fingerprint
[deleted]
cool. although I think with 4mb window it would be more efficient. 100mb seems excessive, then I assume you wouldn't need a sliding window. (if it works well enough for 100mb)
the problem happens with any fixed window spacing regardless of the block size.
If you create a block every Xmb... inserting a single byte at the beginning of the file will change every subsequent block.
If you create a block every Xmb... inserting a single byte at the beginning of the file will change every subsequent block.
You're technically speaking wrong, but I'm sure the author doesn't want to reimplement block storage devices... So the spirit of the message is probably correct
Oh I'm not talking about disks... this is based on how SnowFS (the library for this project) splits up big files into chunks:
https://github.com/Snowtrack/SnowFS/blob/main/src/common.ts#...
The intent is a simple form of delta encoding, the hope is that many chunks will be common between two versions.
https://github.com/Snowtrack/SnowFS/blob/main/src/common.ts#...
The intent is a simple form of delta encoding, the hope is that many chunks will be common between two versions.
I should clarify this. The 100 MB window in SnowFS is currently unrelated to compression as it is only used to compare if a block changed. Each block gets a hash. This is a fallback used for some file formats where the mtime timestamp cannot be trusted. Some files have a change in the first block e.g. 100 MB and that is faster to compare than an entire 8GB file. But this window size is dynamic and can be changed and used for compression in the future
Ahh this is my bad. For some reason I assumed the blocks were part of the storage scheme, but I see they only are used to compute hash, and that the whole file is added to zip. Sorry for the misunderstanding!
[deleted]
I built tech like this at a startup in 2010. It was aimed at musicians but basically the same idea - transparent version control of all of their DAW projects and easy collaboration (sending of huge projects, hard a decade ago). In this case it actually indexed all of the drives (musicians use attached media a lot) and maintained an online searchable archive with previews as well. We’d listen for file system notifications and there was enough logic to understand project directory structure, flac-compressing the audio (while preserving metadata so the it’s could be perfectly reconstructed).
This was only a means to an end though - it’s clear deeper integration with these products works better for a number of reasons, and a much better collaboration workflow isn’t just sending entire directories around. Look at Figma for an example of that - they reinvented the entire product space around collab.
Eventually the company (https://gobbler.com/) pivoted the business model away from this space into plugins.
I’m not convinced the opaque file approach can provide / capture enough value to be a sustainable business over the next 5-10 years.
This was only a means to an end though - it’s clear deeper integration with these products works better for a number of reasons, and a much better collaboration workflow isn’t just sending entire directories around. Look at Figma for an example of that - they reinvented the entire product space around collab.
Eventually the company (https://gobbler.com/) pivoted the business model away from this space into plugins.
I’m not convinced the opaque file approach can provide / capture enough value to be a sustainable business over the next 5-10 years.
There is a lot going on in the space right now and there will be certainly a big shift in the next years. But as long as Blender, Cinema 4D, Photoshop, Pixelmator, Affinity are not running in the cloud I rather feel like DCC software packages and tablets will introduce a change first
Yeah the company I work for (https://www.copia.io/) is doing similar things for Programmable Logic Controllers, which historically have binary project files.
Also a means to an end given the current sad reality. But most vendors are slowly moving towards better VCS support.
Also a means to an end given the current sad reality. But most vendors are slowly moving towards better VCS support.
I thought it was just a pretty UI on top of git, the bit about "It's good for large binary files!" is hidden under the link to the SnowFS github project...
Since the HN headline says "3D artists" I was expecting to see an app that has a 3D viewer that can show 2 versions of a 3d scene on top^W^W inside each other and you can see toggle to see the differences...
Since the HN headline says "3D artists" I was expecting to see an app that has a 3D viewer that can show 2 versions of a 3d scene on top^W^W inside each other and you can see toggle to see the differences...
> top^W^W
Is this an abbreviation or a reference to a command or what does it mean?
Is this an abbreviation or a reference to a command or what does it mean?
> 3d scene on top^W^W inside each other
Ctrl-W is delete previous word on some terminals, so I wrote "3d scene on top [of] each other" and decided saying "3d scene inside each other" made more sense.
Ctrl-W is delete previous word on some terminals, so I wrote "3d scene on top [of] each other" and decided saying "3d scene inside each other" made more sense.
On Mac, it would be nice if you could work with files completely inside Snowtrack instead of having to jump to Finder and back. Also, dragging a file onto Snowtrack seems like it should add the file to the current project. Otherwise, I love the idea for a simple and straightforward VCS tool.
Thanks for your feedback! Better D&D handling was already requested a few times and will certainly be added in the next versions
Can definitely see the usefulness of this for artists, especially if there is integration into the various art programs.
Unless I am mis-understanding this is only a solution for organizing local files? Ideally there would be a distributed version control component or integration with an existing version control system. I can see this lulling users into a false sense of security that their local data is backed-up and safe.
Unless I am mis-understanding this is only a solution for organizing local files? Ideally there would be a distributed version control component or integration with an existing version control system. I can see this lulling users into a false sense of security that their local data is backed-up and safe.
Thanks for your input! Snowtrack has no collaboration features built in yet. The public beta of Snowtrack is available since today. The project has a lot of fields to discover and will evolve over time. There are certainly countless of areas untouched yet
That's really cool! What's the business model here? A lot of artists, especially in free-software/free-culture communities could use tooling like that.
If you're willing to make it a proper donation-run project with a commitment to free-software, i'm sensing a great future like GIMP/Blender has :) Good luck on it, can't wait to try it out once it's packaged on Debian
If you're willing to make it a proper donation-run project with a commitment to free-software, i'm sensing a great future like GIMP/Blender has :) Good luck on it, can't wait to try it out once it's packaged on Debian
Thank you! The project consists out of two projects. SnowFS is the underlying core and is available as an open-source project (MIT). Snowtrack is the UI and the commercial product build around it and will be available as a Freemium product
This looks really interesting. I read through the docs on the site though, and it doesn't say anything about diffs and merges. Those are fundamental VCS operations so I hope it supports them. I recognize merging is complex for 2d/3d files but if it claims to be a VCS then I'd think it would have to support it, otherwise what use are branches?
How do you define a diff or merge for such files? Having a history available and branches (e.g. alternative versions of the same image) that can't be merged back seems to me to be still a version-control system.
It depends on how deep the file parser is. For 2d, the zero-level feature would just be pixelwise image diff of the rendered result. For photoshop or inkscape, you could diff layers, show added/deleted/changed adjustments, etc.
For 3d, to take Blender as an example, a zero-level diff feature would be to export (e.g. as glTF) and show which scene objects differ (meshes, textures, lights, etc.) More detailed parsing would give more details, e.g. image-diffing modified textures.
As I noted, merging is nontrivial. But it seems important -- if I save a branch, go back to the trunk and do more work, how do I add in the changes I made on the branch? Or even know what those changes were? Good diffs would help of course.
Merging would certainly have to be product-specific. For some open-source tools one could imagine hooking into their undo/redo logic, or parsing the product files into a temporary line-based format, but from outside it seems much more challenging than line-based merge in git (which is already imperfect for many applications).
For 3d, to take Blender as an example, a zero-level diff feature would be to export (e.g. as glTF) and show which scene objects differ (meshes, textures, lights, etc.) More detailed parsing would give more details, e.g. image-diffing modified textures.
As I noted, merging is nontrivial. But it seems important -- if I save a branch, go back to the trunk and do more work, how do I add in the changes I made on the branch? Or even know what those changes were? Good diffs would help of course.
Merging would certainly have to be product-specific. For some open-source tools one could imagine hooking into their undo/redo logic, or parsing the product files into a temporary line-based format, but from outside it seems much more challenging than line-based merge in git (which is already imperfect for many applications).
I think this is one of those things that superficially sound not that complicated but when you look closer, it is a gargantuan complex task. If I think only about Photoshop and PSD, this seems terrible to implement and maintain.
Also, I guess that the need is not really there, because otherwise, Adobe or some other company would have implemented this in the past 30 years.
Also, I guess that the need is not really there, because otherwise, Adobe or some other company would have implemented this in the past 30 years.
Thank you for your feedback. You brought up very good points.
Snowtrack is available as a public beta since today and will evolve from here through user-requests and feedback. Snowtrack supports a simple before/after compare but this feature will be heavily extended with the next versions.
About merge operations: The majority of solo artists and designers don't work in many branches. Most projects just go forward and branches are only useful if a rollback is needed. That being said, Snowtrack does offer a simplified "cherry-pick". But merge operations as you might know them from Git/Perforce are not implemented yet as it takes time to see how users adapt the software.
Keep in mind, most artists and designers are new to VCS or have never heard of it before and therefore it is crucial to find the right balance between too little and too many features. Any second that keeps the artist away from creating their artwork is wasted and therefore and the goal is to not interfere with their workflow or to add additional "management tasks"
About merge operations: The majority of solo artists and designers don't work in many branches. Most projects just go forward and branches are only useful if a rollback is needed. That being said, Snowtrack does offer a simplified "cherry-pick". But merge operations as you might know them from Git/Perforce are not implemented yet as it takes time to see how users adapt the software.
Keep in mind, most artists and designers are new to VCS or have never heard of it before and therefore it is crucial to find the right balance between too little and too many features. Any second that keeps the artist away from creating their artwork is wasted and therefore and the goal is to not interfere with their workflow or to add additional "management tasks"
[deleted]
Love the idea. I was thinking about creating something similar (without any knowledge on how to achieve that, I'm not a good programmer) for music creation and especially collaboration.
When writing a song with someone else, this can become really handy, as we sometimes don't know if some idea might work.
I would totally be a customer (if its not bound to a monthly payment). But as of now, it's lacking linux support (why do I have to be that special unicorn all the time?).
By the way: BandLab has something like this integrated already, but I want to use my own DAW.
When writing a song with someone else, this can become really handy, as we sometimes don't know if some idea might work.
I would totally be a customer (if its not bound to a monthly payment). But as of now, it's lacking linux support (why do I have to be that special unicorn all the time?).
By the way: BandLab has something like this integrated already, but I want to use my own DAW.
I received a lot of feedback by people with music background. I recently talked to someone from the music production industry and it was interesting to hear that they have the same workflow and problems as the graphics industry. Snowtrack could be used with any DAW but the missing thumbnails are a downer at the moment
https://web.archive.org/web/20210608090928/https://snowtrack...
"Snowtrack is based on our open-source community project": https://github.com/snowtrack/snowfs
"Snowtrack is based on our open-source community project": https://github.com/snowtrack/snowfs
Your repo has a section "Why not Git/Git-LFS, libgit2, or SVN?". Can i suggest you add a comparison to Mercurial's largefiles extension:
https://www.mercurial-scm.org/wiki/LargefilesExtension
Since, last time i looked into it, which was a while ago, that was the best option for this.
https://www.mercurial-scm.org/wiki/LargefilesExtension
Since, last time i looked into it, which was a while ago, that was the best option for this.
I would also be curious how it compares to Perforce which is widely used for version control of binary art assets (pretty much ubiquitous in the games industry).
Great idea! I've to admit my knowledge about Mercurial is very rusty. Could you help out on that? May I invite you to share some insights in the discussion board of the project? https://github.com/Snowtrack/SnowFS/discussions
It's been such a long time since i looked at it that i don't think i have much useful to say, i'm afraid!
Hmm, version requirements seem odd? "Apple macOS 14.0 (or higher)" we're only on 10.15.x or 11, with 12 on the horizon...[1]
1: https://en.wikipedia.org/wiki/MacOS_version_history#Version_...
1: https://en.wikipedia.org/wiki/MacOS_version_history#Version_...
I’m from the future. ;-) But I will fix it, thanks for the info
My question is, if you're running this with blender, say, does it simply save the same file over and over again internally or in a hidden way somewhere? If that's the case, that's a lot of waste of space for me.
This site fails to load in Firefox, with an error about there being no overlapping TLS ciphers available. Using the up to date public stable release of Firefox, no other TLS issues elsewhere, I wonder what has gone wrong.
Thanks for the feedback! I will look into this
I checked this on Windows and macOS with the latest version of Firefox and I have no issues opening the page
Works fine in Firefox on Linux for me.
After looking at the website and source repo, I still don't understand if this VCS shows delta differences between different versions of the same file (for example a picture).
Does this only work at the project snapshot level? My artists prefer P4 over git with the major reason being the ability to work on single files at a time (ie sparse check out).
Auto-commit is neat.
Auto-commit is neat.
Unfortunately, Snowtrack has no collaboration workflow yet. The underlying file storage system is prepared to support collaboration in the future, but it's too early for this
This looks great! Are you planning to support Linux?
Thank you! I've already prepared the foundation for a Linux build, and it's planned in the near future but I don't have a build available yet
Wow, reviewing. I am in the games industry.
What would a workflow look like with Figma?
Snowtrack focuses on local file versioning with tools that are locally installed (e.g. Photoshop, Blender, Cinema 4D, Pixelmator, ...)
figma already has pretty robust version tracking built in, though it's not necessarily too discoverable
This is a major pain point in my company, so I'm curious about the experiences elsewhere: do you mean that Figma tracks changes between "saves", ala google docs, and one can go back in time and see the whole thing as it was yesterday, or you mean it's possible to see a comparison of what changed, akin to Confluence or (of course) version control systems?
Just the Time Machine part I'm afraid. Diffing would be fantastic though for sure, I can think of several times that would have been useful in my team too.