What is Local first development(alexop.dev)
alexop.dev
What is Local first development
https://alexop.dev/posts/what-is-local-first-web-development/
55 comments
Was hoping this to be about development happening on my host OS instead of containers/orchestrators.
The phrasing threw me off too. I've usually heard of the OP's topic referred to as "local first software", not development.
I was hoping this to be about development happening in containers/VMs that can run on my local computer or elsewhere, as I wish.
Sync is still an unsolved problem. The solutions that claim to do it aren't very satisfactory.
Reliable Sync from a syntactic (e.g. rich text) or data structure level is possible already with CRDTs, there are many excellent solutions (y.js, automerge etc.).
Semantic sync (for example when 2 people have modified a draft of a novel and want to sync): how to auto sync both versions so the resulting text makes sense and has no duplicates, is indeed much harder, and requires manual revisions (like git revisions), possibly with LLM suggestions.
Semantic sync (for example when 2 people have modified a draft of a novel and want to sync): how to auto sync both versions so the resulting text makes sense and has no duplicates, is indeed much harder, and requires manual revisions (like git revisions), possibly with LLM suggestions.
Text editing CRDTs fall in the category of things that are technically possible, in an incredibly useless way - just like it's possible to automatically "resolve" every Git merge conflict by randomly splatting the conflicting lines around in the file.
Most collaborative text editors use CRDTs, OT, or a derivative nowadays. For the most part, the paradigm works fine: changes aren't just "randomly splatted" around the file but appear in their expected causal context. Strange things can happen when paragraphs are deleted or moved around, but this is more of a UI problem than an intrinsic limitation, and I'm not sure what alternative you're thinking of other than locking or throwing up a horrible "pick a version" dialog box.
It's more of a UI/UX problem than a hard limitation. For uses cases that require collaboration with offline editing, the person that last merges might be given a split view to sync their changes.
Again, syntactically possible, semantically harder, but it depends on your use-case plus additional constraints (eg max offline time). Maybe not a fully automatic system, but still very useful.
Again, syntactically possible, semantically harder, but it depends on your use-case plus additional constraints (eg max offline time). Maybe not a fully automatic system, but still very useful.
CRDTs with LLM suggested conflict resolution might just be the thing.
Next best thing is keeping conflicts unresolved, but marked as such as a first-class citizen of the document content (see pijul, jj).
Next best thing is keeping conflicts unresolved, but marked as such as a first-class citizen of the document content (see pijul, jj).
But then you don’t get convergence which is a requirement and LLMs will be just as good for Git as they will be for CRDTs.
I postulate they may handle written language quite well, within reasonable bounds. Re convergence, turns out it's much more convenient to let conflict resolution be deferred.
You can get convergence with the same mechanism that OT uses for compaction: assign a special server role that does it.
Fully-generic character-level textual sync is an unsolved problem (and likely, always will be).
That's also not typically the best way to sync one's concrete data models, and if your data has any sort of structure, you can likely produce a merge operation that works Well Enough™ in practice
That's also not typically the best way to sync one's concrete data models, and if your data has any sort of structure, you can likely produce a merge operation that works Well Enough™ in practice
IME, and experiences may vary, "Well Enough" merge function usually means corrupt data, confusing outputs, incoherent semantics, and frustrating debugging sessions at any kind of scale beyond playing around. YMMV.
I find actual lock on edit or last-save-wins to work best of what I have seen for structured data. You can add nice visual tools that show who is editing (so you can wait for them) and pass the object lock to someone else etc, but this way at least there is no merge mechanism (that generally knows very little about the ins/outs of the data) which will merge two semantically incompatible fields into one record.
For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. In attempts to make these things for an offline first healthcare app which is mostly structured records based, I found that it is very rare for 2 (or more) to work on the same data, but I guess it really depends on the use case and with unstructured data it's not a nice way of working.
For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do. In attempts to make these things for an offline first healthcare app which is mostly structured records based, I found that it is very rare for 2 (or more) to work on the same data, but I guess it really depends on the use case and with unstructured data it's not a nice way of working.
This is what people actually do in practice when collaborating on large non-code documents.
The example I’m familiar with is assets in the video games industry. Everyone used Perforce (or maybe SVN) and artists/designers lock the 3D model or whatever they’re editing. People ask each other to give up a lock all the time.
The example I’m familiar with is assets in the video games industry. Everyone used Perforce (or maybe SVN) and artists/designers lock the 3D model or whatever they’re editing. People ask each other to give up a lock all the time.
> For offline-first, the user would be alerted that someone changed the record(s) while you changed them as well and ask what to do.
There are few sync strategies worse than "you worked hard on this, but now you have to choose to either throw away your change or everyone else's changes". May any product doing that fail miserably.
There are few sync strategies worse than "you worked hard on this, but now you have to choose to either throw away your change or everyone else's changes". May any product doing that fail miserably.
And your great solution? As others say; there are none. Talking about structured content in an offline first context. What is your strategy that won't break anything while doing a merge?
Just wait until the vibe coders use an LLM to decide which data in a merge conflict it likes better.
Not at all true. One good solution exists.
> Effortless Collaboration: The app supports easy collaboration, even in offline scenarios.
mmm, yes, i see
mmm, yes, i see
Like resolving Git merge conflicts.
well, see, if you just hand wave away the really difficult part, and ignore the other really difficult thing (we have a perfect track record over 40+ years of users being terrible at sysadmin of their own computers), then this local first thing is super easy.
edit: to be clear, local first is interesting in theory, but I need a little more than draw the rest of the fucking owl.
edit: to be clear, local first is interesting in theory, but I need a little more than draw the rest of the fucking owl.
The author clearly is referring to cloud apps that require one to be online with their collaborator to edit. The alternative is a locally encrypted self-congratulated neural mesh network where one computes their own inputs to the hard drive.
Another option for offline collaboration I'd like to suggest is sharing a keyboard with a friend. It's local-first.
Or, well, like git. You can use an entirely email based workflow with git if you want to.
Here's an overview: https://drewdevault.com/2018/07/02/Email-driven-git.html
Here's an overview: https://drewdevault.com/2018/07/02/Email-driven-git.html
Actual title:
What is Local-first Web Development?
What is Local-first Web Development?
the problem with this is that most browsers employ persistency rules for OPFS that are not very transparent, at least for the user, or not very predicable, at least for the developer. In other words, a user can find his/her data wiped out all of a sudden.
Here some info: https://developer.mozilla.org/en-US/docs/Web/API/Storage_API...
There is also a discussion on HN: https://news.ycombinator.com/item?id=39222328
You can use the file system API as an alternative, which is permanent, but it's slow, requires the user to remember where the file was stored, and not recommended for things such as databases.
This makes the whole purpose of it a bit pointless in my opinion. What is your experience with it?
Here some info: https://developer.mozilla.org/en-US/docs/Web/API/Storage_API...
There is also a discussion on HN: https://news.ycombinator.com/item?id=39222328
You can use the file system API as an alternative, which is permanent, but it's slow, requires the user to remember where the file was stored, and not recommended for things such as databases.
This makes the whole purpose of it a bit pointless in my opinion. What is your experience with it?
> Picture using these apps offline with automatic synchronization when you’re back online. This is the essence of local-first web development – a revolutionary approach that puts users in control of their digital experience.
I had to laugh very hard at the "revolutionary approach". How is it revolutionary, if software was developed "local-first" for decades?
Techbros really do come up with the oldest ideas and call them revolutionary.
I had to laugh very hard at the "revolutionary approach". How is it revolutionary, if software was developed "local-first" for decades?
Techbros really do come up with the oldest ideas and call them revolutionary.
I laugh (maniacally) every time I need to change a jinja template generating a nightmarish yaml and remember the glory days of xml with affection.
Revolutions always have been the young one's business for a reason, I guess.
Revolutions always have been the young one's business for a reason, I guess.
XML had XML Schema and DTDs, making automatic validation a breeze.
Its only issue was that it was a massive pain to type by hand and no good GUI editors really emerged.
And some developers didn't understand the format and shoved stuff into attributes that should've been elements and vice versa.
Its only issue was that it was a massive pain to type by hand and no good GUI editors really emerged.
And some developers didn't understand the format and shoved stuff into attributes that should've been elements and vice versa.
lol i remember those days. most XML documents were missing DTDs and schemas (or that third type of schema whose name escapes me) and because the data structure of an XML document didnt map cleanly to lists and hashmaps you needed an ugly hacky xpath to grab whatever data you needed.
Dont even get me started on the mess that was serialisation and deserialization.
It was awful, overengineered design-by-committee mess.
This was only obvious to me in retrospect after I realized that these things didnt have to be complicated.
Dont even get me started on the mess that was serialisation and deserialization.
It was awful, overengineered design-by-committee mess.
This was only obvious to me in retrospect after I realized that these things didnt have to be complicated.
> or that third type of schema whose name escapes me
RRELAX NG, perhaps?
RRELAX NG, perhaps?
>no good GUI editors really emerged.
In those good old times both Oxygen and XML Spy were good enough, so I even purchased a license of Oxygen for myself and never felt the money were wasted.
>And some developers didn't understand the format and shoved stuff into attributes that should've been elements and vice versa.
This happened at such scale where we can say that it was UX problem of XML itself.
In those good old times both Oxygen and XML Spy were good enough, so I even purchased a license of Oxygen for myself and never felt the money were wasted.
>And some developers didn't understand the format and shoved stuff into attributes that should've been elements and vice versa.
This happened at such scale where we can say that it was UX problem of XML itself.
I suppose "local-first development" is a misnomer. It's really about the syncing part. Currently we have traditional applications that write data locally and therefore don't require an internet connection to work, and we have online stuff that updates a shared state "live". The "revolutionary" part would be having both of those things. It's a surprisingly hard thing to do.
The current state of the art here is git. So we're basically talking about making git automatic and easy to use for the majority of the population. That's not something we've been "doing for decades".
The current state of the art here is git. So we're basically talking about making git automatic and easy to use for the majority of the population. That's not something we've been "doing for decades".
I can think of so many pieces of software that does that: having a local state, having a remote state, and keeping them synchronized whenever internet is available. It's how email apps work. That's how all cloud drives work, and Dropbox is more than a decade old at this point. It's how notes apps work. Etc. etc.
Really can't see how this can be regarded as a recent idea.
Really can't see how this can be regarded as a recent idea.
I'm someone who has used and continues to use desktop applications like that. I'm old enough to know what "work offline" and an email "outbox" is. Those are not the same things.
The always-online apps bring a lot that is not possible with such software, even simple stuff like editing the same file at the same time. Try that with tools like Dropbox and you'll get conflicts that you need to resolve. It's totally possible to get offline software if you either give up on many features or accept complicated conflict resolution etc. The basic options are either immutability, like email, or conflict resolution, like git. This is about not making those sacrifices.
"Local first" is a misnomer and that misnomer does probably reflect the different experience of youngsters, but if you can get past that there is something new here that is interesting and challenging.
The always-online apps bring a lot that is not possible with such software, even simple stuff like editing the same file at the same time. Try that with tools like Dropbox and you'll get conflicts that you need to resolve. It's totally possible to get offline software if you either give up on many features or accept complicated conflict resolution etc. The basic options are either immutability, like email, or conflict resolution, like git. This is about not making those sacrifices.
"Local first" is a misnomer and that misnomer does probably reflect the different experience of youngsters, but if you can get past that there is something new here that is interesting and challenging.
Just try to do simultaneous edits offline in all those "Etc. etc.", see it automatically fail, then the obvious recency might become apparent
Many applications work like that, particularly ones made between 2000 and 2010. Operating systems have whole stacks of SDKs and libraries dedicated to that exact behavior (e.g., CloudKit in OS X).
Google's Keep Notes works well enough for me.
It might be helpful to read what "local first" means in this context: https://www.inkandswitch.com/local-first/
Techbros really do come up with the oldest ideas and call them revolutionary.
"Let's invent a thing inventor, said the thing-inventor-inventor after being invented by the thing inventor."
— bill wurtz, 2017, in the video "history of the entire world, i guess"
— bill wurtz, 2017, in the video "history of the entire world, i guess"
Not sure I really understand the "development" part of it even after reading the article.
My main gripe with this is how do you charge users? They can just put the browser in offline mode and continue to use the app forever.
Also it's very hard to follow up bugs or other errors if users are often offline. I giess you can queue up errors being sent and so on but still. Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others. How do you solve merge conflicts?
I really like offline first web apps, but it is way harder and more expensive to build I think. For a startup it means more time before you can deploy your app and where I live there is pretty much fast internet everywhere so it kinda is solving an issue that very few customers will face.
Also it's very hard to follow up bugs or other errors if users are often offline. I giess you can queue up errors being sent and so on but still. Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others. How do you solve merge conflicts?
I really like offline first web apps, but it is way harder and more expensive to build I think. For a startup it means more time before you can deploy your app and where I live there is pretty much fast internet everywhere so it kinda is solving an issue that very few customers will face.
That's literally how all software development used to work. All of these, except merge conflicts, are solved by expiring licenses, collecting logs and uploading them when asked or when connectivity comes back and it doesn't matter if it doesn't, it's the user's choice.
Merge conflicts are truly the only unsolved issue and for good reasons. Notably offline IDEs work together with on-demand-online version control software to solve this problem, and also have been since forever. Hard part is getting non-text data to merge; you usually implement file-level checkout/checkin logic (see perforce, sharepoint, etc.)
Merge conflicts are truly the only unsolved issue and for good reasons. Notably offline IDEs work together with on-demand-online version control software to solve this problem, and also have been since forever. Hard part is getting non-text data to merge; you usually implement file-level checkout/checkin logic (see perforce, sharepoint, etc.)
Well, Yes. But users today are used to everything being saved remotely, especially on a website. I think customers would be very angry if suddenly their data were gone because a sync didn't go through and data were lost because they cleaned their cache on perhaps another site that then affected all sites.
People do have higher expectations on software than they used to have. There are also other issues with syncable states than the OP wrote about. Just imagine that you upgrade your api, you will have to be very backwards compatible for a long time or force users to update their web apps when they come online again.
It's easy to mess up service workers and it's possible to put stuff in a state which is almost unable to recover from unless the users affected clear their cache and then risk losing all of their data. Stuff like that can't happen with a normal web app.
While all of these things are certainly possible, I also do think that the development time is longer. You will have a hard time to compete with businesses that don't care about local first stuff since they can pump out features in a higher rate than you can because you have to care about states in a much deeper sense.
It makes sense for some apps to be local first, but not really for that many imho.
People do have higher expectations on software than they used to have. There are also other issues with syncable states than the OP wrote about. Just imagine that you upgrade your api, you will have to be very backwards compatible for a long time or force users to update their web apps when they come online again.
It's easy to mess up service workers and it's possible to put stuff in a state which is almost unable to recover from unless the users affected clear their cache and then risk losing all of their data. Stuff like that can't happen with a normal web app.
While all of these things are certainly possible, I also do think that the development time is longer. You will have a hard time to compete with businesses that don't care about local first stuff since they can pump out features in a higher rate than you can because you have to care about states in a much deeper sense.
It makes sense for some apps to be local first, but not really for that many imho.
> But users today are used to everything being saved remotely, especially on a website.
That's just a matter of training. They were trained that way. 30 years ago users were trained to "save often".
That's just a matter of training. They were trained that way. 30 years ago users were trained to "save often".
Absolutely, but then you have to be the guy training users when everyone else is training them differently.
> While all of these things are certainly possible, I also do think that the development time is longer.
I agree, but I don't think dev time is longer due to some intrinsic reality that local-first is technically harder. Nearly 2 decades of work has been put towards tailoring building blocks for SaaS systems. Similar building blocks can be made for local-first.
I agree, but I don't think dev time is longer due to some intrinsic reality that local-first is technically harder. Nearly 2 decades of work has been put towards tailoring building blocks for SaaS systems. Similar building blocks can be made for local-first.
"My main gripe with this is how do you charge users?
Sometimes, I feel so old. Recently, I read about a junior dev who did not understand that you can build websites that aren't SPAs and was confused when Chrome's debugger cleared itself on a page reload.
People have been charging users for offline apps since before "offline" was a concept that needed to exist because back then, everything* was offline.
* with exceptions.
Sometimes, I feel so old. Recently, I read about a junior dev who did not understand that you can build websites that aren't SPAs and was confused when Chrome's debugger cleared itself on a page reload.
People have been charging users for offline apps since before "offline" was a concept that needed to exist because back then, everything* was offline.
* with exceptions.
> My main gripe with this is how do you charge users? They can just put the browser in offline mode and continue to use the app forever.
Let users pay a price to download the application, do some magic key activation (that could work offline too) like countless of professional software? Ableton, Spine2D and Cascadeur are some recent software I've purchased that works just like that.
> Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others.
Yeah, I think if you have an existing application/architecture/design and you try to shoehorn in local-first with sync in there, there will be some additional work. But if you design your data structures with that in mind, it gets a lot easier. So as always, way easier for greenfield projects than migrating existing ones.
> How do you solve merge conflicts?
Depends on your application. In many cases, CRDTs (just one example) can automatically resolve things based on however you set it up. And for when that doesn't work and you need manual solving, that gets easier too as you can easily see exactly where it doesn't merge, and it gets a lot easier to build the tooling you'd expose to the user to resolve it.
Let users pay a price to download the application, do some magic key activation (that could work offline too) like countless of professional software? Ableton, Spine2D and Cascadeur are some recent software I've purchased that works just like that.
> Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others.
Yeah, I think if you have an existing application/architecture/design and you try to shoehorn in local-first with sync in there, there will be some additional work. But if you design your data structures with that in mind, it gets a lot easier. So as always, way easier for greenfield projects than migrating existing ones.
> How do you solve merge conflicts?
Depends on your application. In many cases, CRDTs (just one example) can automatically resolve things based on however you set it up. And for when that doesn't work and you need manual solving, that gets easier too as you can easily see exactly where it doesn't merge, and it gets a lot easier to build the tooling you'd expose to the user to resolve it.
> They can just put the browser in offline mode and continue to use the app forever.
You are so optimistic about browser's garbage collector! What you have said is possible, but I need a whole computer devoted to use some web-browser software in such a way. Any another web-page can make the "system" to crash and bye-bye foreverness.
It was better 10 years ago when I could buy a 1-day trial to some expensive newspaper, then open all the long-reads and read everything effectively free. Now any web-page tries to be very much smarter than the user and I would rather never use any proprietary "app" for not even seeing what the vendor has prepared to such a digital scavenger.
You are so optimistic about browser's garbage collector! What you have said is possible, but I need a whole computer devoted to use some web-browser software in such a way. Any another web-page can make the "system" to crash and bye-bye foreverness.
It was better 10 years ago when I could buy a 1-day trial to some expensive newspaper, then open all the long-reads and read everything effectively free. Now any web-page tries to be very much smarter than the user and I would rather never use any proprietary "app" for not even seeing what the vendor has prepared to such a digital scavenger.
People have figured that out for desktop applications years ago.
> My main gripe with this is how do you charge users? They can just put the browser in offline mode and continue to use the app forever.
Charge users to buy the product. Don't charge users to use the product. I know it's a revolutionary way to think.
> Also it's very hard to follow up bugs or other errors if users are often offline.
Yes, so?
> I giess you can queue up errors being sent and so on but still.
Have you asked the user if they want those errors, which often contain private information, to be sent?
> Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others. How do you solve merge conflicts?
There's tens, if not hundreds, of solutions to sync. Why don't you take a look at one of the existing solutions?
> I really like offline first web apps
Really? It sounded like you wanted to charge users and didn't like the idea of users going offline. Kind've hypocritical to then say you like offline-first.
> more expensive to build I think.
Not at all. You should be building and deploying in containers. Containers can and should be 100% local first. You download the dependencies (`FROM` statements) and then disconnect from wifi. If you can't build after that then you're doing it wrong.
> For a startup it means more time before you can deploy your app
Not if you're doing it right.
> where I live there is pretty much fast internet everywhere
I'm happy for you! Unfortunately you seem to be the type of person who wants to reduce cost. That means remote workers. Remote workers often live where their internet is very sub-par and/or metered. Local-first (and especially offline-first) will help reduce cost even more.
> it kinda is solving an issue that very few customers will face
I assume you're in the USA. You shouldn't take the FCC's connectivity reports at face value especially after certain people have gutted the FCC's ability to provide reliable connectivity data. You also shouldn't assume that every customer around you has access to fast internet because, unfortunately, last-mile internet service providers have a tendency of monopolistically abusing their customers.
If you're not in the USA... well I'd still be willing to bet that you're wrong unless your customers are exclusively in an urban environment.
Charge users to buy the product. Don't charge users to use the product. I know it's a revolutionary way to think.
> Also it's very hard to follow up bugs or other errors if users are often offline.
Yes, so?
> I giess you can queue up errors being sent and so on but still.
Have you asked the user if they want those errors, which often contain private information, to be sent?
> Syncing means that you probably have to have a complicated logic, especially if the data you are seeing can be modified by others. How do you solve merge conflicts?
There's tens, if not hundreds, of solutions to sync. Why don't you take a look at one of the existing solutions?
> I really like offline first web apps
Really? It sounded like you wanted to charge users and didn't like the idea of users going offline. Kind've hypocritical to then say you like offline-first.
> more expensive to build I think.
Not at all. You should be building and deploying in containers. Containers can and should be 100% local first. You download the dependencies (`FROM` statements) and then disconnect from wifi. If you can't build after that then you're doing it wrong.
> For a startup it means more time before you can deploy your app
Not if you're doing it right.
> where I live there is pretty much fast internet everywhere
I'm happy for you! Unfortunately you seem to be the type of person who wants to reduce cost. That means remote workers. Remote workers often live where their internet is very sub-par and/or metered. Local-first (and especially offline-first) will help reduce cost even more.
> it kinda is solving an issue that very few customers will face
I assume you're in the USA. You shouldn't take the FCC's connectivity reports at face value especially after certain people have gutted the FCC's ability to provide reliable connectivity data. You also shouldn't assume that every customer around you has access to fast internet because, unfortunately, last-mile internet service providers have a tendency of monopolistically abusing their customers.
If you're not in the USA... well I'd still be willing to bet that you're wrong unless your customers are exclusively in an urban environment.