Offline is just online with extreme latency(blog.jim-nielsen.com)
blog.jim-nielsen.com
Offline is just online with extreme latency
https://blog.jim-nielsen.com/2023/offline-is-online-with-extreme-latency/
249 comments
I agree with the sentiment, but for entirely different reasons.
An offline capable application that sometimes tries to sync can be called "Online with Extreme Latency", but that is not what true ~~Scotsman~~ offline is.
Offline/online is first and foremost about data locality and data ownership. In an online application (regardless of latency!) the source of truth is "the server". Offline application itself is the source of truth and "the server" is a slave.
The OP seems to be talking about thin vs fat clients. Fat client is still a client - it fetches data. Offline application is the data source for "the server", it's the server that has to adapt to local changes not the other way around. Naturally, this is problematic since now you have multiple sources of truth. However, shifting source of truth to "the server" does create online application with extreme latency - fat client.
An offline capable application that sometimes tries to sync can be called "Online with Extreme Latency", but that is not what true ~~Scotsman~~ offline is.
Offline/online is first and foremost about data locality and data ownership. In an online application (regardless of latency!) the source of truth is "the server". Offline application itself is the source of truth and "the server" is a slave.
The OP seems to be talking about thin vs fat clients. Fat client is still a client - it fetches data. Offline application is the data source for "the server", it's the server that has to adapt to local changes not the other way around. Naturally, this is problematic since now you have multiple sources of truth. However, shifting source of truth to "the server" does create online application with extreme latency - fat client.
I believe thinking about it as "offline vs online" obscures what's actually difficult about offline operation. It's not storage or ownership of data that's difficult, it's conflict resolution.
The longer you're disconnected, the more things diverge, the better your merging/resolution schemes/algorithms need to be.
There are a lot of ways of thinking about and approaching the problem, some of which work worse or better under different circumstances. You see it in distributed consensus, multi-master systems, version control systems, etc.
There's ongoing work in the form of things like operational transforms and CRDTs, but clearly their needs to be a lot more progress in the area.
The longer you're disconnected, the more things diverge, the better your merging/resolution schemes/algorithms need to be.
There are a lot of ways of thinking about and approaching the problem, some of which work worse or better under different circumstances. You see it in distributed consensus, multi-master systems, version control systems, etc.
There's ongoing work in the form of things like operational transforms and CRDTs, but clearly their needs to be a lot more progress in the area.
A good example of this is are mobile applications used in logistics for tracking movements within a terminal. Often the wifi environment is terrible and the operators will spend significant time not connected. They often have fat clients with local storage, and when the reconnect the sync bidirectionally - they upload whatever they have done since last connection, and they download changes others have made that may affect them in the meantime.
A lot of it is also about accessibility to someone with limited network access.
When you are developing with an always-on connection and a high-performing device, it's easy to ignore all the scenarios where someone doesn't have either one.
When you are developing with an always-on connection and a high-performing device, it's easy to ignore all the scenarios where someone doesn't have either one.
There is tool to mimic that : the venerable “Charles” proxy for instance.
But I think it’s part of webTooling in the browser
But I think it’s part of webTooling in the browser
I think I agree with the analysis. It's why aspects like data verifiability are potentially more important than fancy peer-to-peer routing and transfer protocols. Once you have Merkle proofs of data, it's just as powerful whether it is on the client or server.
Cryptographic verifiability make data locality into an optimization problem not a trust problem: https://fireproof.storage/documentation/how-the-database-eng...
Cryptographic verifiability make data locality into an optimization problem not a trust problem: https://fireproof.storage/documentation/how-the-database-eng...
Trust and anti-corruption are just two of many properties you need for data locality, arguably less important. Conflict resolution or consensus is still very, very difficult. P2P routing and transfer is a bit of a red herring. You still need consensus between your local store and the remote store. Fireproof does it using IPFS underneath which in my experience tends to be quite slow, but is a working solution.
Isn't that more like "decentralized vs centralized" though? In my experience talking with coworkers about online vs offline apps, the article's definition is what we use. Granted, I've worked exclusively in Web dev for years so maybe it's all contextual.
Yes and no, IMO. While these are different classifications, some concepts can be cross applied.
Decentralized essentially means absence of built-in omniscient entity - each end every connection is treated as one between equally correct peers. It does not mean that a pair of entities cannot establish hierarchy, it simply means they do not have to.
Take git for example. There is nothing forcing us to treat github/gitlab/etc as the omniscient node even if it is central point of data exchange - you are free to disagree with the upstream changes. Local data is always there, all local overwrites are explicit and so on. Contrast this with something like online game. The server will tell the client what their state is - there is a central entity controlling the truth.
Web can be interesting in this context, since there are multiple "tiers" of nodes: user applications, backend servers, databases. Data control and conflict resolution properties usually are different among nodes in the same tier and between tiers.
Decentralized essentially means absence of built-in omniscient entity - each end every connection is treated as one between equally correct peers. It does not mean that a pair of entities cannot establish hierarchy, it simply means they do not have to.
Take git for example. There is nothing forcing us to treat github/gitlab/etc as the omniscient node even if it is central point of data exchange - you are free to disagree with the upstream changes. Local data is always there, all local overwrites are explicit and so on. Contrast this with something like online game. The server will tell the client what their state is - there is a central entity controlling the truth.
Web can be interesting in this context, since there are multiple "tiers" of nodes: user applications, backend servers, databases. Data control and conflict resolution properties usually are different among nodes in the same tier and between tiers.
I don't view things like the article states, and I likely won't for a number of reasons other commenters have stated, plus one more:
If offline is online with with extreme latency, then "extreme latency" must include "infinitely long latency". But accommodating infinitely long latency requires a different approach than if you assume that a server will be contacted at some point.
If offline is online with with extreme latency, then "extreme latency" must include "infinitely long latency". But accommodating infinitely long latency requires a different approach than if you assume that a server will be contacted at some point.
An app I was designing about six years back would have been something that some would use in their yard and others would use in a national park.
In the yard you have extreme latency. Sooner or later you'll get thirsty and you'll walk into WiFi distance.
In a national park there's no connectivity. You won't be able to get or receive anything until long after the moment has passed. None of the answers you get will be relevant anymore for most people (what percent of people who visit a national park visit the same one over and over again?).
Then I realized screen contrast wasn't 'there' yet so I shelved it, got busy, and there it remains collecting dust.
In the yard you have extreme latency. Sooner or later you'll get thirsty and you'll walk into WiFi distance.
In a national park there's no connectivity. You won't be able to get or receive anything until long after the moment has passed. None of the answers you get will be relevant anymore for most people (what percent of people who visit a national park visit the same one over and over again?).
Then I realized screen contrast wasn't 'there' yet so I shelved it, got busy, and there it remains collecting dust.
> Offline/online is first and foremost about data locality and data ownership.
not necessarily if the data is stored in a proprietary format.
not necessarily if the data is stored in a proprietary format.
Yes, but I think most people care about availability of data over true ownership of data.
If I’m running a business and my internet goes out, there’s a huge selling point to a local DB that allows my operations to keep chugging along until Comcast figures out their mess. Internet returns, my DB syncs with a server offsite, and now I have that backup.
If I’m running a business and my internet goes out, there’s a huge selling point to a local DB that allows my operations to keep chugging along until Comcast figures out their mess. Internet returns, my DB syncs with a server offsite, and now I have that backup.
It doesn't matter if it's a proprietary format. Proprietary formats can be reverse engineered and modified and converted. There is a world of difference between a proprietary file that I have online in a form that I can't access directly (and I have no idea how it's being accessed, changed or stored by the service itself or third parties) and a file that I have on my harddrive that I have complete control over.
[deleted]
Those cases should both be solved with optimistic UI, so there’s no difference.
You can have a little checkmark to indicate that it’s synced, like many chat apps do.
> In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though.
Yeah, can attest to this. I’m working on an app[1] that strikes the trifecta: p2p, real-time and offline first. All of those things combined makes the amount of tooling, design patterns and resources available shrink to a tiny sliver compared to a typical web-based tech stack. I have researched probably 100 projects that sound promising but almost all have been a poor fit for one reason or another. I opted to build almost everything from scratch.
Kudos to the JS ecosystem. They are way ahead in this space, with rxdb, watermelon, yjs, automerge etc. Unfortunately I couldn’t use any of them because I use a split-language stack.
[1]: https://payload.app/
You can have a little checkmark to indicate that it’s synced, like many chat apps do.
> In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though.
Yeah, can attest to this. I’m working on an app[1] that strikes the trifecta: p2p, real-time and offline first. All of those things combined makes the amount of tooling, design patterns and resources available shrink to a tiny sliver compared to a typical web-based tech stack. I have researched probably 100 projects that sound promising but almost all have been a poor fit for one reason or another. I opted to build almost everything from scratch.
Kudos to the JS ecosystem. They are way ahead in this space, with rxdb, watermelon, yjs, automerge etc. Unfortunately I couldn’t use any of them because I use a split-language stack.
[1]: https://payload.app/
to add to this -
rotate CRUD mindset to CQRS, now you have a start event and an end event, this is the extent of the pending sync state which can be afforded to the user at any granularity - document level, message level, form or field level, etc.
The cost is that other view queries don't reflect the pending event, only the view that issued the command has an association to the event. Which is usually the UX you want. Consider a master/detail form app where you insert a new record into a collection, but the view is sorted/filtered/paginated and your new record does not match the criteria and therefore vanishes. That is never the right UX. A less surprising alternative UX is to limit record creation to a specific form for the business rules of creation, and that form exists in the context of a specific collection view with business rules around newly created entities of that kind, with an invariant that newly created entities will always appear at the top or bottom of that collection (e.g. new messages are always added to the bottom of the chat history view). Now the optimistic update bypasses the database and simply writes through to the view optimistically and then when the ack is eventually received it seamlessly changes state without any UI jank.
Now you can send many new messages rapidly and if the nth message fails and the rest of the messages went through you get a properly located error. With a CRUD mindset, probably the form is simply disabled until each individual insert succeeds - which means your chat app cannot keep up with you typing!
rotate CRUD mindset to CQRS, now you have a start event and an end event, this is the extent of the pending sync state which can be afforded to the user at any granularity - document level, message level, form or field level, etc.
The cost is that other view queries don't reflect the pending event, only the view that issued the command has an association to the event. Which is usually the UX you want. Consider a master/detail form app where you insert a new record into a collection, but the view is sorted/filtered/paginated and your new record does not match the criteria and therefore vanishes. That is never the right UX. A less surprising alternative UX is to limit record creation to a specific form for the business rules of creation, and that form exists in the context of a specific collection view with business rules around newly created entities of that kind, with an invariant that newly created entities will always appear at the top or bottom of that collection (e.g. new messages are always added to the bottom of the chat history view). Now the optimistic update bypasses the database and simply writes through to the view optimistically and then when the ack is eventually received it seamlessly changes state without any UI jank.
Now you can send many new messages rapidly and if the nth message fails and the rest of the messages went through you get a properly located error. With a CRUD mindset, probably the form is simply disabled until each individual insert succeeds - which means your chat app cannot keep up with you typing!
When you need a single source of truth you cannot use optimistic UI.
E.g. if the user is a realtor, then she can't tell the customer "you have now bought the house" if there is no online connection and you're waiting for a sync. You can fill out and upload the form async, but commitment must be online.
You cannot tell the customer "you might have (or might not have) bought the house, we will only know later".
E.g. if the user is a realtor, then she can't tell the customer "you have now bought the house" if there is no online connection and you're waiting for a sync. You can fill out and upload the form async, but commitment must be online.
You cannot tell the customer "you might have (or might not have) bought the house, we will only know later".
> You cannot tell the customer "you might have (or might not have) bought the house, we will only know later".
Sure you can - in fact, that's what realtors call offers. You place an offer, and you only find out later if you bought the house, often days later.
Sure you can - in fact, that's what realtors call offers. You place an offer, and you only find out later if you bought the house, often days later.
Technically correct, but you're missing the point.
Eventually you do need to be able to commit some sort of transaction and get confirmation from the counterparty.
Eventually you do need to be able to commit some sort of transaction and get confirmation from the counterparty.
This is correct but you can do tricks to change who the counterparty is, which is often useful. If, instead of putting up a yard sign and saying "this house is for sale" and waiting for someone to come by and complete the transaction, the homeowners entered a contract with a brokerage to sell the house to anyone who is willing to abide by a list of terms (minimum price, occupancy date, etc.), then the brokerage can respond immediately and say "great, the house is yours". This is essentially the equivalent of sending a sell order to a stock broker.
[deleted]
You can if you acquire a distributed lock in advance, as per https://electric-sql.com/blog/2022/05/03/introducing-rich-cr...
In this case the optimistic UI might be good to have a step like “this request is pending”. So when the form is submitted the data is ready to send and will send when it can. Ideally with an indicator that the user is offline and needs to be online to sync.
That's true even if you're example isn't perfect. Some types of transactions have business requirements that demand immediate consistency or responses. Contraindications certainly don't invalidate ideas, though. Use the right to for the job.
> Those cases should both be solved with optimistic UI, so there’s no difference.
Ah, yes, "in theory, the theory and the practice are the same". Unfortunately, in practice, the theory and the practice are not the same, and those cases are regularly not solved with optimistic UI (even though they should), so there is difference.
Ah, yes, "in theory, the theory and the practice are the same". Unfortunately, in practice, the theory and the practice are not the same, and those cases are regularly not solved with optimistic UI (even though they should), so there is difference.
Git has one interface for disconnected commits and another one for connected ones. Most of the command set is bifurcated around this. Sure there are places where you do the same act for local and remote but those are more the exception than the rule. It's very exposed. They've done very little no create any sort of abstraction across them, and that's probably the right answer.
First I heard of "optimistic UI" was with GraphQL and Apollo, but I had seen the behavior that they someone, perhaps them, gave this name for a lot before.
I don't find the concept "optimistic UI" to be more useful for talking about it or the implementation by Apollo to be more elegant. It's fine but it doesn't solve any problem except one that was created by Apollo's binding of server data to the client, React-style.
I don't find the concept "optimistic UI" to be more useful for talking about it or the implementation by Apollo to be more elegant. It's fine but it doesn't solve any problem except one that was created by Apollo's binding of server data to the client, React-style.
A related concern would be surfacing the "online" part as explicit resources and actions for the user instead of implicit, background magic. If you are sufficiently offline in your lifestyle, you need to plan your communication phases.
I recently got into using a GPS sports watch. It is the kind of thing I would want to use in an offline fashion, i.e. go somewhere off the grid and track my hikes or bike rides. These devices are designed to function offline for a stretch of time, but they have a requirement to eventually sync to an online system. They will eventually fill up with recorded sensor data and you want to offload that somewhere else before clearing local device storage. More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly.
Unfortunately, the manufacturers have been infected with smartwatch expectations. They started designing it as if it is always online, and the syncing functions are implicit background behaviors. When doing sporadic syncs and going back offline, it is hard to influence it to "get latest ephemeris" when you known you have internet connectivity and will be going offline again. Worse, these results are localized and it implicitly gets ephemeris for its current location. The UI doesn't allow you to indicate a destination and pre-load the right data to allow fully offline function on arrival.
I recently got into using a GPS sports watch. It is the kind of thing I would want to use in an offline fashion, i.e. go somewhere off the grid and track my hikes or bike rides. These devices are designed to function offline for a stretch of time, but they have a requirement to eventually sync to an online system. They will eventually fill up with recorded sensor data and you want to offload that somewhere else before clearing local device storage. More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly.
Unfortunately, the manufacturers have been infected with smartwatch expectations. They started designing it as if it is always online, and the syncing functions are implicit background behaviors. When doing sporadic syncs and going back offline, it is hard to influence it to "get latest ephemeris" when you known you have internet connectivity and will be going offline again. Worse, these results are localized and it implicitly gets ephemeris for its current location. The UI doesn't allow you to indicate a destination and pre-load the right data to allow fully offline function on arrival.
> More importantly, satellite positioning receivers need a cached "ephemeris" file that helps them predict which satellites will be overhead at a given time to operate efficiently, accurately, and quickly.
Interesting, I've never heard of that particular optimization for GNSS before. I know GPS transmits ephemeris information in each frame, since that's the input data for the positioning calculations. I've got a number of Garmin watches, and they've always been able to get a position fix even after being disconnected for weeks.
I find GNSS implementations very interesting, which manufacturer is making their watches like this?
Interesting, I've never heard of that particular optimization for GNSS before. I know GPS transmits ephemeris information in each frame, since that's the input data for the positioning calculations. I've got a number of Garmin watches, and they've always been able to get a position fix even after being disconnected for weeks.
I find GNSS implementations very interesting, which manufacturer is making their watches like this?
Garmin watches do! Go into the System -> About menu and there is a page showing ephemeris status.
Their documentation states that this may expire after approximately 30 days or if you travel more than 200 miles, while it will update during syncing.
I've seen it expire in less than two weeks with daily use of GPS but phone syncing disabled. It will still get position, but it can be the difference between an almost immediate fix after opening an activity menu or a delay for tens of seconds to minutes. Distance and pace measurements also seem to be lower quality when operating without a current ephemeris file.
Their documentation states that this may expire after approximately 30 days or if you travel more than 200 miles, while it will update during syncing.
I've seen it expire in less than two weeks with daily use of GPS but phone syncing disabled. It will still get position, but it can be the difference between an almost immediate fix after opening an activity menu or a delay for tens of seconds to minutes. Distance and pace measurements also seem to be lower quality when operating without a current ephemeris file.
> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online.
Isn't that exactly what the article argues for though?
> This kind of idea would move you away from a product full of API calls to one based on data synchronization.
Isn't that exactly what the article argues for though?
> This kind of idea would move you away from a product full of API calls to one based on data synchronization.
> You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online.
I'm not sure you got the point. The design guideline that "Offline Is Just Online with Extreme Latency" already reflects very specific architectural requirements, and state transitions in the application life cycle. We're talking event-driven architectures, batching events, flushing events in offline/online transitions or even when minimizing windows, pulling events when going online, etc etc etc.
I'd go even further and claim that this whole "pessimistic vs optimistic UI" thing is just "adequate vs broken UI design",regardless whether the app is even expected to go online.
I'm not sure you got the point. The design guideline that "Offline Is Just Online with Extreme Latency" already reflects very specific architectural requirements, and state transitions in the application life cycle. We're talking event-driven architectures, batching events, flushing events in offline/online transitions or even when minimizing windows, pulling events when going online, etc etc etc.
I'd go even further and claim that this whole "pessimistic vs optimistic UI" thing is just "adequate vs broken UI design",regardless whether the app is even expected to go online.
The really hard part is conflict resolution. You updated something 5 times while offline, but another user deleted that something between updates 2 and 3, or made a divergent update of their own. There are so many potential scenarios like this in a multi-user app. It's a huge rabbit hole, and you can end up in situations where it's impossible to sync in a user-friendly way. Essentially you are taking on all the challenges of distributed databases. Maybe it's worth it, but you should know what you're getting into.
Online-only collaboration systems already have to do this. The local view has to synchronize state between other local views and the canonical version on the server. Offline changes are online edits with high latency.
Yes, but the problem gets much harder to resolve the longer you allow client states to diverge.
I think you've got it spot on.
I've also worked on systems like that where a mobile user could loose connection at any time and they are indeed very complex to get right.
I'm not sure there's a perfect way to do it but we ended having have certain functions that had to be done online and others where the user built a "request" for service that was handled optimistically with failures sent to the user's inbox later.
I've also worked on systems like that where a mobile user could loose connection at any time and they are indeed very complex to get right.
I'm not sure there's a perfect way to do it but we ended having have certain functions that had to be done online and others where the user built a "request" for service that was handled optimistically with failures sent to the user's inbox later.
Sounds like it is a useful concept: you just used it to introduce your own.
I think that optimistic UI isn't more complex to build: it's just less familiar.
In a broader sense, UI/UX design culture places way too much focus and value on familiarity.
The best UI/UX designs I have ever experienced are when software design diverges from the norm and tries a new approach.
I think that optimistic UI isn't more complex to build: it's just less familiar.
In a broader sense, UI/UX design culture places way too much focus and value on familiarity.
The best UI/UX designs I have ever experienced are when software design diverges from the norm and tries a new approach.
> I think that optimistic UI isn't more complex to build: it's just less familiar.
Having multiple records of state that have to be resolved when systems come online is complex. Anything requiring consensus is difficult to do.
Having multiple records of state that have to be resolved when systems come online is complex. Anything requiring consensus is difficult to do.
The same complexity is present in both approaches. The only difference is cadence.
Not exactly - there’s a bunch of non-trivial stuff you need to worry about when both local and remote states represent sources of truth. Things like CRDTs and event sourcing make it easier, but there’s still more complexity than dealing with only one source of truth.
In an online-first experience, any local state being held is (for the most part) an optimization or is intentionally ephemeral. You can just toss it away at a performance penalty if it ever gets too hairy.
In an offline-first experience, you need to be very careful that you treat everything like a source of truth. You also need to deal with schema migrations and business logic migrations, since you need those to partially live on the client.
In an offline-first experience, you need to be very careful that you treat everything like a source of truth. You also need to deal with schema migrations and business logic migrations, since you need those to partially live on the client.
Was required to use a MS sql server database for a project that would go offline for 5-10 minutes every hour. (Getting admins to fix it was a no go).
Now of course I was judged on uptime of my app that relied on it.
Finally just cloned the data to a local MySQL and refreshed data frequently. Database team totally clueless that app servers were hosting their own databases.
Now of course I was judged on uptime of my app that relied on it.
Finally just cloned the data to a local MySQL and refreshed data frequently. Database team totally clueless that app servers were hosting their own databases.
You'd have to make sure the user knew whether it finally worked even weeks later or never sent. They'll forget.
Abstracting this away from the user is what meteor was built to solve, and did it very well for most use cases.
Online is just offline with
extreme latency for any new data fetch/processing (an http roundtrip or more for each interaction with your data)
someone else being in control of the programs you use, forcing changes and updates whenever they like, renting them as a service,
your data hostage, snooped at, sold to advertisers
a limited UI experience based on the DOM and browser APIs
crappy OS integration
(often) ads
extreme latency for any new data fetch/processing (an http roundtrip or more for each interaction with your data)
someone else being in control of the programs you use, forcing changes and updates whenever they like, renting them as a service,
your data hostage, snooped at, sold to advertisers
a limited UI experience based on the DOM and browser APIs
crappy OS integration
(often) ads
Right.
There's a great deal of technically unnecessary complexity that many people here handwave away because "that's just how it is."
It's kind of funny how, e.g. above, the emphasis is on "UI," when that's not it at all. It's the underlying infrastructure that's often encrusted layers of unnecessary (but profitable) stupid.
There's a great deal of technically unnecessary complexity that many people here handwave away because "that's just how it is."
It's kind of funny how, e.g. above, the emphasis is on "UI," when that's not it at all. It's the underlying infrastructure that's often encrusted layers of unnecessary (but profitable) stupid.
This ^
Anyone who thinks UI is where the latency and complexity resides should do a 3-mo residency at a cloud provider running actual infrastructure--not VMs and the like, but the actual routers, backbone links, service provider peering, and servers. It'll change your perspective on complexity for sure.
Anyone who thinks UI is where the latency and complexity resides should do a 3-mo residency at a cloud provider running actual infrastructure--not VMs and the like, but the actual routers, backbone links, service provider peering, and servers. It'll change your perspective on complexity for sure.
> (often) ads
This is a much bigger problem than people realize. Instead of working on improving user experience, we've got people explicitly working against user experience by injecting ads into it, for the simple reason that they have your attention and can then sell it. It's the attention economy, and it sucks. Fuck ads.
This is a much bigger problem than people realize. Instead of working on improving user experience, we've got people explicitly working against user experience by injecting ads into it, for the simple reason that they have your attention and can then sell it. It's the attention economy, and it sucks. Fuck ads.
There would be no user experience if there were no ads. Working on ads can improve the user experience by making them more relevant.
User Experience exists in many spaces that lack ads. Ads don't inherently improve UX, nor are they required for a good experience.
My point was that if there was no ads the app would shut down and would not be usable.
I exclusively use apps without ads, and they all still exist, so I have no clue what you're talking about.
Most desktop apps still don't have ads, and their UX is generally superior to the web.
I think people are really only interested in commercial messages when they come at a much lower frequency than they do these days. Other than that, it’s just noise that we suffer through in order to get the information or content that we want.
>Other than that, it’s just noise that we suffer through in order to get the information or content that we want.
The trailing part that's left off is "for free". If you want all the content we receive for free today, someone has to pay for it. So either people pay with their attention or their wallet, and consumers have by and large made their choice of ad supported services over paid alternatives.
The trailing part that's left off is "for free". If you want all the content we receive for free today, someone has to pay for it. So either people pay with their attention or their wallet, and consumers have by and large made their choice of ad supported services over paid alternatives.
I pay for software and the developers force in ads later. It's not either or, it's just racing to the bottom, forever.
> (often) ads
This one doesn't have much to do with online vs. offline except that you have to be online to see the "latest and greatest" ads. Stale ads can perfectly well be shown offline.
This one doesn't have much to do with online vs. offline except that you have to be online to see the "latest and greatest" ads. Stale ads can perfectly well be shown offline.
> Stale ads can perfectly well be shown offline.
They can, but an Android app I use regularly simply won't show any ads if it cannot access the internet - I get its premium version for free just by pressing a button. Meanwhile I've seen the user-hostile version of that on an iPad years ago, where an app simply locked itself until it was able to download ads.
They can, but an Android app I use regularly simply won't show any ads if it cannot access the internet - I get its premium version for free just by pressing a button. Meanwhile I've seen the user-hostile version of that on an iPad years ago, where an app simply locked itself until it was able to download ads.
Traditionally they weren't on offline software though. It's an online-era thing, and it thrives on online-ness, both for getting the "new" ads and for sending impressions and telemetry back. Even on modern offline-software-with-ads it's usually a glorified web view.
...and more surveillance.
Whoa now. Don't go after the cash cow! It's so much harder to rent-seek when people just run regular programs that don't require 10 gigs of RAM and internet access. But where's the career promotion in that?
He is not talking about offline applications but online applications with offline functionality.
Since the rise of cloud applications and electron based applications I'm back at writing faster than the application can process my input. And I'm not a fast typist.
Since the rise of cloud applications and electron based applications I'm back at writing faster than the application can process my input. And I'm not a fast typist.
> Since the rise of cloud applications and electron based applications I'm back at writing faster than the application can process my input. And I'm not a fast typist.
If the application you're using (the one(s) you're referring to when you say "cloud applications") is sending the key input to the backend and waits to render it client-side until it received a response, it's doing something horribly wrong and if that's how they write things, you probably want to avoid that for a multitude of reasons. Please name and shame though, so others can avoid it too :)
Same goes for Electron-based applications, there is no reason they'd block showing the text you inputted, and if they just use <input/> without doing anything egregious, you really shouldn't be able to type faster than characters appearing on the screen, something is really, really wrong then, and I don't think it's because of Electron.
If the application you're using (the one(s) you're referring to when you say "cloud applications") is sending the key input to the backend and waits to render it client-side until it received a response, it's doing something horribly wrong and if that's how they write things, you probably want to avoid that for a multitude of reasons. Please name and shame though, so others can avoid it too :)
Same goes for Electron-based applications, there is no reason they'd block showing the text you inputted, and if they just use <input/> without doing anything egregious, you really shouldn't be able to type faster than characters appearing on the screen, something is really, really wrong then, and I don't think it's because of Electron.
1Password’s new UI does just that.
I enter some text into a field quickly, hit Tab, then watch some of the trailing characters (or the entire field content) disappear.
This keeps occurring and disgusting me to no end.
This is why I'm sticking with 1Password 7. They can pry it from my cold dead hands.
> If the application you're using (the one(s) you're referring to when you say "cloud applications") is sending the key input to the backend and waits to render it client-side until it received a response
I'm 99% sure nobody does that, that would be insane :D.
It's most likely caused by React and the use of controlled input components without thinking about performance. If your input sets it's value from React state, then you're updating the state on every keypress and re-rendering the component. In the most simple case, it's probably fine. But depending on your component hierarchy, it's pretty easy to end up in situation with over 50ms of typing latency.
I'm 99% sure nobody does that, that would be insane :D.
It's most likely caused by React and the use of controlled input components without thinking about performance. If your input sets it's value from React state, then you're updating the state on every keypress and re-rendering the component. In the most simple case, it's probably fine. But depending on your component hierarchy, it's pretty easy to end up in situation with over 50ms of typing latency.
> I'm 99% sure nobody does that, that would be insane :D.
I imagine you haven't seen how people implement autocomplete on web forms ;).
I imagine you haven't seen how people implement autocomplete on web forms ;).
> I'm 99% sure nobody does that, that would be insane :D.
I agree, hence the surprise!
Although parent explicitly calls out "cloud applications" so I'm guessing online has something to do with it.
And in the context of online/offline, it wouldn't matter if it's internet connected or not, using controlled input components with state changes going through a deep hierarchy would introduce that sort of delay no matter if you're online or offline.
I agree, hence the surprise!
Although parent explicitly calls out "cloud applications" so I'm guessing online has something to do with it.
And in the context of online/offline, it wouldn't matter if it's internet connected or not, using controlled input components with state changes going through a deep hierarchy would introduce that sort of delay no matter if you're online or offline.
> Please name and shame though, so others can avoid it too :)
MS Teams does this when starting a new chat - you select the recipient and start typing and Teams will do two unexpected things shortly afterwards:
MS Teams does this when starting a new chat - you select the recipient and start typing and Teams will do two unexpected things shortly afterwards:
1) Clear the text you typed, and
2) Reset the cursor to the start of the lineAnd this is one of the many reasons why i very much dislike MS Teams!
VS Code is still much slower than necessary despite editing files locally - on my Thinkpad from 2016 I got extremely inconsistent input latency that felt like a shaky SSH session, in better cases it seems to be closer to 30ms[0]. For reference: gVim's latency hovers around 1ms and even intelliJ can still be 10x faster[1].
[0] https://github.com/microsoft/vscode/issues/161622#issuecomme...
[1] https://pavelfatin.com/typing-with-pleasure/#windows
[0] https://github.com/microsoft/vscode/issues/161622#issuecomme...
[1] https://pavelfatin.com/typing-with-pleasure/#windows
Gmail has had this issue for years, if you type in the search bar a bit fast or the system is a bit busy, some of your key presses will trigger keyboard shortcuts and randomly select/archive/mark as read/open emails in the current view. This is maddening because you can end up missing emails completely as there is only a single level of undo.
Seen that on mattermost, it is poorly written React or other SPA. It is one of reason why SPA get so much hate around.
I issues with Outlook Web App (M365) often. Dropped characters are not so common, but it is common for the cursor to bug out and insist on returning the to start of a line with each new character.
Are you using Firefox? I had that problem consistently with OWA, but I recently switched to primarily Safari on an old MacBook and don’t recall having it happen recently.
It's more if I edit data grids or multiple fields where I switch between them with the tab key.
On desktop applications they may lack in the response on the screen if they or the computer are to slow, but with cloud apps sometimes keystrokes go missing and the entered data is in the wrong field.
The operating system recognises all keystrokes, but the browser seems to skip some.
On desktop applications they may lack in the response on the screen if they or the computer are to slow, but with cloud apps sometimes keystrokes go missing and the entered data is in the wrong field.
The operating system recognises all keystrokes, but the browser seems to skip some.
This is increasingly happening with desktop apps as they get rewritten to be async throughout without a good understanding of what this implies. The old single-threaded UI model with a message pump was hard on the developer (if they wanted to keep the UI response) by requiring them to manage all asynchrony explicitly with threads, but at the same time it also forced to think things through more. Now it's often just throwing async/await or equivalent around mindlessly, and it leads to the same exact kind of bugs you describe.
Allow me to remind you how this happens.
https://news.ycombinator.com/item?id=13940014
https://news.ycombinator.com/item?id=13940014
I think that’s the take and from my own experience, it kind of sucks and feels like we’re running worst versions of basic software than we used to (old man rant?).
For example, I use Apple Music every time I leave my apartment. Put my headphones on and start some music. All my songs are downloaded locally so it should be fine, however, as soon as I close my front door until after I step off the elevator, no music will even attempt to play. That’s because Apple Music is blocking itself on some sort of network call/drm check because it thinks I’m online and it should work.
If I go into airplane mode and hit play, everything works perfectly.
For example, I use Apple Music every time I leave my apartment. Put my headphones on and start some music. All my songs are downloaded locally so it should be fine, however, as soon as I close my front door until after I step off the elevator, no music will even attempt to play. That’s because Apple Music is blocking itself on some sort of network call/drm check because it thinks I’m online and it should work.
If I go into airplane mode and hit play, everything works perfectly.
"online applications with offline functionality" or "offline applications with online functionality"?
In the nitty gritty they are the same thing, but the way you say it changes the way you picture it and what parts you make sync vs async
In the nitty gritty they are the same thing, but the way you say it changes the way you picture it and what parts you make sync vs async
FTA:
> Unfortunately, some of the open technologies for building more local-first, p2p applications just aren’t there yet. Like Peter’s point on webRTC.
I guffawed at this. I mean, seriously. In the days before the internet was widespread, like mid-90s, there were actually multi-player games that worked over LANs, both IP and IPX. Somehow, people were able to configure multiple computers to talk to each other with this funny numbers called "addresses" or something. DNS existed of course, but you could also add to /etc/hosts, Novell had a thing, everyone had a thing.
The problem is that somewhere around the mid 2000s or early 2010s we forgot that anything networking-related could be based around non-web protocols and stacks. What a disservice we've done ourselves.
Not that I'm advocating people type in IP addresses or have to set up their own DNS servers or admin their own LANs, but wow, it's like part of our collective brains is just missing.
> Unfortunately, some of the open technologies for building more local-first, p2p applications just aren’t there yet. Like Peter’s point on webRTC.
I guffawed at this. I mean, seriously. In the days before the internet was widespread, like mid-90s, there were actually multi-player games that worked over LANs, both IP and IPX. Somehow, people were able to configure multiple computers to talk to each other with this funny numbers called "addresses" or something. DNS existed of course, but you could also add to /etc/hosts, Novell had a thing, everyone had a thing.
The problem is that somewhere around the mid 2000s or early 2010s we forgot that anything networking-related could be based around non-web protocols and stacks. What a disservice we've done ourselves.
Not that I'm advocating people type in IP addresses or have to set up their own DNS servers or admin their own LANs, but wow, it's like part of our collective brains is just missing.
I think it’s more than forgetting. I didn’t forget for sure. But what is easily forgotten is why we moved from local p2p to internet brokered services. It’s mostly simplicity. By hosting all the state and discovery, match making, etc, in a remote service outside the local network and RF environment, you dramatically simplify everything to “configure the local tcp stack.” The 50’s-2010 world wasn’t awesome and things were super fragile, tedious, poor compatibility, etc etc. The move to service based approach meant protocols were homogenous and compatible, implementations were simple, state management easy, reliability near perfect, etc. You even get to ensure everyone has basically the same software since all the real software is in the service and there’s exactly one version on earth. In the bad old days you referred to, every device around you had some ragged edge version dependency graph of everything from the device to its network card through to the end user experience. It was a miracle anyone ever played Red Alert on the LAN.
People always talk about “how expensive” it is to send packets to Oregon to turn on your lightbulb in seattle. But the cost is already fully loaded and paid for. If you didn’t notice it, maybe that’s because it’s actually not more expensive? Yes latencies are higher, and that’s literally the only downside I can imagine. But it is a lot more reliable, flexible, simpler to deploy in any environment, robust.
People always talk about “how expensive” it is to send packets to Oregon to turn on your lightbulb in seattle. But the cost is already fully loaded and paid for. If you didn’t notice it, maybe that’s because it’s actually not more expensive? Yes latencies are higher, and that’s literally the only downside I can imagine. But it is a lot more reliable, flexible, simpler to deploy in any environment, robust.
> It’s mostly simplicity.
You make points below that are good, but let's be clear. You're talking about simplicity of interop and configuration that's gotten better, rather than simplicity of the software stack. There are so many variables that have changed by adopting the web stack that I don't think any other dimension of simplicity has been achieved.
If we had instead focused networking protocol design on the simplicity of configuration, discovery, and naming, then we'd have been better off. It's not like you need an electron stack, an apache stack, and a cloudy (planet megascale!) datastore for that.
You make points below that are good, but let's be clear. You're talking about simplicity of interop and configuration that's gotten better, rather than simplicity of the software stack. There are so many variables that have changed by adopting the web stack that I don't think any other dimension of simplicity has been achieved.
If we had instead focused networking protocol design on the simplicity of configuration, discovery, and naming, then we'd have been better off. It's not like you need an electron stack, an apache stack, and a cloudy (planet megascale!) datastore for that.
I think the simplicity story is more than that. First, state management is a tricky thing to do in any environment, but in an environment of a billion random variables it’s tedious if not impossible. I’d assert that each of those complexities you highlight are actually solutions to a specific challenge in peer to peer interaction between computers that are simpler in the aggregate than if they didn’t exist and we had just focused on, say, making Bluetooth not be a massive annoyance. The ability to simply ensure a tcp stack is configured then be done with the entire end to end problem of peer to peer collaboration across platforms, devices, stacks, local RF environment, LAN configuration, and it includes durability and available assurances, etc, are enormous.
I also think it’s kinda not true that protocols have stayed stationary in that time. Certainly Bluetooth has improved, there’s lot do innovation on small device adhoc networks, there even a lot of innovation on internet protocols over the last 5 or so years. My observation (having been at Netscape at the time) was the real demise of network and internet protocols until late was the MSFT destruction of Netscape. We were a huge driver of protocol innovation and had demonstrated there’s gold in them thar networks. We hired a lot of protocol and standard developers and built products that implemented as close as our talent allowed to the spec. It was a heady time. Microsoft realized at the time this push really threatened their core business model. They so publicly and roundly pwn’ed us out of existence that it created a massive chill in anything internet tech related (most of us went into e-commerce as a result). It wasn’t until sort of recently Google self servingly started bullying standards out the door that things sort of unwedged again and despite the kind of sad way Google primed the pump, things are really exciting right now on the internet.
I’ll note that our Mozilla play was a really important effort on our part to ensure that culture we had built around open internet standards and technologies could stay afloat into the present. I’ve been happy to watch Gecko and Xul and others grow from their sapling state to where it is now. It also prevented Microsoft from establishing a real hegemony in internet standards. But most of all, I am so happy to see a real C/C++ replacement candidate emerge (Rust) from that effort. It was a real thumb in Microsoft’s eye to one up their $1B free browser anticompetitive play with taking our then worthless IP and establishing a long lived foundation around it.
I also think it’s kinda not true that protocols have stayed stationary in that time. Certainly Bluetooth has improved, there’s lot do innovation on small device adhoc networks, there even a lot of innovation on internet protocols over the last 5 or so years. My observation (having been at Netscape at the time) was the real demise of network and internet protocols until late was the MSFT destruction of Netscape. We were a huge driver of protocol innovation and had demonstrated there’s gold in them thar networks. We hired a lot of protocol and standard developers and built products that implemented as close as our talent allowed to the spec. It was a heady time. Microsoft realized at the time this push really threatened their core business model. They so publicly and roundly pwn’ed us out of existence that it created a massive chill in anything internet tech related (most of us went into e-commerce as a result). It wasn’t until sort of recently Google self servingly started bullying standards out the door that things sort of unwedged again and despite the kind of sad way Google primed the pump, things are really exciting right now on the internet.
I’ll note that our Mozilla play was a really important effort on our part to ensure that culture we had built around open internet standards and technologies could stay afloat into the present. I’ve been happy to watch Gecko and Xul and others grow from their sapling state to where it is now. It also prevented Microsoft from establishing a real hegemony in internet standards. But most of all, I am so happy to see a real C/C++ replacement candidate emerge (Rust) from that effort. It was a real thumb in Microsoft’s eye to one up their $1B free browser anticompetitive play with taking our then worthless IP and establishing a long lived foundation around it.
> In the days before the internet was widespread, like mid-90s, there were actually multi-player games that worked over LANs, both IP and IPX. Somehow, people were able to configure multiple computers to talk to each other with this funny numbers called "addresses" or something.
In the scheme of things very few people were able to set this up. It's definitely unfortunate that so much software no longer works without a server component you dont control, but one of the major reasons for it is that the typical user can now do it because they don't have to set up the networking or understand much about it. A little later software companies realised they'd actually rather keep control and stopped distributing the server component at all.
In the scheme of things very few people were able to set this up. It's definitely unfortunate that so much software no longer works without a server component you dont control, but one of the major reasons for it is that the typical user can now do it because they don't have to set up the networking or understand much about it. A little later software companies realised they'd actually rather keep control and stopped distributing the server component at all.
The bar is higher now which is what has really changed. Your thing has to work on every network, through multiple levels of NAT, without copy-pasting ip addresses or updating system files, often across multiple platforms.
WebRTC is the focus because it won by a huge margin, other p2p protocols are a footnote. Everything uses WebRTC whether it actually has a web client or not.
WebRTC is the focus because it won by a huge margin, other p2p protocols are a footnote. Everything uses WebRTC whether it actually has a web client or not.
What happen is that dads realize that it's easier to write , maintain, and monetize multi-client software that runs primarily on the dev's computer and pushes some of the UI to the client, than multi-client software that runs primarily in the client computers.
What's old is new again.
Remember when there was no "cloud" and just locally executing software with local persistence?
I want very much to return to that. There are a few use cases for me, personally, where SaaS / "the cloud" makes sense but for 99% of what I use a computer for, I want local only. I don't want my application to require an active Internet connection. I don't want it sending data to someone else's computer. Probably most importantly, I don't want the software that I pay for to change on me without my direct opt-in.
The fact that we have articles about offline-first, cloud is "optional" is hilarious when this was just normal every day software only a few short years ago. I can't stand how trend-happy our industry is. "The Cloud" and "SaaS" solved a few problems for a few people ... but we went all in as if there were no alternatives and those of us who don't require mobility and would prefer that we can choose when to "upgrade" and what data to "share" were left behind.
Remember when there was no "cloud" and just locally executing software with local persistence?
I want very much to return to that. There are a few use cases for me, personally, where SaaS / "the cloud" makes sense but for 99% of what I use a computer for, I want local only. I don't want my application to require an active Internet connection. I don't want it sending data to someone else's computer. Probably most importantly, I don't want the software that I pay for to change on me without my direct opt-in.
The fact that we have articles about offline-first, cloud is "optional" is hilarious when this was just normal every day software only a few short years ago. I can't stand how trend-happy our industry is. "The Cloud" and "SaaS" solved a few problems for a few people ... but we went all in as if there were no alternatives and those of us who don't require mobility and would prefer that we can choose when to "upgrade" and what data to "share" were left behind.
I fondly remember locally-executing software with local persistence by default.
To provide a counterpoint, though, during that time my entire family shared one desktop PC. Looking around me, right now: work macbook, personal linux laptop, smartphone, steam deck. I suspect that the workflows that were "fine" in 1995 would really wear on me today. Especially the ones that involve migrating documents from place-to-place for collaboration while trying to maintain a canonical master copy somehow.
Today, because of de-facto reliance on the cloud, "setting up" a new machine - regardless of its OS - takes me about 20 minutes. If my laptop fell off of my bike, that would suck, but I wouldn't irretrievably lose important data.
There are downsides to the current cloud-first paradigm too, of course. But I don't think it's _all_ downside.
To provide a counterpoint, though, during that time my entire family shared one desktop PC. Looking around me, right now: work macbook, personal linux laptop, smartphone, steam deck. I suspect that the workflows that were "fine" in 1995 would really wear on me today. Especially the ones that involve migrating documents from place-to-place for collaboration while trying to maintain a canonical master copy somehow.
Today, because of de-facto reliance on the cloud, "setting up" a new machine - regardless of its OS - takes me about 20 minutes. If my laptop fell off of my bike, that would suck, but I wouldn't irretrievably lose important data.
There are downsides to the current cloud-first paradigm too, of course. But I don't think it's _all_ downside.
Yeah I never said it was "all" downside.
I never migrated the majority of what I do to "the cloud" and I have multiple devices, laptops etc. in my house. Setting up a new Linux install takes me about 20 minutes. Wouldn't know about Windows. Then again, I probably use way fewer "services" / "apps" than most people. It's funny, I've always been a very tech savvy computer nerd. I've developed software for a living for 25 years. But the older I get and the more the industry changes the less I find I use "modern tech" as a consumer.
I'm also not thinking back to 1995. This all started, in my memory, following the "mobile revolution" at the tail-end of the 00s. That's when more and more software that I use every day stopped selling perpetual licenses and started charging monthly subscriptions. Everything went "mobile first" and "cloud first" and it got more and more persistent as the 2010s went on.
On a positive note, there is some software that used to be inaccessible due to price, like Avid's Pro Tools, that I can now afford thanks to the switch in pricing models.
But even if it took me an entire weekend to set up a new device, I think that I would still prefer that over needing literally everything to have Internet access. It removes the control from me, the user, and places it in the hands of "the corporation." They can change the software without my consent. They can suffer outages (to be fair we can have local hardware failures too but it's under my control which makes a difference). I remember almost returning my PS4 when it required me to connect it to the Internet just to be able to use it on first boot.
I never migrated the majority of what I do to "the cloud" and I have multiple devices, laptops etc. in my house. Setting up a new Linux install takes me about 20 minutes. Wouldn't know about Windows. Then again, I probably use way fewer "services" / "apps" than most people. It's funny, I've always been a very tech savvy computer nerd. I've developed software for a living for 25 years. But the older I get and the more the industry changes the less I find I use "modern tech" as a consumer.
I'm also not thinking back to 1995. This all started, in my memory, following the "mobile revolution" at the tail-end of the 00s. That's when more and more software that I use every day stopped selling perpetual licenses and started charging monthly subscriptions. Everything went "mobile first" and "cloud first" and it got more and more persistent as the 2010s went on.
On a positive note, there is some software that used to be inaccessible due to price, like Avid's Pro Tools, that I can now afford thanks to the switch in pricing models.
But even if it took me an entire weekend to set up a new device, I think that I would still prefer that over needing literally everything to have Internet access. It removes the control from me, the user, and places it in the hands of "the corporation." They can change the software without my consent. They can suffer outages (to be fair we can have local hardware failures too but it's under my control which makes a difference). I remember almost returning my PS4 when it required me to connect it to the Internet just to be able to use it on first boot.
> But the older I get and the more the industry changes the less I find I use "modern tech" as a consumer.
It may be a "get out of my lawn!!" reaction, but I find modern software distasteful.
I want to use software that enable me to invent and do nice things. Not software that locks me in a pre-designed process. Even if most of the time I don't go inventing, and if the pre-designed process is nice, that difference still sores me.
It may be a "get out of my lawn!!" reaction, but I find modern software distasteful.
I want to use software that enable me to invent and do nice things. Not software that locks me in a pre-designed process. Even if most of the time I don't go inventing, and if the pre-designed process is nice, that difference still sores me.
You aren't alone. Open ended systems are usually far more powerful and interesting than rigid ones. I need an extremely compelling reason to even consider picking up a new online-only/saas or similar tool.
Investing time and energy into a tool that can be ruined or lost at any point in the future, or might turn out to be too limited, is a huge risk. It doesn't take getting burned too many times to start getting a lot more wary.
Investing time and energy into a tool that can be ruined or lost at any point in the future, or might turn out to be too limited, is a huge risk. It doesn't take getting burned too many times to start getting a lot more wary.
All good points.
I suspect preferences will hinge on peoples' budgets for personal responsibility. As my non-digital responsibilities have increased, I've found it nice to be able to delegate to "the cloud" - even at the loss of independence & control.
If there were a personally-owned "cloud" setup, I would prefer that. A box that plugs into my fiber connection and provides the equivalents of the cloud services I use, with data stored locally and backed up automatically to a secure server. A man can dream.
I suspect preferences will hinge on peoples' budgets for personal responsibility. As my non-digital responsibilities have increased, I've found it nice to be able to delegate to "the cloud" - even at the loss of independence & control.
If there were a personally-owned "cloud" setup, I would prefer that. A box that plugs into my fiber connection and provides the equivalents of the cloud services I use, with data stored locally and backed up automatically to a secure server. A man can dream.
Such a thing exists, but building and maintaining that come at a pretty high cost to your personal time. There’s very little that we do in the cloud that doesn’t have an on-prem (at home) equivalent. You could even rent servers at a CoLo or something and provide yourself regional resiliency, etc.
There's no reason why it should come at a pretty high cost to one's personal time, though. A plug-and-play box with a well-defined API for storage and sync is not an insurmountable engineering problem. The economics of it is why we don't have one, yet.
Hear hear. I often dream of having a home setup with everything but my web browser blocked from accessing the internet, and exception granted as they are necessary.
Alas, that's a pipe dream because of the way things work now, and I don't necessarily want to inflict a luddite lifestyle on my wife and children. But the allure is still there to setup some crackpot scheme that will keep me from idly pulling out my phone to check whatever just because my brain is idle for 5 seconds.
Alas, that's a pipe dream because of the way things work now, and I don't necessarily want to inflict a luddite lifestyle on my wife and children. But the allure is still there to setup some crackpot scheme that will keep me from idly pulling out my phone to check whatever just because my brain is idle for 5 seconds.
I keep seeing this argument here but I just don't buy it. Most companies don't want to operate all the infrastructure they need to run their core workloads. Managed cloud services are such a massive boon to small to medium sized companies. They're great even for large companies for many scenarios.
The irony of many libraries and frameworks trying to recreate the offline-first syncing and replication that Lotus Notes had more or less solved should be better known.
Bonus points for the irony of LotusScript being ECMAScript, which is also JavaScript.
Oh, and Notes generating the native app as a web app natively was pretty fluttery.
I have to stop, because I can no longer remember why people were switching away from Notes so furiously. And there was probably a good reason like IIS.
NotesSQL was a great little ODBC driver to make relational db calls from the NoSQL backend of Notes. Many folks owe their sharpness in the universal standards of ANSI SQL to get data ported to MySQL.
Edit:
https://en.m.wikipedia.org/wiki/HCL_Domino
Bonus points for the irony of LotusScript being ECMAScript, which is also JavaScript.
Oh, and Notes generating the native app as a web app natively was pretty fluttery.
I have to stop, because I can no longer remember why people were switching away from Notes so furiously. And there was probably a good reason like IIS.
NotesSQL was a great little ODBC driver to make relational db calls from the NoSQL backend of Notes. Many folks owe their sharpness in the universal standards of ANSI SQL to get data ported to MySQL.
Edit:
https://en.m.wikipedia.org/wiki/HCL_Domino
> Remember when there was no "cloud" and just locally executing software with local persistence?
Remember it? That's still my reality, because for almost everything I do the cloud and SaaS are not good solutions.
Remember it? That's still my reality, because for almost everything I do the cloud and SaaS are not good solutions.
> solved a few problems for a few people
Absolute rubbish. The proliferation of SaaS has been absolutely huge for business productivity, in fact, I would argue that my professional services business doesn't exist at anywhere near its profitability without it.
> I don't want my application to require an active Internet connection. I don't want it sending data to someone else's computer. Probably most importantly, I don't want the software that I pay for to change on me without my direct opt-in.
What software specifically? How to collaborate on literally anything business wise without a internet connection?! Now, is there a case for offline software? ABSOLUTELY. But to call this whole thing a fad and a trend is absurdly disingenuous.
EDIT:
Just saw your reply below:
I have multiple devices, laptops etc. in my house. Setting up a new Linux install takes me about 20 minutes. Wouldn't know about Windows. Then again, I probably use way fewer "services" / "apps" than most people. It's funny, I've always been a very tech savvy computer nerd. I've developed software for a living for 25 years. But the older I get and the more the industry changes the less I find I use "modern tech" as a consumer.
Ahhh makes more sense, you're a developer. I get that YOU might not need SaaS tools, but this is a very myopic take.
Absolute rubbish. The proliferation of SaaS has been absolutely huge for business productivity, in fact, I would argue that my professional services business doesn't exist at anywhere near its profitability without it.
> I don't want my application to require an active Internet connection. I don't want it sending data to someone else's computer. Probably most importantly, I don't want the software that I pay for to change on me without my direct opt-in.
What software specifically? How to collaborate on literally anything business wise without a internet connection?! Now, is there a case for offline software? ABSOLUTELY. But to call this whole thing a fad and a trend is absurdly disingenuous.
EDIT:
Just saw your reply below:
I have multiple devices, laptops etc. in my house. Setting up a new Linux install takes me about 20 minutes. Wouldn't know about Windows. Then again, I probably use way fewer "services" / "apps" than most people. It's funny, I've always been a very tech savvy computer nerd. I've developed software for a living for 25 years. But the older I get and the more the industry changes the less I find I use "modern tech" as a consumer.
Ahhh makes more sense, you're a developer. I get that YOU might not need SaaS tools, but this is a very myopic take.
You realize I was speaking for myself, right? I never once said that SaaS doesn't provide any value to anyone.
But you need to understand that even as a self-employed business owner (product company, not contractor btw) for 15 years, I didn't take advantage of many SaaS tools at all.
Collaboration is niche. B2B productivity tools are a big industry (ironically I actually work for a company that makes a popular B2B collaborative SaaS product) but in the global scheme of things it's still a narrow niche when you consider the broad field of computing and software in general. Not everyone needs to be mobile, not everyone needs to collaborate. There are loads of software applications that can be local only, but there are few options because of the way our industry works.
And your post is timely because I was just thinking to myself this morning about how the biases that are introduced by Product Owners who come up with these ideas in work-related contexts, even if they are not the POs of B2B or "collaborative" software, still permeate non-B2B, non-"collaborative" software. They are in "work mode" and so they tend to think in terms of "How do I interact with this software. What do I need?" rather than actually considering the use cases for their every day users.
Take gaming, for example. Multi-player gaming is quite popular and those who seek it out ought to have options there. BUT ... I stopped gaming in large part because the industry went all in on online multiplayer and single player gaming options became more difficult to find. It's not that they don't exist anymore, it's just that I don't want a console that requires an Internet connection to play a story-based game like The Last of Us.
It's the following that pisses me off:
- The "all or nothing" mentality of our industry
- The ability for companies to change software that I use and pay for without me being able to decide if it's worth "upgrading" or not
- The lack of choices
I think it's pretty rich that we're both accusing each other of the same thing. You made the argument that because collaborative B2B tools are useful in a business context that it is "myopic" of me to want something that doesn't cater to that niche. I can make the exact same argument right back at you.
But you need to understand that even as a self-employed business owner (product company, not contractor btw) for 15 years, I didn't take advantage of many SaaS tools at all.
Collaboration is niche. B2B productivity tools are a big industry (ironically I actually work for a company that makes a popular B2B collaborative SaaS product) but in the global scheme of things it's still a narrow niche when you consider the broad field of computing and software in general. Not everyone needs to be mobile, not everyone needs to collaborate. There are loads of software applications that can be local only, but there are few options because of the way our industry works.
And your post is timely because I was just thinking to myself this morning about how the biases that are introduced by Product Owners who come up with these ideas in work-related contexts, even if they are not the POs of B2B or "collaborative" software, still permeate non-B2B, non-"collaborative" software. They are in "work mode" and so they tend to think in terms of "How do I interact with this software. What do I need?" rather than actually considering the use cases for their every day users.
Take gaming, for example. Multi-player gaming is quite popular and those who seek it out ought to have options there. BUT ... I stopped gaming in large part because the industry went all in on online multiplayer and single player gaming options became more difficult to find. It's not that they don't exist anymore, it's just that I don't want a console that requires an Internet connection to play a story-based game like The Last of Us.
It's the following that pisses me off:
- The "all or nothing" mentality of our industry
- The ability for companies to change software that I use and pay for without me being able to decide if it's worth "upgrading" or not
- The lack of choices
I think it's pretty rich that we're both accusing each other of the same thing. You made the argument that because collaborative B2B tools are useful in a business context that it is "myopic" of me to want something that doesn't cater to that niche. I can make the exact same argument right back at you.
>>> but we went all in as if there were no alternatives and those of us who don't require mobility and would prefer that we can choose when to "upgrade" and what data to "share" were left behind
> You realize I was speaking for myself, right?
So "us" is just you? Your "us" is a very small minority, which is the point of the people who are responding to you.
> You realize I was speaking for myself, right?
So "us" is just you? Your "us" is a very small minority, which is the point of the people who are responding to you.
> You realize I was speaking for myself, right? I never once said that SaaS doesn't provide any value to anyone.
> solved a few problems for a few people
Are you trolling?
> solved a few problems for a few people
Are you trolling?
I personally like that these days my biggest concern about a computer suddenly dying on me, or getting lost/stolen, or a drive failing is the cost of replacing it. Pretty much any important data to me is painlessly synced and it takes some amount stress out of my life knowing that I have basically no chance of losing data.
Well try and start an offline only software business right now, and you’ll see the request pouring in.
“How do I share this with other people (who don’t pay for the software)?”
“Can we work collaboratively on this?”
Etc etc.
People are used to things being in the cloud already. And I am not talking about tech people.
Once you’ve experienced live collab, there is just no turning back…
“How do I share this with other people (who don’t pay for the software)?”
“Can we work collaboratively on this?”
Etc etc.
People are used to things being in the cloud already. And I am not talking about tech people.
Once you’ve experienced live collab, there is just no turning back…
You hit the crux though: collaboration. If your product's essence is collaboration you'll need to support this, but if it's an enhancement the local experience is often so much better, and we can solve the connectivity part on the side.
>> People are used to things being in the cloud already. And I am not talking about tech people.
Most people use their phones for almost everything, and this is almost all local execution, local persistence and huge amounts of cloud-based collaboration. I wish desktops were on this trajectory.
>> People are used to things being in the cloud already. And I am not talking about tech people.
Most people use their phones for almost everything, and this is almost all local execution, local persistence and huge amounts of cloud-based collaboration. I wish desktops were on this trajectory.
We have articles about offline-first, cloud-optional software because those articles are primarily discussing tools written using web technology (HTML and Javascript) and often running in your web browser.
Alternate take:
I work on my documents on my computer, my phone and my iPad.
My IDE is hosted in AWS Cloud 9 most of the time because I travel a lot and while my internet may be spotty, it’s good enough to connect to the hosted environment and I get full speed internet between my IDE and the Internet when I need to download something to it (packages, Docker containers, etc)
My photos and videos automatically get backed up to iCloud, Google Drive, Amazon Drive and OneDrive.
If I dropped my phone into the ocean, I can go to the Apple Store, buy a new phone and it’s just like having my old one.
I work on my documents on my computer, my phone and my iPad.
My IDE is hosted in AWS Cloud 9 most of the time because I travel a lot and while my internet may be spotty, it’s good enough to connect to the hosted environment and I get full speed internet between my IDE and the Internet when I need to download something to it (packages, Docker containers, etc)
My photos and videos automatically get backed up to iCloud, Google Drive, Amazon Drive and OneDrive.
If I dropped my phone into the ocean, I can go to the Apple Store, buy a new phone and it’s just like having my old one.
I agree with this in spirit, but there are many advantages to a services model. However, building services is exceptionally complicated. I'm building a simple platform which people can run at home or in cloud. I have an unlisted tech talk dry run https://www.youtube.com/watch?v=bWYgChA_aYA
I miss offline readers. Push a button, download a payload of data updates, read it all offline.
I will almost always choose to use the web app over downloading a client onto my system. Keeps things clean. The exception is when the web app quality sucks.
I’m sympathetic to your point but most apps I regularly use require internet anyways.
I’m sympathetic to your point but most apps I regularly use require internet anyways.
It's a bit like when smart phones happed we started doing ui "mobile first" as it easer to scale up a mobile UI to desktop, it's easier to start "offline first" and move to constantly online (even collaborative) than to start online first and add "offline" eventual consistency later.
There is much chat about real-time collaborative apps, but the reality is that most people are working async most of the time. If you start with the concept of offline first, with eventual consistency for collaborative work, you implement all of the same conflict resolution as if you are implementing a real-time app with conflict resolution. Then adding real-time collaboration later is easy.
This is where CRDTs (conflict free replicated datatypes) work better than OT (operational transforms). OT need more supervision from the server, the further you diverge from the servers state the more likely there will be a corruption. CRDTs are inherently designed to work with massive diversion and no central authority, really they are offline first.
There is much chat about real-time collaborative apps, but the reality is that most people are working async most of the time. If you start with the concept of offline first, with eventual consistency for collaborative work, you implement all of the same conflict resolution as if you are implementing a real-time app with conflict resolution. Then adding real-time collaboration later is easy.
This is where CRDTs (conflict free replicated datatypes) work better than OT (operational transforms). OT need more supervision from the server, the further you diverge from the servers state the more likely there will be a corruption. CRDTs are inherently designed to work with massive diversion and no central authority, really they are offline first.
Yeah and riding 1000cc bike is just like driving the bicycle with Extreme Speed /s
> In the case of webRTC, I've seen so many projects do this where they’re like, “It‘s peer-to-peer! You just connect to the signaling server and then the things talks!” That’s not really peer-to-peer cause you've got a server. If you've already got a server...my advice is: don't mess around with peer-to-peer stuff, just use that server.
The prevalence of NAT really did a number on ability to have properly p2p communication. And, well, rampant insecurity of default user setups making most internet connection be at the very least behind "only allow outgoing traffic or traffic initiated by user" type of firewall.
Even if you make all that nice code with each user having it nice and secure private key and having to accept someone's public key to start communicating there is still problem with distribution and ascertaining that user is really who they are.
And even if they are, well, they can be compromised, so your local app also have to be very secure or else the potential malware can just go between your contacts and compromise everyone using the app.
Or, you can just use SaaS and don't worry about it, or at the very least not worry about more than one app having ability to compromise your system
> In the case of webRTC, I've seen so many projects do this where they’re like, “It‘s peer-to-peer! You just connect to the signaling server and then the things talks!” That’s not really peer-to-peer cause you've got a server. If you've already got a server...my advice is: don't mess around with peer-to-peer stuff, just use that server.
The prevalence of NAT really did a number on ability to have properly p2p communication. And, well, rampant insecurity of default user setups making most internet connection be at the very least behind "only allow outgoing traffic or traffic initiated by user" type of firewall.
Even if you make all that nice code with each user having it nice and secure private key and having to accept someone's public key to start communicating there is still problem with distribution and ascertaining that user is really who they are.
And even if they are, well, they can be compromised, so your local app also have to be very secure or else the potential malware can just go between your contacts and compromise everyone using the app.
Or, you can just use SaaS and don't worry about it, or at the very least not worry about more than one app having ability to compromise your system
NAT is definitely the villain but their are other things as well. You still need a discovery system to find who you want to connect with. An ip isn’t very good because it moves around, and DNS is too slow to be operationally viable.
And even if you solve those you have an N^2 problem in the naive solution, and to reduce that you need mesh networks which just don’t have the QoS we’ve come to expect (to my meager knowledge).
Imo we should be able to find a better middle ground, because full decentralization is insanely hard. If we had federated standards and tooling like web and email, but for modern use cases, I’d be a happy man.
Despite (or perhaps because) the tech craze over the last decade, the developments of such open and useful standards leaves a lot to be desired. Almost everything cool these days is some app with perhaps an api that is branded as a “platform”. We’re basically treading water.
And even if you solve those you have an N^2 problem in the naive solution, and to reduce that you need mesh networks which just don’t have the QoS we’ve come to expect (to my meager knowledge).
Imo we should be able to find a better middle ground, because full decentralization is insanely hard. If we had federated standards and tooling like web and email, but for modern use cases, I’d be a happy man.
Despite (or perhaps because) the tech craze over the last decade, the developments of such open and useful standards leaves a lot to be desired. Almost everything cool these days is some app with perhaps an api that is branded as a “platform”. We’re basically treading water.
Someday, in the far, far 90s, we had LAN. And everything worked.
Now, you need 5Mbit speeds just to collaborate. Funny how the tech "progressed".
Now, you need 5Mbit speeds just to collaborate. Funny how the tech "progressed".
In other news, old man yells at cloud. You conveniently ignore the changed reality: since the 90ies, the number of active internet users, malware, services, goods sold, and customer expectations have skyrocketed.
What you have in stock is nostalgia, my friend. Everything worked? Like Windows 95 crashing if you looked at it sideways, Linux requiring you to compile drivers for pretty much any device yourself, like fifteen competing printer protocols all incompatible with each other? You might want to get a reality check :)
What you have in stock is nostalgia, my friend. Everything worked? Like Windows 95 crashing if you looked at it sideways, Linux requiring you to compile drivers for pretty much any device yourself, like fifteen competing printer protocols all incompatible with each other? You might want to get a reality check :)
In the late 90s, I used to collaborate with about 10 people, all in the same office on the same LAN, for production of a magazine that sold IT stuff.
We used the "Excel Shared Spreadsheet" functionality. The file was hosted on a network drive. It corrupted or crashed so frequently - even when only one person had it open - that we took to copying it to your local machine, making the network copy read-only, editing the local copy, then copying back onto the network drive. Even this was buggy, so possession of a stuffed octopus represented a lock on the file. Microsoft themselves got involved at one point, as we were a reseller for their products. Their solution was no better.
So if there was a halcyon period for collaboration at some point in the past, I'd like to know when it was. When I see 100 people live editing the same Google Sheet, I think perhaps we are in the golden age.
We used the "Excel Shared Spreadsheet" functionality. The file was hosted on a network drive. It corrupted or crashed so frequently - even when only one person had it open - that we took to copying it to your local machine, making the network copy read-only, editing the local copy, then copying back onto the network drive. Even this was buggy, so possession of a stuffed octopus represented a lock on the file. Microsoft themselves got involved at one point, as we were a reseller for their products. Their solution was no better.
So if there was a halcyon period for collaboration at some point in the past, I'd like to know when it was. When I see 100 people live editing the same Google Sheet, I think perhaps we are in the golden age.
Please tell me you called it locktopus
Heh, we had a similar process for modifying the schema for an Access database or… something adjacent to it. There was a bunch of XML that needed to be versioned and if two people modified it, committed to SVN, and merged, there was almost certainly going to be huge breakage. I went out for a cigarette after one of the moments of frustration, saw a nice maybe 2kg rock, and the Lock Rock was born :D
so young man yells at old man yelling at cloud? TBF, software in the past handled fluctuating connectivity much much better than todays, probably because back then this was normal. Once you got a thing working (a printer driver, say) it typically worked.
Today, with spotty internet, software you have or could have on your device might or might not work. Just recently I have seen a website which loaded, but was non-functional until 2 mins later a cookie warning was also loaded.
Nothing in there is about
> the number of active internet users, malware, services, goods sold
but about not building for applications only loading half (because we are still waiting for some obscure JS or CSS from not the original domain that noone really never needs.)
Today, with spotty internet, software you have or could have on your device might or might not work. Just recently I have seen a website which loaded, but was non-functional until 2 mins later a cookie warning was also loaded.
Nothing in there is about
> the number of active internet users, malware, services, goods sold
but about not building for applications only loading half (because we are still waiting for some obscure JS or CSS from not the original domain that noone really never needs.)
At least the printer protocols have stayed true to themselves, the more the merrier.
Picking a random example: "AirPrint®, Brother iPrint&Scan, Mopria™ , Cortado Workplace, Wi-Fi Direct®".
Will a lists like that be enough to overcome your fear and skip the costly upsell to exactly the same printer, but it also understands PCL6? (likely a bit flipped in the firmware...) You decide. It's like a reverse confidence trick: "look, it's 2023 already and it's awesome, you really don't need PCL6 anymore. Trust me."
Picking a random example: "AirPrint®, Brother iPrint&Scan, Mopria™ , Cortado Workplace, Wi-Fi Direct®".
Will a lists like that be enough to overcome your fear and skip the costly upsell to exactly the same printer, but it also understands PCL6? (likely a bit flipped in the firmware...) You decide. It's like a reverse confidence trick: "look, it's 2023 already and it's awesome, you really don't need PCL6 anymore. Trust me."
Haha I got a WiFi direct printer. It made no indication of working with Linux but it just worked out of the box. It came with a driver disk, and it's needed to get it working with windows ..
Nice to hear! I'm not really doing Linux on the desktop, but in this age of almost-but-not-quite paper free, I like to think of printers in decades. And on that time scale, "no special fiery hoops to jump through on Linux" is really the only thing that has any value as a predictor of future usefulness. After some googling my vague conclusion was that a full complement of wireless protocols would indeed be a tolerable substitute of PCL6, but I'm glad for the confirmation. (even if I recently dropped out of the printer market again, turns out that the day I gave up on my ca 2001 Samsung I just wasn't sufficiently desperate to tease it into accepting an empty page as something to print on)
"paper free" sounds un-fun. I have a brother laser, that replaced a 2 decade old HP laser that my youngest broke when he was 2 or 3. We go through about 3 reams a year, more when one of us is going for a degree. I print maybe 5 things a month on average, usually recipes. I wouldn't forego a printer for any reason.
you can use gimp to take images and make them into coloring book pages, you can print instructions, calendars, drawings, diagrams; prompts for writing, your own writing to make sure that the "screen" isn't tricking your brain making you miss editing/grammar errors. Coloring pages, maps of countries, and other educational things are more fun if you can hold them and put them on your wall, if my kids are any indication. Having a scanner and a copier and a networked printer in a single box for what they cost - that makes it difficult to see the aversion.
I have had fancy e-readers since the original kindle, and before that a Clie and two palm pilots with acrobat and an epub (or whatever) reader. and i still prefer dead wood. I read more books on paper than e-ink. I read more on a computer screen than books, though.
you can use gimp to take images and make them into coloring book pages, you can print instructions, calendars, drawings, diagrams; prompts for writing, your own writing to make sure that the "screen" isn't tricking your brain making you miss editing/grammar errors. Coloring pages, maps of countries, and other educational things are more fun if you can hold them and put them on your wall, if my kids are any indication. Having a scanner and a copier and a networked printer in a single box for what they cost - that makes it difficult to see the aversion.
I have had fancy e-readers since the original kindle, and before that a Clie and two palm pilots with acrobat and an epub (or whatever) reader. and i still prefer dead wood. I read more books on paper than e-ink. I read more on a computer screen than books, though.
> You conveniently ignore the changed reality: since the 90ies, the number of active internet users, malware, services, goods sold, and customer expectations have skyrocketed.
Now our printers, cars, and soon fridges are DRM-locked and doing surveillance.
And they can stop working at any time if the manufacturer goes bankrupt or simply stop supporting them.
Now our printers, cars, and soon fridges are DRM-locked and doing surveillance.
And they can stop working at any time if the manufacturer goes bankrupt or simply stop supporting them.
in the 90s, there was microsoft messenger. real time, point to point (and maybe ptmp) chat, audio, and video. What replaced it? I have to run a coturn (STUN) server just to be able to talk to my friend in a private way, which means now i gotta deal with some provider for hosting both coturn and, say, Matrix (Synapse).
I also run a PBX, just in case coturn decides to stop working from lack of use. Now i gotta make sure i start the PBX app (3cx) once a week otherwise android helpfully removes notifications for me!
AIM, ICQ, etc were great - and i'm discounting "presence", because that was a small part of it. realtime communication for "free". I've been running chat servers for over a decade, and right now, there's three people on my matrix server, me, wife, and my best friend. there's five people on my PBX, same three plus my youngest child without a SIM card, and grandma, so grandma can call kid and vice versa. I've run ircd, matrix, rocket.chat, mattermost, a couple of the jabber systems, ventrilo, teamspeak, hand-rolled garbage.
we've had people sign up and chat once, but as soon as notifications stop working we never see them again. Heck, i have 3 PUSH accounts just to make sure that notifications go through everywhere.
Meanwhile, 20-25 years ago, load up trillian or AIM/ICQ and there was everyone, literally everyone. I know facebook ate that lunch, but i refuse to have any facebook apps on my phone, and most people i know refuse as well, so that's a non-starter. Texting is still spotty, because VoWiFi tends to just magically stop working if you leave the wifi area a couple times in a day, so texts will sit in limbo until you realize. Phone calls are the same way (except via PBX, those always seem to work for some reason...)
running out of IPv4 and the ubiquitousness of CGNAT have killed what the internet was. And that's sad; "nostalgia" isn't necessarily a negative thing. It used to be better, at least in the West. Maybe wechat and signal and telegram and whatever work better for the rest of the world now than any of our stuff from 25 years ago, i wouldn't know, because i know exactly zero people on any of those services - i've checked, once every couple of years.
I also run a PBX, just in case coturn decides to stop working from lack of use. Now i gotta make sure i start the PBX app (3cx) once a week otherwise android helpfully removes notifications for me!
AIM, ICQ, etc were great - and i'm discounting "presence", because that was a small part of it. realtime communication for "free". I've been running chat servers for over a decade, and right now, there's three people on my matrix server, me, wife, and my best friend. there's five people on my PBX, same three plus my youngest child without a SIM card, and grandma, so grandma can call kid and vice versa. I've run ircd, matrix, rocket.chat, mattermost, a couple of the jabber systems, ventrilo, teamspeak, hand-rolled garbage.
we've had people sign up and chat once, but as soon as notifications stop working we never see them again. Heck, i have 3 PUSH accounts just to make sure that notifications go through everywhere.
Meanwhile, 20-25 years ago, load up trillian or AIM/ICQ and there was everyone, literally everyone. I know facebook ate that lunch, but i refuse to have any facebook apps on my phone, and most people i know refuse as well, so that's a non-starter. Texting is still spotty, because VoWiFi tends to just magically stop working if you leave the wifi area a couple times in a day, so texts will sit in limbo until you realize. Phone calls are the same way (except via PBX, those always seem to work for some reason...)
running out of IPv4 and the ubiquitousness of CGNAT have killed what the internet was. And that's sad; "nostalgia" isn't necessarily a negative thing. It used to be better, at least in the West. Maybe wechat and signal and telegram and whatever work better for the rest of the world now than any of our stuff from 25 years ago, i wouldn't know, because i know exactly zero people on any of those services - i've checked, once every couple of years.
But have you tried to move any users over to, "wechat and signal and telegram and whatever"? Getting users onto your own system wasn't trivial. If you want to move to a more supported system like, say, Signal, then you have to pay the platform switching costs, which includes moving your group of people. Which involves having the social capital to do so, which is unfortunate for the less social of us.
There are many things that killed what the Internet was in the 20-30 years hence. IPv4 exhaustion is regrettable, but entirely predicted, and for those in the west, not a huge deal. I can pay any number of cloud providers a small fee to host me a box with a publicly routable IPv4 address. CGNAT is annoying, but the number of libraries which will punch a hole in that NAT has also expanded by a huge margin since the 90's. So those up things are unfortunate, but I think there are many other things that changed the Internet, some for the better, some for the worse.
There are many things that killed what the Internet was in the 20-30 years hence. IPv4 exhaustion is regrettable, but entirely predicted, and for those in the west, not a huge deal. I can pay any number of cloud providers a small fee to host me a box with a publicly routable IPv4 address. CGNAT is annoying, but the number of libraries which will punch a hole in that NAT has also expanded by a huge margin since the 90's. So those up things are unfortunate, but I think there are many other things that changed the Internet, some for the better, some for the worse.
> But have you tried to move any users over to, "wechat and signal and telegram and whatever"?
no, because everyone uses something different and there's no "trillian" or meebo for all the shiny-tech. If we're going to play "encrypted chat", that no one can agree what that means, or what service to use, may as well use my own. Hopefully matrix becomes more popular (it probably won't, unfortunately*), but if not, i still won't use the likes of telegram or FB messenger or tiktok DMs or anything of that nature. there's not a good IRC client for phones, last few times i checked over the past 15 years.
* Matrix has the curse of being good enough on the client side, but kind of a PITA on the server side. Synapse, the reference implementation is in python, which is single threaded, so joining a large room "as-is" out-of-the-box is not feasible. You have to split each of the synapse elements into a microservice that gets its own python instance, but the documentation was (and probably still is) quite sparse. I'm sure there's a discord where i can get help.
no, because everyone uses something different and there's no "trillian" or meebo for all the shiny-tech. If we're going to play "encrypted chat", that no one can agree what that means, or what service to use, may as well use my own. Hopefully matrix becomes more popular (it probably won't, unfortunately*), but if not, i still won't use the likes of telegram or FB messenger or tiktok DMs or anything of that nature. there's not a good IRC client for phones, last few times i checked over the past 15 years.
* Matrix has the curse of being good enough on the client side, but kind of a PITA on the server side. Synapse, the reference implementation is in python, which is single threaded, so joining a large room "as-is" out-of-the-box is not feasible. You have to split each of the synapse elements into a microservice that gets its own python instance, but the documentation was (and probably still is) quite sparse. I'm sure there's a discord where i can get help.
>Linux requiring you to compile drivers for pretty much any device yourself
It's not much better on modern Fedora. The drivers do come precompiled but you have to reboot sometimes twice per week to install updates.
It's not much better on modern Fedora. The drivers do come precompiled but you have to reboot sometimes twice per week to install updates.
I rarely do updates and don't get notifications for them. Windows OTOH will demand I update, or decide to when I don't want it to.
Nothing forces you to reboot it twice per week.
Agree, they deliver updates and fixes, its good to have the option.
[deleted]
But how many ads did your Windows 95 start menu have?
Now you need 5Mbit speeds so that your Windows' boot time is not slowed down.
Too bad you can't use LAN to work from home
Well, every WAN is a LAN when looked from a sufficient distance.
DSL & Cable encapsulates Ethernet. City-wide networks are again Ethernet. In fact, everything is Ethernet with some routers between.
So, you're actually using a mixed-carrier Enthernet LAN with a couple of NATs in the way. You can build another LAN on your LAN with VPN, too.
DSL & Cable encapsulates Ethernet. City-wide networks are again Ethernet. In fact, everything is Ethernet with some routers between.
So, you're actually using a mixed-carrier Enthernet LAN with a couple of NATs in the way. You can build another LAN on your LAN with VPN, too.
I know you can use ethernet to build a network but if you build a global network with 90s technology, you'll get what?... the internet like we know today but worse. What problem was solved?
That was my point responding to OP. We actually progressed and we are facing new challenges.
That was my point responding to OP. We actually progressed and we are facing new challenges.
We're still using 90s technology. We only have faster switches, quicker routers, a couple of standards for faster connections, and some tools which disguise or repurpose deep packet inspection as "software defined networking".
A country wide router still can stay in service for a decade. Residential FTTx installations consist of a TelCo class switch, maybe a router, compact DSLAMs if you have DSL service and a couple of fibers in, tens of fibers out for FTTH customers.
It's faster hardware on the same stack. Nothing more, nothing less.
...and, oh, link-speed (transparent) DPI equipment in key places for lawful interception.
A country wide router still can stay in service for a decade. Residential FTTx installations consist of a TelCo class switch, maybe a router, compact DSLAMs if you have DSL service and a couple of fibers in, tens of fibers out for FTTH customers.
It's faster hardware on the same stack. Nothing more, nothing less.
...and, oh, link-speed (transparent) DPI equipment in key places for lawful interception.
Theoretically, if an entire major city's connection with the outside world was cut off, could all the infrastructure within the city function as a really big city-wide LAN?
Yes. You might need a dhcpd on a laptop though, if your city's IP management is outside the said city (Cable/DSL modems needs their own IPs, unless they are static).
...and possibly a small DNS server (or a hosts file somewhere easy to get).
That's all you need.
If you want to navigate via IPs only, you can do with a DHCPd.
...and possibly a small DNS server (or a hosts file somewhere easy to get).
That's all you need.
If you want to navigate via IPs only, you can do with a DHCPd.
It's electrified rocks all the way down.
> Too bad you can't use LAN to work from home
Depends on what you consider "work from home". You might not be able to chat with your co-workers without a WAN connection (unless you go really old-school and use a telephone), but you can for instance develop software from home without any working network connection (and git makes it easier by allowing you to create commits while offline; see also the "git bundle" subcommand). I've done so occasionally when the Internet connection goes kaputt.
Depends on what you consider "work from home". You might not be able to chat with your co-workers without a WAN connection (unless you go really old-school and use a telephone), but you can for instance develop software from home without any working network connection (and git makes it easier by allowing you to create commits while offline; see also the "git bundle" subcommand). I've done so occasionally when the Internet connection goes kaputt.
That's kinda my point replying to OP. We've indeed progressed. I'm not sure if you are denying that?
You use VPNs with more or less the same result.
I'm excited about ElectricSQL as an enabler in this space: https://electric-sql.com/docs/overview/technical-intro
They're using SQLite on device with a sync service online. Their team have credentials that give me confidence: https://electric-sql.com/about/team
They're using SQLite on device with a sync service online. Their team have credentials that give me confidence: https://electric-sql.com/about/team
I have high hopes for ElectricSQL given that the co-inventors of CRDTs are on the team! It's like building a generative AI startup with authors of the transformer paper.
Thanks! We’ve just opened up a call for design partners[0] if you (or any teams out there) might be interested in collaborating with us :)
[0] https://next.electric-sql.com
[0] https://next.electric-sql.com
James is speaking at the next local-first meetup. https://lfw.dev
I'd go one step further, and say offline is just real-time with extreme latency. This is what we realized when re-tooling our app for offline use – anything we build that enables offline use will by its nature enable real-time collaboration.
I’d frankly be happy with more apps just being offline tolerant.
Surely google maps knows I live in NYC by now but if I’m between stations on the subway the odds of me being able to pan around at various zoom levels is surprisingly low. Surely basic semi-static highway and geographic information could be locally stored (this basic information used to fit in a spiral bound book we’d keep in the back seats of our cars).
iMessage just gives up if a message can’t go out rather than queuing it.
Heck most weather apps won’t even show me the previously downloaded forecast if there’s no network, they just show “-“ or a sad cloud image.
Surely google maps knows I live in NYC by now but if I’m between stations on the subway the odds of me being able to pan around at various zoom levels is surprisingly low. Surely basic semi-static highway and geographic information could be locally stored (this basic information used to fit in a spiral bound book we’d keep in the back seats of our cars).
iMessage just gives up if a message can’t go out rather than queuing it.
Heck most weather apps won’t even show me the previously downloaded forecast if there’s no network, they just show “-“ or a sad cloud image.
Re: Google
I assume they still have the "offline maps" feature, but you set it up and select regions by yourself. I have no idea why they haven't automated that using their location tracking and AI features.
Re: iMessage Yes that's a shame. Only excuse I can imagine is there should be some timeout by how much a message is allowed to be delayed. And because of SMS delivery etc this would probably cause a lot of complexity (cellular has its own queueing system, at least here, and it's unreliable)
I assume they still have the "offline maps" feature, but you set it up and select regions by yourself. I have no idea why they haven't automated that using their location tracking and AI features.
Re: iMessage Yes that's a shame. Only excuse I can imagine is there should be some timeout by how much a message is allowed to be delayed. And because of SMS delivery etc this would probably cause a lot of complexity (cellular has its own queueing system, at least here, and it's unreliable)
Android GMaps has done this for years already.
Nearly everything about that article depressed me. I want just offline applications and if I need it in the very limited times I do, opt-in to cloud. All this "everything" is online is bullshit.
Yes and storage is just a high latency message to yourself in the future.
This means everything that is in use in high latency communications can be used for storage.
Reminded me of this SIGBOVIK paper - "Harder drives: hard drives we didn't want or need"
http://tom7.org/harder/\
http://tom7.org/harder/\
Okay, that section on chainsaws was brilliant, and is somehow the sanest part of the paper: the following text gets progressively crazier.
Nah. Offline is online with extreme immediacy. Get out there and DO IT NOW!
If you're interested in this sort of thing, there's a good group of folks who are building local-first libraries and software congregating at https://localfirstweb.dev/
They have had a few meetups so far which have been really good (including PVH speaking at the first one). And many of the local-first builders are hanging in their Discord.
They have had a few meetups so far which have been really good (including PVH speaking at the first one). And many of the local-first builders are hanging in their Discord.
>>>It almost sounds like a form of resilient design (i.e. progressive enhancement) if you think about it — the cloud as an optional, layered enhancement of your application. Now that’s a paradigm shift!
This is literally what was pioneered by Lotus Notes 32 years ago. When everything was local, their rich-content databases could work just fine locally and also replicate on any desired schedule with other copies of the same databases/apps. The model was explicitly Seldom-Connected, so you could run stand-alone in a remote location for weeks and send/receive updates whenever you got to a phone line, or could have it collecting/distributing updates every N seconds on a persistent connection.
A vastly superior model to everything I've seen since, and it is too bad that they, and especially IBM blew it with the weird programmability models and corporate-extract-every-dollar approach.
Acting as if this is a somehow new concept is just a display of cluelessness, but it is good to see good concepts getting some attention (again).
This is literally what was pioneered by Lotus Notes 32 years ago. When everything was local, their rich-content databases could work just fine locally and also replicate on any desired schedule with other copies of the same databases/apps. The model was explicitly Seldom-Connected, so you could run stand-alone in a remote location for weeks and send/receive updates whenever you got to a phone line, or could have it collecting/distributing updates every N seconds on a persistent connection.
A vastly superior model to everything I've seen since, and it is too bad that they, and especially IBM blew it with the weird programmability models and corporate-extract-every-dollar approach.
Acting as if this is a somehow new concept is just a display of cluelessness, but it is good to see good concepts getting some attention (again).
My pet peeve are mobile apps that work with good connectivity (mobile or wifi), and also work with no connectivity at all, but when there is patchy connectivity take a minute or so to decide which mode to work in. Spotify I'm looking at you. Especially on screens that absolutely should not need to request data from the server to render information.
This sounds like something one might say during an LSD trip
> Why can’t we do this easily yet?
Well, because it's distributed computing. Distributed computing is hard... really hard! Especially for applications requiring any level of real-time-ish interactivity between different users.
Two computers talking to each other in a room via a server in Virginia is not absurd. The physical distance is abstracted over completely, the application engineer needn't think about that at all, that's the beauty and point of protocols like tcp/ip.
What cannot be abstracted over is the single source of truth in the networked system (the server) that makes a whole class of really hard problems go away. So as much as you can describe the remote server as a problem, because of latency perhaps, it is a vastly lesser problem than the distributed computing ones that'd be introduced in the alternative!
That is the basic reason why things are the way they are, it's really not absurd or surprising at all, in my opinion.
Well, because it's distributed computing. Distributed computing is hard... really hard! Especially for applications requiring any level of real-time-ish interactivity between different users.
Two computers talking to each other in a room via a server in Virginia is not absurd. The physical distance is abstracted over completely, the application engineer needn't think about that at all, that's the beauty and point of protocols like tcp/ip.
What cannot be abstracted over is the single source of truth in the networked system (the server) that makes a whole class of really hard problems go away. So as much as you can describe the remote server as a problem, because of latency perhaps, it is a vastly lesser problem than the distributed computing ones that'd be introduced in the alternative!
That is the basic reason why things are the way they are, it's really not absurd or surprising at all, in my opinion.
With apps I do this. I download the entire rest api. Often 5-10 mb.Entire api is versioned, every content update leads to new live version. And app works offline and super fast. Queue the api updates by default. And cache images / videos. Pretty doable. And api is rarely queried.
We are beyond fucked in terms of any effort to build local-first software.
Notably:
1. product people and interaction designers don't understand or care about the difference, which isn't strictly their fault, because
2. users don't understand (heck some even want a team of engineers making sure the DB doesn't drop their writes all day every day) and only care when the software can reasonably be used offline anyway, which means
3. standards, protocols, and platforms have baked in dependence on servers (oauth, tls, ip, mobile device native apis, the goddamned entire browser, etc.) for example, you can't do anything on iOS in the background unless you have a server sending the devices silent push notifications every 5 seconds ... also
4. true offline with anything more than 1 client is a distributed system and instantly hard, so
5. the entire tech community is not trained to build offline applications, we're trained to glue react UIs onto CRUD backends because thats how you 10x your SAAS revenue in valuation over 3 years.
I am lucky enough to have worked on a few products where we did have to care about offline and it still ended up being relegated to the realm of "obscure edge case", for all intents and purposes. So you end up continually justifying why you're spending resources on supporting these weirdos who don't have reliable internet connections.
Sigh.
/rant
These days, I do honestly think that some things just need to be online. The internet is meant to be a globally addressable network. You're not supposed to have these little islands of accidentally private because-we-ran-out-of-addresses-and-hacked-the-shit-out-of-the-protocol-(NAT) networks. You're not supposed to be afraid of someone sending port 22 on your laptop an ssh handshake because the crypto is good. The whole IP world is supposed to be zero trust. We dropped IPv6 scopes because they were a crazy idea for a different world where we all ran federated networks and whatever.
The real point of "local first" in our modern world is about data ownership. If we can do anything as a community to help move the needle and get to a better place, it's building applications where clients truly own the data and the server just provides convenience and redundancy. Give clients cryptographically strong identities. Use webauthn. Sign requests. Trust users with their own data. Help users be sovereign and we solve lots of privacy and centralization issues all at once.
/soapbox
Notably:
1. product people and interaction designers don't understand or care about the difference, which isn't strictly their fault, because
2. users don't understand (heck some even want a team of engineers making sure the DB doesn't drop their writes all day every day) and only care when the software can reasonably be used offline anyway, which means
3. standards, protocols, and platforms have baked in dependence on servers (oauth, tls, ip, mobile device native apis, the goddamned entire browser, etc.) for example, you can't do anything on iOS in the background unless you have a server sending the devices silent push notifications every 5 seconds ... also
4. true offline with anything more than 1 client is a distributed system and instantly hard, so
5. the entire tech community is not trained to build offline applications, we're trained to glue react UIs onto CRUD backends because thats how you 10x your SAAS revenue in valuation over 3 years.
I am lucky enough to have worked on a few products where we did have to care about offline and it still ended up being relegated to the realm of "obscure edge case", for all intents and purposes. So you end up continually justifying why you're spending resources on supporting these weirdos who don't have reliable internet connections.
Sigh.
/rant
These days, I do honestly think that some things just need to be online. The internet is meant to be a globally addressable network. You're not supposed to have these little islands of accidentally private because-we-ran-out-of-addresses-and-hacked-the-shit-out-of-the-protocol-(NAT) networks. You're not supposed to be afraid of someone sending port 22 on your laptop an ssh handshake because the crypto is good. The whole IP world is supposed to be zero trust. We dropped IPv6 scopes because they were a crazy idea for a different world where we all ran federated networks and whatever.
The real point of "local first" in our modern world is about data ownership. If we can do anything as a community to help move the needle and get to a better place, it's building applications where clients truly own the data and the server just provides convenience and redundancy. Give clients cryptographically strong identities. Use webauthn. Sign requests. Trust users with their own data. Help users be sovereign and we solve lots of privacy and centralization issues all at once.
/soapbox
Offline does not necessarily mean connected AT ALL, witch means not necessarily "local first" but "only local". Means on my iron, so under my control.
For the local first part, witch means "casually connected systems", the extra layer of complexity offer another layer of control and resilience: an autonomous car, capable of limited inertial navigation, with full local maps, routing capabilities etc can move even without GPS. Sure, it's more costly and complex than a connected-only car, but it's FAR safer to use.
Similarly an autonomous ship is far safer than a connected only one, for obvious reason.
Similarly in human terms local people with local power are FAR more resilient and effective than meat-based dysfunctional robots tied to some procedure. A VERY small simple example: I'm moved to France from Italy some years ago, now I decide to convert my perfectly valid Italian driving license in a local one. There is an automated (long) procedure. When I start it I've have had to give a scan of my Italian driving license witch happen to be expired and so got renewed BEFORE the procedure complete. In a resilient system I simply can upload the new one, sure this act trigger some extra actions, but extra actions who will happen anyway, only just after a bigger delay. Did you remember the Toyoda model vs the Ford one? Well, local first it's the same. At any scale.
For the local first part, witch means "casually connected systems", the extra layer of complexity offer another layer of control and resilience: an autonomous car, capable of limited inertial navigation, with full local maps, routing capabilities etc can move even without GPS. Sure, it's more costly and complex than a connected-only car, but it's FAR safer to use.
Similarly an autonomous ship is far safer than a connected only one, for obvious reason.
Similarly in human terms local people with local power are FAR more resilient and effective than meat-based dysfunctional robots tied to some procedure. A VERY small simple example: I'm moved to France from Italy some years ago, now I decide to convert my perfectly valid Italian driving license in a local one. There is an automated (long) procedure. When I start it I've have had to give a scan of my Italian driving license witch happen to be expired and so got renewed BEFORE the procedure complete. In a resilient system I simply can upload the new one, sure this act trigger some extra actions, but extra actions who will happen anyway, only just after a bigger delay. Did you remember the Toyoda model vs the Ford one? Well, local first it's the same. At any scale.
I've always thought of it as: Offline is Persistent, Online is Ephemeral.
e.g. physical books can last centuries, an online blog - not so much.
e.g. physical books can last centuries, an online blog - not so much.
Couldn't agree more.
I just developed a local-first read later software using CRDT technology. aka
https://hamsterbase.com/
1. all data is stored locally, one page corresponds to one CRDT file. CRDT file is a single source of information
2. use sqlite as cache for query and full text search.
3. use the version of CRDT as the file name. It is guaranteed that there will be no duplicate file names and never file content changes. Only files are deleted and added.
In this architecture
Users could synchronize their data with any intermediate media. Such as hard disk, iCloud, http interface, webdav. It is guaranteed that there will always be no conflicts.
I just developed a local-first read later software using CRDT technology. aka
https://hamsterbase.com/
1. all data is stored locally, one page corresponds to one CRDT file. CRDT file is a single source of information
2. use sqlite as cache for query and full text search.
3. use the version of CRDT as the file name. It is guaranteed that there will be no duplicate file names and never file content changes. Only files are deleted and added.
In this architecture
Users could synchronize their data with any intermediate media. Such as hard disk, iCloud, http interface, webdav. It is guaranteed that there will always be no conflicts.
These ideas seem pretty doable for a single user note taking app but as soon as you include another collaborator, you've a whole another problem of different levels of conflicts, CRDTs and what not.
Becomes lot more harder when you have some 100 agents in the field, taking orders on their mobile devices and you have to do full double entry book keeping even when offline.
Becomes lot more harder when you have some 100 agents in the field, taking orders on their mobile devices and you have to do full double entry book keeping even when offline.
And... you've got a large list of customer, contact and historical info to allow access to. Sending that down to people so they have it locally becomes at least a bit of a privacy/security concern.
This solution is suitable for processing documents. For example, figma uses a technique similar to CRDT.
It is not suitable for processing orders.
It is not suitable for processing orders.
if two different users modify the same version of the same document in different ways, how do you resolve that conflict without losing information?
I have recorded all the operations of a document. Each operation has an id .
When the document is loaded into memory, it has a unique uuid as the agent id.
If the current version is
hash(a1 b2 c3)
If two people edit the document at the same time the name will become
hash(a1 b2 c3 e4)
hash(a1 b2 c3 d4 d5)
After merging, the document name will become
hash(a1 b2 c3 d4 e4 d5)
a1, b2 is "Lamport timestamp" https://en.wikipedia.org/wiki/Lamport_timestamp
the full version (a1 b2 c3 e4) is “vector clock” https://en.wikipedia.org/wiki/Vector_clock
When the document is loaded into memory, it has a unique uuid as the agent id.
If the current version is
hash(a1 b2 c3)
If two people edit the document at the same time the name will become
hash(a1 b2 c3 e4)
hash(a1 b2 c3 d4 d5)
After merging, the document name will become
hash(a1 b2 c3 d4 e4 d5)
a1, b2 is "Lamport timestamp" https://en.wikipedia.org/wiki/Lamport_timestamp
the full version (a1 b2 c3 e4) is “vector clock” https://en.wikipedia.org/wiki/Vector_clock
if d4 and e4 are conflicting edits, then how do you resolve them?
say (a1 b2 c3) is "abc" -- (a1 b2 c3 d4 d5) is "abxx" and (a1 b2 c3 e4) is "aby"
what is (a1 b2 c3 d4 e4)? it isn't "abxxy" or "abxy" or "abxxy" or anything else, because d4 and e4 are concurrent/conflicting updates, which have no single well-defined resolution (without additional information)
meaning, this is a conflict
if you just say that this resolves to (a1 b2 c3 e4) then this is LWW but you've lost the information in d4 and d5, so whoever was agent d has written information that was good for a little while, but then ultimately lost after merge, right? so this isn't consistent in any useful way?
say (a1 b2 c3) is "abc" -- (a1 b2 c3 d4 d5) is "abxx" and (a1 b2 c3 e4) is "aby"
what is (a1 b2 c3 d4 e4)? it isn't "abxxy" or "abxy" or "abxxy" or anything else, because d4 and e4 are concurrent/conflicting updates, which have no single well-defined resolution (without additional information)
meaning, this is a conflict
if you just say that this resolves to (a1 b2 c3 e4) then this is LWW but you've lost the information in d4 and d5, so whoever was agent d has written information that was good for a little while, but then ultimately lost after merge, right? so this isn't consistent in any useful way?
On my last project, we built a configuration/collection tool for sensors installed in remote locations. For certain operations, our failure mode was "the user will have to go online so the app can resolve things, then can go back to the remote location and try again".
After folks explained that this could require a multi-day, multi-tens-of-thousands-of-dollars journey for the customer, we re-architected parts of the system to be tolerant of offline failures but still resilient and secure and saved the customer a potential round trip back to the field.
After folks explained that this could require a multi-day, multi-tens-of-thousands-of-dollars journey for the customer, we re-architected parts of the system to be tolerant of offline failures but still resilient and secure and saved the customer a potential round trip back to the field.
One thing that seems to have gone out the window, to a certain extenet, is the idea of DR, Business Continuity and resiliency for a few key reasons:
1. Cloud is not cheap, so doing a proper DR in a loud environment is going to price out small/medium businesses.
2. Most customers don't have staff skilled in the realms of "what if" disasters, and have not tooled to spin up a copy quickly in a new region/provider should their site get nuked.
3. Most cloud customers have zero plan for what to do when cutoff from their cloud providers, because of above.
-
It would be cool to pay for "ghost infrastructure" (I dont know if this exists already)
But... the idea would be that as you build out your VPC/whatever environment, the cloud provider creates a ghost version of that infra in every other reegion (meaning all the configs and everything are mirroed to a repo at the other locations and allow one to spin up a complete copy of primary infa wherever one likes, and youre basically paying a small amount for a clean, up-todate DR copy of the environment as "a cost of doing business" but in the event you needto, load a snapshot of everything (sans actual data - thats extra) to just launch the environment...
We built this manually in the past (manually meaning the ability to launch an instant VPC and migrate traffic to it... but this was a few years back and I am sure that its far more mature in the real world now...
So what options do people have today?
1. Cloud is not cheap, so doing a proper DR in a loud environment is going to price out small/medium businesses.
2. Most customers don't have staff skilled in the realms of "what if" disasters, and have not tooled to spin up a copy quickly in a new region/provider should their site get nuked.
3. Most cloud customers have zero plan for what to do when cutoff from their cloud providers, because of above.
-
It would be cool to pay for "ghost infrastructure" (I dont know if this exists already)
But... the idea would be that as you build out your VPC/whatever environment, the cloud provider creates a ghost version of that infra in every other reegion (meaning all the configs and everything are mirroed to a repo at the other locations and allow one to spin up a complete copy of primary infa wherever one likes, and youre basically paying a small amount for a clean, up-todate DR copy of the environment as "a cost of doing business" but in the event you needto, load a snapshot of everything (sans actual data - thats extra) to just launch the environment...
We built this manually in the past (manually meaning the ability to launch an instant VPC and migrate traffic to it... but this was a few years back and I am sure that its far more mature in the real world now...
So what options do people have today?
This resonates a lot but there are substantial issues with offline-first for many apps, like permissions.
Maybe a simpler way to put it - if what you are building requires client/server for security guarantees, cloud is IMO an easier delivery model than forcing your customer run their own database. Cloud is infinitely better operationally for these users.
If they want to control the versioning, then on-prem is the answer.
If permission modelling is no issue, offline first plus sync is optimal I think.
Maybe a simpler way to put it - if what you are building requires client/server for security guarantees, cloud is IMO an easier delivery model than forcing your customer run their own database. Cloud is infinitely better operationally for these users.
If they want to control the versioning, then on-prem is the answer.
If permission modelling is no issue, offline first plus sync is optimal I think.
SaaS has ruined the development landscape.
I think that boils down to CAP theorem and whether or not you want to shift the problems onto the user.
With collaborative tools like chat or shared document editing. It's far easier for the system to deal with and for the user to conceptualise an availability error like not being connected to the WiFi or the cloud provider being down for a moment. Than it would be to deal with consistency errors like conflict resolution and asynchronous communication, eg: high volume chat channels/threads where responses arrive out of order or too late to be relevant.
It's just easier to switch to a different tool like IM or phone when one cloud tool is down so you can quickly discus the topic there, that to postpone your mental train for when everything is eventual consistent. Or to switch to the 'offline' tools we already have, like e-mail, which require you to rethink your train of though to the different medium.
Also central authorization and authentication are not to be overlooked. Those problems also suffer heavily when partitioned.
It's just easier to switch to a different tool like IM or phone when one cloud tool is down so you can quickly discus the topic there, that to postpone your mental train for when everything is eventual consistent. Or to switch to the 'offline' tools we already have, like e-mail, which require you to rethink your train of though to the different medium.
Also central authorization and authentication are not to be overlooked. Those problems also suffer heavily when partitioned.
The older I get, the more skeptical I get about any article / comment / whatever that uses the word "just" in this way. "X is just Y", "Do do X you just need to do Y". It's almost always a trite oversimplification to an extent that it falls apart as soon as you start thinking about it in detail - as the comments here suggest.
This seems to miss the fact that the cloud providers basic service is a highly discoverable highly available way of advertising endpoints and managing highly durable and resilient state in a distributed system. Yes you can do that locally. It’s just really fragile and difficult. The ability to collaborate between two local machines by using a well known service remotely that’s always available means even when the neighboring laptops lid is shut and reopened things work because the only real dependency that laptop has was its tcp stack. The other laptop doesn’t have to deal with the interruption, try to figure out when it’s come back online, renegotiate connectivity, figure out any state drift and reconcile state, etc etc. Bonus is it works the same when you take your laptop to the bathroom to take a crap as it did when it was 2ft away. I’m not even going to go into the pitiful state of adhoc local protocols and hardware stacks.
I made a version of an offline first app using PouchDB and CouchDB running on a Digital Ocean server. And a version that used a CouchDB installed on a local server (Raspberry Pi).
There are a few ways to do that using those tools, and it's very easy to implement. I thought it was great, especially using a locally installed CouchDB, but my users didn't really care, and I mean not a bit.
I don't have any big corporate users but there are some pretty big businesses and I could not convince them to run a DB on their side. They did not want to fuss with it.
Some of them have been using my apps for over a decade and there's not been any serious issues of downtime so there is no sense of urgency even though I've made it clear that can happen, and the importance of having their data onsite, in their possession.
I use it for my own accounts on my apps. CouchDB makes that very easy to do by itself. It's a built-in feature.
There are a few ways to do that using those tools, and it's very easy to implement. I thought it was great, especially using a locally installed CouchDB, but my users didn't really care, and I mean not a bit.
I don't have any big corporate users but there are some pretty big businesses and I could not convince them to run a DB on their side. They did not want to fuss with it.
Some of them have been using my apps for over a decade and there's not been any serious issues of downtime so there is no sense of urgency even though I've made it clear that can happen, and the importance of having their data onsite, in their possession.
I use it for my own accounts on my apps. CouchDB makes that very easy to do by itself. It's a built-in feature.
IF you can solve data sync for your application, local first is simple & easy to develop and feels very responsive for users. Lots of business applications are now in that range. 10MB of ERP data is like 1 year of operating a medium sized company is the equivalent of a picture of my cat.
[deleted]
Sometimes you can tell that you’re offline. Sometimes you can tell that you’re online. The real trouble is when you have an unreliable connection. That’s the worst of all available worlds, when systems fall apart in a wide variety of troublesome ways.
oh, like my starlink. I can't use it for VoWiFi, because it drops for a couple seconds every minute. So i use it to download, only. It also works for any VoD streaming service (youtube, netflix, etc), but will not work for live streams. Can't use it for gaming, then again with CGNAT on my main ISP i can't use it for gaming really, either.
However stuff like matrix works fine on starlink, because it will eventually be able to phone home and do all of the queued transfers, and that's fine.
However stuff like matrix works fine on starlink, because it will eventually be able to phone home and do all of the queued transfers, and that's fine.
[deleted]
Does your laptop have 200 terabytes of memory, 50 petabytes of storage, and 10,000 CPUs? No? Then it's not really the same thing is it?
There are things you can never do on your laptop. Offline is limiting, in many ways. You know how I know? I grew up in the 90s. Online is a magical godsend of capability and functionality. Offline is cute. Online is what we dreamed of.
Should you have offline modes? Of course. Should you have some offline-first applications? Of course. Is offline just online with extreme latency? Of course it isn't. Use your brain. Think.
There are things you can never do on your laptop. Offline is limiting, in many ways. You know how I know? I grew up in the 90s. Online is a magical godsend of capability and functionality. Offline is cute. Online is what we dreamed of.
Should you have offline modes? Of course. Should you have some offline-first applications? Of course. Is offline just online with extreme latency? Of course it isn't. Use your brain. Think.
Alexa, write me a witty response mentioning IP over Avian Carriers.
The other way to see it is that offline is the outmost possible edge computing.
Or that offline is zero network latency for the stuff you have on your computer.
Or that offline is, when functional, like online but without surrendering all control over your computing to someone else.
All these metaphors have their own limited scope where they are true and most appropriate.
Or that offline is zero network latency for the stuff you have on your computer.
Or that offline is, when functional, like online but without surrendering all control over your computing to someone else.
All these metaphors have their own limited scope where they are true and most appropriate.
There is _one_ thing that can allow "two machines in the same room to talk to each other without sending a request to a server farm in Virginia". It does require the _first_ request to populate the app: Web Bluetooth. Unfortunately it's only implemented in Chrome,but it does work.
P2P in browser was improved a lot in the past years. Your two machines could now connect via WebRTC and then send data to each other directly, no need for bluetooth. This works great and has a low latency. With rxdb you could even realtime-sync the whole application state [1] without connection to a server farm in virginia.
[1] https://rxdb.info/replication-p2p.html
[1] https://rxdb.info/replication-p2p.html
We tried Web Bluetooth in a project. It was extremely flaky. Notably it also only works as a server, so you cant peer two browsers over it either.
Speaking of this, anyone have suggestions for an offline-tolerant distributed filesystem? Basically a dozen devices which get a few minutes/hours of internet access per day/week and not necessarily at the same time.
I've considered Git but not sure how much overhead it adds.
I've considered Git but not sure how much overhead it adds.
Conflict resolution is the issue here.
Effectively sync systems like DropBox and iCloud do this by flagging conflicting edits when you come back online and duplicate files (new a new file name) so there is no data loss.
For a lower level file system though that's not really possible as you may be running databases, or other data models, that need to understand conflict resolution themselves.
Effectively sync systems like DropBox and iCloud do this by flagging conflicting edits when you come back online and duplicate files (new a new file name) so there is no data loss.
For a lower level file system though that's not really possible as you may be running databases, or other data models, that need to understand conflict resolution themselves.
For my personal use case I don't really care about conflicts - they could be crudely resolved by a timestamp or even randomly. Having backups in the event of a conflict would be good. Files are never updated incrementally, just added, renamed or deleted.
why would you share a database over a distributed filesystem, other than for backup purposes where you only have a single source so you should never get a conflict?
See cryptocurrencies.
That's intractable. Conflict resolution has to be done differently for different types of data, so it has to be done at a higher level than filesystem, unless you want a special filesystem that only supports a specific kind of file or has some other very significant restrictions.
i don't mean to blindly +1 but this is the actual correct answer and i think it deserves to be emphasized
Not a distributed filesystem but syncthing can easily do what you're asking for.
Seems promising, thank you.
> Speaking of this, anyone have suggestions for an offline-tolerant distributed filesystem? Basically a dozen devices which get a few minutes/hours of internet access per day/week and not necessarily at the same time. I've considered Git [...]
If you've considered git, I would suggest looking at git-annex (https://git-annex.branchable.com/), which is tuned for that kind of mostly-offline use case.
If you've considered git, I would suggest looking at git-annex (https://git-annex.branchable.com/), which is tuned for that kind of mostly-offline use case.
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum
The latency, however ...
https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum
The latency, however ...
LTO-9 tapes do 45TB compressed, 18TB uncompressed. And uses around 0.23 cubic decimeter of storage.
A European station wagon with the rear seats folded takes 1555 liters, with 95% utility that is 6422 tapes. That wagon can hold 289 Petabyte of compressed data
From Amsterdam to Rotterdam, it takes about an hour to get your data to its destination. That is roughly 640 Terabits per second. For comparison, AMS-IX peaked at 11 Terabits per second last year.
A European station wagon with the rear seats folded takes 1555 liters, with 95% utility that is 6422 tapes. That wagon can hold 289 Petabyte of compressed data
From Amsterdam to Rotterdam, it takes about an hour to get your data to its destination. That is roughly 640 Terabits per second. For comparison, AMS-IX peaked at 11 Terabits per second last year.
A fair comparison with AMS-IX would perhaps need to factor in the time spent reading/writing the tapes though. Compressed LTO-9 tapes can be fully read in about 12.5 hours, so assuming for sake of argument that your car contains 6422 compressed tapes with a total of 289PB of data on them already, and that you have (a rather expensive) array of 26 LTO-9 drives at your disposal in Rotterdam, you'll need about 130 days to fully read their data. The hour of driving across Holland is rather insignificant compared to that. All in all, you get 'only' 26 GB per second, limited to that only by having 26 drives.
when I design new software I bias/default to CLI, monolithic, local-hosted, and offline-only, first.
Then only if and when I have a good pressing reason to give it a GUI, or make it client/server, or require connectivity, or be cloud-hosted, do I carry out a refactor/redesign to add those features. And even after I apply the refactor/redesign I strive hard to maintain a UX configuration which retains all those original default qualities. Maintaining them side by side with the added alternate modes.
Maybe its an age/era thing. Because I'm increasingly surprised when folks DONT use this approach. Felt like a design no-brainer.
The private term I've coined for this philosophy is CLIFMO.
Then only if and when I have a good pressing reason to give it a GUI, or make it client/server, or require connectivity, or be cloud-hosted, do I carry out a refactor/redesign to add those features. And even after I apply the refactor/redesign I strive hard to maintain a UX configuration which retains all those original default qualities. Maintaining them side by side with the added alternate modes.
Maybe its an age/era thing. Because I'm increasingly surprised when folks DONT use this approach. Felt like a design no-brainer.
The private term I've coined for this philosophy is CLIFMO.
an email client is a good example: you can read and send emails also offline (at least with most clients that keep a decent cache) and only synchronise when connected. But that works for some services, not all.
"Build-time rendering is just pre-computed request-time rendering" lead to a whole lot of innovations in the JavaScript ecosystem and allowed a continuum of intermediate approaches to emerge
Mike St Johns talked about DNSSEC bootstrap timers in the context of a VM on the shelf, brought live 4+ years later: how do you update the Trust Anchor for a cold VM image which is very old?
Similarly, files are just a one-way pipe to an unknown process to be executed at some indeterminate future time (ranging from "now" to "never").
It's worth mentioning that the blog's author is not an expert on the matter. I really don't agree with the title or the article.
Offline is just online with 0 degree lightcone.
In related news …
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.Sounds like an opportunity for a new JS framework, because nobody wants to write this stuff themselves
Except for all the people who died before the internet was invented. They were completely offline.
Latency, bandwidth, signal to noise, etc. are all different by orders of magnitude
Rxdb pouch and couchdb. All forgotten tools to build wonderful offline products.
Funny, I think online is just offline with extreme latency and variance...
What do we do about timeouts when dealing with extreme latency though.
Just retry each X seconds. Also browsers have a navigator.onLine property which emits an event at the exact time the client is online again. I am doing this with rxdb and it works great.
Online is just offline with the caching layer missing.
TL;DR: This is about offline communications and not the world in general. It also assumes that communication is achievable. In the real world (offline and online), we have problems with that. When you attempt to communicate online you are often redirected "through the gift shop" so you can be shown the exact number of ads such that you statistically won't give up and pick up the phone (and I'm sure phonecalls will have pre-roll ads at some point too).
In fact, ad-hoc random person-to-person communications online are gatekept by "the algorithm" which would rather answer your question directly to keep you from meeting another warm body online.
In fact, ad-hoc random person-to-person communications online are gatekept by "the algorithm" which would rather answer your question directly to keep you from meeting another warm body online.
Well, nice, but who is going to make the money?
> "A lot of the time we're using the techniques to build aircraft carriers when what we really need is a bicycle: something simple, something personal, something that can take you where you need to go and that responds to you alone (and is a heck of a lot easier to maintain and operate)."
sounds like
"Linux is a semi-truck with 20 gears to operate. Windows is more like a car. TempleOS is a motorbike. If you lean over too far, you'll fall off. Don't do that."
sounds like
"Linux is a semi-truck with 20 gears to operate. Windows is more like a car. TempleOS is a motorbike. If you lean over too far, you'll fall off. Don't do that."
damn bruh, you just opened my eyes
No?
yeah, no
offline means no expectation of a response, that is very different than an extremely delayed response
offline means no expectation of a response, that is very different than an extremely delayed response
The point is to bridge online and offline and make apps more robust
If it works locally and there's no need for syncing or communication in the first place then yeah, problem solved!
If it works locally and there's no need for syncing or communication in the first place then yeah, problem solved!
Like saying "blindness is the same as a really long blink". One has an expectation of seeing again, the other has no guarantees.
Relevant xkcd:
https://xkcd.com/2224/
https://xkcd.com/2224/
&#^
For example, say you have a form. If you built it thinking online first you'll probably have some pessimistic UI which shows a spinner and waits for the server to respond with ok/error. You can't simply think, okay since we're offline, more latency -> show spinner for longer. You have to re-architect things so that the UI is optimistic, commits to a local database and that local database is synced up to the server when you come online.
In my experience optimistic UI is way more complex to build. Many times the complexity is worth it though.