APIs All the Way Down(notboring.substack.com)
notboring.substack.com
APIs All the Way Down
https://notboring.substack.com/p/apis-all-the-way-down
24 comments
We have a similar issue at work, where some official, business-essential data that changes each week is only available via a JavaScript-driven web page.
My coworker used Selenium to write an extractor to get the data automatically, so one of us doesn't have to sit each weekend and do it manually.
My coworker used Selenium to write an extractor to get the data automatically, so one of us doesn't have to sit each weekend and do it manually.
When I built beatsaver[1] for the modding community of beatsaber, I pretty much started with the data layout first, then the API for it. Ensuring everyone else would have access to the database as a source of truth to be built on. Its still around and has shaped the modding community early on to be as open as it can with API doing their one thing, and their one thing really well.
[1] https://github.com/beatsaver/beatsaver
[1] https://github.com/beatsaver/beatsaver
It looks like everyone has blissfully forgot that API access is often one of the first things to go/get significantly limited once companies reach monopoly velocity (i.e. competition exit velocity).
APIs are awesome, but we need APIs that don't break. I think that happens one of two ways:
- APIs built on scraping (EULAs make this untenable -- companies will just login-wall everything that isn't already)
- Standards-compliant APIs that lock companies into complying to be able to even compete with the others that are standards-compliant (ex. if your API doesn't comply with the SimpleAccountingV1 standard, no one would dare use it)
APIs are awesome, but we need APIs that don't break. I think that happens one of two ways:
- APIs built on scraping (EULAs make this untenable -- companies will just login-wall everything that isn't already)
- Standards-compliant APIs that lock companies into complying to be able to even compete with the others that are standards-compliant (ex. if your API doesn't comply with the SimpleAccountingV1 standard, no one would dare use it)
There's a lot of advantages to API-first.
1. You're not bound to one supplier, you can integrate multiple.
2. You don't need to manage the security for a large application, instead, the responsibility is distributed to each API.
3. Scaling becomes a non-issue (unless you abuse the rate-limits/grow too fast).
4. The cost of sysops and development goes waaay down.
The dis-advantages: it might be way more expensive, but at least you'll have shipped! (And of course GDPR/data control might be an issue)
1. You're not bound to one supplier, you can integrate multiple.
2. You don't need to manage the security for a large application, instead, the responsibility is distributed to each API.
3. Scaling becomes a non-issue (unless you abuse the rate-limits/grow too fast).
4. The cost of sysops and development goes waaay down.
The dis-advantages: it might be way more expensive, but at least you'll have shipped! (And of course GDPR/data control might be an issue)
> You don't need to manage the security for a large application, instead, the responsibility is distributed to each API.
Wrong. Third-Party risk management is your responsibility as a service provider. Your customers will want to know what kind of services your application depends on. I would recommend being very thorough in deciding who you will partner with and send data to. If you are a small startup, it most likely doesn't matter, but once you start getting serious business opportunities then data governance is the game.
Wrong. Third-Party risk management is your responsibility as a service provider. Your customers will want to know what kind of services your application depends on. I would recommend being very thorough in deciding who you will partner with and send data to. If you are a small startup, it most likely doesn't matter, but once you start getting serious business opportunities then data governance is the game.
You're right, that's why I added GDPR/Data control might be an issue.
If you read it again, you can see I'm talking about management of security, not the legal issues of it.
If you read it again, you can see I'm talking about management of security, not the legal issues of it.
Agree with all of these points, but you’ll need the understanding and competence to rapidly refactor your pain points when you encounter them.
Tangentially, I believe low and no code platforms are perfect for prototyping gluing APIs together to prove out concepts, and once you’ve demonstrated traction (whatever that looks like for your product), it’s time to refactor out into your own business line app to control your product’s destiny.
Tangentially, I believe low and no code platforms are perfect for prototyping gluing APIs together to prove out concepts, and once you’ve demonstrated traction (whatever that looks like for your product), it’s time to refactor out into your own business line app to control your product’s destiny.
which no code platforms have you had success with? I saw AWS got in the game with honeycode.
Any of the well known products are fine products to be using. Which is right for your use case is dependent on the use case and product features.
"Shopify is a really smart company. It wouldn’t tie its own hands for no good reason."
Don't underestimate companies ability to do 'only one thing' and 'screw everything else up'.
Shopify has evolved on top of 'a neat thing done well early on' and everything is about scaling that.
I've worked at very successful large tech companies who were actually terrible at doing any other tech than the product they sold.
Why isn't Google doing a Stripe API? Why isn't MS? Why isn't Apple? Why isn't Volkswagen?
They are really separate businesses.
Don't underestimate companies ability to do 'only one thing' and 'screw everything else up'.
Shopify has evolved on top of 'a neat thing done well early on' and everything is about scaling that.
I've worked at very successful large tech companies who were actually terrible at doing any other tech than the product they sold.
Why isn't Google doing a Stripe API? Why isn't MS? Why isn't Apple? Why isn't Volkswagen?
They are really separate businesses.
They may be just didn't find a way to make ads better with payments APIs but seriously, I think the only reason why google didn't get into payments is the customer service burden also, why would the company which depends on ADs wouldn't want to see the result(purchases). Or may be they do have a product but no plan like always.
That's a very good article.
I actually believe in the "API" model (I call it the "Dependency" or "Modular" model).
Dependencies are how small teams do big stuff.
But dependencies are also where truly Jurassic-scale disasters are born.
I think that it is important to exercise moderation in our dependencies, and accept that using them is a major decision.
I actually believe in the "API" model (I call it the "Dependency" or "Modular" model).
Dependencies are how small teams do big stuff.
But dependencies are also where truly Jurassic-scale disasters are born.
I think that it is important to exercise moderation in our dependencies, and accept that using them is a major decision.
Let's remember there's such huge difference between using a library, and using some interface operated by different organization. One you are in control, the other you aren't.
Perhaps learned helpless with package management has taught people to feel alienated about both of them...
Perhaps learned helpless with package management has taught people to feel alienated about both of them...
> But dependencies are also where truly Jurassic-scale disasters are born.
It's interesting this link showed up on the same day as How Complex Systems Fail.
https://news.ycombinator.com/item?id=25550685
It's interesting this link showed up on the same day as How Complex Systems Fail.
https://news.ycombinator.com/item?id=25550685
> APIs are an increasingly important piece of how software is built.
API's were always how software has been built .. Web API's, on the other hand ...
API's were always how software has been built .. Web API's, on the other hand ...
> API's were always how software has been built ..
For software designed to be used by other developers, sure. For software in general, that's not my experience.
> Web API's, on the other hand ...
The ellipses suggests a natural conclusion, but none come to mind. Do you mind completing the thought?
For software designed to be used by other developers, sure. For software in general, that's not my experience.
> Web API's, on the other hand ...
The ellipses suggests a natural conclusion, but none come to mind. Do you mind completing the thought?
> For software designed to be used by other developers, sure. For software in general, that's not my experience.
The only software I can think of for users that doesn't use an API is maybe on an embedded microcontroller. Everything uses APIs. File system APIs, GUI APIs, timer APIs, network APIs. Even standard output is usually exposed via an API in most languages.
> The ellipses suggests a natural conclusion, but none come to mind. Do you mind completing the thought?
The conclusion is that software wasn't always built with web APIs, in contrast from APIs in general.
The only software I can think of for users that doesn't use an API is maybe on an embedded microcontroller. Everything uses APIs. File system APIs, GUI APIs, timer APIs, network APIs. Even standard output is usually exposed via an API in most languages.
> The ellipses suggests a natural conclusion, but none come to mind. Do you mind completing the thought?
The conclusion is that software wasn't always built with web APIs, in contrast from APIs in general.
> The only software I can think of for users that doesn't use an API is maybe on an embedded microcontroller. Everything uses APIs.
I understand that all software uses APIs, but TFA is about API-first software, meaning an API (designed for other software) as the primary interface to a product.
> The conclusion is that software wasn't always built with web APIs, in contrast from APIs in general.
Everything on the web has an API (or user agents wouldn't work), which is why I'm asking. APIs are how the web works.
I understand that all software uses APIs, but TFA is about API-first software, meaning an API (designed for other software) as the primary interface to a product.
> The conclusion is that software wasn't always built with web APIs, in contrast from APIs in general.
Everything on the web has an API (or user agents wouldn't work), which is why I'm asking. APIs are how the web works.
Furthermore, networked APIs were commonplace way before REST and JSON were a thing. Even in the HTTP API era we're at the 'second generation', with the first one being SOAP or XML-RPC. Before that there was a long history of CORBA, DCOM+, DCE/RPC, etc, etc...
According to the article's description of APIs, microcontrollers have their own APIs as well, in the form of machine code. Of course in a different sense there's a technical distinction between APIs and ABIs, but in this sense they're the same thing.
This is the future of my company’s vision and technical product line (an in-house homegrown ERP that is 30 years old). Our company used to attempt do everything in-house, mostly due to technical (in)expertise and lack of accomplishing what the customer truly wants with the tools available. The previous model I guess would be “database-driven” design, since every new tech stack that was built, whether it was for an e-commerce website or warehouse management system, would require database access first.
Since I joined we’ve been SaaSifying the business model, and we’re going about it with an API first design. The ERP, running legacy code written in Clarion (I would be very surprised if anyone here has heard of it!), would expose its procedures and functions via a .NET layer, which would open up our ERP via an API.
With this approach, we’ve been able to 1) divide up responsibilities extremely clearly within our small team, 2) establish new integrations with relevant places we’ve never been able to before such as Shopify and 3) move existing features that the ERP has to a subscription model such as shipping rates, EDI document transfers, and automatic invoicing.
Needless to say I am very much for API and modular design. It’s better from a team organization perspective, since the responsibilities (and automated testing) is clear. From a non-technical perspective, our sales & support reps can see and understand when features are missing or where the bottlenecks are once we draw out how the API system is architected.
Since I joined we’ve been SaaSifying the business model, and we’re going about it with an API first design. The ERP, running legacy code written in Clarion (I would be very surprised if anyone here has heard of it!), would expose its procedures and functions via a .NET layer, which would open up our ERP via an API.
With this approach, we’ve been able to 1) divide up responsibilities extremely clearly within our small team, 2) establish new integrations with relevant places we’ve never been able to before such as Shopify and 3) move existing features that the ERP has to a subscription model such as shipping rates, EDI document transfers, and automatic invoicing.
Needless to say I am very much for API and modular design. It’s better from a team organization perspective, since the responsibilities (and automated testing) is clear. From a non-technical perspective, our sales & support reps can see and understand when features are missing or where the bottlenecks are once we draw out how the API system is architected.
From January 1st to send international small packets "it has become obligatory to transmit data, such as the sender's name and address and names of the contents, to the destination postal service in advance converted into an electronic format called Electronic Advance Data (EAD)." By and in itself this is a good thing.
So the Japanese post (JP Post) has put up a website to do this: problem, there is no API... Everything has to be done by clicking and confirming. It's not even possible to register shipments in bulk, they have to be done one by one. About 10 actions per customer per shipment.
I run a (very) small tea business [1], similar to and inspired by Bemmu's Candy Japan [2] after I met him at a Hacker News meetup in Osaka. It's tiny, 100 to 150 shipments per month, but registering a hundred shipments by hand is not something I want to do. One unsatisfactory answer is of course to reach for good old Selenium. And it works, but I hope JP Post will not update its website too often... (unless it is to add an API, of course).
[1] https://tomotcha.com
[2] https://www.candyjapan.com