Ask HN: What are your biggest pains integrating and building APIs?
10 comments
We use a number of 3rd party APIs, and the problem we hit was knowing which to trust, which were reliable and which could be counted on.
I'm obviously not talking about your Googles / Twilios / Auth0s / etc. I'm talking about the longer tail of services that we don't want to build ourselves, but that there isn't instantly a big name service that provides it.
We would try one or two, and then find that they would work great in testing, but when you put any kind of load on them then they would regularly fall down. Similarly, when you reach out for help - while you're in the buying phase, they're very responsive - but as soon as you need support, you're looking at days not hours for a reply.
I'm obviously not talking about your Googles / Twilios / Auth0s / etc. I'm talking about the longer tail of services that we don't want to build ourselves, but that there isn't instantly a big name service that provides it.
We would try one or two, and then find that they would work great in testing, but when you put any kind of load on them then they would regularly fall down. Similarly, when you reach out for help - while you're in the buying phase, they're very responsive - but as soon as you need support, you're looking at days not hours for a reply.
Too many vendors with me-too products that claim they're going to help me integrate and build APIs. I see it as the easiest thing in software development, am I supposed to get help from people who think it's the hardest?
[deleted]
Integrating: a whole zoo of conventions on both sides of req/res. Auth methods (header, bearer, in-payload), signing (cf of ordered field concatenation), excessive encodings and/or encryption, success reports. Underconstrained and incomplete descriptions (does 200 always imply {status:"success",…}?). Examples with misleading bogus data in requests and empty responses, or non-reproducibility. Either no testing endpoints, or test.example.com works different from api.example.com. Hateoas continuations from json to html implying you’re a browser. Inconsistencies in call schemes which prevent factoring out most of the above into a single interface.
A special place in hell for APIs with multi-step setups for an access token (get jwt to get jwt which expires in few minutes but ofc cannot be renewed in advance), multi-step setups for a request envelope, then a cryptic payload, then polling (not too often!) for “ready” identifiers (which disappear once fetched, so please cache not-yours ones), then a separate request for fetching a response by identifier, deleted once fetched. Telcos and banks love this bullshit.
-
Building: needing to take various platforms and traditions into account (e.g. some php shops just don’t understand what long-poll or jsonl/ndjson streaming is and have a higher chance of hitting the auth wall for days). Webhook-based messaging requires valid domain names in development or setting up a complex local infrastructure. Overall boring prologues and epilogues for api calls in standard http libraries (try, rate, auth, parse params/body, validate, 2 lines of actual code, catch, decide if error is worth 500 or 4xx with a message, log details, report back). Testing with curl with auth/signing enabled just sucks.
A special place in hell for APIs with multi-step setups for an access token (get jwt to get jwt which expires in few minutes but ofc cannot be renewed in advance), multi-step setups for a request envelope, then a cryptic payload, then polling (not too often!) for “ready” identifiers (which disappear once fetched, so please cache not-yours ones), then a separate request for fetching a response by identifier, deleted once fetched. Telcos and banks love this bullshit.
-
Building: needing to take various platforms and traditions into account (e.g. some php shops just don’t understand what long-poll or jsonl/ndjson streaming is and have a higher chance of hitting the auth wall for days). Webhook-based messaging requires valid domain names in development or setting up a complex local infrastructure. Overall boring prologues and epilogues for api calls in standard http libraries (try, rate, auth, parse params/body, validate, 2 lines of actual code, catch, decide if error is worth 500 or 4xx with a message, log details, report back). Testing with curl with auth/signing enabled just sucks.
At least for testing webhooks locally: there is https://play.svix.com and ngrok (and probably others).
I do integrate API sometimes into my products. My latest API integration as well as API redistribution is at https://trackmybot.com which is using the POST method.
Before this, I'd built https://bunnyapi.com which used a purely GET method.
Once you learn how to build an API, it's straightforward and not a pain at all. I use https://curlconverter.com/ to do the API which returns it in any language you need it in.
Then just read in the request, check the key if you have one, along with all the other instructions.
Before this, I'd built https://bunnyapi.com which used a purely GET method.
Once you learn how to build an API, it's straightforward and not a pain at all. I use https://curlconverter.com/ to do the API which returns it in any language you need it in.
Then just read in the request, check the key if you have one, along with all the other instructions.
F****. That curl converter is what I needed all this time!
Integrating: usually none because I've never worked with an API that didn't involve building a ton of BL around their use to fit our business use cases
Building: the biggest pain is usually the language we are working in. the spec/protocol/etc don't matter, if they're popular they'll be supported in most languages. however, some languages just make building APIs hard-mode and annoying AF. even though I'm not the biggest fan of nodeJS anymore, building JSON apis with typecsript was a breeze compared to any other language. Have built both public REST and GraphQL apis.
Building: the biggest pain is usually the language we are working in. the spec/protocol/etc don't matter, if they're popular they'll be supported in most languages. however, some languages just make building APIs hard-mode and annoying AF. even though I'm not the biggest fan of nodeJS anymore, building JSON apis with typecsript was a breeze compared to any other language. Have built both public REST and GraphQL apis.
Biggest pain point is every organization has their own implementation details about the REST-like (or not) API, payload structure, paging, rate limits, error handling, etc. There is no solution to this, if someone comes up with yet another standard it will just be yet a different way people will implement things [0].
A woodworking analogy: API's are the glue between applications, sometimes its a nice dovetail joint between the apps with a little glue and a lot of craftsmanship and things fit nicely. Sometimes it looks like Homer Simpson built it after a 12 pack of Duffs with glue all over the place and maybe a rusty bent nail holding things together. In the end it doesn't really matter since you are usually forced to integrate with an API so you just deal with the varying levels of quality.
Here are some things to not do (and how to deal):
- Error responses in HTML (have good handling for when JSON.parse fails)
- Missing/incorrect documentation on some endpoints (call the API and iterate to see how it responds)
- When changing API versions, change the ID of the things in the API (scripts to update old to new ID's, feature flag what ID to use, add yet another column to a table like v4_id)
[0] https://xkcd.com/927/
A woodworking analogy: API's are the glue between applications, sometimes its a nice dovetail joint between the apps with a little glue and a lot of craftsmanship and things fit nicely. Sometimes it looks like Homer Simpson built it after a 12 pack of Duffs with glue all over the place and maybe a rusty bent nail holding things together. In the end it doesn't really matter since you are usually forced to integrate with an API so you just deal with the varying levels of quality.
Here are some things to not do (and how to deal):
- Error responses in HTML (have good handling for when JSON.parse fails)
- Missing/incorrect documentation on some endpoints (call the API and iterate to see how it responds)
- When changing API versions, change the ID of the things in the API (scripts to update old to new ID's, feature flag what ID to use, add yet another column to a table like v4_id)
[0] https://xkcd.com/927/
I'd love to get some more insights on your experience with this topic.
Do you integrate 3rd party APIs in your product?
Did you build your own private or public API?
What pains are you running into integrating or building these APIs?