Building a High-Performance OpenAPI Parser in Go(speakeasy.com)
speakeasy.com
Building a High-Performance OpenAPI Parser in Go
https://www.speakeasy.com/blog/building-speakeasy-openapi-go-library
10 comments
Disclaimer: I work at Speakeasy but not the author.
It probably needs better wording because it's sort of the wrong complexity metric. Many customers have gigantic OpenAPI documents with large numbers of deep and wide JSON Schemas that contain things like allOf/oneOf/anyOf sub-schemas, all of which need to be parsed into an object model for use by downstream tooling (e.g. code generation). For those customers, we want generation time to be super speedy and since this is a core aspect of Speakeasy, it made a ton of sense to us to take full control of OpenAPI parsing and optimize it.
It probably needs better wording because it's sort of the wrong complexity metric. Many customers have gigantic OpenAPI documents with large numbers of deep and wide JSON Schemas that contain things like allOf/oneOf/anyOf sub-schemas, all of which need to be parsed into an object model for use by downstream tooling (e.g. code generation). For those customers, we want generation time to be super speedy and since this is a core aspect of Speakeasy, it made a ton of sense to us to take full control of OpenAPI parsing and optimize it.
> Some were fast but modeled the spec loosely, making it hard to build correct tooling on top. Others were closer to the spec but used untyped maps everywhere, which made large refactors and static analysis painful.
Correctness and types were the real reasons?
Correctness and types were the real reasons?
One observation I've had recently. Postman files seem more popular the OpenAPI specs lately. Major SaaS companies will produce a postman file but not an OpenAPI spec. Two examples: Salesforce and Notion
This is really unfortunate because Postman requires you to have an account and log in to download or export these to another format.
Prediction: Postman produces a paid MCP for API lookup in the near future
This is really unfortunate because Postman requires you to have an account and log in to download or export these to another format.
Prediction: Postman produces a paid MCP for API lookup in the near future
I just wanted an API client with some basic features:
- history
- grouping/folders
- some very basic api key management
Is that too much to ask or does every company need to indefinitely grow?
- history
- grouping/folders
- some very basic api key management
Is that too much to ask or does every company need to indefinitely grow?
Bruno ticks the boxes for this https://www.usebruno.com/
No affiliation, just a long term fan after years of frustration with Postman and Insomnia.
No affiliation, just a long term fan after years of frustration with Postman and Insomnia.
Postman is my goto example for saas enshitification.
Something that should have just stayed foss.
Something that should have just stayed foss.
> For example, in OpenAPI 3.1, the type field of a schema can be a single string (e.g., "string") or an array of strings (e.g., ["string", "null"]).
> In a statically typed language like Go, this is usually handled by using interface{} (which loses type safety) or complex pointer logic.
Having worked on JSON Schema parsing in go very recently, I disagree with this assessment. You create a `Type` in one of a few (2?) ways, depending on your specific needs. The simple method being that it's a `[]string` under the hood with a custom UnmarshalJSON receiver function. If reproducing the exact input structure is important you can cover that by making `Type` into a struct with a `[]string` and a `bool` to track if it was originally a single or an array. Then you have custom MarshalJSON and UnmarshalJSON receiver functions. That is, in fact, how I've seen multiple existing go JSON Schema libraries handle that variable type. No use of `any` or complex pointers.
> In a statically typed language like Go, this is usually handled by using interface{} (which loses type safety) or complex pointer logic.
Having worked on JSON Schema parsing in go very recently, I disagree with this assessment. You create a `Type` in one of a few (2?) ways, depending on your specific needs. The simple method being that it's a `[]string` under the hood with a custom UnmarshalJSON receiver function. If reproducing the exact input structure is important you can cover that by making `Type` into a struct with a `[]string` and a `bool` to track if it was originally a single or an array. Then you have custom MarshalJSON and UnmarshalJSON receiver functions. That is, in fact, how I've seen multiple existing go JSON Schema libraries handle that variable type. No use of `any` or complex pointers.
Off topic: Something on that web page causes Firefox on my MBA2020 to use 133% of CPU, 30% of GPU Helper, the fan goes to full speed, and scrolling is slow and janky. I can barely read the article.
When I go to Reader mode, the CPU goes down to less than 20%, scrolling works great, and the fan goes off.
Did they implement scrolling using JavaScript?
When I go to Reader mode, the CPU goes down to less than 20%, scrolling works great, and the fan goes off.
Did they implement scrolling using JavaScript?
[deleted]
Doesn't really strike me as the load that requires writing a high-performance solution from scratch, especially on modern hardware.