Show HN: Hasura GraphQL Data Connector SDK to add GraphQL API to any data source(hasura.io)
hasura.io
Show HN: Hasura GraphQL Data Connector SDK to add GraphQL API to any data source
https://hasura.io/blog/hasura-graphql-data-connectors/
3 comments
Just curious -- what are the performance trade-offs here? I assume you are basically querying each individual database and then doing a series of joins by doing additional queries? Or are you doing something more sophisticated with cache-ing, etc.?
Do you mean in Hasura's case, or for Data Connector SDK?
With the SDK, it's just an API contract -- you are saying "I have a web service that implements these endpoints you can call to ask for schema/metadata info + perform a query".
Whatever business logic happens inside of the connector service is opaque, so it's up to the author to implement.
For Hasura's own DB-to-DB joins, we do what's called a "Data-shipment Join". It's similar to what you mention.
DB's are queried in relationship top-down order, and join column values (this is over-simplifying) materialized as a sort of a temp-table in the target DB, which is used to JOIN, down the chain.
It's O(n) where n = (number of DBs in the query), which is best-case given the nature of the problem.
For single-DB joins, we compile these to single aggregated queries using JSON_ARRAY_AGG and JSON_BUILD_OBJECT equivalents in the target dialect.
With the SDK, it's just an API contract -- you are saying "I have a web service that implements these endpoints you can call to ask for schema/metadata info + perform a query".
Whatever business logic happens inside of the connector service is opaque, so it's up to the author to implement.
For Hasura's own DB-to-DB joins, we do what's called a "Data-shipment Join". It's similar to what you mention.
DB's are queried in relationship top-down order, and join column values (this is over-simplifying) materialized as a sort of a temp-table in the target DB, which is used to JOIN, down the chain.
It's O(n) where n = (number of DBs in the query), which is best-case given the nature of the problem.
For single-DB joins, we compile these to single aggregated queries using JSON_ARRAY_AGG and JSON_BUILD_OBJECT equivalents in the target dialect.
I work at Hasura -- specifically on the feature mentioned here.
This is the culmination of the better part of a year of work. I'd like to talk about why I find this problem and new functionality interesting:
The SDK allows people to author services that provide automatic GraphQL on top of any data. It extends the core functionality of Hasura (generating GQL from databases) into anything that can provide data.
This can be CSV files, other API's (think Airtable, etc), physical sensors like Raspberry Pi's or IoT Devices, or databases Hasura doesn't yet support -- MongoDB, Oracle, etc.
But the killer feature is the ability to integrate with Hasura cross-source joins and authorization layer to get federated querying with permissions on top of any datasource.
Really looking forward to what the future has in store for Data Connectors and hope other folks find it useful + interesting as well.