Why Graphiti?(graphiti.dev)
graphiti.dev
Why Graphiti?
https://www.graphiti.dev/guides/why
46 comments
Without remarking on the substance of this post, I just wanted to say I enjoyed the writing style and the design of this post tremendously. I was actually quite surprised by how much I enjoyed it haha.
On the other hand, I'm on the other side on this. I kept thinking, "get to the point, what are you trying to say?"
The amount of times I've used APIs claiming to be RESTful but actually aren't consistent... If you use something like Prisma you don't need to worry about the boilerplate or having to maintain consistency when creating a GQL API.
I love the ability to browse introspection via playground most of all.
My two annoyances right now are: lack of input unions and lack of recursive fetching for tree/node based resources
I love the ability to browse introspection via playground most of all.
My two annoyances right now are: lack of input unions and lack of recursive fetching for tree/node based resources
I've wanted this for a while actually. GraphQL is great, but it lacks convention. Which makes it hard for API discovery and code generation. I noticed that the author uses Ruby, which makes total sense, as Rails is a shining example of how conventions in REST can make generating an API incredibly easy.
I actually tried to make a GraphQL based framework in Ruby a while back that emphasized convention. By having convention, I was able to generate, say, a field that searches for a resource by ID. Not revolutionary stuff, but nice if you don't want to rewrite that boilerplate a bunch.
From what I've seen, there's a cycle of structure vs less structure in paradigms. SOAP was too unstructured, so REST was more structured. REST is too structured so now GraphQL. I suppose it's only a matter of time before a structured alternative to GraphQL comes out.
I actually tried to make a GraphQL based framework in Ruby a while back that emphasized convention. By having convention, I was able to generate, say, a field that searches for a resource by ID. Not revolutionary stuff, but nice if you don't want to rewrite that boilerplate a bunch.
From what I've seen, there's a cycle of structure vs less structure in paradigms. SOAP was too unstructured, so REST was more structured. REST is too structured so now GraphQL. I suppose it's only a matter of time before a structured alternative to GraphQL comes out.
GraphQL doesn't not have support for dictionaries and JSON. I'd rather just use JSON schema + Mongo directly. If there is a way to batch multiple HTTP requests ... then GraphQL would be an overkill.
Neither of those is exactly true. GraphQL has support for user-defined scalars and we’ve defined a `JSON` scalar that is used to transfer arbitrary JSON objects as JSON-encoded strings (we also have a `map` scalar that renders as a JSON object at its field, but accepts a JSON-encoded string for input formats; we are shifting away from that because _some_ clients don’t have the flexibility of producing inputs like that).
This looks like an effort to bring to REST APIs some of the benefits GraphQL offers, mainly by adding extensive conventions on top of traditional HTTP endpoints. Comes with an implementation in Ruby.
The idea has some merit. That said, I am not sure I have spotted the discoverability features equivalent to what’s possible in GraphQL world with a GraphiQL console.
The idea has some merit. That said, I am not sure I have spotted the discoverability features equivalent to what’s possible in GraphQL world with a GraphiQL console.
> This looks like an effort to bring to REST APIs some of the benefits GraphQL offers, mainly by adding extensive conventions on top of traditional HTTP endpoints.
This was my take-away as well. There are a couple of full-featured frameworks in GraphQL land -- Hasura[0] and Prisma[1] spring to mind, among others -- that enforce conventions and in exchange allow you to specify a sort_by array and operate on a standardized payload without exposing too much non-essential configuration.
---
[0] https://hasura.io/all-features
[1] https://www.prisma.io/with-graphql
This was my take-away as well. There are a couple of full-featured frameworks in GraphQL land -- Hasura[0] and Prisma[1] spring to mind, among others -- that enforce conventions and in exchange allow you to specify a sort_by array and operate on a standardized payload without exposing too much non-essential configuration.
---
[0] https://hasura.io/all-features
[1] https://www.prisma.io/with-graphql
I must admit the main thing that puts me off GraphQL is that it's not built on JSON. Maybe that's silly, but it just seems like a move away from what we know and love.
Also the graphql tools seem to be based around the idea of a fixd schema, and I'm looking for something a little more flexible.
The schema is not defined in json, which is good, json can't have comments. Graphql requests and responses are json, and you can get a json representation of the schema using the introspection api.
I for one do not like json, the only formats I can think of that I like less are xml and yaml. I'd prefer a protocol with a binary implementation, it must support comments, I like type annotations, and a schema language would be great.
I've been working with Amazon's ion lately and I've enjoyed it. It has text and binary representations, fields have types (in binary), it can be represented as json (you lose annotations)
I for one do not like json, the only formats I can think of that I like less are xml and yaml. I'd prefer a protocol with a binary implementation, it must support comments, I like type annotations, and a schema language would be great.
I've been working with Amazon's ion lately and I've enjoyed it. It has text and binary representations, fields have types (in binary), it can be represented as json (you lose annotations)
> move away from what we know and love
I didn't get the memo where we decided that we love JSON.
I didn't get the memo where we decided that we love JSON.
Sure, but there are tradeoffs to consider: JSON may be difficult and inconvenient to use for human interaction, but on the other hand it's also really slow for machine reading and writing. Plus you have the added bonus of it being an inefficient use of bandwidth.
[deleted]
Especially when there are almost consensual better alternatives https://hjson.org
Uhh, no. Part of the reason JSON is useful is that it's easy to write a correct parser in comparison to most other things - adding a bunch of the complexity of YAML back to JSON seems a bad idea. This makes it possible to use JSON as a serialisation layer for network protocols without introducing too much additional risk of parsing-related vulnerabilities.
When it comes to modern web development I’m more of an observer than a contributor, so my opinion might not carry a lot of weight. However for me all these abstractions seems to get closer and closer to querying a database using SQL directly.
A carefully designed database schema would be able to support all of these use cases in a way that (at least for me) seems a lot simpler than wrapping it in new abstractions. Inserting multiple objects in the same transaction?, already implemented. Updating several fields at the same time as well? Getting only the entities you are interested in together with their subentities?, well that’s what a relational database does.
As I said, I’m not a web developer, and I haven’t touched on use cases where a lot of different things need to happen on server operations, but all the examples in the article would be easily solvable with SQL.
A carefully designed database schema would be able to support all of these use cases in a way that (at least for me) seems a lot simpler than wrapping it in new abstractions. Inserting multiple objects in the same transaction?, already implemented. Updating several fields at the same time as well? Getting only the entities you are interested in together with their subentities?, well that’s what a relational database does.
As I said, I’m not a web developer, and I haven’t touched on use cases where a lot of different things need to happen on server operations, but all the examples in the article would be easily solvable with SQL.
GraphQL came from Facebook, where it's used to wrap other services and aggregate the results into a tree of data. I've also used it this way.
As I understand, most of the constraints (writes are done in isolation, you don't have transactions, etc.) are there because they're consistent with an architecture where you can't actually implement those things. Transactions are something that happens further down the chain of services, and a single mutation may or may not be implemented using transactions that may or may not be distributed across multiple services. A mutation could cause information to be sent to message queues, stored in various data stores with different technologies, cached in multiple places, persisted to a data warehouse, or other side effects, and transactions are an implementation detail.
You could use foreign data wrappers in postgres, or some other equivalent, but most of the good bits of SQL would take a lot of work or wouldn't even be possible.
As I understand, most of the constraints (writes are done in isolation, you don't have transactions, etc.) are there because they're consistent with an architecture where you can't actually implement those things. Transactions are something that happens further down the chain of services, and a single mutation may or may not be implemented using transactions that may or may not be distributed across multiple services. A mutation could cause information to be sent to message queues, stored in various data stores with different technologies, cached in multiple places, persisted to a data warehouse, or other side effects, and transactions are an implementation detail.
You could use foreign data wrappers in postgres, or some other equivalent, but most of the good bits of SQL would take a lot of work or wouldn't even be possible.
PostgREST is a project that does that automatically for PostgreSQL. It takes your schema of your db and creates an api based on table relationships and provides security through a database design pattern of enforcing row level security. Side effects are challenging. For instance sending an email on account creation requires creating a listener on the postgres notify event but it is interesting to think about even if not 100% practical.
https://postgrest.org/en/v6.0/
https://postgrest.org/en/v6.0/
I wonder if the people at edgedb (https://edgedb.com/) have the right approach then. They used Postgres as a foundation, and used their experience in SQL to make a DSL that is similarly expressive, but but with less caveats and some helpers for very common use cases.
It's a very young project, and as a heavy ORM using I'm not the best person to judge, but I like the way they think.
It's a very young project, and as a heavy ORM using I'm not the best person to judge, but I like the way they think.
I personally think fundamentally what a lot
Of people miss in the API standards discussions (debates?) is that GraphQL, JSONApi et al are not just about getting around the single operation per request/call, they are about abstracting your API interfaces always from a tightly coupled schema. I think this because most APIs are built in these circumstances (and this assumes that your endpoints are actually RESTful):
* the schema/Endpoint already exists and it’s being updated to a newer standard (like GraphQL or OpenAPI) and they simply use the schema of the existing data source (usually a database) and these are just coupled to the data source. This gains more efficient operations and possibly documentation and validation (all good things!) but misses another goal (will explain momentarily)
* That an api is being built from scratch and one of these schema based standards is being used as a stand in for the database store. In this case the tight coupling is just backward (your data source will likely be built to match the API)
What I seem to not see is anyone talking serious about how these standards and associated technologies or implementations aren’t being leveraged to create an abstract interface to data points regardless of data source structure or schema. The strength of GraphQL, OpenAPI (formerly Swagger), JSONApi and others isn’t just in pure description and validation, it’s in that it allows you to build descriptive and normalized APIs that can in effect act as middleware over any data source and keeps your data sources independent from API concerns.
The end result being you shouldn’t have to change your API schemas because you changed your data store or how you want to store your data. So you never have to version your APIs. It’s suppose to make your APIs inclusive of change.
* the schema/Endpoint already exists and it’s being updated to a newer standard (like GraphQL or OpenAPI) and they simply use the schema of the existing data source (usually a database) and these are just coupled to the data source. This gains more efficient operations and possibly documentation and validation (all good things!) but misses another goal (will explain momentarily)
* That an api is being built from scratch and one of these schema based standards is being used as a stand in for the database store. In this case the tight coupling is just backward (your data source will likely be built to match the API)
What I seem to not see is anyone talking serious about how these standards and associated technologies or implementations aren’t being leveraged to create an abstract interface to data points regardless of data source structure or schema. The strength of GraphQL, OpenAPI (formerly Swagger), JSONApi and others isn’t just in pure description and validation, it’s in that it allows you to build descriptive and normalized APIs that can in effect act as middleware over any data source and keeps your data sources independent from API concerns.
The end result being you shouldn’t have to change your API schemas because you changed your data store or how you want to store your data. So you never have to version your APIs. It’s suppose to make your APIs inclusive of change.
You'd have to keep users of your application in sync with database users though. And you wouldn't get complex backend logic out of the box without something like PL/(pg)SQL or database extensions (all of which are a bit unwieldy compared to an HTTP server).
Doesn't work that well when your databases are distributed and you're using microservices. You don't want to be doing join's across microservices and most setups wouldn't allow you to anyways. GraphQL provides that orchestration layer for you.
Let's say it provides more like a standardized common interface for you. You still have to code the glue to make all those data aggregations, cache handling and permission checks.
In fact, because it's such a new tech, and since it's embedded into very few things, you have a lot to write by hands and perf on single servers will suffer compared to regular SQL + Restish.
The concept is interesting for big distributed heterogeneous systems though, especially when you have many clients.
In fact, because it's such a new tech, and since it's embedded into very few things, you have a lot to write by hands and perf on single servers will suffer compared to regular SQL + Restish.
The concept is interesting for big distributed heterogeneous systems though, especially when you have many clients.
This is exactly what Hive is. https://mapr.com/products/apache-hive/
It provides a full SQL query engine on top of unstructured data so you can assemble results.
Personally I would never expose the raw DB in any capacity to the front end systems for security and sanity reasons. DB schemas need to be free to change all the time and you wouldn't want to break entire services from a column type change.
It provides a full SQL query engine on top of unstructured data so you can assemble results.
Personally I would never expose the raw DB in any capacity to the front end systems for security and sanity reasons. DB schemas need to be free to change all the time and you wouldn't want to break entire services from a column type change.
I think the primary reason not to do this is security—and I don't just mean that letting attackers make Postgres queries directly is a bad idea (it is, of course, but you could imagine other ways to implement this). You want any interface you expose to the public to be simple enough that you can be reasonably confident that none of the messages you accept can cause bad things to happen. SQL is so computationally powerful that you can't really get that assurance.
> You want any interface you expose to the public to be simple enough that you can be reasonably confident that none of the messages you accept can cause bad things to happen.
Yes, that's why you expose a simple set of views as your public interface for each role, and apply the dirt simple RBAC that every multiuser RDBMS supports to limit users to the appropriate set of views.
> SQL is so computationally powerful that you can't really get that assurance.
The computational power of SQL is mostly irrelevant, because the dirt simple security model let you limit database effects, so that all the computational power means is that people can tie up resources with poorly (or maliciously) crafted queries, if you don't apply the simple mitigations every multiuser RDBMS has against those.
People not bothering to learn about the database engines they are using leads to putting (often poor) ad hoc security in the web layer in front of a database that isn't properly secured because the app in front is excessively trusted, making this less, rather than more, secure than if people just learned to understand heir DB and exposed it directly instead of trying to acheive security through cargo cult complexity.
It's possible to enhance database security with defense in depth and and additional layers, but none of that starts with misunderstanding RDBMSs so badly as to think that they are, in an app stack, one of the hard parts to secure.
Yes, that's why you expose a simple set of views as your public interface for each role, and apply the dirt simple RBAC that every multiuser RDBMS supports to limit users to the appropriate set of views.
> SQL is so computationally powerful that you can't really get that assurance.
The computational power of SQL is mostly irrelevant, because the dirt simple security model let you limit database effects, so that all the computational power means is that people can tie up resources with poorly (or maliciously) crafted queries, if you don't apply the simple mitigations every multiuser RDBMS has against those.
People not bothering to learn about the database engines they are using leads to putting (often poor) ad hoc security in the web layer in front of a database that isn't properly secured because the app in front is excessively trusted, making this less, rather than more, secure than if people just learned to understand heir DB and exposed it directly instead of trying to acheive security through cargo cult complexity.
It's possible to enhance database security with defense in depth and and additional layers, but none of that starts with misunderstanding RDBMSs so badly as to think that they are, in an app stack, one of the hard parts to secure.
I am like you as well except the database might be mongo,elasticsearch,etc... Plus processing of the data might need to be done before the client gets it or the response might be based on db data but the fields might not be db mappable because they're dynamic (e.g.: 'baseItemCost' vs 'itemCostAdjusted' adjusted to item availability,demand,delivery cost,etc...)
I completely agree!
I really don't understand why we don't simply ship SQL queries around, maybe a very simple subset of SQL so that it either works on most vendors or that it can be easily manipulated and adapt to different storage backend.
Can somebody enlight me?
I really don't understand why we don't simply ship SQL queries around, maybe a very simple subset of SQL so that it either works on most vendors or that it can be easily manipulated and adapt to different storage backend.
Can somebody enlight me?
it is unlikely that all the data sources you will be querying now are directly available in an SQL database.
For sure they are not available in a /rest or GraphQL engine neither.
SQL should be only the syntax used to communicate with the backend, the real implementation would be in whatever make sense.
SQL should be only the syntax used to communicate with the backend, the real implementation would be in whatever make sense.
That would require you to map the SQL to something else which would then map back to SQL or mongo or w/e. For example if you're trying to join data from multiple microservices you can't execute the join because most microservices setups are not going to allow joins. The joins would have to be mapped to service to service calls and at this point you're just rebuilding an orchestration layer which has already been solved by graphql.
So we invented a query language (GraphQL) to solve an orchestration problem?
Sorry but I don't buy this reason. Especially not to the question: "Why we don't use SQL as a query language?" especially because REST was around way earlier than GraphQL.
Your reasoning could be great to answer the question: "Why we like GraphQL where there are a lot of microservices?"
BTW, there is nothing inherent in the language in itself, if each microservices would be speaking SQL, you would simply need to optmize your SQL query, decompose your optimized SQL query creating simpler queries one for each microservice, send each of those queries to the relative microservices, compute back the final result.
If you replace "microservices" with "table" here you get the standard way an SQL engine works :)
Please, it is not that I am against GraphQL, but it honestly seems like some engineer at facebook was annoyed by SQL, and decide to re-invent it. And that we are all following him or she just because they worked at facebook, which seems unwise.
Maybe the use of types? But what is the difference between a type and something that belong to a table?
Anyway, I am looking now at [GraphQL: The Documentary (Official Release)][1]
[1]: https://www.youtube.com/watch?v=783ccP__No8&feature=youtu.be
Sorry but I don't buy this reason. Especially not to the question: "Why we don't use SQL as a query language?" especially because REST was around way earlier than GraphQL.
Your reasoning could be great to answer the question: "Why we like GraphQL where there are a lot of microservices?"
BTW, there is nothing inherent in the language in itself, if each microservices would be speaking SQL, you would simply need to optmize your SQL query, decompose your optimized SQL query creating simpler queries one for each microservice, send each of those queries to the relative microservices, compute back the final result.
If you replace "microservices" with "table" here you get the standard way an SQL engine works :)
Please, it is not that I am against GraphQL, but it honestly seems like some engineer at facebook was annoyed by SQL, and decide to re-invent it. And that we are all following him or she just because they worked at facebook, which seems unwise.
Maybe the use of types? But what is the difference between a type and something that belong to a table?
Anyway, I am looking now at [GraphQL: The Documentary (Official Release)][1]
[1]: https://www.youtube.com/watch?v=783ccP__No8&feature=youtu.be
> So we invented a query language (GraphQL) to solve an orchestration problem? Sorry but I don't buy this reason.
Exactly. I came here to say the same thing. This seems like an attempt to justify the existence of a thing after-the-fact. And it's a reason which isn't even the stated reason of the project itself!
Exactly. I came here to say the same thing. This seems like an attempt to justify the existence of a thing after-the-fact. And it's a reason which isn't even the stated reason of the project itself!
There is nothing about GraphQL that makes me think someone invented it because they were annoyed by SQL and decided to reinvent it, generally reinvention means a lot of similarity only badly done. You might argue GraphQl is badly done in comparison to SQL, but surely not that it is similar.
Honestly I don't use it so often, but it seems that any query in graphql could be easily and automatically translated to a SQL query. Am I wrong?
There are definitely trying to solve the same problem, aren't they? What are the advantages of GraphQL over SQL?
Please, I really want to know the answer to this question.
The previous answer was "the infrastructure we build around GraphQL is better for microservices", ok I can see that. But it is the only answer? It seems like a little weak reason to create a language so that later you can create a new infrastructure around it.
Am I missing something?
Why you don't find GraphQL similar to SQL?
There are definitely trying to solve the same problem, aren't they? What are the advantages of GraphQL over SQL?
Please, I really want to know the answer to this question.
The previous answer was "the infrastructure we build around GraphQL is better for microservices", ok I can see that. But it is the only answer? It seems like a little weak reason to create a language so that later you can create a new infrastructure around it.
Am I missing something?
Why you don't find GraphQL similar to SQL?
Normally when I see a language that was created to fix problems somebody had with another language, then the two languages tend to be syntactically similar. I don't see any syntactic similarity.
Secondly I don't believe that GraphQl is meant to map to the relational algebra, which is the genesis of SQL.
GraphQL's benefit seems to be that you should be able to get all the data you need and nothing more with one query, and get it back in the form you want it. In the earlier comment that started this thread the commenter said something like that's what SQL gives you, but from my experience if you want to get back complicated data (3 joins, many properties with same names) your SQL statement is going to be a lot more difficult to write and organize than your GraphQL query. Writing the queries is generally really straightforward and it is quite quickly clear how to structure a query to get what you want, and indeed if it is even possible to structure the query. Ease of declaring what you want is a valid reason for having a new language.
Structuring a query in SQL or most query languages are generally more difficult than in GraphQL - in my experience. This of course does not mean there are not places where it can get complicated - just generally not at query construction time.
I can't really say if having a new infrastructure is a weak reason for creating a new language, I guess that depends on the infrastructure that one needs. Languages are generally really powerful tools for doing things, so if you want to do difficult things with higher productivity than previously you might feel encouraged to create a new language as part of your endeavors.
Secondly I don't believe that GraphQl is meant to map to the relational algebra, which is the genesis of SQL.
GraphQL's benefit seems to be that you should be able to get all the data you need and nothing more with one query, and get it back in the form you want it. In the earlier comment that started this thread the commenter said something like that's what SQL gives you, but from my experience if you want to get back complicated data (3 joins, many properties with same names) your SQL statement is going to be a lot more difficult to write and organize than your GraphQL query. Writing the queries is generally really straightforward and it is quite quickly clear how to structure a query to get what you want, and indeed if it is even possible to structure the query. Ease of declaring what you want is a valid reason for having a new language.
Structuring a query in SQL or most query languages are generally more difficult than in GraphQL - in my experience. This of course does not mean there are not places where it can get complicated - just generally not at query construction time.
I can't really say if having a new infrastructure is a weak reason for creating a new language, I guess that depends on the infrastructure that one needs. Languages are generally really powerful tools for doing things, so if you want to do difficult things with higher productivity than previously you might feel encouraged to create a new language as part of your endeavors.
I really like your reasoning, thanks.
While I agree that GQL queries are simpler to write, it is also true that they are much less powerful.
I personally don't believe that it is a valid reason for creating a new language and all this infrastructure, but for some kind of project I can see the attractiveness.
While I agree that GQL queries are simpler to write, it is also true that they are much less powerful.
I personally don't believe that it is a valid reason for creating a new language and all this infrastructure, but for some kind of project I can see the attractiveness.
Exposing that much of your internal implementation details could be very problematic.
Sorry, I realize my first message wasn't as clear as I wanted.
You would not expose the internal implementation, you would simply accept SQL queries, maybe against a separate view of your SQL database, or maybe against a custom backend.
SQL shuould be the query languange, not the implementation.
You would not expose the internal implementation, you would simply accept SQL queries, maybe against a separate view of your SQL database, or maybe against a custom backend.
SQL shuould be the query languange, not the implementation.
That’s exactly what I meant as well. SQL may be the implementation, but if so with a well designed exposed schema, not with the application internals.
The biggest reason for graphql is in my opinion the enforced schema. REST apis so often had a nice written document describing the api and its fields, but nowhere was stated that some attributes can be null in some rare edge cases. In graphql the spec states an attribute can be null and i will not get some code crashing in a few months because of some undocumented behaviour.
I also miss that in GraphQL.
With swagger I can exactly define how an object looks like. Not only if something is null or string, but also what the minLength the string is or what enum-values it can have.
At the risk of being a language advocate I wish the reference server implementation had been in Typescript rather than Ruby.
I'm not sure how this differs from some of the more fully thought through hypermedia frameworks. Although it is nice that there's convention for flattening included linked resources down to something a bit more manageable.
For me, GraphQL really shines when going beyond CRUD. In my experience, many client applications aren't just querying and filtering, they're acting as a gateway for complex server processes and need to present a lot of data at once. These advantages aren't going to come through in simple CRUD examples.
GraphQL, in its current state, has some frustrating limitations, but I still think the future looks more like it than REST.
For me, GraphQL really shines when going beyond CRUD. In my experience, many client applications aren't just querying and filtering, they're acting as a gateway for complex server processes and need to present a lot of data at once. These advantages aren't going to come through in simple CRUD examples.
GraphQL, in its current state, has some frustrating limitations, but I still think the future looks more like it than REST.