Exactly. It is intentionally open-ended: new "principles" can emerge, split, or retire as the taxonomy evolves. This is just version 1. The grid is a visual index and the fun part is mappin systems to "molecules" in different domains like OS, databases, computer architecture, distributed systems, programming languages, networking, and more..
Author here, great question :) If principles are the elements, we can think of each system as a "molecule" with some imagination. For example, an SQL database system has many principles:
1. Abstraction Lifting (Al) + Policy/Mechanism Separation (Pm): SQL states high-level intent with precise semantics, and logical operators are decoupled from physical operators.
2. Equivalence-based Planning (Ep) + Invariant-Guided Transformation (Ig): We apply algebraic rewrites that preserve semantics (e.g., join reordering, predicate pushdown) under stated invariants.
3. Cost-based Planning (Cm): We choose concrete physical operators and join orders using a cost model and so on..
Totally fair.. I am not claiming periodicity here :) I just wanted to use the "periodic table" as a visual metaphor. The goal is to outline a mostly orthogonal set of system design principles and illustrate cross-domain connections to students so that it is easier to compare trade-offs and discuss designs more precisely.
Author here, appreciate the share :) I was not expecting this to get so much attention.
To clarify: this is indeed just a taxonomy of classic system-design principles. The periodic-table styling is a familiar metaphor; there is no claim that principles repeat periodically. The goal was to outline a mostly orthogonal set of design principles and highlight cross-domain connections across computer systems so it is easier to discuss designs precisely. Thanks for all the thoughtful feedback!
It is based on the feedback shared by an undergraduate student -- pretty subjective. For instance, it is possible to graduate with a CS degree without even taking an introductory course on systems.
As we do not have ground truth, we only qualitatively checked for accuracy -- no quantitative metrics. We did find a significant drop in accuracy with GPT 3.5 as opposed to GPT 4.
Are you measuring accuracy with data wrangling prompts? Would love to learn more about that.
Yes, there is definitely a human-in-the-loop element here.
It would be great if you could share an example of the inconsistent output problem -- we also faced it. GPT-4 was much better than GPT-3.5 in output quality.
Another interesting aspect was the money spent on LLMs. We could have directly used GPT-4 to generate the "golden" table; however, it's a bit expensive — costing $60 to process the information of 1000 users. To maintain accuracy while reducing costs significantly, we set up an LLM model cascade in the EvaDB query, running GPT-3.5 before GPT-4, leading to a 11x cost reduction ($5.5).
Thanks for your kind words, @skeptrune! It was certainly a fun project.
We found some interesting insights. In the Langchain community, ~40% of the stargazers are from India. In the GPT4All community, we found that web developers love open-source LLMs -- more so that machine learning folks :)
Curious if there is any reason why you would not use it?