HackerTrans
TopNewTrendsCommentsPastAskShowJobs

delaguardo

no profile record

Submissions

Show HN: Transit-format (JSON/MessagePack) reader/writer in C

github.com
6 points·by delaguardo·الشهر الماضي·0 comments

A fast EDN (Extensible Data Notation) reader written in C11 with SIMD boost

github.com
112 points·by delaguardo·قبل 8 أشهر·38 comments

comments

delaguardo
·الشهر الماضي·discuss
Now I have a reason to get reMarkable :)
delaguardo
·قبل 3 أشهر·discuss
You should split this question in two: - is JVM usage increasing or decreasing in 2026 and - is Java usage increasing or decreasing in 2026

JVM usage (which is the main host for Clojure) as a platform is increasing modestly, driven by modernization (Loom, GraalVM, Valhalla on the horizon)
delaguardo
·قبل 7 أشهر·discuss
From the spec: about symbols - "Symbols are used to represent identifiers" about keywords - "Keywords are identifiers that typically designate themselves."

more than that: in this reader implementation nil, true, false are also valid identifiers with special handling to turn them into nil and bool.

I encourage you to reread original specification and then continue after "this insanity" :)
delaguardo
·قبل 8 أشهر·discuss
And also with experimental things that might eventually find its way to Clojure :) Check out digit delimiters and text block features:

  {:pi  3.141_592_653_589
   :c   299_792_458
   :hex 0xDE_AD_BE_EF
   :tb  """
        #!/usr/bin/env bb

        (require '[babashka.http-client :as http])

        (defn get-url [url]
          (println "Downloading url:" url)
          (http/get url))
        """}
delaguardo
·قبل 8 أشهر·discuss
> I meant that it would have to decode the entire EDN string literal containing the base64 data before decoding the base64

yes, any edn reader implementation will read the complete base64 string from the example before giving this string to a custom reader. I understand now what you explain. However, I don't know what I can do about it. I use edn daily, it works great to me, and I have no immediate plans to replace it with something else.

Anyway, the example you shared looks interesting, I'll definitely read more about it. Thank you.
delaguardo
·قبل 8 أشهر·discuss
> From what I can understand from the specification, the EDN decoder will still need to run and cannot be streamed if the official specification is used

Sorry, you understand it wrong

There is no enclosing element at the top level. Thus edn is suitable for streaming and interactive applications.

> but I don't know if the existing one does

This implementation does not do streaming for now, but it understands a concept of "reading one complete" element from buffer. The only missing part is buffer managment.

> So, the extensibility is still restricted.

Could you explain how it is restricted if you are allowed to run whatever you want during reading of edn document? You can even do IO, no restrictions at all!

Consider this:

#init/postgres {:db-spec {:host "..." :port 54321 ,,,} :specs {:user ,,,}} [#user/id 1 #user/id 2 #user/id 3]

This allows you to have a program that can lookup postgres database during reading of a document validating every returned object using provided spec (conforming the value)

> In my opinion, ASN.1 (and ASN.1X) does it better.

Please show how it does better. I'm very curious
delaguardo
·قبل 8 أشهر·discuss
What do you mean under "syntax typing" and complications in the syntax?

> The whole world seems to run on JSON

That is true, and I don't like that :)

From my perspective JSON syntax is too "light" and that translates to many complications typically in the form of convention: {"id": {"__MW__type": "LONG NUMBER", "value": "9999999999999999999999999"}}.
delaguardo
·قبل 8 أشهر·discuss
It should be easy to add source info for every token, some of them already keep both (size and offset) I can create a branch for that.

> I'm curious how much AI was used in the creation of edn.c

A fair amount. This is my first big public project written in pure C. I did consult LLM about best practices for code organisation, memory management, difference in SIMD instructions between platforms, etc. All the things Clojure developer typically don't think about (luxury of a hosted language). Ultimately, the goal was to learn some part of C programming, working reader is a side effect of that.

> These days, I like to get a measure of that for every library I use.

Btw, I'm curious, what kind of measuring you are looking for?
delaguardo
·قبل 8 أشهر·discuss
I can do a lot without applying schema at all. For that I only need handful of types defined in EDN specification and Clojure programming language.
delaguardo
·قبل 8 أشهر·discuss
Thanks for the link!

Yes, EDN is a textual format intended to be human-readable. There is also a format called Transit used to serialise EDN elements. Unlike raw EDN, Transit is designed purely for program-to-program communication and drops human readability in favor of performance. It can encode data into either binary (MessagePack) or text (JSON), but in both cases, it preserves all EDN data types and originates from the Clojure language.

https://github.com/cognitect/transit-format
delaguardo
·قبل 8 أشهر·discuss
> EDN seems to lack a proper format for binary data

The best part of EDN that it is extendable :)

#binary/base64 "SGVsbG8sIHp6bzM4Y29tcHV0ZXIhIEhvdyBhcmUgeW91IGRvaW5nPw=="

This is a tagged literal that can be read by provided (if provided) custom reader during reading of the document. The result could be any type you want.
delaguardo
·قبل 8 أشهر·discuss
One of a key design principles in EDN is to be exclusively data exchange format. Which is true even for JSON where json-schema is something that sits on top of JSON itself. Same goes to EDN - in Clojure there is clojure.spec that adds schema like notation, validation rules and conformation. https://clojure.org/about/spec , something like this could be implemented in other languages as well.
delaguardo
·قبل 8 أشهر·discuss
Wow, that is a greate news!) Thanks for looking at it from this perspective! There are some benchmarks already available in the project - https://github.com/DotFox/edn.c/blob/main/bench/bench_integr...

you can run it locally with `make bench bench-clj bench-wasm`

Let me know if I can do anything to help you with support in jank.
delaguardo
·قبل 8 أشهر·discuss
I think you can use metadata to model html attributes but in clojure people are using plain vector for that. https://github.com/weavejester/hiccup

tl;dr first element of the vector is a tag, second is a map of attributes test are children nodes:

[:h1 {:font-size "2em" :font-weight bold} "General Kenobi, you are a bold one"]
delaguardo
·قبل 8 أشهر·discuss
Yes, plan is there but didn't have time yet. Most likely will be available next week