I learned some haskell as my hobby language a few years back. It was very cool and forced me to think about programming differently (and finally grok recursion). It never felt like a good language for data analysis to me though (maybe that's cause this library wasn't around? lol). This isn't meant a criticism of this library, instead, I'm curious the use cases the author, if you're around or a user, has in mind. Congrats on the v1 release!
I find this perspective both scary and exciting. I'm curious, how do you validate the LLM's output? If you have a way to do this, and it's working. Then that's amazing. If you don't, how are you gauging "work best"?
Embeddings from things like one-hot, count vectorization, tf-idf, etc into dimensionality reduction techniques like SVD and PCA have been around for a long time and also provided the ability to compare any two pieces of text to each other. Yes, neural networks and LLMs have provided the ability for the context of each word to affect the whole document's embedding and capture more meaning, potentially that pesky "semantic" sort even; but they still are fundamentally a dimensionality reduction technique.
I remember my college roommate, around 2010, using Notepad++. I assumed it was fairly old at that point given how simple and solid it looked. Way to go, Notepad++ for building something that's standing the test of time!