joshring · HackerTrans

vor 8 Monatendiscuss

Originally (maybe over a year ago) I had similar issues. But now Zitadel is one `enable = true;` option[1] away and in the official nixpkgs repo so you shouldn't really have this issue anymore. I was able to use it pretty easily with the built in service and postgres service[2] (note mine is encapsulated in a nixos container but otherwise the inner config is all you really need).

[1]: https://search.nixos.org/options?channel=25.05&query=zitadel

[2]: https://git.joshuabell.xyz/ringofstorms/dotfiles/src/branch/...

joshring

vor 2 Jahrendiscuss

Well, as someone building something similar I have been looking around at how people are tackling the problem of varied index approaches for different files, and again how that can scale.

I haven't read the code on your github but the readme mentions using qdrant/pgvector. I'm curious how you will tackle having that scale to billions of files with tens/hundreds/etc? different indexing approaches for each file. It doesn't feel tennable to keep it in a single postgres instance as it will just grow and grow forever.

Think even a very simple example of more indexes per file: having chunk sizes of 20/500/1000 along with various overlaps of 50/100/500. You suddenly have a large combination of indexes you need to maintain and each is basically a full copy of the source file. (You can imagine indexes for BM25, fuzzy matching, lucene, etc...)

You could be brute force ish and always run every single index mode for every file until a better process exists to only do the best ones for a specific file. But even if you narrowed it down a file could want 5 different index types searched and ranked for Retrieval step.

I want to know how people plan to shard/make it possible to have so many search indexes on all their data and still be able to query against all of it. Postgres will eventually run out of space even on the beefiest cloud instance fairly quickly.

The second biggest thing is then to tackle how to use all of those indexes well in the Retrieval step. Which indexes should be searched against/weighted and how given the user query/convo history?

joshring

vor 2 Jahrendiscuss

Nice, thanks for the reply. Glad to hear you are looking into these challenges and plan to tackle some of them. Will keep my eye on the repo for some of these improvements in the future.

And totally agree, the scaling out of ingesting large quantities of data is a hard challenge as well and it does make sense to work on that problem space too. Sounds like that is a higher priority at the moment which is totally fine.

joshring

vor 2 Jahrendiscuss

Is there a roadmap for planned features in the future? I wouldn't call this a "powerful tool for addressing key challenges in deploying RAG systems" right now. It seems to do the most simple version of RAG that the most basic RAG tutorial teaches someone how to do with a pretty UI over it.

The most key challenges I've faced around RAG are things like:

- Only works on text based modalities (how can I use this with all types of source documents, including images)

- Chunking "well" for the type of document (by paragraph, csvs including header on every chunk, tables in pdfs, diagrams, etc). The rudimentary chunk by character with overlap is demonstrably not very good at retrieval

- the R in rag is really just "how can you do the best possible search for the given query". The approach here is so simple that it is definitely not the best possible search results. It's missing so many known techniques right now like:

    - Generate example queries that the chunk can answer and embed those to search against.

    - Parent document retrieval

    - so many newer better Rag techniques have been talked about and used that are better than chunk based

    - How do you differentiate "needs all source" vs "find in source" questions? Think: Summarize the entire pdf, vs a specific question like how long does it take for light to travel to the moon and back?

- Also other search approaches like fuzzy search/lexical based approaches. And ranking them based on criterial like (user query is one word, use fuzzy search instead of semantic search). Things like that

So far this platform seems to just lock you into a really simple embedding pipeline that only supports the most simple chunk based retrieval. I wouldn't use this unless there was some promise of it actually solving some challenges in RAG.

joshring

vor 3 Jahrendiscuss

I'll have to try this out. Currently use Amethyst + Hammerspoon scripts for my window tools. Like others in this thread, Amethyst occasionally loses track of all windows and requires a restart (esp after monitor (dis)connection).

Amethyst does a decent job at the layouts I care about.

I primarily use AwesomeWM in linux on my personal computers which has the amazing super key drag/resizing behavior for windows. I use Hammerspoon to replicate this behavior[0][1] and it works quite well.

Eventually I want to replace Amethyst and just do everything in Hammerspoon as it seems quite plausible to do window layouting with it. Will give Yabai a try as well in the meantime.

[0]: https://github.com/RingOfStorms/setup/blob/master/home/confi... [1]: https://share.joshuabell.link/ezgif-1-ef0d3e0728.gif

joshring

vor 3 Jahrendiscuss

This was also an issue I had, but there is actually very good support for this hidden by an invisible feature. Simply add an `*@domain` as your email address identity. When you select that as your from address the fastmail UI gives you an input box to use whatever email you want, you don't need to make a new identity each time. For lookup, I just use my password manager.

joshring

vor 6 Jahrendiscuss

I agree, all of the "do it all" database tools have always hurt in the long term in every system I've worked on that uses them (TypeOrm, Prisma, etc..)

From what I can tell it is always better to just learn and understand the database you are using and create it by hand. You may spend a tiny bit longer with basic crud boiler plate, but it will benefit you greatly in the long term to just have solid native migrations setup. And a framework that simply works with your database schema, rather than having an opinion forced on your database.

In Prisma, Typeorm, etc you are a slave to what the ORM wants the database to look like. Tools like Objection, jOOQ, etc are all much easier to work with in the long term and allow you to tune your database by hand and without random framework constraints.

Prisma is great if you plan on never maintaining past your MVP, so I guess it makes sense that startups use it and get stuff out the door quickly, but I don't find it as a long term solution or for it to ever be able to handle complicated situations that _will happen_ to your database in the future.

Edited: Spelling

comments