HackerTrans
トップ新着トレンドコメント過去質問紹介求人

roampal

no profile record

投稿

[untitled]

1 ポイント·投稿者 roampal·6 か月前·0 コメント

1% vs. 67%: What happened when we stopped trusting embeddings alone

roampal.ai
16 ポイント·投稿者 roampal·6 か月前·7 コメント

RAG accuracy jumped from 10% to 60% when I added outcome scoring

roampal.ai
11 ポイント·投稿者 roampal·7 か月前·7 コメント

Show HN: I made Claude Code learn from its mistakes

github.com
4 ポイント·投稿者 roampal·7 か月前·0 コメント

Show HN: Roampal – a local memory layer that learns from outcomes

github.com
1 ポイント·投稿者 roampal·7 か月前·4 コメント

コメント

roampal
·7 か月前·議論
You're right, it is a form of tagging technically. The difference is you're already saying "thanks that worked" or "nah that's wrong" anyway. No extra step, it just listens.
roampal
·7 か月前·議論
Ran a 4-way comparison test across 200 query-memory pairs:

- Baseline RAG (embedding similarity only): 10%

- RAG + reranker: 20%

- Outcomes only (no reranker): 60%

- RAG + outcome scoring (mature memories with 20+ uses): 60%

"Accuracy" = correct memory ranked #1 for the query. The outcome scoring uses Wilson score lower bound - memories that consistently get positive feedback from the "user" get boosted, ones that fail get demoted.

Test methodology: https://github.com/roampal-ai/roampal/blob/main/dev/benchmar...
roampal
·7 か月前·議論
Fair point, the install instructions at the end were meant as a "here's how to try it if interested" but I can see how it reads as pushy. The core of the post is about the outcome scoring approach itself. Should've led with more depth on the methodology. Thanks for the feedback.
roampal
·7 か月前·議論
Absolutely, go for it!

Run whatever benchmarks you have. I would love to see how it stacks up against RL methods.

Ping me if you need help with anything.

Thanks!
roampal
·7 か月前·議論
Dude, you literally wrote the exact motivation paragraph for Roampal right around the same time I posted this

Thorndike's Law of Effect is the entire reason I built the outcome-scoring (+0.2 for worked, −0.3 for failed) and shift weighting toward proven memories. You're not half-baked — you're 100% right. I just happened to ship the PoC first.

Would love to hear your take on the cold-start problem and whether those reward magnitudes feel right in practice. Shooting you a connection request on LinkedIn if you want to swap notes.