HackerTrans
TopNewTrendsCommentsPastAskShowJobs

roampal

no profile record

Submissions

[untitled]

1 points·by roampal·6 mesi fa·0 comments

1% vs. 67%: What happened when we stopped trusting embeddings alone

roampal.ai
16 points·by roampal·6 mesi fa·7 comments

RAG accuracy jumped from 10% to 60% when I added outcome scoring

roampal.ai
11 points·by roampal·7 mesi fa·7 comments

Show HN: I made Claude Code learn from its mistakes

github.com
4 points·by roampal·7 mesi fa·0 comments

Show HN: Roampal – a local memory layer that learns from outcomes

github.com
1 points·by roampal·7 mesi fa·4 comments

comments

roampal
·7 mesi fa·discuss
You're right, it is a form of tagging technically. The difference is you're already saying "thanks that worked" or "nah that's wrong" anyway. No extra step, it just listens.
roampal
·7 mesi fa·discuss
Ran a 4-way comparison test across 200 query-memory pairs:

- Baseline RAG (embedding similarity only): 10%

- RAG + reranker: 20%

- Outcomes only (no reranker): 60%

- RAG + outcome scoring (mature memories with 20+ uses): 60%

"Accuracy" = correct memory ranked #1 for the query. The outcome scoring uses Wilson score lower bound - memories that consistently get positive feedback from the "user" get boosted, ones that fail get demoted.

Test methodology: https://github.com/roampal-ai/roampal/blob/main/dev/benchmar...
roampal
·7 mesi fa·discuss
Fair point, the install instructions at the end were meant as a "here's how to try it if interested" but I can see how it reads as pushy. The core of the post is about the outcome scoring approach itself. Should've led with more depth on the methodology. Thanks for the feedback.
roampal
·7 mesi fa·discuss
Absolutely, go for it!

Run whatever benchmarks you have. I would love to see how it stacks up against RL methods.

Ping me if you need help with anything.

Thanks!
roampal
·7 mesi fa·discuss
Dude, you literally wrote the exact motivation paragraph for Roampal right around the same time I posted this

Thorndike's Law of Effect is the entire reason I built the outcome-scoring (+0.2 for worked, −0.3 for failed) and shift weighting toward proven memories. You're not half-baked — you're 100% right. I just happened to ship the PoC first.

Would love to hear your take on the cold-start problem and whether those reward magnitudes feel right in practice. Shooting you a connection request on LinkedIn if you want to swap notes.