dial481

dial481

27 hari yang laludiscuss

This is badly needed. Jira is horrendous.

Can your plugin system support custom workflow triggers yet?

dial481

28 hari yang laludiscuss

[flagged]

dial481

28 hari yang laludiscuss

[flagged]

dial481

3 bulan yang laludiscuss

MemPalace's benchmark claims have been picked apart and debunked in the project's GitHub issues:

github.com/milla-jovovich/mempalace/issues/27 github.com/milla-jovovich/mempalace/issues/29 github.com/milla-jovovich/mempalace/issues/39 github.com/milla-jovovich/mempalace/issues/125 github.com/milla-jovovich/mempalace/issues/242

(and others)

TL;DR: When independent third parties ran actual end-to-end QA instead of retrieval metrics, the scores dropped dramatically.

dial481

4 bulan yang laludiscuss

That's encouraging to hear from someone with IR experience, thanks. Agree completely.

dial481

4 bulan yang laludiscuss

We audited the LoCoMo benchmark (one of the most cited eval for LLM agent memory) and found 99 score-corrupting errors in 1,540 questions (6.4%). Separately, we tested the LLM judge with adversarially generated wrong answers, it accepted 62.81% of vague-but-topical wrong answers. Some published system scores barely clear that bar. Full audit with methodology, all 99 errors documented, and reproducible scripts.

Submissions

[untitled]

Show HN: Allelix – Annotate your DNA against 7 public databases offline

[untitled]

Show HN: Proposal for a real long-term AI memory benchmark

Milla Jovovich's MemPalace Claims 100% on LoCoMo. Its Benchmarks.md Disagrees

LoCoMo AI Benchmark: 6.4% of answer key wrong, judge accepts 63% of fake answers

comments