Question
Is the public Memory Palace retrieval benchmark claim supported by committed evidence?
Short finding
Evidence-backed claim: the public Memory Palace retrieval benchmark claim is aligned with committed evidence. Raw LongMemEval rows: 500. Raw LongMemEval Recall@5: 0.966. This is evidence-backed committed artifact alignment, not independent scientific proof.
What FoldEngine checked
- Public repository surface
- Committed LongMemEval benchmark result artifact
- Benchmark documentation
- Retrieval implementation anchors
What FoldEngine did not check
- Independent scientific validation
- Independent benchmark rerun
- Full code review
- Security audit
- Legal or compliance review
- Private systems or private repository content
Evidence boundary / receipt-style summary
- Artifact kind
- claim_evidence_probe
- Evidence surface
- Committed public artifacts
- Key metric recorded
- 500 rows; Recall@5 0.966
- Proof boundary
- Not independent scientific proof
Seed candidates
Seed A — Tighten benchmark claim language
Hypothesis: Naming the exact artifact, split, and metric prevents over-interpretation of benchmark results.
Value: high · Risk: low
Evidence required: README claim references specific dataset split, metric name, and evaluation artifact.
Verification: Claim language matches committed evaluation output exactly.
Decision receipt: PROCEED
Benchmark claim is evidence-backed by committed artifacts. Proceed with the caveat that stronger rerank/perfect-score wording should be tightened.
- Decision
- Proceed
- Expires
- 30 days from issuance
Governed next steps
-
Priority 1 — Seed A: Tighten claim language
Unlock: Access to README and evaluation artifacts.
Exit: Claim language references exact artifact, split, and metric.