A · Filesystem-native agent
Strong baseline with files, search, shell/tool access and freedom to organize or rewrite persistent information.
research://memorybench
SymbioRAP MemoryBench
An architecture-neutral benchmark for persistent agent memory: measure continuity, temporal correctness, adaptation, latency and cost without rewarding one storage design by construction.
RESEARCHPUBLISHED · v1.0.0-m14
research://thesis
A memory benchmark is only useful if strong non-memory baselines are allowed to compete.
MemoryBench avoids a weak “chat history vs memory product” comparison. The benchmark gives a best-of-breed filesystem-native agent real tools to search, organize, rewrite and consolidate its own persistent workspace, then compares that baseline with SynapseFS and SynapseFS + SymbioRAP Memory under matched model, information and compute conditions.
focus://memorybench
Research focus
B · SynapseFS agent
Same agent conditions with deterministic persistence, stable identities, evidence and version semantics.
C · SynapseFS + Memory
Adds episodic/semantic memory, temporal graph, consolidation and provenance-aware context selection.
Matched conditions
Model, information access, task set and compute budget are controlled so architecture effects remain interpretable.
architecture://working-model
Benchmark dimensions span quality, time and operating cost.
The benchmark protocol is designed to expose trade-offs rather than produce a single flattering score. It measures whether an architecture remains correct over time, adapts to changed facts, retrieves the right evidence and does so within practical latency and token/compute budgets.
Accuracy
Correct task answers and evidence use across short and long horizons.
Temporal consistency
Correct handling of superseded facts, changed preferences and time-dependent state.
Adaptation
Ability to revise behavior when new evidence conflicts with earlier assumptions.
Latency & cost
Retrieval/consolidation overhead, token use and compute required to maintain memory quality.
publication://m14
Open benchmark evidence, not a leaderboard
MemoryBench v1.0.0-m14 publishes the benchmark method, comparator sources, normalized reference evidence and a machine-readable claim matrix. Windows and Linux are treated as separate test populations; unlike semantic classes are not collapsed into a single winner score.
PUBLISHED · v1.0.0-m14
evidence://reference
Published reference evidence
Promotion evidence is retained with negative results and variance. These measurements describe the tested hosts and configurations, not universal hardware or product rankings.
| Population | H6 ingest | H6 query | H8 · 1 worker | H8 · 32 workers |
|---|---|---|---|---|
| Windows x64 | 154.78 ops/s CV 37.70% | 1758.49 q/s CV 8.73% | 264.00 ops/s | 749.48 ops/s CV 1.45% |
| Linux ARM64 | 137.71 ops/s CV 4.67% | 551.55 q/s CV 15.91% | 129.13 ops/s | 1143.26 ops/s CV 15.77% |
SQLite
Windows 3.53.4 · 1132.88 ops/s · PASS
Linux 3.45.1 · 1304.90 ops/s · PASS
RocksDB
Windows 11.8.1 · 1175.84 ops/s · PASS
Linux 8.9.1 · 866.83 ops/s · PASS
FAISS
Windows 1.14.3 · default NEGATIVE RESULT
Linux 1.14.1 · 180.95 q/s · PASS
SQLite and RocksDB are matched durable single-key KV comparators. FAISS is retrieval-only. Their throughput values are not semantically equivalent to full Memory Engine ingest.
FAISS default correctness passed on the tested Linux configuration. The tested Windows default configuration retained transient retrieval failures; a separately labeled Windows single-thread diagnostic passed. This is published as configuration-specific evidence, not as a general claim about FAISS.
open://reproduce
Reproduce and audit the benchmark
The public repository contains frozen workload definitions, the neutral adapter contract, SQLite/RocksDB/FAISS comparator sources, verification tooling, methodology and M14 reference results. The proprietary SynapseFS / SymbioRAP Memory Engine implementation is not published.
claims://boundary
What M14 does — and does not — support
Supported by the frozen evidence
- 58/58 native Windows correctness tests and 62/62 native Linux correctness tests passed.
- 100/100 abrupt-termination recovery runs passed on each tested platform.
- H6 task accuracy remained 1.0 across all promotion repetitions.
- At 32 workers, median physical WAL syncs per ingest fell below the fixed four WAL records per ingest, showing measured group batching on both tested hosts.
- Genuine SQLite and RocksDB durable-KV comparator runs completed with correctness PASS on both tested platforms.
Not supported by this publication
- No claim that SymbioRAP or SynapseFS is universally faster than SQLite or RocksDB.
- No claim that the Memory Engine is faster than FAISS; FAISS is retrieval-only.
- No Windows-versus-Linux performance ranking; the hosts and external-library versions differ.
- No universal “best-of-breed” claim from these two reference machines.
research://questions
Open research questions
Which tasks genuinely require a dedicated memory layer instead of disciplined filesystem organization?
How much improvement comes from persistence semantics alone before semantic memory is added?
Which consolidation policies improve long-horizon accuracy without creating stale or overconfident memory?
How should benchmark tasks resist model memorization and architecture-specific shortcuts?
status://research
SymbioRAP Research
The points shown describe research and implementation directions, not guarantees of finished product functionality.
connect://research
Contribute to the research architecture
For research, funding and technology partners, we share architecture questions, benchmark design and technical work states in qualified collaboration.
Next useful step
Tell us your idea – a few bullet points are enough.
You do not need to define the technical solution. Describe what you want to build or improve and we will map feasibility, a sensible first scope and next steps.
Get a free idea assessment
Four fields are enough. You will receive a concrete response on how to start small and extend the project later.