Memory Papers
Memory Architecture

Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents

AuthorsNatchanon Pollertlam, Witchayut Kornsuwannawit

2026
arXivPDF

TL;DR

Beyond the Context Window uses a Mem0 fact store versus long-context GPT-5-mini to show a 26% cost saving after 20 turns at 100k tokens.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Persistent agents face growing context costs at 100k tokens and beyond

Beyond the Context Window shows that at a context length of 100k tokens, the memory system only becomes cheaper after approximately ten interaction turns.

This means long-context GPT-5-mini agents that resend full histories incur a growing per-turn cost, making long-running persistent assistants economically fragile despite strong factual recall.

HOW IT WORKS

Beyond the Context Window — Mem0 fact memory versus long-context GPT

Beyond the Context Window wires Conversation Segmentation, Fact Extraction, Embedding and Storage, and Retrieval Mechanism into a Mem0-based memory pipeline against long-context GPT-5-mini.

You can think of Beyond the Context Window like RAM and disk: long-context GPT-5-mini rereads the whole log, while Mem0 keeps a compact card catalog of atomic facts.

This design in Beyond the Context Window enables a one-time write cost plus near-fixed read cost, something a plain context window with prompt caching cannot structurally achieve.

DIAGRAM

Turn by turn interaction between user, memory system, and long-context GPT-5-mini

This diagram shows how Beyond the Context Window routes each user turn either through Mem0 retrieval or through long-context GPT-5-mini with full history.

DIAGRAM

Evaluation and cost analysis pipeline across three benchmarks

This diagram shows how Beyond the Context Window evaluates accuracy and cumulative cost on LongMemEval, LoCoMo, and PersonaMem v2.

PROCESS

How Beyond the Context Window Handles a Multi turn Conversation Session

  1. 01

    Conversation Segmentation

    Beyond the Context Window uses Conversation Segmentation with batch_size 10 and 8,000 character limits to preserve temporal order before Fact Extraction.

  2. 02

    Fact Extraction

    Beyond the Context Window runs Fact Extraction with GPT-5-nano to distill long conversations into atomic flat typed facts suitable for Embedding and Storage.

  3. 03

    Embedding and Storage

    Beyond the Context Window performs Embedding and Storage using text-embedding-3-small into a 1536 dimensional pgvector HNSW index for efficient Retrieval Mechanism.

  4. 04

    Retrieval Mechanism

    Beyond the Context Window applies the Retrieval Mechanism with top k 20 facts, then GPT-5-mini reads them to answer new questions at a roughly fixed per turn cost.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Accuracy comparison across three benchmarks

    Beyond the Context Window reports that LC GPT-5-mini reaches 92.85% on LoCoMo and 82.40% on LongMemEval, while the Mem0 Memory System scores 57.68% and 49.00% respectively.

  • 02

    Cost model incorporating prompt caching

    Beyond the Context Window builds a cost model where long-context GPT-5-mini pays $0.0265 on turn one and $0.0036 on cached turns at 101,601 tokens.

  • 03

    Break even analysis for persistent agents

    Beyond the Context Window shows the Mem0 Memory System becomes cheaper after approximately ten turns at 100k tokens and after nine turns at 200k and 500k tokens.

RESULTS

By the Numbers

LoCoMo accuracy

92.85%

+35.17 over Memory System

PersonaMem v2 accuracy

62.48%

vs LC GPT-OSS-120B at 60.50%

LongMemEval accuracy

49.00%

33.40 below LC GPT-5-mini at 82.40%

Cost at 20 turns

$0.0700

26% cheaper than LC GPT-5-mini at $0.0947

Beyond the Context Window evaluates on LongMemEval, LoCoMo, and PersonaMem v2, showing that Mem0 based memory trades 33.4 percentage points of LongMemEval accuracy for 26% lower cost at 20 turns and 101,601 tokens.

BENCHMARK

By the Numbers

Beyond the Context Window evaluates on LongMemEval, LoCoMo, and PersonaMem v2, showing that Mem0 based memory trades 33.4 percentage points of LongMemEval accuracy for 26% lower cost at 20 turns and 101,601 tokens.

BENCHMARK

Accuracy on three datasets for Memory System and long context baselines

Accuracy (%) across LoCoMo, PersonaMem v2, and LongMemEval.

KEY INSIGHT

The Counterintuitive Finding

Beyond the Context Window finds that at 101,601 tokens, the Mem0 Memory System becomes cheaper than long-context GPT-5-mini after only ten interaction turns.

This is surprising because long-context GPT-5-mini also enjoys a 90% prompt caching discount, yet its per turn cost still grows enough to lose to a one time Mem0 write.

WHY IT MATTERS

What this unlocks for the field

Beyond the Context Window unlocks a principled way to choose between Mem0 style fact memory and long-context GPT-5-mini based purely on expected turn counts and context length.

Builders can now design persistent assistants, tutors, and customer support agents that hit explicit cost break even targets, instead of guessing whether to rely on retrieval or huge context windows.

~12 min read← Back to papers

Related papers

Memory Architecture

A Control Architecture for Training-Free Memory Use

Yanzhen Lu, Muchen Jiang et al.

· 2026

TAG routes low-confidence steps to uncertainty-based routing, filters them with guarded acceptance with rollback, chooses between bank selection across rule and exemplar memory, and prunes via evidence-based retirement inside a unified control loop. On SVAMP and ASDiv, TAG reaches 81.0% and 85.2% accuracy, improving over the 74.0% and 77.5% no-memory baselines while a compute-matched Retry baseline stays flat.

arXiv:2604.18206Read explainer

Questions about this paper?

Paper: Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents

Answers use this explainer on Memory Papers.

Checking…