Boost Memory Retrieval: Memento-Inspired Quality System

The Problem with Current Memory Retrieval

Memory retrieval is a crucial aspect of information management. In the current landscape, many memory services rely heavily on quantity-based retrieval. This means that when you search, the system primarily focuses on semantic similarity – how closely related the search terms are to the stored memories. However, this approach has a significant drawback: all memories are treated equally, regardless of their actual usefulness or quality. This can lead to a frustrating experience for users, as the most relevant and valuable information may be buried under a mountain of less helpful or even outdated content. Imagine searching for a solution to a problem you're facing and having to sift through numerous results, only to find that the truly helpful answer is several pages down. This scenario highlights the need for a more intelligent system that can prioritize high-quality memories and offer a more streamlined and effective search experience. A system that can learn and adapt, ensuring that the most valuable information is always at your fingertips.

Limitations of Existing Systems

Several limitations exist within the current memory retrieval systems. Here’s a breakdown:

• Lack of Quality Differentiation: There's no mechanism to distinguish between memories that are truly helpful and those that are not. All memories are treated as equally valuable, leading to a diluted search experience.
• Inability to Learn from User Behavior: The system doesn't learn from user interactions. It cannot identify which memories are frequently accessed, rated positively, or deemed most useful by users. This lack of feedback prevents the system from improving over time.
• Inefficient Search Results: Valuable memories can be lost in a sea of less relevant or outdated information. This makes it difficult for users to quickly find the information they need.
• Ineffective Archiving: Low-quality or outdated memories are not archived quickly enough. This leads to information clutter and can slow down search results.

These limitations underscore the need for a more intelligent and adaptive memory retrieval system.

Introducing the Memento-Inspired Quality System

To overcome the limitations of the current system, we propose a Memento-inspired Quality System. This system draws inspiration from the case-based reasoning framework of Memento, which has demonstrated impressive results. The core idea is to move from quantity-based retrieval to value-based retrieval. The goal is to ensure that the system prioritizes high-quality, outcome-labeled memories.

The Power of Outcome Labeling and Quality Scoring

The fundamental principle behind this system is to label memories with outcomes and assign them a quality score. This allows the system to differentiate between valuable and less valuable information. This approach is based on the insight that prioritizing high-quality, outcome-labeled retrieval is more effective than relying on large-scale similarity searches. By focusing on quality, the system can provide more relevant results.

Core Features of the Quality System

This Memento-inspired quality system introduces several core features designed to enhance memory retrieval:

1. Outcome Labeling: Memories are tagged with outcomes indicating success, failure, or neutrality. This provides direct feedback on the memory's usefulness.

   # Tag memories with success/failure outcomes
   metadata = {
       "outcome": "positive" | "negative" | "neutral",
       "user_rating": 1 | 0 | -1,  # thumbs up/neutral/down
       "feedback_source": "user" | "consolidation" | "automatic"
   }
   

2. Quality Scoring: A composite score is calculated based on multiple signals, including user ratings, access frequency, retrieval ranking, and relevance derived from consolidation processes. This creates a multi-faceted view of each memory's value.

   # Composite quality score (0.0-1.0) from multiple signals
   quality_score = (
       user_rating * 0.30 +         # Explicit feedback
       access_frequency * 0.30 +    # How often retrieved
       retrieval_ranking * 0.20 +   # Position in results
       consolidation_score * 0.20   # Relevance from consolidation
   )
   

3. Quality-Boosted Search: Search results are reranked to prioritize high-quality memories. This ensures that the most valuable information appears at the top of the search results.

   # Rerank results to prioritize high-quality memories
   final_score = (0.7 * semantic_similarity) + (0.3 * quality_score)
   

4. Quality-Based Consolidation: The system uses quality scores to determine how long memories are preserved. High-quality memories are retained for longer periods, while low-quality memories are archived more quickly. This ensures that the system remains efficient and relevant.

By implementing these features, the Memento-inspired Quality System aims to transform memory retrieval, making it more intelligent, adaptive, and user-friendly.

Implementation Plan: A Phased Approach

The implementation of the Memento-inspired Quality System will be executed in phases, ensuring a smooth transition and minimizing disruption to the existing system. This phased approach allows for rigorous testing, feedback gathering, and iterative improvements.

Phase 1: Foundation (Weeks 1-2)

The initial phase focuses on establishing the core components of the new quality system.

• Extend Memory model with quality fields (in metadata)
• Implement calculate_quality_score() function
• Update storage backends (SQLite-vec, Cloudflare, Hybrid)
• Write unit tests

Phase 2: API Layer (Week 3)

This phase focuses on creating the necessary tools and endpoints for interacting with the quality system.

• Add rate_memory MCP tool (thumbs up/down)
• Add get_memory_quality MCP tool (quality metrics)
• Add HTTP endpoints: /api/memories/{hash}/rate
• Write API integration tests

Phase 3: UI Implementation (Week 4)

This phase is dedicated to integrating the quality system into the user interface, making it accessible and user-friendly.

• Add rating buttons to dashboard memory cards
• Add quality badges (color-coded scores)
• Add quality analytics view (distribution chart)
• Write UI tests (Playwright)

Phase 4: Integration (Week 5)

This phase involves integrating the new quality features into the existing system, ensuring seamless functionality.

• Update consolidation with quality-based forgetting
• Implement quality-boosted search (opt-in)
• Add configuration options (feature flags)
• Performance testing

Phase 5: Documentation & Release (Week 6)

The final phase focuses on documenting the new features and releasing the updated system.

• Write user guide (docs/guides/memory-quality-guide.md)
• Update API documentation
• Create changelog entry
• Release as v8.45.0

This structured implementation plan ensures a successful rollout of the Memento-inspired Quality System, providing a more efficient, user-friendly, and intelligent memory retrieval experience.

Technical Design: Schema, API, and Storage Updates

To ensure a smooth and backward-compatible implementation, the technical design incorporates several key elements:

Schema (Backward Compatible)

No database migration is required. New quality fields are stored within the metadata JSON of the Memory model. This approach ensures that existing data structures remain unchanged, minimizing the risk of data loss or compatibility issues.

@dataclass
class Memory:
    content: str
    content_hash: str
    metadata: Dict[str, Any]  # ← Quality fields stored here

    @property
    def outcome(self) -> Optional[str]:
        return self.metadata.get('outcome')

    @property
    def quality_score(self) -> float:
        return self.metadata.get('quality_score', 0.5)  # Default neutral

    @property
    def user_rating(self) -> Optional[int]:
        return self.metadata.get('user_rating')

API Examples

The API provides clear and concise methods for users to interact with the system and provide feedback on the quality of memories.

MCP Tool: Rate Memory

The rate_memory function allows users to mark memories as helpful or unhelpful.

# Mark memory as helpful
rate_memory("abc123...", 1, "This solved my problem!")

# Mark memory as unhelpful
rate_memory("def456...", -1, "Outdated information")

Dashboard UI

The UI will incorporate rating buttons, allowing users to quickly provide feedback directly from the dashboard.

<div class="memory-quality">
    <span class="quality-badge">★ 0.85</span>
    <button onclick="rateMemory(hash, 1)">👍</button>
    <button onclick="rateMemory(hash, -1)">👎</button>
</div>

Storage Layer Updates

SQLite-vec: Quality-Boosted Retrieval

The storage layer will be updated to incorporate the quality score into the retrieval process. This ensures that high-quality memories are prioritized in search results.

async def retrieve(query: str, quality_boost: bool = True):
    # 1. Get semantic similarity results (over-fetch 3x)
    results = vector_search(query, limit=n_results * 3)

    # 2. Calculate quality scores
    for result in results:
        quality = calculate_quality_score(result.memory)
        result.final_score = 0.7 * semantic + 0.3 * quality

    # 3. Rerank and return top N
    results.sort(key=lambda x: x.final_score, reverse=True)
    return results[:n_results]

Consolidation Integration

The consolidation process will be updated to use quality scores to determine how long memories are retained. This ensures that low-quality memories are archived more quickly, keeping the system efficient and relevant.

async def _identify_forgetting_candidates(memories):
    for memory in memories:
        quality = calculate_quality_score(memory)

        # Quality-aware forgetting rules
        if quality < 0.3:
            forget_after = 30  # days
        elif quality < 0.5:
            forget_after = 90
        elif quality < 0.7:
            forget_after = 180
        else:  # High quality
            forget_after = 365

These technical design elements ensure a robust, efficient, and user-friendly quality system.

Performance Expectations and Configuration

The Memento-inspired Quality System is designed to enhance memory retrieval without significantly impacting performance. The system’s design carefully considers the balance between added functionality and potential performance overhead. The implementation team has established key metrics to evaluate the success of the system and ensure its effectiveness. Gradual rollout using feature flags will allow for testing and iterative improvements.

Performance Expectations

To ensure the system meets performance standards, the following metrics will be closely monitored:

Metric	Current	With Quality	Change
Search latency	~30ms (SQLite-vec)	~50ms	+66% (acceptable)
Retrieval precision	~50% helpful	~70% helpful	+40% (target)
Memory retention	Fixed decay	Quality-based	Smarter

These expectations provide a benchmark for assessing the system's performance and ensuring that it meets its objectives. The goal is to provide a noticeable improvement in retrieval precision while keeping the increase in search latency to a minimum.

Configuration

The system will be configurable, allowing administrators to enable or disable features as needed. Feature flags will be used to control the rollout of new features and enable quality-boosted search gradually. This approach allows for testing and adjustments before fully integrating the changes.

# Feature flags (gradual rollout)
export MCP_QUALITY_SYSTEM_ENABLED=true          # Enable quality tracking
export MCP_QUALITY_BOOST_ENABLED=false          # Opt-in for search boost
export MCP_QUALITY_BOOST_WEIGHT=0.3             # 30% quality, 70% semantic

These configurations provide flexibility and control, allowing for fine-tuning and optimization of the system.

Success Metrics and Comparison with Memento

Key Success Metrics

The success of the Memento-inspired Quality System will be measured by several key metrics. These metrics will help to assess the system's effectiveness in improving memory retrieval and enhancing user experience. Measuring these metrics allows for continuous improvement and ensures the system meets its goals.

1. Retrieval Precision: The percentage of top-5 results rated helpful should increase to >70% (up from ~50%).
2. User Engagement: >30% of memories should be rated by users within 3 months.
3. Quality Distribution: 20-30% of memories should have a quality score ≥0.7, indicating high-quality memories.
4. Search Latency: Search latency for SQLite-vec with quality boost should remain below 50ms.

Comparison with Memento

The Memento-inspired Quality System adopts key principles from the Memento framework while adapting them to the specific needs of the memory service. This approach allows for a streamlined implementation, focusing on the most impactful features while minimizing complexity.

What We Adopted ✅

• Outcome labeling: Positive/negative/neutral tags to provide direct feedback on memory usefulness.
• Quality scoring: A composite score from usage and feedback, offering a multi-faceted view of memory value.
• Small memory philosophy: Prioritizing the best memories rather than simply storing the most.
• Value-based retrieval: Prioritizing quality over similarity to ensure more relevant results.

What We Skipped ❌

• Parametric memory: The current system relies on embeddings, which have proven effective.
• Planner-executor: This is a different domain; we are focusing on storage, not an agent framework.
• Full training pipeline: No infrastructure is in place yet, so a complete training pipeline is not feasible at this time.

Adaptation Rationale

The system adapts Memento's philosophy (outcome-based memory) but not its implementation (RL-trained retriever). This provides:

• 80% of benefits (quality-aware retrieval)
• 20% of complexity (no training pipeline)

This approach allows us to realize the benefits of Memento's core principles without overcomplicating the system. By focusing on the essentials, we can deliver a more effective and user-friendly memory retrieval experience.

Future Enhancements and Related Work

Future Enhancements (Post-v8.45.0)

The Memento-inspired Quality System is designed to be extensible, allowing for future enhancements and improvements. Further improvements could greatly increase performance.

• v8.50.0+: Neural Reranking
  • Train a cross-encoder on collected feedback.
  • Implement two-stage retrieval (embeddings + reranker).
  • This requires 1000+ (query, memory, helpful?) tuples.
• v8.55.0+: Contextual Quality
  • Implement quality scores that vary by context.

  quality_scores = {
      "general": 0.8,
      "python": 0.9,     # High quality for Python queries
      "javascript": 0.5  # Lower for JS queries
  }
  

These enhancements will further improve the system's ability to provide relevant and high-quality memories.

Related Work

• Memento Repository: https://github.com/Agent-on-the-Fly/Memento
• Memory Analysis: Stored in memory service (Dec 5, 2025)
• Detailed Plan: /tmp/memento-adoption-plan.md (45KB implementation guide)

These resources provide additional information and context for understanding the Memento-inspired Quality System.

Timeline and Benefits Summary

Timeline

The project is expected to be completed within six weeks, with a target release date of v8.45.0. This timeline provides a structured approach to ensure the successful implementation of the new features. It also allows for continuous improvement.

• Week 1-2: Foundation (model, storage, tests)
• Week 3: API layer (MCP tools, HTTP endpoints)
• Week 4: UI (dashboard rating interface)
• Week 5: Integration (consolidation, search boost)
• Week 6: Documentation & release

Benefits Summary

The Memento-inspired Quality System offers a range of benefits that will transform the MCP Memory Service and greatly improve the user experience.

1. ✅ Learn from feedback: The system improves based on user feedback.
2. ✅ Smarter retrieval: Prioritizes high-quality memories in search results.
3. ✅ Efficient consolidation: Archives low-quality memories faster.
4. ✅ Backward compatible: No database migration is required, ensuring a smooth transition.
5. ✅ Extensible: Provides a foundation for future ML enhancements, such as neural reranking.

This system transforms the MCP Memory Service from a static storage layer to a learning memory system. It evolves and improves based on user interactions, aligning with Memento's vision of continual learning without LLM fine-tuning. This enhances the value of the service and provides a much better user experience.

For more information on the principles behind this system, check out the Memento project on GitHub: Memento.