Memory System

Overview

NeuronAgent provides a comprehensive memory system for AI agents, enabling persistent storage and retrieval of information across sessions. The memory system uses HNSW-based vector search for efficient similarity search and supports hierarchical organization for better context management.

Key Features

• Hierarchical Memory: Multi-level memory organization for better recall and context management
• Long-Term Memory: HNSW-based vector search for efficient context retrieval
• Memory Promotion: Background worker for promoting important memories to long-term storage
• Event Streaming: Real-time event capture and summarization
• Context Retrieval: Intelligent context loading from messages and memory
• Vector Search: Semantic search across stored memories using embeddings

Hierarchical Memory

The hierarchical memory system organizes memories across multiple levels, allowing agents to maintain both short-term and long-term context. This structure improves recall and enables more efficient memory management.

Memory Levels

• Episodic Memory: Session-specific memories for immediate context
• Semantic Memory: General knowledge and facts stored long-term
• Procedural Memory: Task-specific knowledge and workflows

Memory Organization

Memories are automatically organized based on usage patterns, importance, and relevance. The system promotes frequently accessed memories to higher levels for faster retrieval.

Long-Term Memory

Long-term memory uses HNSW (Hierarchical Navigable Small World) indexes for efficient vector similarity search. This enables agents to retrieve relevant context from large memory stores in milliseconds.

Vector Embeddings

All memories are automatically embedded using configured embedding models, enabling semantic search across stored information. The system supports multiple embedding models and can be configured per agent.

Memory Storage

Memories are stored in PostgreSQL with vector indexes, providing both durability and fast retrieval. The system automatically manages index creation and optimization.

Memory Promotion

The memory promotion system automatically identifies and promotes important memories to long-term storage based on usage patterns, recency, and relevance scores.

Promotion Criteria

• Access Frequency: Frequently accessed memories are promoted
• Recency: Recent memories with high relevance scores
• Importance: Memories marked as important by the agent or user
• Contextual Relevance: Memories that are contextually relevant to current tasks

Background Worker

A dedicated background worker continuously analyzes memory usage patterns and promotes memories automatically. This ensures that important information is always available in long-term storage.

Event Streaming

The event streaming system captures real-time events from agent interactions and automatically summarizes them for storage in memory. This enables agents to maintain awareness of ongoing activities.

Event Types

• Message Events: User and agent messages
• Tool Events: Tool execution results
• State Changes: Agent state transitions
• Error Events: Errors and exceptions

Event Summarization

Events are automatically summarized using LLM integration, creating concise memory entries that capture essential information while reducing storage requirements.

Context Retrieval

The context retrieval system intelligently loads relevant context from both messages and memory, ensuring agents have access to the most relevant information for their current task.

Retrieval Strategies

• Semantic Search: Vector similarity search for relevant memories
• Temporal Relevance: Recent memories are prioritized
• Contextual Filtering: Memories are filtered based on current context
• Relevance Scoring: Memories are ranked by relevance to the current query

Context Window Management

The system automatically manages context windows, ensuring that the most relevant information is included while staying within token limits for LLM interactions.

API Reference

The memory system is accessible through the NeuronAgent REST API and can be managed using the Memory tool in agent workflows.

Memory Endpoints

• POST /api/v1/memories - Create a new memory
• GET /api/v1/memories - List memories with filtering
• GET /api/v1/memories/:id - Get a specific memory
• PUT /api/v1/memories/:id - Update a memory
• DELETE /api/v1/memories/:id - Delete a memory
• POST /api/v1/memories/search - Search memories by similarity
• POST /api/v1/memories/promote - Promote memories to long-term storage

Memory Tool

The Memory tool provides direct access to memory operations within agent workflows, enabling agents to store, retrieve, and manage memories as part of their execution.