Intelligent Agent Memory System

The Agent Memory System (api/agent_autopilot.py, ~1,930 lines) replaces flat last-N memory with a structured three-tier hierarchy exposed via a Model Context Protocol (MCP) server (api/mcp_memory_server.py). It enables the Autopilot and Co-Pilot LLM agents to retain proven rules across sessions, transfer knowledge between symbols, and provide the LLM with only relevant context — with a Memory Researcher fetching historical context from PostgreSQL and a Strategy Advisor generating recommendations.

---

Architecture Overview

Key Components

Component	File	Lines	Purpose
Autopilot Loop	api/agent_autopilot.py	1118–1930	Full orchestration: vision → memory research → strategy generation → block validation → backtest → learn → synthesize → evaluate → backtrack
MCP Server	api/mcp_memory_server.py	—	TCP JSON-RPC server exposing search_agent_memory tool (consumed by the AI Assistant generator and the Slack agent)
Memory Researcher	api/agent_autopilot.py	467–730	Queries agent_memories (PostgreSQL) for rules + exact/cross-asset insights and summarizes them via LLM
Strategy Advisor	api/agent_autopilot.py	733–1049	Two-turn LLM dialogue: turn 0 selects tags for memory search, turn 1 analyzes context and writes recommendations
Critic Sub-Agent	api/agent_autopilot.py	1052–1115	Standalone strategy logic validator (currently uncalled by any route or loop)
Block Validation	api/agent_autopilot.py	1387–1423	Programmatic check for restricted blocks (pro_only, kline_only)
Rule Synthesis	api/agent_autopilot.py	285–378	Promotes 3+ similar insights into a permanent rule
Lifecycle Evaluation	api/agent_autopilot.py	381–453	Reinforces success, decays/deprecates failures
Tag Classification	api/agent_autopilot.py	202–282	LLM assigns tags, strategy type, outcome to each insight

---

Database Model

Sources: models.py

The table is auto-created at startup (api/depthsight_api.py:1469, inside the FastAPI lifespan). Extended fields (tags, symbol, strategy_type, outcome, confidence, validated_count, config_hash) were added via migration alembic/versions/2a487db2c451_add_memory_tags.py, and community fields (visibility, source_node_uuid, author_user_id, community_confirmations) via 3c4d5e6f7a8b_add_community_memory_fields.py.

Memory Types

Type	TTL	Description
rule	Permanent	Cross-asset rules promoted from insights
strategy_insight	90/30/60 days	Tagged conclusions from backtest runs
optimization	60 days	Autopilot final optimization results

Only strategy_insight, rule, and optimization are currently written by production code (api/agent_autopilot.py:1662/1779/291/363/1899); preference, market_context, and observation are declared in the model comment (api/models.py:159) but not written by any code path.

TTL Policies for Insights

Outcome	TTL
Success (PnL > 0, trades >= 5)	90 days
Failure	30 days
Optimization result	60 days
Rules	Permanent (expires_at = None)

---

MCP Memory Server

The memory system is exposed as a Model Context Protocol tool via a custom TCP-based JSON-RPC 2.0 server (api/mcp_memory_server.py, 359 lines).

Protocol

Aspect	Detail
Transport	TCP (not stdio)
Protocol	JSON-RPC 2.0
Port	8100 (configurable via MCP_MEMORY_PORT)
Host	127.0.0.1 (dev) / 0.0.0.0 (Docker)
Version	2025-03-26

Tool: search_agent_memory

The tool (api/mcp_memory_server.py:149-246) performs a cascade search with an overall budget of 8 items:

Sources: mcp_memory_server.py

Community memories are included only when the user's share_community_memories flag is set (lines 161–166). Cross-asset transfer candidates are fetched with a budget + 5 over-fetch and filtered to insights whose symbol differs from the target (lines 198–209). The output lines are formatted inline (lines 211–246), tagging shared rows with [COMMUNITY 🌐].

Context Format

**Universal Rules:**
- [conf: 85%] Volume spike > 2x required for breakout
- [COMMUNITY 🌐] [conf: 70%] Prefer 2x volume confirmation on breakouts

**ETHUSDT Insights:**
- [SUCCESS] Ascending triangle breakout with ADX confirmation

**Cross-Asset Transfer:**
- ⚡ [Transfer from BTCUSDT] [SUCCESS] Similar momentum setup

Deployment

In Docker, the MCP server runs as a separate container:

Sources: docker-compose.yml

Other services (api, websocket, bot, celery_worker, celery_beat, slack_agent) connect via MCP_MEMORY_HOST=mcp-memory.

Environment Variables

Variable	Default	Description
MCP_MEMORY_PORT	8100	TCP port
MCP_MEMORY_HOST	127.0.0.1	Host (use 0.0.0.0 in Docker)
AI_ADVISOR_MODEL	—	Model for Strategy Advisor agent (falls back to the provider default: gemini-3-flash-preview / qwen-max / google/gemini-3-flash-preview)

---

Sub-Agent Critic

The Critic sub-agent (api/agent_autopilot.py:1052-1115) is a standalone LLM validator that checks strategy JSON for logical flaws. It is bypassed in the autopilot loop (replaced by Memory Researcher + Strategy Advisor + programmatic block validation) and is not currently wired to any REST route or called from anywhere else.

Critic Checklist

The intended checklist lives in api/prompts/critic_system.md (loaded once at agent_autopilot.py:462 but currently unused); run_critic_agent() instead sends an inline prompt (lines 1072–1078) asking for {"approved", "reason", "critical_flaw"}:

Check	Description
Inverted SL/TP	Stop Loss above entry for long, or Take Profit below
Contradictory filters	Both uptrend AND downtrend conditions simultaneously
Missing weights	Zero or missing foundation weights
Missing partial exits	Strategy defines exits but no partial exit rules

Output Format

{"approved": true}

On rejection:

{"approved": false, "reason": "Stop Loss is set above entry price for long position.", "critical_flaw": "inverted_stop_loss"}

Sources: agent_autopilot.py

Model Selection

Configurable via AI_CRITIC_MODEL env var:

Provider	Default Model
Gemini	gemini-3-flash-preview
Qwen	qwen-max
OpenRouter	google/gemini-3-flash-preview

Auto-Approve Fallback

If the critic LLM call fails entirely, the system auto-approves the strategy to avoid blocking the pipeline — either because the response was not valid JSON (lines 1103–1107):

{"approved": True, "reason": "Critic hallucinated, auto-approved.", "critical_flaw": None}

or because the call raised an exception (lines 1111–1115):

{"approved": True, "reason": f"Critic offline or errored, bypassing. ({e})", "critical_flaw": None}

---

Memory Researcher

Before strategy generation, the Memory Researcher (api/agent_autopilot.py:467-730) queries the agent_memories table (PostgreSQL) for relevant historical context — active rules, exact-symbol insights, and cross-asset transfers — and summarizes it with an LLM:

Sources: agent_autopilot.py

The returned context is an LLM-written summary of the retrieved memories (the deterministic three-section format shown in the MCP section above is produced by the MCP search_agent_memory tool itself, not by the Researcher).

---

Strategy Advisor

After the Memory Researcher returns context, the Strategy Advisor (api/agent_autopilot.py:733-1049) conducts a two-turn LLM dialogue:

1. Turn 0 — Tag Selection: The advisor proposes tags for memory search (e.g., ["momentum", "breakout", "ethusdt"]). These tags are used to query additional context.
2. Turn 1 — Recommendation: The advisor analyzes the full context (tags + memory + best historical configs) and writes a strategy recommendation.

Sources: agent_autopilot.py

Configurable via AI_ADVISOR_MODEL env var (default: the provider's default model — gemini-3-flash-preview / qwen-max / google/gemini-3-flash-preview).

---

Block Validation

After strategy generation, a programmatic block validator (api/agent_autopilot.py:1387-1423) checks for restricted blocks:

Block Type	Restriction
pro_only	Only allowed if user has Pro subscription
kline_only	Only allowed on supported timeframes

If validation fails, the autopilot retries strategy generation with the list of violating blocks as feedback.

---

Autopilot Loop

The full agent cycle is orchestrated in run_autopilot_loop() (api/agent_autopilot.py:1118-1930). It accepts a max_iterations parameter (default 5; the string until_profitable runs up to 30 iterations):

Sources: agent_autopilot.py

Loop Steps

Step	Description
Vision	Separate LLM call analyzes chart screenshot (before the loop); the observations are posted to the event stream as a status message
Memory Research	Fetch rules + exact/cross-asset insights from the agent_memories table
Strategy Advisor	Two-turn LLM: selects tags → generates strategy with full context
Block Validation	Programmatic check for pro_only / kline_only violations; retries on failure
Backtest	Celery task executes the backtest asynchronously
Learn	Save insight with tags, outcome, confidence; success = 90d TTL, failure = 30d TTL
Synthesize	If 3+ insights for this strategy type, promote to a rule
Evaluate	Reinforce success (+0.1 confidence, capped at 1.0) or decay failure (−0.2, community rules −0.05); deprecate at the thresholds below
Backtrack	Compare risk-adjusted (Calmar) performance to best so far; reset strategy if worse, update baseline if better

Screenshot & Vision Analysis

Before the loop begins, an optional chart screenshot is analyzed in a separate LLM call:

1. Vision analysis — a separate LLM call analyzes the screenshot for chart patterns and posts the observations to the event stream as a status message (lines 1263–1269). The vision output is not passed to the Strategy Advisor.
2. Optional — both image_base64 and image_mime_type are nullable parameters; the whole step is skipped when no image is supplied.

Sources: agent_autopilot.py

---

Rule Lifecycle: Reinforcement & Forgetting

Two mechanisms control memory decay:

1. TTL-Based Forgetting

Each memory has an expires_at timestamp. Expired rows are:

• Filtered out of all queries (expires_at IS NULL OR expires_at > now())
• Purged by delete_expired_memories() CRUD function

2. Confidence-Based Deprecation

Which rules are applied is decided by a single LLM JSON call using api/prompts/rule_evaluation.md (lines 405–421), which returns the list of applied_rule_ids:

Sources: agent_autopilot.py

Event	Effect
Backtest confirms rule	confidence += 0.1 (capped at 1.0), validated_count += 1
Backtest contradicts rule	confidence -= 0.2, validated_count -= 1
Backtest contradicts a community rule	confidence -= 0.05, validated_count -= 1
Confidence <= 0.3 (<= 0.2 for community rules)	Rule marked deprecated (expires_at = now())
validated_count <= -2 (<= -5 for community rules)	Rule marked deprecated

Rule Promotion (Synthesis)

When 3+ insights with the same strategy type exist, the system promotes them into a permanent rule (run_rule_synthesis(), lines 285–378; it bails out when fewer than 3 insights exist, lines 294–295, and delegates duplicate-checking to the LLM prompt, which receives the existing rules at lines 332–336):

Sources: agent_autopilot.py

---

MCP Integration with AI Providers

The AI Assistant (api/ai_assistant.py) converts MCP tools into each provider's native function-calling format:

Sources: ai_assistant.py

Provider	Mechanism	Configuration
Gemini	Tools dict from _mcp_tools_to_gemini_tool() with "function_calling_config": {"mode": "ANY", ...}	Forces a function call on turn 0, JSON response on subsequent turns
OpenRouter / Qwen	OpenRouter function-calling format	Detects native tool_calls or text-fallback regex call:(?:default_api:)?search_agent_memory{...}
Fallback	Regex call:(?:default_api:)?search_agent_memory{...}	For models that don't support native function calling

The user_id is injected server-side into every MCP tool call, so the model does not need to supply it.

---

Tag Classification

Tags are assigned by an LLM call at insight save time (tag_strategy_insight()):

Sources: agent_autopilot.py

The prompt (api/prompts/tag_insight.md) instructs the LLM to:

1. Extract the strategy type from structural blocks
2. Prefer reusing existing tags for consistency
3. Assign confidence based on PnL and trade count
4. Return JSON with strategy_type, tags, outcome, confidence

If the LLM call fails, the system falls back to extracting strategy_type from block names and building tags from [symbol, strategy_type, filter types].

---

REST API

In addition to the MCP interface, memories can be managed via REST:

Sources: ai.py

The response can be filtered by visibility (community / private, lines 117–128), and bulk delete lives at DELETE /memories (lines 229–236) with single delete at DELETE /memories/{memory_id} (lines 238–255).

Community memory endpoints: PUT /api/v1/ai/memories/community-sharing toggles sharing (line 132), POST /api/v1/ai/memories/{id}/share shares a private memory (line 174), and POST /api/v1/ai/memories/deduplicate merges duplicates (line 257). Shared rows get visibility='community' and are included in MCP search only when the user's share_community_memories flag is set (api/mcp_memory_server.py:161-175); new successful insights are queued for community promotion when the user has opted in (api/agent_autopilot.py:1684).

---

Why This Wins

Judge Criteria	What We Show
Memory Agent	Three-tier hierarchy with MCP tool interface
Persistent Memory	Lives in PostgreSQL across sessions
Learning from Experience	Rules auto-extracted from 3+ similar insights
Forgetting	TTL expiry + confidence-based deprecation
Transfer Learning	Knowledge from ETH applied to BTC, marked ⚡
Memory Researcher	Fetches rules + insights for strategic context before generation
Strategy Advisor	Two-turn LLM dialogue: selects tags → generates strategy
Screenshot Vision	Separate vision analysis step on first generation pass
MCP Protocol	Standardized tool interface compatible with any MCP host