ML Pipeline & Oracle
Detailed analysis of the Gaussian Mixture Model (GMM) market regime classification, three-sensor feature engineering, live prediction with caching, and regime-aware trading exits.
The AI Oracle (bot_module/oracle.py, ~132 lines) is a machine learning engine that classifies the current market phase into distinct "regimes" (e.g., Calm Range, Volatile Trend, Extreme Shock). Strategies use this state to gate entries (oracle filter block), filter tradable symbols (DYNAMIC_ORACLE mode), and react to regime changes (breakeven / loss-cut exits).
Beyond the offline GMM, the platform ships River-based online learning (bot_module/model_pipeline.py, bot_module/ml_strategy.py), a Compass strategy that consumes Oracle regimes (bot_module/compass_strategy.py), and Celery jobs tasks.generate_dataset / tasks.train_model (tasks.py) for dataset generation and model training.
GMM Market Regime Classification
DepthSight uses an unsupervised Gaussian Mixture Model (GMM) from scikit-learn to cluster market states. Unlike raw price action, GMM analyzes the underlying statistical distributions of the market, identifying hidden structural patterns.
Model Loading
The Oracle loads a pre-trained GMM model once at instantiation (lines 14โ23):
Sources:The model is trained offline (the training script train_oracle.py is referenced in a comment at bot_module/oracle.py:28 but is not part of this repository) and distributed as a .joblib file. No online training occurs in the Oracle โ the model is purely for inference.
The Three Sensors
The GMM model does not look at prices directly. It is fed three custom features (called "sensors") computed by engineer_features() (lines 25โ81). These sensors capture different dimensions of market behavior:
Sensor 1: Memory (Market Memory / Forgetting Speed)
Measures the ratio of short-term to long-term volatility:
Sources:| Value | Interpretation |
|---|---|
< 1.0 | Current volatility is below long-term average (calm) |
| ~ 1.0 | Normal market conditions |
| > 1.0 | Volatility spike โ transition to high-panic regime |
Purpose: Detects volatility regime changes. A sudden jump from 0.8 to 2.5 signals a market panic event (flash crash, news reaction).
Sensor 2: News Background (News Sentiment Asymmetry)
Averages sentiment signals over a 12-hour window (720 candles on 1m timeframe):
Sources:| Value | Interpretation |
|---|---|
| > 0.0 | Net positive sentiment (bullish bias) |
| ~ 0.0 | Neutral / mixed sentiment |
< 0.0 | Net negative sentiment (bearish bias) |
Purpose: Captures macro-sentiment shifts that often precede sustained trends. A news-aware strategy can avoid buying during negative sentiment spikes.
If sentiment columns are not available (e.g., the data source doesn't provide them), the sensor falls back to 0.0 gracefully.
Sensor 3: Complexity (Complexity Drift / Normalized ATR)
Measures normalized ATR โ how much price drift or noise is occurring relative to the absolute price level:
Sources:| Value | Interpretation |
|---|---|
| Low (0.001โ0.005) | Tight range, low noise (ideal for scalping) |
| Medium (0.005โ0.02) | Normal market drift |
| High (> 0.02) | High noise, erratic movements (tighten SL) |
Purpose: Evaluates market "friction" โ high complexity regimes warrant wider stop-losses to avoid noise-triggered exits, while low complexity regimes allow tighter risk management.
Feature Engineering Output
The three sensors are combined into a feature DataFrame:
Sources:The explicit .copy() detaches the feature frame from the source DataFrame, and infinite / NaN values are sanitized to ensure the GMM model always receives valid input.
Live Prediction & Caching
The Oracle evaluates regimes at the end of each closed candle via get_current_regime() (lines 83โ132):
Caching Strategy
| Component | Behavior |
|---|---|
| Cache Key | _last_kline_timestamp โ the index of the most recent candle |
| Cache Hit | Returns cached (regime, confidence) tuple immediately |
| Cache Miss | Re-runs feature engineering + GMM inference, updates cache |
| Empty Data | Returns (-1, 0.0) โ signals "unknown regime" |
| NaN Guard | If features produce NaN after sanitization, returns (-1, 0.0) |
This optimization ensures the CPU-intensive GMM inference runs at most once per candle, even when hundreds of strategies query the Oracle simultaneously.
Regime-Aware Strategy Execution
How Strategies Consume Oracle Output
The Oracle's output (regime_index, confidence_pct) is stored in StrategyConfig at the database level:
| Column | Type | Description |
|---|---|---|
oracle_regime | Integer (nullable) | Current GMM regime index (0 to N-1) |
oracle_confidence | Float (nullable) | Confidence percentage of the prediction |
Strategies can reference the Oracle state through two mechanisms:
- Oracle Filter Block: A visual builder block that gates entry conditions โ trades are only allowed when
oracle_regimeequals the required regime andoracle_confidencemeets the configured minimum (bot_module/strategy.py:4670-4702). - Regime-Change Reaction: when the
breakeven_on_regime_changeoption is enabled,Strategy.check_on_screener_update()(bot_module/strategy.py:3230-3314) compares the position's entry regime with the screener's currentoracle_regime: winning positions get their stop moved to breakeven, and losing positions are closed byTradingController.
Regime Change Exit
When the breakeven_on_regime_change option is enabled, a regime transition triggers a per-position reaction:
Regime flip detected on screener update (breakeven_on_regime_change = true)
โ
Winning trade โ stop moved to breakeven (MOVE_SL)
Losing trade โ TradingController.close_position(reason="REGIME_CHANGE_LOSS_CUT")
This is a per-position safeguard, not a blanket liquidation: when the regime flips under a losing trade, that position is cut; winning positions are protected by a breakeven stop.
Model File Management
| Aspect | Implementation |
|---|---|
| Training | Offline (training script not included in this repository; referenced from bot_module/oracle.py:28) |
| Serialization | joblib.dump() / joblib.load() |
| Storage | data/oracle_model.joblib (ORACLE_MODEL_PATH, bot_module/config.py:890) |
| Versioning | File-based โ replace the .joblib file and restart |
| Error Handling | FileNotFoundError raised if model path does not exist |
The model is pre-loaded during the FastAPI lifespan to eliminate cold-start latency on the first prediction request.
AI Co-Pilot Assistant
Technical architecture of the generative AI strategy assistant โ prompt hydration with live market context, multimodal image ingestion, multi-provider abstraction (Google Gemini, OpenAI, Qwen, OpenRouter, Vertex Agent Builder), response validation, and security filtering.
Intelligent Agent Memory System
MCP-based memory for the Autopilot and Co-Pilot LLM agents โ three-tier hierarchy, sub-agent critic review, automatic rule synthesis, confidence-based forgetting, and screenshot-enhanced strategy generation.