Backtesting and Optimization

Dual Backtesting Engines

Technical analysis of the Fast Vector Backtester and Event-Driven DepthSight Backtester — their architectures, trade simulation strategies, KPI calculations, and when to use each.

⏱️ 8 min read📊 Level: Intermediate

DepthSight integrates two separate backtesting engines designed to solve different phases of strategy development: the Fast Vector Backtester (for high-speed testing and genetic parameter optimization) and the Event-Driven DepthSight Backtester (for realistic live simulation with L2 order book precision).


Fast Vector Backtester

The FastVectorBacktester (bot_module/fast_vector_backtester.py, ~7,470 lines) is built using vectorized operations via pandas, numpy, and optional Numba. It processes entire arrays of historical data at once rather than step-by-step.

Rendering diagram...

Run Cycle

Sources:

Data Preparation (_prepare_data, line 4454)

The preparation phase handles:

  1. Multi-Timeframe Support: Indicators are computed on their specific timeframe (5m, 15m, 1h) and broadcast to the 1m main index via _broadcast_to_1m() (line 2958). The broadcast cache (self.broadcasted_cache, line 872) is a per-instance cache that prevents redundant recalculation of already-broadcast series during the run.

  2. Dynamic Indicator Extraction: Calls _extract_indicators_from_json() (line 4635) to recursively walk the strategy JSON and extract all required indicator names, periods, and timeframes. Both entryConditions and filters are walked:

Sources:
  1. Supported Indicator Calculations: EMA, SMA, RSI, NATR, ATR, ADX, MACD, Bollinger Bands, Stochastic — each on the appropriate timeframe DataFrame.

Signal Generation (_generate_signals, line 4751)

The signal pipeline processes conditions through three layers:

Layer 1 — Entry Condition Tree: Uses the unified condition_core evaluation engine. The strategy JSON's entryConditions tree is recursively evaluated using AND/OR logic gates, producing a boolean mask:

Sources:

Before any evaluation, _validate_causal_shifts (line 1307) recursively walks the condition tree and rejects negative historical shifts, preventing lookahead bias.

Layer 2 — Filters: The filters section evaluates independently. If any filter fails, the candle is rejected regardless of entry conditions. Tracks which specific filter node caused each rejection via _evaluate_condition_tree_with_failures().

Layer 3 — Foundation Weight System (lines 4783–4811): Each condition carries a configurable weight. A total_weight is calculated across all triggered foundations, and only when total_weight >= effective_threshold is the entry mask accepted:

Sources:

Layer 4 — Oracle Integration (line 4816): If enabled, the GMM regime signal is overlaid as an additional filter.

Condition Types Supported

The system evaluates ~30 condition types via _evaluate_condition_tree (line 4838):

CategoryConditions
Time Filterstrading_session, time_filter
Trend Filterstrend_filter, adx_filter, btc_state_filter, correlation
Volatility Filtersvolatility_filter, natr_filter, rel_vol_filter, market_activity
Entry Conditionsma_cross_condition, bollinger_bands_condition, stochastic_condition, rsi_condition, macd_condition, trend_direction, tape_condition, value_comparison, price_vs_level, volume_confirmation, classic_pattern, local_level, significant_level, level_touch_analyzer, volatility_squeeze, price_action_analyzer, price_consolidation, round_level, open_interest, return_to_level

Trade Simulation (_simulate_trades_vectorized_v2, line 5539)

Despite the "vectorized" name, this method uses a vectorized-sequential hybrid approach — signal detection is vectorized, but each individual trade is simulated sequentially with per-trade state:

Sources:

Key simulation features:

  • Entry: Next candle open + configurable slippage (default 0.05%)
  • Stop Loss: ATR-multiplier, percentage, or fixed price
  • Take Profit: R/R multiplier, ATR-multiplier, fixed price, or percentage
  • Partial Exits: Multiple sorted targets filled independently
  • Grid Management: Initializes grid orders, fills them on subsequent candles
  • DCA: Percentage, ATR-based, or custom-condition step triggers
  • Trailing Stop: Percentage-based ratcheting
  • Breakeven: Via first TP hit or by R/R threshold
  • Phantom Trade Tracking: After BE exit, simulates original TP/SL for missed opportunity analysis
  • Funding Rate PnL: 8-hour funding period tracking between executions

KPI Calculation (_calculate_kpis, line 7221)

Max drawdown is computed from the equity curve via cummax (line 7330), and the risk-adjusted ratios are clamped to a ±10 range:

Sources:

Sortino follows the same pattern with downside deviation and the same ±10 clamp (lines 7377–7391).

Returns: total_trades, total_pnl_pct, total_pnl, win_rate, max_dd, profit_factor, sharpe_ratio, sortino_ratio, consistency_score, total_commission, equity_curve, analytics_report.

ParameterDefaultDescription
initial_balance10000.0Starting capital (production default, BACKTEST_INITIAL_BALANCE)
commission_pct0.06%Per-trade commission (BACKTEST_COMMISSION_PCT)
slippage_pct0.05%Entry/exit slippage assumption (BACKTEST_SLIPPAGE_PCT)
base_timeframe"1m"Primary candle resolution

DepthSight Backtester (Event-Driven)

The DepthSightBacktester (bot_module/depthsight_backtester.py, ~5,540 lines) uses an event-driven design that mimics the live trading runtime. It processes events sequentially, updating technical state candle-by-candle and evaluating L2 order book snapshots tick-by-tick.

Architecture

Rendering diagram...

Run Cycle (run_async(), line 3516)

The main loop iterates over candles sequentially:

Sources:

Each iteration builds the pair_info_enriched market state (line 3798), slices historical data to prevent lookahead bias, fetches the L2 snapshot via get_book_snapshot_at (line 331), parses bookDepth columns into market_data_full["depth_analysis"] (lines 3914–4011), processes limit orders and open-position management (SL, TP, grid, liquidation), and finally generates signals through the Strategy instance's check_signal (lines 4732–4821) before handling entries. On completion it returns _calculate_final_kpis_for_mode().

L2 Order Book Integration

The event-driven backtester reads compressed .bin.zst order book snapshots via L2HistoricalDataReader (line 255), using LRU caching with msgpack + zstandard compression. It parses the bookDepth columns (depth_m1..m5 bids, depth_p1..p5 asks) into market_data_full["depth_analysis"] as structured bids/asks lists:

Sources:

Market impact is simulated using simulate_market_order_execution with actual order book snapshots, providing realistic slippage calculations.

ML Integration Features

The event-driven backtester supports optional ML confirmation:

  • ML Inference: Signals are scored through ModelPipeline.predict_one() / predict_proba_one() (bot_module/model_pipeline.py, lines 168/181). When ML confirmation mode is enabled, the backtester loads the confirmation model and syncs its active features into a FeatureExtractor (lines 550–568).
  • ML Training Mode: Generates training data with y_true labels via Numba-optimized _get_ml_target_label_numba() (line 87).
  • Strategy-Symbol Dynamic Risk: Tracks per-(symbol, strategy) rolling PnL, win rate, and consecutive losses to dynamically adjust risk multipliers during backtest (_update_and_adjust_strategy_symbol_performance, line 2878).

KPI Calculation

Returns additional metrics beyond the vector backtester:

  • L2 slippage tracking (entry + exit slippage in USD)
  • Average slippage per trade, average total slippage %
  • Equity curve with proper timestamps (daily resampled)
  • ML-specific metrics in ML training mode

Summary Comparison

MetricFast Vector BacktesterEvent-Driven Backtester
SpeedExtremely High (~100k candles/sec)Medium-Low (simulates sequential time)
Data ResolutionCandle OHCLVTick-by-tick & L2 Order Book
Execution EmulationSimplistic (approximate slippage)Precise (L2-accurate slippage)
L2 Book IntegrationNoYes (.bin.zst snapshots)
ML ConfirmationOracle filter onlyFull ModelPipeline inference
Funding RateSimplified 8h calculationPer-position tracking
DB PersistenceFull BacktestTrade + BacktestTradeExecution recordsFull BacktestTrade + BacktestTradeExecution records
Best Used ForGenetic optimization, initial hypothesis testingFinal risk validation, live strategy parity
Code Locationbot_module/fast_vector_backtester.pybot_module/depthsight_backtester.py

Via the REST API, engines are addressed by the identifiers vector (alias turbo) and kline (alias precision) — normalized in api/schemas.py (lines 21–26) and dispatched in tasks.py (lines 900–908).