Backtest Tools

The backtest tools family contains a single tool — backtest_run — that runs a deterministic backtest over a stored StrategySpec and a date range. The engine is trusted, audited numpy code; the agent never provides executable code. Same inputs always produce identical outputs, down to the content hash.

`backtest_run`

Run a deterministic backtest over a StrategySpec and date range. Returns a run_id, the equity curve, full statistics, final portfolio weights, and a risk attestation. The run is stored as an 8-file artifact bundle (see Artifact tools).

Arguments

Argument	Type	Required	Description
`spec`	object	Yes	A `StrategySpec` dict (same shape accepted by `propose_strategy`).
`range`	object	Yes	Date range with `start` and `end` (ISO 8601 dates).

What it does

Validates the spec against the frozen StrategySpec contract (same validation as propose_strategy).
Resolves the universe to concrete symbols and fetches the required OHLCV data through the data layer, recording provenance.
Locks the environment — the numpy version, data hashes, and engine version are captured in environment_lock.json so the run is reproducible.
Runs the backtest engine: signal → weights → risk → execution, rebalanced at the spec’s horizon.
Computes statistics and a risk attestation.
Stores the full result as an 8-file artifact bundle with SHA-256 content hashes.
Returns the run_id and a summary.

Return value

Field	Type	Description
`run_id`	string (UUID)	The backtest run identifier.
`strategy_id`	string (UUID)	The strategy this run tested (if proposed first).
`equity_curve`	array	Array of `{ date, equity }` points.
`stats`	object	Full statistics block (see below).
`final_weights`	object	Final portfolio weights keyed by symbol.
`risk_attestation`	object	Risk constraints checked against the run.
`artifact_bundle`	object	Pointer to the 8-file artifact bundle.
`content_hash`	string	SHA-256 hash of the canonicalized result.

Statistics block

The stats object contains a comprehensive set of performance and risk metrics:

Metric	Type	Description
`sharpe`	float	Annualized Sharpe ratio (risk-free = 0).
`sortino`	float	Annualized Sortino ratio (downside-deviation denominator).
`cagr`	float	Compound annual growth rate.
`max_drawdown`	float	Maximum peak-to-trough drawdown (as a fraction, e.g. -0.32).
`win_rate`	float	Fraction of profitable periods.
`turnover`	float	Average daily turnover (fraction of portfolio traded per day).
`var_95`	float	Value-at-Risk at 95% confidence (daily, as a fraction).
`cvar_95`	float	Conditional Value-at-Risk (expected shortfall) at 95% confidence.

VaR and CVaR are computed from the daily return distribution using historical simulation. They are not parametric assumptions — they reflect what actually happened in the backtest window.

Example call

{
  "spec": {
    "name": "NVDA momentum (50/200 SMA cross)",
    "universe": { "type": "single", "symbol": "NVDA" },
    "signal": { "type": "sma_cross", "fast": 50, "slow": 200, "direction": "long_only" },
    "weights": { "type": "fixed_fraction", "fraction": 1.0 },
    "risk": { "max_weight": 1.0, "stop_loss_pct": null, "max_positions": 1 },
    "execution": { "slippage_bps": 5, "fee_bps": 1 },
    "horizon": "daily"
  },
  "range": { "start": "2022-01-01", "end": "2024-12-31" }
}

Example response

{
  "run_id": "run_01HQKX2J3K4M5N6P7R8S9T0V2A",
  "strategy_id": "strat_01HQKX2J3K4M5N6P7R8S9T0V1Y",
  "equity_curve": [
    { "date": "2022-01-03", "equity": 100000.00 },
    { "date": "2022-01-04", "equity": 100120.50 },
    { "date": "2024-12-31", "equity": 312450.75 }
  ],
  "stats": {
    "sharpe": 1.84,
    "sortino": 2.41,
    "cagr": 0.46,
    "max_drawdown": -0.28,
    "win_rate": 0.54,
    "turnover": 0.012,
    "var_95": -0.031,
    "cvar_95": -0.048
  },
  "final_weights": {
    "NVDA": 1.0
  },
  "risk_attestation": {
    "max_weight_constraint": 1.0,
    "max_weight_observed": 1.0,
    "max_weight_breached": false,
    "max_positions_constraint": 1,
    "max_positions_observed": 1,
    "max_positions_breached": false,
    "attested_at": "2025-01-15T14:34:55.412Z"
  },
  "artifact_bundle": {
    "bundle_id": "art_01HQKX2J3K4M5N6P7R8S9T0V2B",
    "files": 8,
    "content_hash": "sha256:b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3"
  },
  "content_hash": "sha256:c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4"
}

Determinism

Determinism is a hard guarantee, not a best-effort property. Two backtests with the same spec, range, engine version, and data hashes must produce identical output. If they don’t, it’s a bug.

Determinism is achieved through three controls:

Fixed random seed. The engine seeds numpy’s RNG with a deterministic value derived from the spec content hash and range. No call to np.random uses OS entropy.
Pinned numpy. The engine runs against a pinned numpy version recorded in environment_lock.json. Floating-point behavior is stable within a numpy major.minor release.
Fixed data window. The OHLCV data is fetched once, hashed, and locked. The data_manifest.json artifact records every symbol, date, and content hash used. Re-running with the same manifest guarantees identical inputs.

The result is that content_hash is a function of (spec, range, engine_version, data_hashes). If all four match, the hash matches. This lets you verify reproducibility across runs, machines, and time.

What can break determinism?

Different data. If the upstream provider restates a bar (e.g. a split adjustment), the data hash changes and the output changes. This is correct behavior — the data_manifest.json makes it visible.
Different numpy version. Floating-point summation order can change between numpy versions. The environment_lock.json pins the version so this doesn’t happen silently.
Different engine version. A new engine release may change the order of operations. The engine version is part of the hash input, so a version change produces a different hash by design.

The risk attestation

The risk_attestation object records whether the run respected every constraint in the spec’s risk block. It is computed after the backtest, against the realized path — not just the final weights. This catches intra-run breaches (e.g. a position that briefly exceeded max_weight before a rebalance). If any constraint was breached, the corresponding *_breached flag is true and the run is flagged in the web UI. The agent should report breaches to the human before requesting paper-trading promotion.

Strategy tools — propose a spec before backtesting
Artifact tools — the 8-file bundle produced by every run
StrategySpec reference — the full schema
Approvals — promote a backtested strategy to paper trading

​backtest_run

​Arguments

​What it does

​Return value

​Statistics block

​Example call

​Example response

​Determinism

​The risk attestation

​Related

`backtest_run`

Arguments

What it does

Return value

Statistics block

Example call

Example response

Determinism

The risk attestation

Related