08Validation
Built to break your strategy.
A strategy earns trust by surviving attempts to disprove it. Evrant tests a frozen configuration on data it has never seen.
- 62% of expectancy keptPass
Out-of-sample
Validation range, 2022-01 to 2024-06
- 6 of 8 profitablePass
Walk-forward
8 rolling windows
- Drawdown within limitPass
Monte Carlo drawdown
2,000 bootstrap paths
- 9% chance of lossPass
Monte Carlo loss
Over 100 trades
- Expectancy −31%Pass
+1 tick slippage
Per side
- Fragile, −52%Warning
1-bar entry delay
Robustness
- Waiting for youSealed
Sealed range
2024-07 onwards
Illustrative run, not a performance claim.
Decide what the strategy never sees.
You set the boundaries between development, validation and sealed data. The engine enforces them on every job.
- One calendar boundary applies across markets
- A one-week embargo keeps straddling trades out
- Checked by the app, then again by the data reader
- Frozen at the first validation job
Common coverage · sample markets · 1-minute
2016-09 → 2026-07
2016-09 → 2021-12
- Bars
- –
- Trades
- –
2022-01 → 2024-06
- Bars
- –
- Trades
- –
2024-07 →
- Bars
- –
- Trades
- –
Off. Validation runs to the end of coverage.
Freezes on the first validation job. Edit freely until then. Later changes create a new plan version, and evidence stays attached to the plan it was produced under.
Gates exist only to protect the evidence.
Move in any direction. These rules exist because breaking them corrupts evidence, not because the workflow needs an order.
- Validate only what is frozen
Validation jobs run against a frozen configuration and execution profile.
A result you cannot pin to one build, parameter set and fill model is not evidence.
- Ranges stay apart
Development jobs cannot read validation or sealed bars. Validation jobs cannot read sealed bars.
Once development code has read the test data, the test measures memory, not edge.
- Sealed data needs a person
Sealed-range evaluation takes an explicit human action and is recorded as an exposure event.
The last unseen data can be spent once. Someone should decide to spend it.
- Candidates are earned
A candidate needs a completed suite on that exact configuration and your recorded decision.
A candidate is a claim about one configuration, so its evidence must be about that one.
- Change it, test it again
Any change to logic, parameters, model or features creates a new configuration to validate.
After a tweak, the old results describe a strategy that no longer exists.
- Paper trading, coming later
When it arrives, paper trading will run only a candidate exactly as it was validated.
Same build, parameters, models and features, or it is a different strategy.
Validation exposures
0
Sealed exposures
0
- clean
base-orb v1
- exposure 1
Out-of-sample run
- exposure 2
Walk-forward · 8 windows
- validation_informed · exposures 2
base-orb v2
- exposure 3
Default suite on v2
- exposure event
Sealed evaluation, by you
Sealed range · 2024-07 → locked
agent → request_job(range: "sealed") denied
Every look at test data is counted.
A strategy changed after seeing validation results is no longer tested out of sample. Evrant records it instead of pretending.
- Every validation job adds an exposure
- Later versions are marked validation_informed
- Sealed data opens once, for you, after two confirmations
- The agent can never request the sealed range
A feature that peeks ahead is rejected.
Leakage rarely looks like cheating. It looks like a centred smoothing window. Switch the definition and move the cut point.
- Every value carries an available_at time
- Strategies see only completed bars
- New feature versions must pass truncation invariance
- ML splits are purged and embargoed
Four tests that try to disprove it.
A versioned suite of tests with pass criteria you set. It runs on one frozen configuration and stays attached to it.
Your pass criteria
Out-of-sample degradation
The frozen configuration runs on the validation range. Evrant compares it with in-sample for expectancy, trade frequency and distribution, drawdown and risk-adjusted return, and asks whether each one persists.
- expectancy persistsCI excludes 0
- trade distributionKS p = 0.31
- OOS / IS expectancy62%
- criterion
Expectancy
62% of IS
Trade rate
91% of IS
Sharpe
63% of IS
Max drawdown
27% deeper
Walk-forward, window by window
Optimise on a training window, freeze the choice by the robust-region rule, test on the next unseen window, roll forward. Each window is stored with its own optimisation, parameters and out-of-sample run. The sealed range is never used.
- windows8 rolling · 30-mo train · 8-mo test
- profitable OOS windows6 / 8 · 75%
- WF efficiency0.58
- criterion
Windows over 2016-09 to 2024-06. The sealed range is never used.
Monte Carlo distributions, not one curve
Five seeded, versioned methods. Resampling methods reuse the stored trade ledger; perturbation methods run real backtests. The result is a distribution of outcomes.
- Trade-order shufflingstored ledger
- Bootstrap resamplingstored ledger
- Execution perturbationreal backtests
- Parameter perturbationreal backtests
- Return perturbationstored ledger
- median after 100 trades17.3R
- drawdown 95th / 99th−18.2R / −23.4R
- P(loss) over 100 trades8.9%
- criterion
2,000 bootstrap paths, seed 7, 36 drawn.
Stress it until it breaks
Raise costs, add slippage, delay entries. Each test reports the change against the unstressed run and the stress level at which profitability disappears. If a small stress removes profitability or halves expectancy, Evrant flags the configuration fragile automatically.
Expectancy per trade
fragile0.20R unstressed
Unstressed run. Move a slider.
- profit disappears at+2.4 ticks / side
- profit disappears at1.9 bars delay
- small-stress thresholds1 tick · 1 bar · $1
Illustrative market, $300 average risk per trade.
The last word is yours.
The top trial is never promoted automatically. The dossier shows every piece of evidence, including what argues against it.
- Configuration
- base-orb · bed514
- Suite · default v1
- 5 pass, 1 warning
- Walk-forward
- 6 / 8 windows · WFE 0.58
- Monte Carlo DD95
- −18.2R · within 20R
- Robustness
- fragile · 1-bar delay
- Exposure
- validation 1 · sealed 0
- Lineage
- clean · first validation
- Configurations tried
- 90
Awaiting your decision. Evrant has reported; it has not decided.
Early access
Bring a strategy you want to break.
Join free for an early-access invitation. No account or payment required. We’ll confirm market coverage before you choose a plan.
You’re on the list. We’ll email you when your invitation is ready.
For your early-access invitation only. Request removal anytime at [email protected]. See our privacy notice.