Methodology
Mechaniq sells evidence, not aspiration. This page describes how every signal is tested before it ships, and what we deliberately don't claim. If a strategy can't survive the tests below, it doesn't ship.
1. The pipeline — six stages, one shape
Every setup at Mechaniq runs the same gauntlet of five gates: Idea → Quick-screen → Walk-forward validation → Trade simulation → Live. The folder a setup lives in tells you which stage it is in. Movement between stages is blocked by a pre-registered gate; nothing skips ahead. Customers see this happen: setups in trade simulation appear on /lab as Currently Testing, promoted setups move to /live, and setups that miss a gate are documented on /lab as Failed. The pipeline is the product.
lab/postmortem/ → /lab2. Pre-registration — the integrity claim
Before a setup enters a stage, the pass/fail criteria for that stage are written down and frozen. Pre-registration means: if you adjust the criterion after seeing the result, that is not the same setup any more — it gets a new slug and a new pre-registration. The mechanic is mundane; the discipline is not. Almost every retail-trading product reports backtest numbers without admitting that the thresholds were chosen after the fact. Our gates live in lab/hypotheses/gates.md with a timestamp. If you want to verify, the repo history shows when each gate was set relative to when each test ran.
3. Stage 0–1 — Idea and quick-screen
Every setup starts as a written idea with an explicit "because". If the rationale cannot be articulated in a paragraph — what economic, structural, or behavioural reason makes the relationship plausible — the idea isn't ready. Stage 1 is a cheap in-sample sanity check (Sharpe, hit rate, sample size against pre-registered thresholds). The point is not to validate; the point is to cut weak ideas before they consume serious work. Most ideas don't survive this stage. That is the system functioning, not failing.
4. Stage 2 — Walk-forward validation
We split the data into chronological folds. Parameters are tuned on fold N and measured on fold N+1 — which the tuning process has never seen. If a signal looks great on the training fold and falls apart on the next, it was overfit. The discipline is simple: parameters are not allowed to see the future. The published walk-forward number is what survives this process across every fold, not the best fold.
5. Stage 2 — Attribution and parameter sensitivity
A composite score that aggregates many features can hide which one actually carries the edge. We compute Spearman ρ between each feature's score and the realised forward return (5/10/30/60-day windows). Features with |ρ| < 0.05 are noise and get cut. The same Spearman ρ is split by SPY regime — a feature that predicts in up-markets but flips sign in down-markets is a red flag, not a robust signal. We also perturb each key parameter by ±10–20%: knife-edge sensitivity (one specific value works, neighbours don't) is treated as overfitting, not edge.
6. Stage 3 — Trade simulation (8-12 weeks)
A strategy that passes walk-forward goes into trade simulation for at least eight weeks in production conditions: same data feeds, same scheduling, same universe — but no alerts are sent to customers. Outcomes are logged and compared against the walk-forward expectation. If the simulated Sharpe stays within tolerance of the walk-forward Sharpe and the signal frequency matches what was expected, the strategy can promote to live. If it degrades materially, it does not. This stage is the buffer between "the test said it works" and "customers receive alerts from it".
7. Runtime regime gate — knowing when to keep quiet
Markets behave differently in uptrends than in downtrends. A breakout strategy that works above SPY's 200-day moving average can fail badly below it. The regime gate is a simple, hard rule: when SPY is below its 200-day SMA, high-conviction alerts (Imminent and Breakout) are suppressed. Watchlist signals continue (the early-radar function) but no triggers fire in a defensive regime. The rule is mechanical, not discretionary. Per-strategy allowlists are derived from walk-forward per-regime Sharpe.
8. What we deliberately do not claim
We do not model bid-ask spread, slippage beyond a fixed friction assumption, dividends, early option exercise, or order-routing delays. Our friction assumption (0.5% per trade) is conservative for liquid US stocks but not universal. A backtest is an estimate, not a guarantee. Real execution will differ. We say this plainly because it is true — and because pretending otherwise is what every other retail product does.
9. The fail record — strategies that didn't make it through
Every strategy that fails a gate is documented on /lab: which stage it failed at, which gate it missed, what the actual numbers were, what we learned. Strategies that ship without facing a gate are strategies whose authors had no integrity test in place. The page shows a running counter — strategies tested, failed, fail rate — and a numbered archive. Past entries are not edited. A high fail rate is the point, not a flaw: it is the evidence that the gates are real.
10. The live record
Every alert sent from Mechaniq's launch day onward is logged with its forward-return outcome (5/10/30/60-day) at /breakout/history. Updates weekly on Sunday 06:30 CET. The track record builds in public, with no editing of past entries. We could show ten years of in-sample backtests — everyone does — but in-sample numbers prove nothing. The honest answer is to start the public clock at zero.