Methodology
Mechaniq sells evidence, not aspiration. This page describes how every signal is tested before it ships, and what we deliberately don't claim. If a strategy can't survive the tests below, it doesn't ship.
1. The pipeline — six stages, one shape
Every setup at Mechaniq runs the same gauntlet of five gates: Idea → Quick-screen → Walk-forward validation → Trade simulation → Live. The folder a setup lives in tells you which stage it is in. Movement between stages is blocked by a pre-registered gate; nothing skips ahead. Customers see this happen: setups in trade simulation appear on /lab as Currently Testing, promoted setups move to /live, and setups that miss a gate are documented on /lab as Failed. The pipeline is the product.
lab/postmortem/ → /lab2. Pre-registration — the integrity claim
Before a setup enters a stage, the pass/fail criteria for that stage are written down and frozen. Pre-registration means: if you adjust the criterion after seeing the result, that is not the same setup any more — it gets a new slug and a new pre-registration. The mechanic is mundane; the discipline is not. Almost every retail-trading product reports backtest numbers without admitting that the thresholds were chosen after the fact. Our gates live in lab/hypotheses/gates.md with a timestamp. If you want to verify, the repo history shows when each gate was set relative to when each test ran.
3. Stage 0–1 — Idea and quick-screen
Every setup starts as a written idea with an explicit "because". If the rationale cannot be articulated in a paragraph — what economic, structural, or behavioural reason makes the relationship plausible — the idea isn't ready. Stage 1 is a cheap in-sample sanity check (Sharpe, hit rate, sample size against pre-registered thresholds). The point is not to validate; the point is to cut weak ideas before they consume serious work. Most ideas don't survive this stage. That is the system functioning, not failing.
4. Stage 2 — Walk-forward validation
We split the data into chronological folds. Parameters are tuned on fold N and measured on fold N+1 — which the tuning process has never seen. If a signal looks great on the training fold and falls apart on the next, it was overfit. The discipline is simple: parameters are not allowed to see the future. The published walk-forward number is what survives this process across every fold, not the best fold.
5. Stage 2 — Attribution and parameter sensitivity
A composite score that aggregates many features can hide which one actually carries the edge. We compute Spearman ρ between each feature's score and the realised forward return (5/10/30/60-day windows). Features with |ρ| < 0.05 are noise and get cut. The same Spearman ρ is split by SPY regime — a feature that predicts in up-markets but flips sign in down-markets is a red flag, not a robust signal. We also perturb each key parameter by ±10–20%: knife-edge sensitivity (one specific value works, neighbours don't) is treated as overfitting, not edge.
6. Stage 3 — Trade simulation (8-12 weeks)
A strategy that passes walk-forward goes into trade simulation for at least eight weeks in production conditions: same data feeds, same scheduling, same universe — but no alerts are sent to customers. Outcomes are logged and compared against the walk-forward expectation. If the simulated Sharpe stays within tolerance of the walk-forward Sharpe and the signal frequency matches what was expected, the strategy can promote to live. If it degrades materially, it does not. This stage is the buffer between "the test said it works" and "customers receive alerts from it".
7. Runtime regime gate — knowing when to keep quiet
Markets behave differently in uptrends than in downtrends. A breakout strategy that works above SPY's 200-day moving average can fail badly below it. The regime gate is a simple, hard rule: when SPY is below its 200-day SMA, high-conviction alerts (Imminent and Breakout) are suppressed. Watchlist signals continue (the early-radar function) but no triggers fire in a defensive regime. The rule is mechanical, not discretionary. Per-strategy allowlists are derived from walk-forward per-regime Sharpe.
8. The data — survivorship-free, honest about its limits
The tests run on end-of-day bars for the entire US market, survivorship-free: delisted and merged names stay in, so a backtest can't quietly drop its own losers — going back two decades. Where a study needs it we add intraday minute data; where it needs company events, it reads primary filings straight from the SEC — insider transactions, buyback announcements, point-in-time share counts. Options are the hard case: reliable historical option prices are difficult to license honestly, so options strategies are validated forward, in paper, against a live options surface from our read-only broker link — never a fragile daily-close proxy. We name our anchor vendors where it builds trust — prices from Tiingo, filings from SEC EDGAR — and keep the rest of the supply chain to ourselves; it isn't the edge. What matters for you is the property, not the logo: survivorship-free, point-in-time, and never marked to a number we couldn't reproduce.
9. Execution costs — measured where we can, honest about the rest
We used to charge a flat friction and disclaim the spread; the options work has moved past that. For the options book — 0DTE especially — we now charge the actual bid-ask spread rather than a flat rate: the forward capture records every leg's live bid/ask and pays the realized crossing cost at entry and at the forced 15:55 exit, with the go/no-go using the conservative 75th-percentile measured spread and no assumed price improvement. Paper trades fill at the real quoted bid/ask plus tiered commission; where historical option quotes simply don't exist, we fall back to a tiered half-spread model rather than pretend spreads are free. Equity and cross-sectional strategies still carry a fixed, deliberately conservative round-trip friction — set per strategy, not one universal number. What we still do not model: dividends, order-routing latency, and slippage beyond these assumptions; early assignment we avoid structurally — the QQQ sleeve exits before the cash close — rather than price it. A backtest is still an estimate, not a guarantee: real execution will differ. We say this plainly because it is true, and because pretending otherwise is what every other retail product does.
10. The fail record — strategies that didn't make it through
Every strategy that fails a gate is documented on /lab: which stage it failed at, which gate it missed, what the actual numbers were, what we learned. Strategies that ship without facing a gate are strategies whose authors had no integrity test in place. The page shows a running counter — strategies tested, failed, fail rate — and a numbered archive. Past entries are not edited. A high fail rate is the point, not a flaw: it is the evidence that the gates are real.
11. The live record
Every alert sent from Mechaniq's launch day onward is logged with its forward-return outcome (5/10/30/60-day) at /breakout/history. Updates weekly on Sunday 06:30 CET. The track record builds in public, with no editing of past entries. We could show ten years of in-sample backtests — everyone does — but in-sample numbers prove nothing. The honest answer is to start the public clock at zero.