Methodology
Methodology: keeping simulations honest
The ground rules MAA research is designed around: point-in-time data, stated assumptions, out-of-sample validation, and honest labels for simulation versus live.
Published · 5 min read
Research
Research notes and methodology.
Written in the open so the process can be criticized. Articles are static, dated, and updated in place with a visible revision date.
Notes and methodology
Short, citable write-ups. Each shows its publication and revision dates.
-
-
Research note
Why agents need a deterministic policy layer
Language-model output is untrusted input. A design note on separating what models may suggest from what code is allowed to approve.
Published · 4 min read
-
Research note
Research note: five ways a backtest can mislead
Look-ahead, survivorship, data snooping, cost errors, and model leakage — the failure modes MAA's methodology is designed to guard against.
Published · 5 min read
Methodology principles
The principles the research process is being designed around. They are commitments about process, not claims about results.
Point-in-time data only
An experiment may only use information that would have been available at the simulated moment — including for language models, whose training data can leak future knowledge.
Assumptions before results
Cost, slippage, and liquidity assumptions are written down before an experiment runs, not tuned afterward.
Out-of-sample by default
Hold-out and walk-forward validation are the default. In-sample fit is never presented as evidence.
Count every attempt
Every experiment is registered, including failures, so the effect of repeated testing can be accounted for.
Reproducible runs
Configuration, data version, code version, and model version are recorded so a run can be repeated.
Honest labels
Each result is labeled by what it is — historical simulation, paper trading, or live — and these are never blended.
How results will be labeled
Three categories are kept strictly separate in all MAA materials.
| Label | What it means | Current status |
|---|---|---|
| Historical simulation | Run on past data with stated assumptions. Subject to hindsight bias, overfitting, and cost-model error. It does not show what would have happened in real trading. | Not yet published |
| Paper trading | Forward simulation against live data feeds with no real capital. Exposes timing and data issues that backtests hide, but still omits real execution. | Planned |
| Live results | Real capital deployed in real markets. | Not applicable. MAA does not trade live. |