A trading hypothesis is a testable claim connecting observable conditions with subsequent outcomes. “This shape is usually followed by a rise” is incomplete without a definition, information-availability time, measurement horizon and comparison criterion.
Define the protocol first
Before calculation, specify the instruments, period, entry and exit rules, position sizing, permitted information and execution-cost model. Choose a primary metric and a reasonable baseline, such as no trading or a simple rule without the proposed feature. Record all variants tried.
Otherwise a fortunate configuration can be selected retrospectively from many unsuccessful ones.
Split chronologically
A hypothetical design uses 2018–2021 for development, 2022 for choosing among predefined candidates and 2023–2024 for a one-time independent evaluation. These dates illustrate separate roles; they are not recommended periods.
Once the independent period has been inspected and the rules changed, it is no longer independent for that version. Fresh evaluation requires genuinely unused observations.
Randomly shuffling neighbouring market cases risks placing the same information in both sets. When outcomes span several bars, exclude training cases whose outcome intervals overlap the evaluation interval.
Walk-forward
Develop using only past data, freeze the rules, evaluate the next interval, then move the window. Define the schedule beforehand and combine evaluation windows rather than selecting only favourable ones.
Trying many walk-forward configurations against the same final history can still overfit.
Report uncertainty, not just a score
Include the result after costs, observation count, drawdown, outcome dispersion and sensitivity to execution assumptions. Dependent trades do not become independent because there are many rows.
Weak or inconclusive evidence is a valid finding. Do not redefine success after observing it.
Bailey and colleagues discuss selecting the best backtest from many alternatives. This article describes research design, not a validated strategy.