Walk-forward backtest: how not to fool yourself

Every quantitative strategy works in the past. That's the single most important point in this entire post: if you torture historical data hard enough, you'll always find a rule that would have delivered exceptional returns. That's overfitting, and almost every retail "quant product" sold in Brazil suffers from it.

The antidote is called walk-forward backtest without look-ahead bias. It's a protocol, not a single technique, and it separates serious backtesting from fantasy. This post explains how it works under the hood, why it matters so much, and how VORTEX QSP implements the protocol.

The problem: why naive backtesting lies

Imagine you want to test the rule "buy the 10 stocks with highest dividend yield on B3, annual rebalance". You download data from 2010 to 2024, calculate, discover it returned +14% p.a. versus +8% for the IBOVESPA. Conclusion: the strategy works, right?

Wrong, for at least three reasons.

1. Look-ahead bias

You used the "universe of B3 stocks" you knew existed in 2024. But in 2010, several of those stocks weren't even listed or had insufficient liquidity. Worse: you knew which stocks survived. That's survivorship bias — you retroactively excluded the failures. The actual result, without that bias, would be significantly worse.

2. Parameter overfitting

Why did you choose "the 10 with highest yield"? Why not 15, or 20, or just the top 5? Why annual rebalance instead of semi-annual? If you tested various combinations and picked the best one, you did data dredging. The rule that wins in-sample almost always is the one that fit the noise of that specific period most closely — and almost never survives outside it.

3. Look-ahead in fundamental data

Q4 2020 earnings were only published in March 2021. If your backtest "rebalances in January 2021 based on Q4'20 earnings", you're using information nobody had. This type of error is absurdly common in poorly constructed backtests.

The solution: walk-forward

Walk-forward backtest reproduces exactly what you would have been able to do in real time. The rule is simple:

At each rebalance, you only use information that was publicly available before that date — and the selection rule is fixed in advance, not chosen based on the result.

In practice this means:

• Dynamic universe. At each rebalance, the universe is "stocks that had liquidity X months prior at that moment", not "stocks we know today were liquid in hindsight".
• As-of-date data. Earnings only enter after their actual publication date. Prices are from the day before rebalancing — not the day after.
• Frozen parameters. You fix all parameters (momentum lookback, number of stocks, factor weights, constraints) before running. Don't change them afterward.
• Realistic costs. Each rebalance debits brokerage fees + bid-ask spread + market impact. At VORTEX QSP we use 12 bps per side, a conservative value for investors using premium brokers on B3.

VORTEX QSP in walk-forward

The backtest published in Performance covers 7.3 years — January 2019 to May 2026 — with monthly rebalances, totaling 89 reblancs. At each one:

1. Determines the universe: IBrX-100 stocks with minimum liquidity in the 6 months prior to rebalance (not after!).
2. Calculates the 5 factors (momentum, low volatility, quality, value, low beta) using exclusively data available up to the rebalance date. Earnings and fundamentals enter only after official publication date.
3. Normalizes cross-section (z-score per factor) and calculates composite score with fixed weights defined in January 2019.
4. Applies hysteresis band (top-15 enters, exits only when falling outside top-25) to control turnover.
5. Applies inverse-variance weighting and sector restrictions.
6. Executes rebalance, debits transaction costs (12 bps/side).
7. Accumulates returns until next month. Repeat.

What walk-forward doesn't solve

Honesty demands acknowledging the method's limits.

Single period

7.3 years sounds like a lot, but in terms of market regimes it's a small sample. 2019-2026 includes pandemic, interest rate shock, polarized elections — but doesn't include hyperinflation, severe currency crisis, or prolonged zero-rate environment. The next cycle may bring regimes the backtest never saw.

Structural market change

The B3 of 2019 is different from the B3 of 2026 — more retail investors, more high-frequency robots, more listed options. Backtest assumes price discovery dynamics are stationary. They may not be.

Unobserved tail risk

VORTEX QSP's Max Drawdown of -33.2% in the backtest happened in 2020-2022 — pandemic + 13% Selic. A more acute scenario could produce larger DDs than that. No backtest, however rigorous, guarantees the magnitude of the worst future scenario.

How to read a backtest others publish

Every time you see a number like "+30% p.a.", ask these questions before believing it:

• Universe: stocks by which liquidity/listing criteria, and is the universe defined dynamically?
• Period: what's the full window? Does it include periods where the strategy would likely suffer (2008, 2020)?
• Costs: were transaction costs debited? How much?
• Parameters: who chose the parameters, and when were they frozen? If the backtest shows retrospective parameter optimization, ignore the result.
• Survival: does the universe include stocks that failed or were delisted? If not, it overstates returns.
• Drawdown disclosure: what's the worst consecutive period? If the presentation only shows cumulative return without decomposing the path, be suspicious.

VORTEX QSP's backtest explicitly answers all these questions in Performance, including the month-by-month table without cherry-picking — good months and bad months appear the same way.

To close

Backtest without walk-forward is an exercise in fiction, and the Brazilian quant industry is full of it. When a product promises you "+40% p.a." without showing methodology, without listing all parameters frozen before the test date, without breaking down month by month — assume the number is optimized retrospectively, not replicable.

Walk-forward doesn't guarantee future performance. It only guarantees that the historical result is honest. And honesty in backtesting is a prerequisite for any serious conversation about risk and return in a systematized product.