← Quantery Blog

Does a complex stock thesis add value?

October 9, 2026 · 8 min readthesis-buildingbacktestingoverfitting

A complex stock thesis adds value only when its extra rules improve the evidence beyond a simpler version of the same idea. Test that directly: freeze the universe and backtest settings, run a one-claim baseline, add one criterion family at a time, then remove each family from the finished thesis. If the apparent edge doesn't survive those comparisons, the extra logic hasn't earned its place.

Complexity isn't the number of lines in the file. It's the number of independent choices that can change which companies qualify. Features and thresholds count. So do gates, score weights, lookback windows, and exceptions. Each choice can express a real belief. Each is also another chance to fit the history you already saw.

What is the simplest fair baseline?

The baseline should preserve the thesis's main economic claim while discarding supporting claims. If your idea is “cheap businesses with dependable cash generation,” a free-cash-flow-yield rule is a fair baseline. An all-market index isn't. That index is the alternative portfolio, useful as a benchmark, but it doesn't tell you whether the debt rule added anything to the value rule.

Write the main claim in one sentence. Pick the smallest observable proxy that can represent it. Keep the universe and filing-quality requirements fixed. Freeze the rebalance cadence, holding rule, test dates, and benchmark too. Now the baseline and the full thesis differ in the rules you want to judge.

A baseline needn't be crude. It still needs null guards and a defensible denominator. It can use a graded score to avoid one cliff. Simple means that only the main claim gets a vote.

This builds on the distinction in how to build a thesis worth testing: selection rules describe companies now, while subsequent returns grade the prediction. The baseline is the smallest set of selection rules that still makes your prediction intelligible.

Don't begin with whichever stripped-down version looked best in an earlier run. That is already a selected result. Declare the baseline before comparing curves, and keep the versions that lose.

Add rule families one at a time

A criterion family is a group of rules meant to capture one business claim. Profitability ratios might form an operating-quality family. Free cash flow relative to net income might form cash backing. Net debt relative to operating output might form balance-sheet resilience. A valuation yield might stand alone as price.

Start with the baseline, then make a sequence:

1. Valuation only.
2. Valuation plus operating quality.
3. Add cash backing.
4. Add balance-sheet resilience.

These labels aren't a recommended formula. They're a test order. Save a separate thesis version after each addition and run every version with identical backtest settings.

Record what each step changes before reading performance. How many qualification events disappear? Which sectors and size ranges lose representation? Does the average survivor count shrink because a volatile quarterly feature now controls the gate? An added rule can improve a headline result by turning a broad thesis into a narrow industry or size bet.

Judge several outputs together. Compare hypothetical excess return against the same benchmark. Read median beside mean event outcomes. Check maximum drawdown, the event-study count, and hit rate. Read the average universe and survivor counts too. Every result excludes trading costs. If a new rule lifts the mean while the median is flat and the qualifying evidence contracts sharply, inspect the events before crediting the rule.

The sequence tests incremental value: what changed when one business claim entered the thesis. It doesn't prove the added claim caused the change. It gives you a controlled comparison worth investigating.

Remove each rule from the finished thesis

Forward addition has a weakness. The value of a rule can depend on what is already present. A leverage test might add little to a broad valuation baseline but matter once the thesis has selected capital-intensive companies. So run the comparison backward too.

Take the finished thesis and remove one criterion family at a time. This is an ablation test. Keep every other rule and every backtest setting fixed. If removing cash backing barely changes qualifications or outcomes, it may be useful context with no need for a vote. Keep it as a displayed feature if it helps research. A prudent-sounding ratio hasn't earned points by sounding prudent.

If removing one family wrecks the result, slow down. That rule may contain the thesis's useful information. It may instead be a disguised sector filter. Data availability or a threshold fitted to one period can have the same effect. Inspect the added and dropped companies. Compare sector mix and company size. Then check dates and missing-data flags. Move that family's thresholds by round amounts and repeat the test.

This is where avoiding duplicate signals matters. Return on equity and net margin can both respond to the same profit change. Free-cash-flow yield and cash conversion both reuse free cash flow. A full thesis may look diversified while one accounting input gets several votes. Remove families, then remove overlapping criteria inside the family. The score should lose a distinct claim each time.

Write the comparison so one change stays visible

The shape below is illustrative. The bundled templates are the reference for exact fields, while the Quantery thesis documentation covers the workflow around the DSL.

# Cash-generative value with optional support: illustrative
params:
  yield_floor: 0.05
  roe_floor: 0.12
  conversion_floor: 0.80
  leverage_max: 2.0

features:
  ni_ttm: ttm(net_income)
  fcf_ttm: ttm(free_cash_flow)
  equity_now: newest(total_equity)
  debt_now: newest(total_debt)
  cash_now: newest(cash)
  ebitda_ttm: ttm(ebitda)
  fcf_yield: if(market_cap > 0, fcf_ttm / market_cap, null)
  roe: if(equity_now > 0, ni_ttm / equity_now, null)
  conversion: if(ni_ttm > 0, fcf_ttm / ni_ttm, null)
  net_debt_ebitda: if(ebitda_ttm > 0,
    (debt_now - cash_now) / ebitda_ttm, null)

criteria:
  valuation:
    rules:
      - { when: "is_null(fcf_yield)", score: 0, flag: no_yield }
      - { when: "fcf_yield >= $yield_floor", score: 1 }
      - { else: 0 }
  operating_quality:
    rules:
      - { when: "is_null(roe)", score: 0, flag: no_roe }
      - { when: "roe >= $roe_floor", score: 1 }
      - { else: 0 }
  cash_backing:
    rules:
      - { when: "is_null(conversion)", score: 0, flag: no_conversion }
      - { when: "conversion >= $conversion_floor", score: 1 }
      - { else: 0 }
  balance_sheet:
    rules:
      - { when: "is_null(net_debt_ebitda)", score: 0, flag: no_leverage }
      - { when: "net_debt_ebitda <= $leverage_max", score: 1 }
      - { else: 0 }

gate:
  mode: strict

For the baseline, keep valuation and use the strict gate. For the next version, add operating_quality and leave the gate strict. Continue in the declared order. After the full run, make one copy without each supporting family and keep the gate strict there too. Every version then requires a point from every family it contains, so each comparison changes one family while preserving the gate policy.

The proposed thresholds are starting assumptions. Changing them after every result creates more candidate theses, whether or not the file count makes that obvious. Bailey, Borwein, López de Prado, and Zhu explain why testing more configurations raises the chance of selecting an overfit backtest in their paper on backtest overfitting and out-of-sample performance. Harvey, Liu, and Zhu reach the related problem in factor research: conventional significance standards become too lenient after researchers test many candidates (their study of multiple testing in expected returns).

When does extra complexity earn a place?

Keep a rule family when four things hold.

First, it has an economic job you can state without referring to the backtest. “Avoid companies whose debt claim can consume weak cash generation” is a job. “Improves the curve” isn't.

Second, it changes the selected companies in the way that job predicts. Read the Go events for each backtest, then run the corresponding thesis version as a scan and inspect its near-misses. A leverage family should distinguish balance sheets. If it mostly removes small companies with patchier data, the data filter is doing the work.

Third, its contribution survives nearby specifications. Move its threshold a little and use another test window. Check a matched benchmark too. A holdout period is the last check after development. Don't use it as fresh material for another repair.

Fourth, the gain is worth the cost. Extra rules can reduce the number of qualification events and make the thesis harder to diagnose. A small hypothetical improvement that excludes most observations or depends on one narrow period is a poor trade.

You don't need every supporting rule to improve returns. A balance-sheet gate may be there to bound a failure mode even if it lowers the average result. Say that in advance, then judge it on the outcome you assigned it, such as drawdown or exposure to highly levered companies. Otherwise every rule gets graded against whichever statistic makes it look useful.

What if the simple version wins?

Believe the comparison long enough to simplify. A smaller thesis is easier to explain and harder to overfit. Its failures are clearer too. It also gives later ideas a clean baseline to beat.

The full version may still be useful as a research view. Features that don't deserve points can remain visible for inspection. Flags can call attention to leverage or poor cash conversion without letting those observations decide membership. Complexity can leave the score and stay in the analysis.

A backtest can't certify that the surviving rule will work in the future. Over the tested period, it can show whether added machinery improved on the main claim. It can also show that one family contributed through removal and perturbation. Those are useful findings. Keep the rules that earned them. Cut the rest. Make the next idea beat the simpler thesis first.

Want to try this on your own rules? Quantery is free for 14 days: the full app, no card required.

← All articles