← Quantery Blog

How to handle missing data in a stock screen

September 25, 2026 · 8 min readscreeningdata-qualitythesis-building

Missing data needs a rule before it needs a replacement. If an input is required to make the thesis claim, a missing value should make the company ineligible. If the input is optional context, preserve the gap and flag it. Don't turn a blank into zero unless zero is what the blank means by definition.

That policy sounds cautious because it is. A stock screen compares unlike filings at market scale, so a clean formula fed by less evidence for one company than another can do as much damage as a bad formula. So put the coverage policy inside the thesis itself, where every result can show it. Then check whether exclusions drift as the test moves through time.

Why do filing fields go missing?

Public filings weren't designed as identical database rows. Companies report the economics of their businesses using accounting concepts that fit those businesses. The SEC requires domestic filers to tag financial-statement information in Inline XBRL, which makes one document both human-readable and machine-readable (SEC, Inline XBRL). Machine-readable doesn't mean every filer uses the same fact in every period.

The SEC has documented the fault line. A filer may create a custom XBRL element only when an appropriate standard tag doesn't exist, and custom tags can reduce comparability between companies. Its August 2026 review examines custom line-item tags in Forms 10-K across fiscal years 2012 through 2025 (SEC, U.S. GAAP XBRL custom tags trend). A machine can read a custom fact, but a market-wide data pipeline still has to decide whether it maps cleanly to the common field your rule expects.

Period-to-period consistency can break too. SEC staff observed filers using different elements for the same income-statement line across successive reports, including a Form 10-Q and a Form 10-K. The staff warned that inconsistent element selection hinders comparability and said the same item should keep the same element unless its accounting definition or nature changes (SEC, Changing tags for the same reported items).

Then there are ordinary business differences. A software company may report no inventory because it has none. A bank's balance sheet doesn't fit a manufacturer's current-ratio test. A newly listed company may lack the prior periods needed for a year-over-year comparison. A filing can be late. A line can move into a footnote, change taxonomy tags, or arrive with a sign that needs review. None of those cases has the same economic meaning, even though all can appear as null (the engine's marker for a missing or unknown value) in a calculation.

When should a missing value exclude a company?

Start with the claim. If the thesis says “operating margin is positive,” it needs revenue and operating income from a compatible period. Without either input, the claim hasn't been tested. Exclude the company from that thesis result.

The same applies to a denominator. A debt-to-assets rule without assets has nothing to say about leverage. Letting the comparison fall through to a favorable score rewards the row for being incomplete.

Use a required-data gate for inputs that define the thesis:

• A valuation rule needs its price measure and the financial numerator.
• A year-over-year rule needs both comparable periods and enough filing history to make the lag meaningful.
• A balance-sheet identity needs all components from the same filing date.
• A trailing cash-flow rule needs a declared policy for incomplete quarters.

That's a statement about one thesis, which can't evaluate the company yet. Another thesis built from available fields may be perfectly valid.

Sector exclusions belong in the same conversation. Working capital and the current ratio explains why “current” doesn't carry the same meaning for every business model. Dropping banks from a manufacturing-style liquidity thesis is better than imputing values that make the ratio look complete.

When should a blank stay blank?

Some missing facts describe absence. Others describe ignorance. Keep those apart.

A filing with no dividend payment may support treating dividends_paid as zero if the field's documented convention says absence means none. The same shortcut is indefensible for revenue, debt, or operating cash flow. A missing debt fact doesn't prove a debt-free balance sheet. A missing cash-flow line doesn't prove zero cash generation.

Preserve null when the business fact is unknown. Add a visible flag that names the gap. Use specific flags such as no_margin, short_history, or stale_filing. A generic “data issue” label hides what you need to inspect and makes counts harder to compare across runs.

Be careful with helper functions. In Quantery, a comparison involving null evaluates false, which keeps a missing value from clearing a positive threshold. But ttm() sums whichever values are present among the newest four quarterly rows and returns null only when all four are absent. That behavior can produce a number from an incomplete window. Trailing versus forward numbers covers the filing-window problem in detail. If four quarters are essential to the claim, check the history and the underlying periods. A non-null trailing value isn't proof that the window is complete.

Don't borrow an older value without saying so. latest(field) searches backward for the newest non-null observation. That's useful for a standalone level when carrying the last reported value is appropriate. It is dangerous when two balance-sheet inputs must come from one filing. Use newest() for both sides so a missing component stays missing and never pair this quarter's liabilities with last quarter's equity.

What does a coverage-aware thesis look like?

The shape below is illustrative. The bundled templates are the reference for exact fields and options, and the Quantery DSL documentation has the full vocabulary.

# Coverage-aware profitability and leverage: illustrative
params:
  margin_floor: 0.05
  leverage_ceiling: 0.50

features:
  revenue_now: newest(revenue)
  operating_income_now: newest(operating_income)
  liabilities_now: newest(total_liabilities)
  equity_now: newest(total_equity)
  assets_now: liabilities_now + equity_now
  debt_now: newest(total_debt)
  margin: if(revenue_now > 0,
    operating_income_now / revenue_now, null)
  leverage: if(assets_now > 0,
    debt_now / assets_now, null)

criteria:
  operating_margin:
    rules:
      - { when: "is_null(margin)", score: 0, flag: no_margin }
      - { when: "margin >= $margin_floor", score: 2 }
      - { when: "margin > 0", score: 1 }
      - { else: 0 }
  balance_sheet:
    rules:
      - { when: "is_null(leverage)", score: 0, flag: no_leverage }
      - { when: "leverage <= $leverage_ceiling", score: 2 }
      - { when: "leverage < 1", score: 1 }
      - { else: 0 }

gate:
  mode: strict

The null rules come first because criterion rules are first-match-wins. An unavailable ratio scores zero and carries a reason. The strict gate then requires both claims to have some support.

Notice the same-filing balance sheet. liabilities_now and equity_now both use newest(), then add to assets_now. If either fact is absent from the newest period, arithmetic with null keeps assets and leverage unresolved. The code refuses to fabricate a balance sheet from different dates.

The example's round thresholds are assumptions to test. What makes a good business remains an open question. The data policy is the less negotiable part. Missing evidence cannot earn a favorable score.

A looser alternative is a score gate that lets one measured strength offset one missing criterion. That can be a valid research design, but call it what it is: the thesis is making a decision with partial evidence. Keep the missing flag in the output and compare that version with the strict one. If the loose version works only because incomplete rows receive enough points elsewhere, you have measured the gate's tolerance as much as the business condition.

How can exclusions bias a backtest?

Strict handling stops incomplete rows from passing, but it doesn't remove bias. It changes the eligible universe. If coverage is worse among small companies, older filings, or a particular industry, the tested portfolio can drift toward large, well-covered firms. A strong result may belong partly to that coverage filter.

Record coverage at every simulated decision date. Ask how many otherwise eligible companies had the inputs, which sectors lost the most rows, and whether the exclusion rate changed through the test. Don't quote one full-period percentage and call the problem solved. Coverage can improve sharply over time, leaving an early backtest period unlike the later one.

Run an ablation test on the rule that consumes the sparse field. Keep the universe, dates, rebalance schedule, and benchmark fixed. Remove that one criterion and compare the eligible set as well as the hypothetical returns. Every backtest is hypothetical and excludes trading costs. The comparison can show whether the sparse input changed who was testable; it can't tell you how the excluded companies would have scored if their facts had existed.

A second useful run restricts the test to dates or cohorts with stable coverage. If the conclusion vanishes, you have learned that the original result depended on the data boundary. How to tell if a backtest result is real gives the broader perturbation method. The dial you're turning here is the evidence requirement itself.

What should you inspect after a scan?

Open some passes and near-misses, then spend real time on the exclusions. Follow each decisive feature back to its filing inputs. Check the fiscal periods and filed dates. Confirm that ratios use compatible dates and that an apparent zero wasn't created by a default.

Quantery exposes data-quality flags and the calculation path because a score without its missing-data policy is underspecified. The audit workflow for a stock screen result walks from a gate decision through the matched rule to the stored inputs. Use it on exclusions too. A company that failed for no_margin tells you something about coverage, while a company that failed because its margin was below the threshold tells you something about the rule.

Keep the failures visible between versions. If a taxonomy mapping or new filing fills a gap later, rerun the unchanged thesis before altering its thresholds. Otherwise a data repair and a rule change land in the same result, and you won't know which moved it.

Missing data is a state the thesis must interpret. Spell out how yours interprets it, and revisit that choice when the evidence says you should.

Want to try this on your own rules? Quantery is free for 14 days: the full app, no card required.

← All articles