How to refresh stock data before a scan
Before rerunning a stock screen, refresh the datasets its rules actually use. Start with a small ticker list, run the relevant data job, check that it finished, and then run the same saved thesis version. Expand to the whole universe only after the small job produced the coverage you expected.
That sequence keeps a routine update from becoming a rebuild of everything in your local lake. A valuation thesis may need current prices and the latest filings. A short-interest thesis needs a different collector. If you refresh every available dataset on every run, you'll wait longer and lose track of which new input changed the result.
Which data should you refresh before a scan?
Read the thesis before opening the data controls. List each feature that can change and trace it to the source beneath it.
A price-to-value ratio needs a recent price plus the filing values in its denominator. A momentum feature needs enough price history to cover its lookback. A cash-flow quality rule needs SEC fundamentals. A rule about short positioning needs short-interest observations. Analyst consensus, options implied volatility, insider transactions, earnings dates, and share float matter only when the thesis refers to them.
This dependency check sounds fussy. It's faster than waiting for an irrelevant collection job, and it prevents a common mistake: assuming a fresh price pull also refreshed the filings. Each dataset keeps its own history. It may also use a different provider and collection status.
For most filing-based screens, begin with these two jobs:
- Prices + actions refreshes daily market bars and the corporate actions needed to interpret that history. Use it when the thesis contains price, market capitalization, return, drawdown, liquidity, or another market-derived feature.
- Fundamentals (SEC) refreshes point-in-time statement data from company filings. Use it when the thesis reads revenue, cash flow, assets, debt, share count, or a ratio built from those fields.
Quantery can collect other datasets separately. Leave them alone unless the thesis needs them. A dataset sitting in the app isn't a requirement to collect it.
If you don't know what a feature depends on, audit one current screen result down to its filing inputs. The explanation chain makes the dependency concrete. It'll also show whether the value is a direct read or a calculation assembled from several periods.
Why should you start with a small ticker list?
A targeted refresh is a diagnostic. Pick a company with the stale or missing input you noticed, plus another company whose coverage already looks normal. Run the relevant job for those names and inspect what changed.
The first company tells you whether collection can fill the gap. The second helps catch a broader source problem. If they don't update, check the job output and provider setup before widening the scope. If the expected rows arrive, you've proved the path on a small sample and can decide whether the rest of the thesis universe needs the same update. You haven't proved the full universe is current yet.
In the data view, turn off Whole universe (latest snapshot) and enter the tickers you want to test. The input accepts a pasted list. Keep this first pass narrow enough that you can inspect each company's result afterward.
Use the whole-universe option when the coming scan genuinely covers that universe and the inputs are stale across it. A market-wide screen with fresh data for a hand-picked group creates an uneven comparison. Companies with current inputs may evaluate differently from companies still carrying older ones, even though the rules are identical.
Scope also controls the claim you can make about the scan. Refreshing a few examples can confirm that the collector works. It can't establish that every eligible company is current. Check data coverage before a backtest for the same reason: the declared universe can be wider than the set the local data can evaluate.
When should you use a full refresh?
The normal price and fundamentals jobs respect freshness. They can skip data that the lake already considers current. That's the useful default. It avoids asking a provider for the same records and keeps routine updates focused on missing or stale material.
The Full refresh control tells the selected collector to ignore that freshness check. For prices, it rebuilds the requested pull instead of trusting the current marker. For fundamentals, it re-pulls the supported filings for the chosen scope.
Use that control when you have a reason:
- A completed job left a gap that the source now appears able to supply.
- A provider response was partial and you want to retry the requested scope.
- A filing was amended or the local record looks inconsistent with EDGAR.
- You've changed data-source settings and need to verify the new path.
Don't make full refresh the ritual before every scan. It'll consume provider allowance and time without adding information. More fetching doesn't make a thesis more current when the source has nothing newer to return.
A refresh can't create a filing that hasn't been submitted. It also can't extend price history beyond the depth available from your market-data plan. When the required history isn't available, shorten the research window or change the rule's missing-data policy. Filling an unknown input with zero only turns absent evidence into a favorable number.
How do you run and verify the data job?
Choose the collector, set the scope, and use the optional starting date only when you're deliberately backfilling history. Then start the job. If another write operation is active, the new work joins the queue instead of fighting for the local database.
Quantery gives every scan, backtest, and data pull access to one lake writer. The queue banner shows what is active and what is waiting. Use Run next when this job should move ahead of other queued work. Turn on Notify when done if you're leaving the app. Promotion changes the order, not the contents of the job.
Wait for the job to finish, then read its status and output. A completed label means the process ended. It doesn't guarantee that every requested company gained every requested period. Provider limits, unsupported symbols, and issuers without the expected filing type can still leave holes.
Open the refreshed companies and read Data coverage again. Check the stored range, latest collection status, filing cadence, and any reported gaps that matter to your feature. For a calculated value, use Data and filings or the number explanation to confirm that the expected periods now feed the calculation. Don't infer a successful update from the job label alone.
If the job failed, keep the failure separate from an empty result. A failed request says collection didn't complete. A successful SEC check with no supported filing says the source returned no usable filing for that company. Retrying can fix the first case. It won't automatically fix the second.
How should you rerun the scan after refreshing data?
Keep the thesis version and scan scope fixed. Rerun the same rules first. If you've also changed a threshold, gate, or universe, a changed survivor list has several possible causes and the refresh taught you very little.
Compare the new run with the prior run. Added and dropped companies tell you where to investigate, while score movement can identify a feature that changed without crossing the gate. Comparing two screen runs is useful here, but the diff doesn't prove causation. Confirm the changed values and their dates before attributing a move to a new filing or price.
Write down the dataset, job scope, refresh setting, and thesis version beside the run. You don't need a grand research journal for this. One plain sentence is enough to preserve what changed between observations.
Be careful with saved historical results. A recorded scan retains the values it saw. A live explanation may use the lake after your refresh, so Recorded (run) and Live (today) can differ. That's expected. Today's filing links explain the live calculation; they don't reconstruct every input behind an older saved value.
A repeatable pre-scan routine
Use the same short routine whenever freshness matters:
- Read the thesis and name the datasets behind its changeable features.
- Check coverage on a stale example and a normal comparison company.
- Run a targeted job without forcing a full refresh.
- Inspect the job output and coverage before widening the scope.
- Refresh the needed universe, then rerun the unchanged thesis.
- Compare runs and investigate the values that moved.
This is less dramatic than rebuilding the lake before every screen. It's also easier to reason about. The data job becomes one controlled change in the experiment, and the next run tells you how your rules responded to that change.
Quantery keeps the data local and the rules editable. That means freshness is yours to manage too. Refresh what the thesis consumes, verify what arrived, and make the screen answer one clean question at a time.
Research tooling, not investment advice. Nothing here is a recommendation to buy, sell, or hold any security. Screens, scores, and backtests are informational only; backtested results are hypothetical, exclude costs such as commissions and slippage, and do not guarantee future results. Verify against primary filings and make your own decisions.
Want to try this on your own rules? Quantery is free for 14 days: the full app, no card required.
← All articles