A Google Colab notebook that downloads about five years of annual company financials per market through yfinance, saves them to parquet every three minutes, and lets you resume from a zip. Research and education only. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Open-Source Tools · How-To · Markets research
How to use colab-market-fundamentals: collect dated annual fundamentals for 20 markets in Colab
Revised
· v1.0.0 · what changed
Code: github.com/herrrickshaw/colab-market-fundamentals
Watch the walkthrough
Watch on YouTube: youtu.be/9NjzvuAPlfE
Why it is useful
- Who benefits: students and researchers who want a table of annual revenue, profit, assets, debt and cash flow for many companies without paying for a data feed.
- What it saves: building a rate-limited, resumable collector yourself. The notebook already handles Yahoo blocking, retries and checkpoints.
- Evidence from the run: a probe of three large companies passed, and a live test of the collector on three German companies (SAP, Siemens, Allianz) returned 12 rows with 13 columns in a few seconds.
- It stores one row per company per fiscal year with the fields needed for a Piotroski-style score, so the output slots straight into a screening notebook.
What it is for
Open collect.ipynb in Colab, press Run all, and download fundamentals.zip. The notebook reads tickers.csv, skips the US by default and the India set is stored with bare symbols, fetches income statement, balance sheet and cash flow for each ticker, and writes one parquet file per market.
The data come from Yahoo Finance through yfinance, so they are third-party and unaudited. Use them for research and teaching. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Who it is for
- Students and analysts doing multi-market fundamentals research
- People who want a reproducible, resumable collection job that runs in the free Colab tier
- Non-programmers who want an AI assistant to walk through each cell
- Investment decisions. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
- Point-in-time backtests. The 'filed' date is an assumption (fiscal year end plus 90 days), not the real filing date.
- Redistributing the collected data. It comes from Yahoo, whose terms apply.
What you need
| Requirement | Why | How to get it |
|---|---|---|
| A Google account | to run the notebook in Colab | colab.research.google.com, free tier is enough for a small test |
| yfinance, curl_cffi, pandas, pyarrow | the notebook installs these in its first cell | automatic in Colab; locally pip install them |
| A Yahoo-friendly IP | Yahoo blocks many datacentre IPs | the notebook probes first; if blocked, Runtime then Restart runtime for a new IP |
| Time | a full run over about 30,000 tickers is long | the README gives no total runtime; start with one market |
Get the code and an AI assistant
You do not have to be a programmer. Get the repository, give it to an AI assistant, and ask in plain language. Any of these routes works:
| Tool | What you need | Best for |
|---|---|---|
| GitHub | Free account to fork or star; git to clone, or the green Code button, then Download ZIP, or open in Codespaces | Getting the code and keeping up with updates |
| Claude Code | Claude subscription or API key; run claude inside the cloned folder | Letting the AI read and run the whole repo |
| Gemini CLI | Google account; run gemini inside the cloned folder | The same hands-on workflow with Gemini |
| Claude.ai, Gemini or ChatGPT in the browser | A chat account; upload README.md and the files you care about (or paste the output of git ls-files) | Understanding the project and asking questions without installing anything |
| Other tools (Copilot Chat, Cursor, Codex CLI) | Open the cloned folder in the tool and point it at README.md first | Editing and extending the code |
git clone https://github.com/herrrickshaw/colab-market-fundamentals.git
cd colab-market-fundamentals
claude # or: gemini
Install
- Open in Colab
https://colab.research.google.com/github/herrrickshaw/colab-market-fundamentals/blob/main/collect.ipynb - Or run locally in a virtualenv
python3 -m venv .venv && source .venv/bin/activate pip install yfinance curl_cffi pandas pyarrow git clone https://github.com/herrrickshaw/colab-market-fundamentals.git
Quick start: commands and what you should see
Cell 1: install and load the ticker universe
U = pd.read_csv('tickers.csv')
print(U.groupby('market').size().to_dict())
The universe size (32,590 tickers, 20 markets) and a count per market.
Cell 4: probe, then collect (or test one market first)
collect_market('DE', ['SAP.DE','SIE.DE','ALV.DE'], workers=3)
'DE: 3 names with data'. A full Run all prints progress per market and writes /content/fundamentals.zip every three minutes.
Sample outputs
Real output from a run on the date shown in the sources line; trimmed to fit.
Universe by market (cell 1, run locally from the committed tickers.csv)
universe: 32590 tickers, 20 markets
{'AU': 1509, 'BR': 243, 'CA': 2091, 'CH': 192, 'CN': 2895, 'DE': 449, 'DK': 104, 'EU': 1067, 'FI': 171, 'HK': 2793, 'IN': 2368, 'JP': 3709, 'KR': 2768, 'SA': 374, 'SE': 564, 'SG': 614, 'TW': 2204, 'UK': 854, 'US': 7441, 'ZA': 180}
The US set is skipped by default in the collection loop.
IP probe and a three-company test (collector code from the notebook, run locally)
probe 3
DE: 3 names, 0 done, 3 to fetch
DE: 3 names with data
(12, 13)
Three large companies, four fiscal years each, 13 columns.
Latest two fiscal years, selected columns (EUR)
ticker fy_end net_income revenue cfo filed
SAP.DE 2024-12-31 3.124e9 3.4176e10 5.207e9 2025-03-31
SAP.DE 2025-12-31 7.161e9 3.680e10 9.156e9 2026-03-31
SIE.DE 2024-09-30 8.301e9 7.5930e10 1.1665e10 2024-12-29
SIE.DE 2025-09-30 9.620e9 7.8914e10 1.3257e10 2025-12-29
Yahoo-sourced figures, shown only to illustrate the table shape; the 'filed' column is fiscal year end plus 90 days.
Columns written per company-year
ticker, fy_end, net_income, revenue, gross_profit, total_assets, current_assets,
current_liabilities, long_term_debt, shares, equity, cfo, filed
Some fields are missing for some companies, for example banks and insurers have no gross profit or current assets.
Ways to use it
| Method | How it works |
|---|---|
| Colab, whole universe | Run all, then download fundamentals.zip and unzip to get one parquet per market. |
| Colab, one market | Edit MARKETS in cell 4 to a single code to test before a long run. |
| Resume | Upload a previous zip in cell 2 and the collector skips tickers already done. |
| Ask an AI assistant | Have the assistant explain each cell or adapt it to add a field. |
Sample prompts for your AI assistant
Copy, paste and edit the parts in [brackets].
- Claude: Read collect.ipynb and explain in plain English how the checkpointing and the IP probe work.
- Gemini: Modify cell 4 so only the DE and SE markets run, and print how many rows each parquet has at the end.
- ChatGPT: I uploaded a market parquet from this notebook. Compute return on equity per company-year and flag missing values.
- GitHub (Copilot Chat): Add a field for 'Total Debt' to the BAL dictionary in the collector and update the output columns.
- Claude: Explain why the 'filed' date here cannot be used for a point-in-time backtest and suggest a safer approach.
Limits and honest caveats
- Yahoo data are third-party, unaudited and can change or vanish; terms of service apply. Samples here are shown only to illustrate the table shape.
- The 'filed' column is an assumption: fiscal year end plus 90 days. It is not the actual filing date, so do not use it for strict point-in-time work.
- Yahoo blocks many datacentre IPs. The notebook probes first and tells you to restart the runtime if the probe returns zero; whether Colab's IP is blocked on a given day is out of the repo's control.
- I ran the collector code locally (probe plus three German tickers), not a full Colab run of 30,000 tickers; total runtime and block rate at scale are not measured. The Colab upload/download cells were not run.
- Banks, insurers and some foreign filers return only part of the 13 columns.
- Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Sources: Ran the notebook's cell 1 logic and collector cell 3 (with probe) locally in a throwaway virtualenv on 19 October 2026 from a clone of github.com/herrrickshaw/colab-market-fundamentals, using yfinance with curl_cffi for three large German companies. Colab-specific cells (upload, download) not run.
- v1.0.0 — 23 October 2026 — first published.
About this article: Researched, written and edited by Umashankar Triplicane Dwarakanathan, with AI research assistance; every figure is meant to trace to the primary source cited. See the Editorial Policy for how sourcing, AI use and corrections work.