Thinking global, living local

How to use colab-market-fundamentals: collect dated annual fundamentals for 20 markets in Colab

October 19, 2026

A Google Colab notebook that downloads about five years of annual company financials per market through yfinance, saves them to parquet every three minutes, and lets you resume from a zip. Research and education only. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.

Open-Source Tools · How-To · Markets research

How to use colab-market-fundamentals: collect dated annual fundamentals for 20 markets in Colab

Skip to article content
Text Size

Revised · v1.0.0 · what changed

32,590tickers in tickers.csv
20market codes
3 of 3IP probe names returned data
12rows for 3 DE tickers (4 years each)

Code: github.com/herrrickshaw/colab-market-fundamentals

Watch the walkthrough

Watch on YouTube: youtu.be/9NjzvuAPlfE

Why it is useful

  • Who benefits: students and researchers who want a table of annual revenue, profit, assets, debt and cash flow for many companies without paying for a data feed.
  • What it saves: building a rate-limited, resumable collector yourself. The notebook already handles Yahoo blocking, retries and checkpoints.
  • Evidence from the run: a probe of three large companies passed, and a live test of the collector on three German companies (SAP, Siemens, Allianz) returned 12 rows with 13 columns in a few seconds.
  • It stores one row per company per fiscal year with the fields needed for a Piotroski-style score, so the output slots straight into a screening notebook.

What it is for

Open collect.ipynb in Colab, press Run all, and download fundamentals.zip. The notebook reads tickers.csv, skips the US by default and the India set is stored with bare symbols, fetches income statement, balance sheet and cash flow for each ticker, and writes one parquet file per market.

The data come from Yahoo Finance through yfinance, so they are third-party and unaudited. Use them for research and teaching. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.

Who it is for

  • Students and analysts doing multi-market fundamentals research
  • People who want a reproducible, resumable collection job that runs in the free Colab tier
  • Non-programmers who want an AI assistant to walk through each cell
Not for.
  • Investment decisions. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
  • Point-in-time backtests. The 'filed' date is an assumption (fiscal year end plus 90 days), not the real filing date.
  • Redistributing the collected data. It comes from Yahoo, whose terms apply.

What you need

RequirementWhyHow to get it
A Google accountto run the notebook in Colabcolab.research.google.com, free tier is enough for a small test
yfinance, curl_cffi, pandas, pyarrowthe notebook installs these in its first cellautomatic in Colab; locally pip install them
A Yahoo-friendly IPYahoo blocks many datacentre IPsthe notebook probes first; if blocked, Runtime then Restart runtime for a new IP
Timea full run over about 30,000 tickers is longthe README gives no total runtime; start with one market

Get the code and an AI assistant

You do not have to be a programmer. Get the repository, give it to an AI assistant, and ask in plain language. Any of these routes works:

ToolWhat you needBest for
GitHubFree account to fork or star; git to clone, or the green Code button, then Download ZIP, or open in CodespacesGetting the code and keeping up with updates
Claude CodeClaude subscription or API key; run claude inside the cloned folderLetting the AI read and run the whole repo
Gemini CLIGoogle account; run gemini inside the cloned folderThe same hands-on workflow with Gemini
Claude.ai, Gemini or ChatGPT in the browserA chat account; upload README.md and the files you care about (or paste the output of git ls-files)Understanding the project and asking questions without installing anything
Other tools (Copilot Chat, Cursor, Codex CLI)Open the cloned folder in the tool and point it at README.md firstEditing and extending the code
git clone https://github.com/herrrickshaw/colab-market-fundamentals.git
cd colab-market-fundamentals
claude      # or: gemini

Install

  1. Open in Colab
    https://colab.research.google.com/github/herrrickshaw/colab-market-fundamentals/blob/main/collect.ipynb
  2. Or run locally in a virtualenv
    python3 -m venv .venv && source .venv/bin/activate
    pip install yfinance curl_cffi pandas pyarrow
    git clone https://github.com/herrrickshaw/colab-market-fundamentals.git

Quick start: commands and what you should see

Cell 1: install and load the ticker universe

U = pd.read_csv('tickers.csv')
print(U.groupby('market').size().to_dict())

The universe size (32,590 tickers, 20 markets) and a count per market.

Cell 4: probe, then collect (or test one market first)

collect_market('DE', ['SAP.DE','SIE.DE','ALV.DE'], workers=3)

'DE: 3 names with data'. A full Run all prints progress per market and writes /content/fundamentals.zip every three minutes.

Sample outputs

Real output from a run on the date shown in the sources line; trimmed to fit.

Universe by market (cell 1, run locally from the committed tickers.csv)

universe: 32590 tickers, 20 markets
{'AU': 1509, 'BR': 243, 'CA': 2091, 'CH': 192, 'CN': 2895, 'DE': 449, 'DK': 104, 'EU': 1067, 'FI': 171, 'HK': 2793, 'IN': 2368, 'JP': 3709, 'KR': 2768, 'SA': 374, 'SE': 564, 'SG': 614, 'TW': 2204, 'UK': 854, 'US': 7441, 'ZA': 180}

The US set is skipped by default in the collection loop.

IP probe and a three-company test (collector code from the notebook, run locally)

probe 3
  DE: 3 names, 0 done, 3 to fetch
  DE: 3 names with data
(12, 13)

Three large companies, four fiscal years each, 13 columns.

Latest two fiscal years, selected columns (EUR)

ticker  fy_end      net_income    revenue       cfo           filed
SAP.DE  2024-12-31  3.124e9       3.4176e10     5.207e9       2025-03-31
SAP.DE  2025-12-31  7.161e9       3.680e10      9.156e9       2026-03-31
SIE.DE  2024-09-30  8.301e9       7.5930e10     1.1665e10     2024-12-29
SIE.DE  2025-09-30  9.620e9       7.8914e10     1.3257e10     2025-12-29

Yahoo-sourced figures, shown only to illustrate the table shape; the 'filed' column is fiscal year end plus 90 days.

Columns written per company-year

ticker, fy_end, net_income, revenue, gross_profit, total_assets, current_assets,
current_liabilities, long_term_debt, shares, equity, cfo, filed

Some fields are missing for some companies, for example banks and insurers have no gross profit or current assets.

Ways to use it

MethodHow it works
Colab, whole universeRun all, then download fundamentals.zip and unzip to get one parquet per market.
Colab, one marketEdit MARKETS in cell 4 to a single code to test before a long run.
ResumeUpload a previous zip in cell 2 and the collector skips tickers already done.
Ask an AI assistantHave the assistant explain each cell or adapt it to add a field.

Sample prompts for your AI assistant

Copy, paste and edit the parts in [brackets].

  • Claude: Read collect.ipynb and explain in plain English how the checkpointing and the IP probe work.
  • Gemini: Modify cell 4 so only the DE and SE markets run, and print how many rows each parquet has at the end.
  • ChatGPT: I uploaded a market parquet from this notebook. Compute return on equity per company-year and flag missing values.
  • GitHub (Copilot Chat): Add a field for 'Total Debt' to the BAL dictionary in the collector and update the output columns.
  • Claude: Explain why the 'filed' date here cannot be used for a point-in-time backtest and suggest a safer approach.

Limits and honest caveats

  • Yahoo data are third-party, unaudited and can change or vanish; terms of service apply. Samples here are shown only to illustrate the table shape.
  • The 'filed' column is an assumption: fiscal year end plus 90 days. It is not the actual filing date, so do not use it for strict point-in-time work.
  • Yahoo blocks many datacentre IPs. The notebook probes first and tells you to restart the runtime if the probe returns zero; whether Colab's IP is blocked on a given day is out of the repo's control.
  • I ran the collector code locally (probe plus three German tickers), not a full Colab run of 30,000 tickers; total runtime and block rate at scale are not measured. The Colab upload/download cells were not run.
  • Banks, insurers and some foreign filers return only part of the 13 columns.
  • Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.

Sources: Ran the notebook's cell 1 logic and collector cell 3 (with probe) locally in a throwaway virtualenv on 19 October 2026 from a clone of github.com/herrrickshaw/colab-market-fundamentals, using yfinance with curl_cffi for three large German companies. Colab-specific cells (upload, download) not run.

Revision history.
  • v1.0.0 — 23 October 2026 — first published.

About this article: Researched, written and edited by Umashankar Triplicane Dwarakanathan, with AI research assistance; every figure is meant to trace to the primary source cited. See the Editorial Policy for how sourcing, AI use and corrections work.

Umashankar Triplicane Dwarakanathan
Contact Us
Umashankar Triplicane Dwarakanathan
Investment Promotion & Energy-Sector Leader · Chennai, Tamil Nadu, India
LinkedIn → GitHub → Email +91 78273 81696
How this site works

Data-led analysis of India's trade, currency and industrial policy. Every article is built from primary official sources, and every figure links back to the release, table or filing it came from.

Sources. DGCI&S TradeStat (imports/exports, HSN-wise) · PIB (government press releases, January 2017 to today, refreshed daily) · RBI (circulars, balance of payments) · MoSPI (CPI/WPI, IIP) · PARIVESH (environmental clearances) · CCIL (bond yields) · BIS (policy rates) · SEBI, NSE/BSE and SEC filings for company data.

Interpretation. Figures carry their vintage and retrieval date; estimates and press-reported numbers are labelled as such; where sources disagree, both are shown. Corrections are made visibly, never silently. Articles are written with AI assistance from the cited sources — AI-generated text can misstate figures even when working from real material, so verify any number that matters to a decision against the linked primary source.

footer

Browse all articles by topic

Every piece on this blog, grouped. Or read the full index.

Agriculture & FertilisersAI ToolsChemicalsClimate & CarbonEnergy & FuelsGas & LNGImport SubstitutionIndustrial PolicyMarkets & FinanceMobility & EVPrices & InflationTextilesTrade & Tariffs

Each topic is a live archive page that updates itself as pieces are labelled. It replaces a hand-kept list that had fallen 18 articles behind.