Ready-made correlation matrices of daily stock returns for six markets (India, US, Japan, Korea, China, Europe), stored compactly so you can load one in a few lines of Python. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Open-Source Tools · How-To · Markets research
How to use market-correlation-matrices: read per-market stock correlation snapshots
Revised
· v1.0.0 · what changed
Code: github.com/herrrickshaw/market-correlation-matrices
Watch the walkthrough
Watch on YouTube: youtu.be/VUgVkKy1FxA
Why it is useful
- Who benefits: students and analysts who want to see how tightly stocks in a market move together, for diversification studies or teaching.
- What it replaces: dense CSV files of up to 837 MB that were never committed. The README says the same information is about 24 times smaller here, in regular git files.
- Evidence from the run: the rebuild script ran end to end in about 17 seconds on a local copy of the price warehouse, and the six summary files load instantly. The rebuilt Europe dense matrix is 1,420 by 1,420, and a lookup of the most similar stocks to a name takes one line.
- The summaries are derived aggregates (mean, median, counts, top pairs) computed from daily price returns, so you can compare markets without any raw price data.
What it is for
For each market the repo stores three files: a dense correlation matrix (float16, with a symbol list), a table of pairs whose correlation is at least 0.6 in absolute value, and a small summary. Correlation is the Pearson measure on daily log returns over the last 252 trading days, using the 3,000 most traded symbols.
It is a descriptive snapshot, not a model and not a signal. Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Who it is for
- Students learning about correlation, diversification and market structure
- Analysts who want a quick cross-market comparison of how closely stocks move together
- Non-programmers who want an AI assistant to explain what the numbers mean
- Trading signals. The README says the pairs file is descriptive, not a signal by itself.
- Current conditions. The committed windows end between 29 June and 21 July 2026, and a rebuild on 23 October 2026 ends 21 to 22 October (China 1 July); windows are not aligned across markets.
- Full-precision work. Values are stored as float16, about three decimal places.
What you need
| Requirement | Why | How to get it |
|---|---|---|
| Python 3 with numpy, pandas, pyarrow | load the matrix and pairs files | pip install numpy pandas pyarrow |
| duckdb and a price warehouse (rebuild only) | compute_correlations.py reads an OHLCV warehouse (a DuckDB file with an ohlcv view) that is not in this repo; the path is hard-coded to ~/repos/global-market-data/warehouse/warehouse.duckdb | pip install duckdb; build your own ohlcv view with columns market, Date, Symbol, Close, Volume and edit the WAREHOUSE path at the top of the script; reading the committed files needs neither |
| Not needed | no API key, no GPU | about 130 MB of disk for the data folders |
Get the code and an AI assistant
You do not have to be a programmer. Get the repository, give it to an AI assistant, and ask in plain language. Any of these routes works:
| Tool | What you need | Best for |
|---|---|---|
| GitHub | Free account to fork or star; git to clone, or the green Code button, then Download ZIP, or open in Codespaces | Getting the code and keeping up with updates |
| Claude Code | Claude subscription or API key; run claude inside the cloned folder | Letting the AI read and run the whole repo |
| Gemini CLI | Google account; run gemini inside the cloned folder | The same hands-on workflow with Gemini |
| Claude.ai, Gemini or ChatGPT in the browser | A chat account; upload README.md and the files you care about (or paste the output of git ls-files) | Understanding the project and asking questions without installing anything |
| Other tools (Copilot Chat, Cursor, Codex CLI) | Open the cloned folder in the tool and point it at README.md first | Editing and extending the code |
git clone https://github.com/herrrickshaw/market-correlation-matrices.git
cd market-correlation-matrices
claude # or: geminiInstall
- Get the code and data
git clone https://github.com/herrrickshaw/market-correlation-matrices.git cd market-correlation-matrices - Create an environment
python3 -m venv .venv && source .venv/bin/activate pip install numpy pandas pyarrow
Quick start: commands and what you should see
Read the per-market summaries
import json
for m in ['IN','US','JP','KR','CN','EU']:
s = json.load(open(f'data/{m}/summary.json'))
print(m, s['window'], s['symbols'], s['mean_pairwise_corr'])One line per market with the window dates, symbol count and mean correlation.
Load a dense matrix and find a stock's closest neighbours
import numpy as np
z = np.load('data/EU/corr_dense.npz', allow_pickle=True)
c, sy = z['corr'].astype(float), list(z['symbols'])
k = sy.index('SIE.DE'); c[k,k] = -9
print([(sy[i], round(c[k,i],3)) for i in np.argsort(-c[k])[:3]])The three symbols most correlated with Siemens (SIE.DE), with their r values.
Sample outputs
Real output from a run on the date shown in the sources line; trimmed to fit.
Market summaries after a rebuild (data/*/summary.json), data to 22 October 2026
python compute_correlations.py (with a session-completeness filter, see limits)
IN ['2025-10-13', '2026-10-22'] 1828 symbols mean r 0.2113 median 0.2118
US ['2025-09-25', '2026-10-21'] 2883 symbols mean r 0.1058 median 0.097
JP ['2025-09-16', '2026-10-21'] 2813 symbols mean r 0.1853 median 0.1746
KR ['2025-09-29', '2026-10-21'] 2468 symbols mean r 0.2528 median 0.2515
CN ['2025-06-18', '2026-07-01'] 2978 symbols mean r 0.1718 median 0.1684
EU ['2025-09-25', '2026-10-21'] 1420 symbols mean r 0.0978 median 0.0837Last trading session in the data: India 22 Oct 2026; US, Japan, Korea, Europe 21 Oct 2026; China only 1 Jul 2026 (stale, so its row is unchanged in substance from the committed one). Korea and India are still the most tightly co-moving markets, Europe the lowest.
Old committed snapshot versus the rebuild
market symbols old->new mean r old->new
IN 2001 -> 1828 0.2253 -> 0.2113
US 2945 -> 2883 0.115 -> 0.1058
JP 2998 -> 2813 0.1703 -> 0.1853
KR 2570 -> 2468 0.227 -> 0.2528
CN 2978 -> 2978 0.171 -> 0.1718
EU 842 -> 1420 0.0524 -> 0.0978Windows moved from ending June/July 2026 to ending October 2026 (China excepted). The larger Europe universe and higher mean are partly because the old build had fewer symbols passing the 200-observation floor; do not read the change as a market regime shift without a like-for-like check.
Closest neighbours in the rebuilt Europe matrix
np.load('data/EU/corr_dense.npz')
SIE.DE -> SIE.F 0.917, SU.PA 0.692, BRGE.L 0.684
ASML.AS -> ASM.AS 0.810, PCT.L 0.720, BESI.AS 0.686
MC.PA -> CDI.PA 0.849, CFR.SW 0.708, RMS.PA 0.687SIE.F is Siemens on the Frankfurt exchange, i.e. the same company on a second venue, so the first real peer for Siemens is SU.PA. Semiconductor equipment (ASM, BESI) and luxury (Dior, Richemont, Hermes) still cluster.
Top Europe pairs and matrix shape
summary.json top_pairs; corr_dense.npz
STMMI.MI / STMPA.PA 0.9998 (same company, Milan and Paris listings)
STLAM.MI / STLAP.PA 0.9991 (same company, Milan and Paris listings)
MT.AS / MTS.MC 0.9985 (same company, Amsterdam and Madrid listings)
REL.L / REN.AS 0.9883 (same company, London and Amsterdam)
Europe dense matrix: (1420, 1420) float16
Europe pairs file: 2031 pairs, columns sym1, sym2, corrVery high pairs are one company listed twice, not two independent stocks. The same holds for US trusts and ETFs (GLD/IAU, GBTC/BITB at 0.9999).
Ways to use it
| Method | How it works |
|---|---|
| Python | np.load the dense file, or pandas.read_parquet the pairs file, then filter by symbol. |
| Cross-market comparison | Compare mean and median r across the six summaries; note the different window ends. |
| Ask an AI assistant | Point the assistant at data/ and ask it to chart the distribution of correlations for a market. |
Sample prompts for your AI assistant
Copy, paste and edit the parts in [brackets].
- Claude: Load data/EU/corr_dense.npz and tell me the ten most correlated stock pairs after removing pairs that share a company name or a dual listing.
- Gemini: Read every data/*/summary.json and write a three-sentence comparison of the six markets, naming the windows.
- ChatGPT: Explain in plain language what a mean pairwise correlation of 0.05 versus 0.23 says about diversification. Do not give investment advice.
- GitHub (Copilot Chat): Write a function that returns the k nearest symbols by correlation for any symbol in a market's npz file, with a test.
- Claude: The README mentions eigenvalue summaries as an open question. Compute the largest-eigenvalue share for the Europe matrix and explain what it means.
Limits and honest caveats
- Snapshot, not live: windows end 21 or 22 October 2026 for India, US, Japan, Korea and Europe, but 1 July 2026 for China (the warehouse copy has no later China bars). Cross-market comparisons therefore mix dates.
- Data-quality finding: in the warehouse copy, many recent days (Sep to Oct 2026) hold only a small share of symbols (for example Europe days with 1 to 30 symbols, Japan days with 8 to 150, Korea days with 3 to 4). The repo script counts any date as a trading session, so the unmodified script's 252-day window absorbs these partial days and its Europe row differs (928 symbols, mean r 0.148). The numbers shown here come from a locally patched copy that counts a date only if at least half the market's usual symbol count traded and requires volume above zero; the unpatched run is reported in the status notes. The repo itself was not changed.
- Some pairs are stale-price artefacts rather than co-movement: in the unpatched run a Korean pair showed r of exactly 1.0 because both series had a constant price and zero volume on recent days. The patch removes zero-volume bars.
- The India universe includes ETFs and index funds (the README says the NSE panel is not ISIN-filtered), and several top India pairs between small caps and large banks (for example 0.96) are unexplained and not validated here. Filter before reading them as stock pairs. The US universe also contains trusts and ETFs.
- Pairwise correlation on thinly traded stocks is noisy even with a 200-observation floor; float16 storage rounds to about three decimals. Europe's low mean is also affected by non-overlapping trading hours across venues.
- Not investment advice: this is research and education material built from public-source or derived data; it is not a recommendation to buy or sell anything.
Sources: Re-ran compute_correlations.py (commit 0d8e98f of github.com/herrrickshaw/market-correlation-matrices) on 23 October 2026 in a throwaway virtualenv in a scratch clone, pointed at a local read-only copy of the author's OHLCV warehouse (last sessions: IN 22 Oct 2026; US, JP, KR, EU 21 Oct 2026; CN 1 Jul 2026), with a session-completeness patch described under limits. Samples are derived aggregates (means, counts, top pairs, neighbours); no raw price rows are shown. Neighbour lookups computed by me from the rebuilt matrix. The GitHub repo was not modified.
- v1.0.0 — 23 October 2026 — first published.
About this article: Researched, written and edited by Umashankar Triplicane Dwarakanathan, with AI research assistance; every figure is meant to trace to the primary source cited. See the Editorial Policy for how sourcing, AI use and corrections work.
