Daily APMC mandi prices from Agmarknet (via data.gov.in) plus World Bank, FAOSTAT and NASA POWER agriculture data, all from open sources and stored in a local DuckDB file.
Open-Source Tools · How-To · Agri & trade trackers
How to use agri-commodity-tracker: India mandi prices and an agriculture panel in one DuckDB file
Revised
· v1.0.0 · what changed
Code: github.com/herrrickshaw/agri-commodity-tracker
Watch the walkthrough
Watch on YouTube: youtu.be/0P80NVYwPCM
Why it is useful
- Analysts, journalists and students who need market-level (not just state-average) Indian crop prices get a daily history without paying for a data feed or registering for a portal.
- It replaces hand-downloading Agmarknet pages: one script pulls the day, one table keeps the history, and re-running a day replaces that day instead of duplicating it.
- The repo already ships CSV snapshots, so the demo needed no network at all: 463,023 rows across 21 Jul to 24 Sep 2026 loaded and queried in seconds.
- The run also showed a trap worth knowing: a naive commodity-average day-on-day comparison put 'Wood' at -39.8%, while a like-for-like market comparison showed prices essentially flat. The post shows both.
What it is for
agri-commodity-tracker collects APMC mandi prices (state, district, market, commodity, variety, minimum, maximum and modal price in rupees per quintal) and a cross-country agriculture indicator panel from open sources, and writes them to data/agri.duckdb. Committed CSV snapshots let you explore the data without running any collector.
It also has collectors for the FCI depot directory, FAOSTAT producer prices and crop production, and NASA POWER monthly rainfall and temperature for 14 Indian states.
Who it is for
- Analysts and researchers studying Indian food price behaviour
- Students learning DuckDB on a real dataset
- Developers building commodity dashboards or alerts
- Trading decisions: this is not investment advice, and prices are mandi spot prices, not futures
- Complete daily coverage: some days are partial (see limits)
- Official statistics: the data is as served by the source portals, unaudited
What you need
| Requirement | Why | How to get it |
|---|---|---|
| Python 3 with duckdb, pycryptodome (and pytest for the tests) | requirements.txt lists duckdb and pycryptodome; the demo also used pandas-free DuckDB SQL | pip install -r requirements.txt |
| A terminal and git | To clone the repo and run commands | Terminal on macOS/Linux, or Git Bash/WSL on Windows |
| Internet, only for live collection | Collectors call data.gov.in, the World Bank and other public APIs | Not needed to query the committed CSV snapshots |
| Optional free data.gov.in API key | The built-in public demo key returns only 10 records per request, so a full daily pull is slow and can hit rate limits | Register at data.gov.in and set DATA_GOV_IN_KEY (see DATA_ACCESS.md) |
Get the code and an AI assistant
You do not have to be a programmer. Get the repository, give it to an AI assistant, and ask in plain language. Any of these routes works:
| Tool | What you need | Best for |
|---|---|---|
| GitHub | Free account to fork or star; git to clone, or the green Code button, then Download ZIP, or open in Codespaces | Getting the code and keeping up with updates |
| Claude Code | Claude subscription or API key; run claude inside the cloned folder | Letting the AI read and run the whole repo |
| Gemini CLI | Google account; run gemini inside the cloned folder | The same hands-on workflow with Gemini |
| Claude.ai, Gemini or ChatGPT in the browser | A chat account; upload README.md and the files you care about (or paste the output of git ls-files) | Understanding the project and asking questions without installing anything |
| Other tools (Copilot Chat, Cursor, Codex CLI) | Open the cloned folder in the tool and point it at README.md first | Editing and extending the code |
git clone https://github.com/herrrickshaw/agri-commodity-tracker.git
cd agri-commodity-tracker
claude # or: gemini
Install
- Clone and enter the repo
git clone https://github.com/herrrickshaw/agri-commodity-tracker.git cd agri-commodity-tracker - Create a virtual environment and install dependencies
python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt pytest
Quick start: commands and what you should see
Run the test suite
python -m pytest -q
10 passed in about 5 seconds.
Collect today's mandi prices (live; needs network, slow with the demo key)
python3 collectors/mandi_prices.py
Rows are written to data/agri.duckdb, replacing any earlier pull for the same date. Not run in this demo, see limits.
Explore the committed snapshot without any network, with DuckDB
python3 -c "import duckdb; print(duckdb.sql(\"select pulled_on, count(*) n, count(distinct commodity) commodities from read_csv_auto('data/mandi_prices_snapshot.csv') group by 1 order by 1 desc limit 3\"))"
A table of the latest pull dates with row and commodity counts.
Sample outputs
Real output from a run on the date shown in the sources line; trimmed to fit.
Coverage of the latest pulls in the committed snapshot
select pulled_on, count(*) n, count(distinct commodity) com, count(distinct state) st from m group by 1 order by 1 desc limit 4
pulled_on n com st
2026-09-24 13375 227 28
2026-09-23 11583 224 24
2026-09-21 11258 221 24
2026-09-20 7987 151 21
Whole file: 463,023 rows, pulled 2026-07-21 to 2026-09-24
Days differ a lot in size because the source feed fills through the day.
Naive day-on-day movers (average of all markets, 24 vs 23 Sep 2026, commodities with 30+ rows)
average modal_price per commodity per pull date, then percent change
commodity prev latest pct
Wood 1005 605 -39.8
Pineapple 4978 5779 +16.1
Tender Coconut 4209 4617 +9.7
Rice 4410 3998 -9.4
Water Melon 1840 1969 +7.0
Prices in rupees per quintal. These big moves are mostly a change in which markets reported that day.
Like-for-like check: same state, market, commodity and variety on both days (median percent change)
join the two days on state+market+commodity+variety
commodity same_markets prev latest median_pct
Soyabean 73 6040 5894 -0.4
Bajra 30 2400 2435 +0.2
Cotton 35 7967 7978 0.0
Garlic 105 17213 17177 0.0
The same data shows essentially flat prices once the market mix is held fixed. Always compare like with like.
Highest Tomato modal prices on 24 Sep 2026
select market, state, modal_price ... where commodity='Tomato' order by modal_price desc limit 3
Manubazar (Tripura) 6200
Damnagar APMC (Gujarat) 5000
Bolpur APMC (West Bengal) 4960
Rupees per quintal.
Ways to use it
| Method | How it works |
|---|---|
| Command line collectors | Run collectors/mandi_prices.py daily (the README suggests a 14:30 IST cron) to grow the price history. |
| SQL on the DuckDB file or CSV snapshots | Query data/agri.duckdb, or read the committed CSVs directly with DuckDB or pandas. |
| Colab notebook | The README links a Colab notebook for a no-install try-out. |
| Through an AI assistant | Open the cloned folder in Claude Code or Gemini CLI and ask for the query in plain English, or upload README.md and a CSV snapshot to a chat assistant. |
Sample prompts for your AI assistant
Copy, paste and edit the parts in [brackets].
- Explain this repository in plain English: which data sources does it collect and what tables does it create?
- Using data/mandi_prices_snapshot.csv, show the price trend of [commodity] in [state] by week, and flag days with unusually few rows.
- Compare day-on-day price change for [commodity] using only markets that reported on both days, and explain why that is safer than averaging.
- Read DATA_ACCESS.md and tell me what I need to do to run the daily collector with a personal data.gov.in key.
- Join the World Bank panel with mandi prices to show how India's agricultural value added share compares with [country], and note the limits of that comparison.
Limits and honest caveats
- Not investment advice. Mandi spot prices only.
- I did not run the live collector: collect.log in the repo ends with an HTTP 429 (Too Many Requests) from the source, and the public demo key serves 10 records per request, so a full pull is slow. Everything shown comes from the committed snapshot and the test suite.
- The snapshot ends on 24 Sep 2026 (pulled dates 21 Jul to 24 Sep 2026), so it is not current as of the post date.
- Per the README, days before 23 Jul 2026 are roughly 10-row stubs because of an earlier pagination bug and cannot be backfilled; the source only serves the current day.
- Daily row counts and state coverage vary, so averages across days mix different market sets. Use like-for-like comparisons.
- The data is as published by Agmarknet via data.gov.in and has not been audited.
Sources: the agri-commodity-tracker README, DATA_ACCESS.md, tests and data/mandi_prices_snapshot.csv; pytest and DuckDB queries run on 19 October 2026 on a fresh clone.
- v1.0.0 — 19 October 2026 — first published.
About this article: Researched, written and edited by Umashankar Triplicane Dwarakanathan, with AI research assistance; every figure is meant to trace to the primary source cited. See the Editorial Policy for how sourcing, AI use and corrections work.