Thinking global, living local

How to use agri-commodity-tracker: India mandi prices and an agriculture panel in one DuckDB file

October 19, 2026

Daily APMC mandi prices from Agmarknet (via data.gov.in) plus World Bank, FAOSTAT and NASA POWER agriculture data, all from open sources and stored in a local DuckDB file.

Open-Source Tools · How-To · Agri & trade trackers

How to use agri-commodity-tracker: India mandi prices and an agriculture panel in one DuckDB file

Skip to article content
Text Size

Revised · v1.0.0 · what changed

10tests passed in the demo
463,023mandi price rows in the committed snapshot
13,375rows in the fullest day (24 Sep 2026), 227 commodities, 28 states
9World Bank agriculture indicators

Code: github.com/herrrickshaw/agri-commodity-tracker

Watch the walkthrough

Watch on YouTube: youtu.be/0P80NVYwPCM

Why it is useful

  • Analysts, journalists and students who need market-level (not just state-average) Indian crop prices get a daily history without paying for a data feed or registering for a portal.
  • It replaces hand-downloading Agmarknet pages: one script pulls the day, one table keeps the history, and re-running a day replaces that day instead of duplicating it.
  • The repo already ships CSV snapshots, so the demo needed no network at all: 463,023 rows across 21 Jul to 24 Sep 2026 loaded and queried in seconds.
  • The run also showed a trap worth knowing: a naive commodity-average day-on-day comparison put 'Wood' at -39.8%, while a like-for-like market comparison showed prices essentially flat. The post shows both.

What it is for

agri-commodity-tracker collects APMC mandi prices (state, district, market, commodity, variety, minimum, maximum and modal price in rupees per quintal) and a cross-country agriculture indicator panel from open sources, and writes them to data/agri.duckdb. Committed CSV snapshots let you explore the data without running any collector.

It also has collectors for the FCI depot directory, FAOSTAT producer prices and crop production, and NASA POWER monthly rainfall and temperature for 14 Indian states.

Who it is for

  • Analysts and researchers studying Indian food price behaviour
  • Students learning DuckDB on a real dataset
  • Developers building commodity dashboards or alerts
Not for.
  • Trading decisions: this is not investment advice, and prices are mandi spot prices, not futures
  • Complete daily coverage: some days are partial (see limits)
  • Official statistics: the data is as served by the source portals, unaudited

What you need

RequirementWhyHow to get it
Python 3 with duckdb, pycryptodome (and pytest for the tests)requirements.txt lists duckdb and pycryptodome; the demo also used pandas-free DuckDB SQLpip install -r requirements.txt
A terminal and gitTo clone the repo and run commandsTerminal on macOS/Linux, or Git Bash/WSL on Windows
Internet, only for live collectionCollectors call data.gov.in, the World Bank and other public APIsNot needed to query the committed CSV snapshots
Optional free data.gov.in API keyThe built-in public demo key returns only 10 records per request, so a full daily pull is slow and can hit rate limitsRegister at data.gov.in and set DATA_GOV_IN_KEY (see DATA_ACCESS.md)

Get the code and an AI assistant

You do not have to be a programmer. Get the repository, give it to an AI assistant, and ask in plain language. Any of these routes works:

ToolWhat you needBest for
GitHubFree account to fork or star; git to clone, or the green Code button, then Download ZIP, or open in CodespacesGetting the code and keeping up with updates
Claude CodeClaude subscription or API key; run claude inside the cloned folderLetting the AI read and run the whole repo
Gemini CLIGoogle account; run gemini inside the cloned folderThe same hands-on workflow with Gemini
Claude.ai, Gemini or ChatGPT in the browserA chat account; upload README.md and the files you care about (or paste the output of git ls-files)Understanding the project and asking questions without installing anything
Other tools (Copilot Chat, Cursor, Codex CLI)Open the cloned folder in the tool and point it at README.md firstEditing and extending the code
git clone https://github.com/herrrickshaw/agri-commodity-tracker.git
cd agri-commodity-tracker
claude      # or: gemini

Install

  1. Clone and enter the repo
    git clone https://github.com/herrrickshaw/agri-commodity-tracker.git
    cd agri-commodity-tracker
  2. Create a virtual environment and install dependencies
    python3 -m venv .venv && source .venv/bin/activate
    pip install -r requirements.txt pytest

Quick start: commands and what you should see

Run the test suite

python -m pytest -q

10 passed in about 5 seconds.

Collect today's mandi prices (live; needs network, slow with the demo key)

python3 collectors/mandi_prices.py

Rows are written to data/agri.duckdb, replacing any earlier pull for the same date. Not run in this demo, see limits.

Explore the committed snapshot without any network, with DuckDB

python3 -c "import duckdb; print(duckdb.sql(\"select pulled_on, count(*) n, count(distinct commodity) commodities from read_csv_auto('data/mandi_prices_snapshot.csv') group by 1 order by 1 desc limit 3\"))"

A table of the latest pull dates with row and commodity counts.

Sample outputs

Real output from a run on the date shown in the sources line; trimmed to fit.

Coverage of the latest pulls in the committed snapshot

select pulled_on, count(*) n, count(distinct commodity) com, count(distinct state) st from m group by 1 order by 1 desc limit 4
pulled_on    n      com  st
2026-09-24   13375  227  28
2026-09-23   11583  224  24
2026-09-21   11258  221  24
2026-09-20    7987  151  21

Whole file: 463,023 rows, pulled 2026-07-21 to 2026-09-24

Days differ a lot in size because the source feed fills through the day.

Naive day-on-day movers (average of all markets, 24 vs 23 Sep 2026, commodities with 30+ rows)

average modal_price per commodity per pull date, then percent change
commodity       prev   latest  pct
Wood            1005    605    -39.8
Pineapple       4978   5779    +16.1
Tender Coconut  4209   4617     +9.7
Rice            4410   3998     -9.4
Water Melon     1840   1969     +7.0

Prices in rupees per quintal. These big moves are mostly a change in which markets reported that day.

Like-for-like check: same state, market, commodity and variety on both days (median percent change)

join the two days on state+market+commodity+variety
commodity   same_markets  prev   latest  median_pct
Soyabean    73            6040   5894    -0.4
Bajra       30            2400   2435    +0.2
Cotton      35            7967   7978     0.0
Garlic      105           17213  17177    0.0

The same data shows essentially flat prices once the market mix is held fixed. Always compare like with like.

Highest Tomato modal prices on 24 Sep 2026

select market, state, modal_price ... where commodity='Tomato' order by modal_price desc limit 3
Manubazar (Tripura)      6200
Damnagar APMC (Gujarat)  5000
Bolpur APMC (West Bengal) 4960

Rupees per quintal.

Ways to use it

MethodHow it works
Command line collectorsRun collectors/mandi_prices.py daily (the README suggests a 14:30 IST cron) to grow the price history.
SQL on the DuckDB file or CSV snapshotsQuery data/agri.duckdb, or read the committed CSVs directly with DuckDB or pandas.
Colab notebookThe README links a Colab notebook for a no-install try-out.
Through an AI assistantOpen the cloned folder in Claude Code or Gemini CLI and ask for the query in plain English, or upload README.md and a CSV snapshot to a chat assistant.

Sample prompts for your AI assistant

Copy, paste and edit the parts in [brackets].

  • Explain this repository in plain English: which data sources does it collect and what tables does it create?
  • Using data/mandi_prices_snapshot.csv, show the price trend of [commodity] in [state] by week, and flag days with unusually few rows.
  • Compare day-on-day price change for [commodity] using only markets that reported on both days, and explain why that is safer than averaging.
  • Read DATA_ACCESS.md and tell me what I need to do to run the daily collector with a personal data.gov.in key.
  • Join the World Bank panel with mandi prices to show how India's agricultural value added share compares with [country], and note the limits of that comparison.

Limits and honest caveats

  • Not investment advice. Mandi spot prices only.
  • I did not run the live collector: collect.log in the repo ends with an HTTP 429 (Too Many Requests) from the source, and the public demo key serves 10 records per request, so a full pull is slow. Everything shown comes from the committed snapshot and the test suite.
  • The snapshot ends on 24 Sep 2026 (pulled dates 21 Jul to 24 Sep 2026), so it is not current as of the post date.
  • Per the README, days before 23 Jul 2026 are roughly 10-row stubs because of an earlier pagination bug and cannot be backfilled; the source only serves the current day.
  • Daily row counts and state coverage vary, so averages across days mix different market sets. Use like-for-like comparisons.
  • The data is as published by Agmarknet via data.gov.in and has not been audited.

Sources: the agri-commodity-tracker README, DATA_ACCESS.md, tests and data/mandi_prices_snapshot.csv; pytest and DuckDB queries run on 19 October 2026 on a fresh clone.

Revision history.
  • v1.0.0 — 19 October 2026 — first published.

About this article: Researched, written and edited by Umashankar Triplicane Dwarakanathan, with AI research assistance; every figure is meant to trace to the primary source cited. See the Editorial Policy for how sourcing, AI use and corrections work.

Umashankar Triplicane Dwarakanathan
Contact Us
Umashankar Triplicane Dwarakanathan
Investment Promotion & Energy-Sector Leader · Chennai, Tamil Nadu, India
LinkedIn → GitHub → Email +91 78273 81696
How this site works

Data-led analysis of India's trade, currency and industrial policy. Every article is built from primary official sources, and every figure links back to the release, table or filing it came from.

Sources. DGCI&S TradeStat (imports/exports, HSN-wise) · PIB (government press releases, January 2017 to today, refreshed daily) · RBI (circulars, balance of payments) · MoSPI (CPI/WPI, IIP) · PARIVESH (environmental clearances) · CCIL (bond yields) · BIS (policy rates) · SEBI, NSE/BSE and SEC filings for company data.

Interpretation. Figures carry their vintage and retrieval date; estimates and press-reported numbers are labelled as such; where sources disagree, both are shown. Corrections are made visibly, never silently. Articles are written with AI assistance from the cited sources — AI-generated text can misstate figures even when working from real material, so verify any number that matters to a decision against the linked primary source.

footer

Browse all articles by topic

Every piece on this blog, grouped. Or read the full index.

Agriculture & FertilisersAI ToolsChemicalsClimate & CarbonEnergy & FuelsGas & LNGImport SubstitutionIndustrial PolicyMarkets & FinanceMobility & EVPrices & InflationTextilesTrade & Tariffs

Each topic is a live archive page that updates itself as pieces are labelled. It replaces a hand-kept list that had fallen 18 articles behind.