datagouv-toolkit

datagouv-toolkit is a Python command-line toolbox for working with datasets published on data.gouv.fr.

The project focuses on the upstream part of a data-analysis workflow: discovering datasets, inspecting their metadata, evaluating and selecting resources, downloading them reproducibly, and handing them off cleanly to analysis tools such as Python, R, DuckDB, or standard Unix utilities.

It deliberately does not try to become another data-analysis environment. Its role is to make the path from an open-data catalogue to usable local resources explicit, inspectable, and reproducible.

Workflow

A typical workflow is:

catalogue exploration
        |
        v
dataset resolution
        |
        v
resource evaluation and filtering
        |
        v
URL or manifest generation
        |
        v
optional download
        |
        v
Python / R / DuckDB / jq / other tools

Unified command-line interface

The package exposes a single main command:

datagouv --help

Its current subcommands cover:

  • catalogue search;
  • dataset resolution;
  • resource filtering and selection;
  • metadata inspection;
  • resource statistics;
  • raw dataset inspection;
  • organisation information;
  • resource downloading;
  • download-and-audit workflows;
  • structural inspection of local CSV files;
  • exploration of local catalogue snapshots.

Reproducible resource selection

Interactive search is convenient for exploration, but ambiguous textual queries are undesirable in automated workflows.

datagouv-toolkit therefore supports explicit filters and non-interactive selection, while allowing stable dataset identifiers to be used when stronger reproducibility is required.

Selected resources can be exposed as plain URLs for Unix pipelines or as a compact JSON manifest containing identifiers, titles, formats, known sizes, and URLs.

For example, URL output can be connected directly to another Unix tool:

datagouv resources \
    "accidents corporels" \
    --producer "Ministère de l'intérieur" \
    --format csv \
    --first \
    --urls | xargs -n1 wget

Structural data audit

The toolkit can inspect a local CSV before deeper analysis:

datagouv inspect-csv data.csv

The audit reports information such as:

  • dimensions;
  • inferred data types;
  • missing values;
  • cardinalities;
  • candidate keys;
  • low-cardinality distributions;
  • duplicate rows;
  • a data preview.

The same information can be emitted as JSON for further processing.

This is intended as structural profiling rather than scientific or statistical interpretation.

Automated workflows

The workflow command combines dataset resolution, resource selection, download, and CSV auditing.

This provides a reproducible bridge between the remote data catalogue and a local analysis environment without embedding the downstream analysis itself inside the toolkit.

Catalogue snapshots

datagouv catalog-stats can explore a local snapshot of the data.gouv.fr catalogue.

This makes it possible to characterise datasets and resources before selecting individual datasets for more detailed work.

Architecture and quality

The project is packaged as a conventional Python package and currently targets Python 3.11 and newer.

Quality checks are centralised through:

make check

The verification workflow includes static analysis, type checking, security checks, tests with coverage, package construction, distribution validation, and an installed-CLI smoke test.

Reuse: BAAC 2005–2024

The analysis of twenty years of French road-accident data remains an important reuse of the toolkit, but it is now deliberately separated from the generic core.

The BAAC case demonstrates how the same data-access and inspection machinery can support a domain-specific reproducible analysis without making the toolkit itself BAAC-specific.

Source code

The project is developed publicly on GitHub:

github.com/fmaillar/datagouv-toolkit