Formats

ExCSV is one language — types, units, stats, SQL — sitting on top of CSV. Four of the variants are just packaging: how that language travels with the rows. The fifth, JSON, is the same document in a different serialization. Pick by whether you can touch the CSV, how big the file is, whether you're shipping one table or a snapshot of several — and whether the consumer speaks CSV or JSON.

Which one

You have… Use Why
A table you produce Plain One artifact. Diff it, email it, pd.read_csv(..., skiprows=n).
A CSV you must not change Sidecar Original bytes stay. Excel / Tableau still open sales.csv.
A large table to send ZIP Smaller. Schema and aggregations sit in the ZIP comment — preview before extract.
A wide table, or several related ones Pack Read amount without loading note. Ship orders + customers as one file.
An API, a config file, or an LLM writing the table JSON Real types, one JSON Schema to validate against. Round-trips back to CSV.

Default

Plain — one file

Header, # lines, then ordinary CSV rows, all in sales.excsv (or sales.extsv if the delimiter is a tab). This is the canonical form: what you export, version, paste into a chat, or drop next to a notebook.

Every CSV reader that ignores # comments already reads the data. pandas: pd.read_csv("sales.excsv", skiprows=n), where n is the number of # lines. grep / awk / cut work on the rows as usual.

Why

  • One artifact. No pair to keep together, no archive to unpack.
  • Readable diffs. Git sees the schema change and the row change in the same file.
  • CLI-native. Meta lines start with #, so Unix tools skip them for free.
  • Pasteable. Drop a snippet into Slack or an LLM — types and totals come with it.

sales.excsv

Schema and rows in one place.

#!excsv version=0.6 delim=comma header=1 rows=2
#@grain: one row per order
#column name=id type=int role=id
#column name=customer type=string
#column name=amount type=decimal unit=USD role=measure
#%sum: ,,750.50
id,customer,amount
1,Acme Corp,500.00
2,Globex Inc,250.50

If the CSV already exists and you can't rewrite it — that's sidecar.

Legacy-friendly

Sidecar — annotate without touching the CSV

sales.csv stays a normal CSV — same bytes, same checksum, still opens in Excel. Next to it, sales.excsv holds only metadata and points at the data with reference=sales.csv. No rows in the sidecar. Tabs use sales.extsv + sales.tsv.

You open whichever you have. Opening the CSV, an ExCSV-aware tool looks for the sibling. Opening the sidecar, it follows reference=. Everyone else just reads the CSV.

Why

  • Don't touch the source. Vendor dumps, regulated extracts, lake files with a hash someone else checks.
  • Existing tools keep working. Tableau, Excel, a random COPY script — they see a CSV, not a new format.
  • Several views of the same data. Different sidecars can describe the same file for different audiences.
  • Stats describe the CSV, not the sidecar. Row count, checksum, #% totals are about the referenced file.

sales.excsv

Metadata only — no data rows.

#!excsv version=0.6 delim=comma header=1 rows=2 reference=sales.csv
#@source: sales_db.orders
#column name=id type=int
#column name=customer type=string
#column name=amount type=decimal unit=USD
#%sum: ,,750.50

sales.csv

Ordinary CSV. Unchanged.

id,customer,amount
1,Acme Corp,500.00
2,Globex Inc,250.50

If you control the export, a plain .excsv is enough.

Distribution

ZIP — compress, still peekable

Wrap a plain file as sales.excsv.zip (or .extsv.zip). Inside: the same inline ExCSV. Outside: a normal ZIP that anything can open. pandas will take it with pd.read_csv("sales.excsv.zip", skiprows=n).

The trick is the ZIP comment. It repeats the header, columns, and #% aggregations, so you can see schema and totals without extracting — useful when a folder is full of 200 MB exports and you need to know which one is Q1 sales.

Why

  • Smaller to send. Email, S3, Slack — same table, less bytes.
  • Preview before unzip. Schema, grain, sums in the archive comment.
  • Still a ZIP. No custom container. Password-protect it the usual way if you need to.
sales.excsv.zip
└── sales.excsv          ← ordinary plain file (header + rows)

Stay on plain if you want zcat | awk without an archive. Several tables is pack. A sidecar stays two files on disk — ZIP is for one plain file.

Columnar · multi-table

Pack — columns and several tables

A .excsv.pack.zip is a small warehouse dump. Instead of storing rows, it stores each column as its own file, and can hold several tables plus the foreign keys between them. Same ExCSV metadata — types, stats, SQL — per table.

Think: you exported orders, customers, and products, and you usually only need amount and date out of a 200-column fact table. A row ZIP makes you decompress everything. A pack lets you open one .col.

Why

  • Selective reads. Pull two columns from a wide table without paying for the rest.
  • Better compression. A file of one type (IDs, amounts, dates) squeezes harder than mixed rows.
  • A snapshot, not a sheet. Related tables and their FKs travel as one file.
  • Cheap extra column. Add a .col, rewrite that table's header. Don't rebuild the whole dump.
sales.excsv.pack.zip
├── _manifest.excsv          ← tables + foreign keys
├── orders/
│   ├── _header.excsv        ← schema + stats for this table
│   ├── 00-id.col
│   ├── 01-customer_id.col
│   └── 02-amount.col
├── customers/
│   └── ...
└── products/
    └── ...

Wide fact tables can split each column into row chunks (default 10k) so a single cell is a short scan, not a full-column read. Small lookup tables stay as one file per column.

Stay on plain or ZIP if you want to stream rows. Pack is in v0.6; the Go CLI doesn't open it yet.

JSON-native

JSON — the same document, without the CSV

sales.excsv.json is not an export or a summary — it's the same ExCSV document with the # stripped off. Every meta line becomes a key: the header is csv, #@ is meta, #column is columns, #% is aggregates, #$ is sql.

Use it where JSON is already the envelope: a REST payload, a config file, a TypeScript frontend — or an LLM that you'd rather constrain to a JSON Schema than trust to hand-write a #!excsv header. Serve it as application/excsv+json.

Why

  • Real types. required=1 is true, enum=a|b|c is an array, an empty cell is null.
  • Validation for free. One JSON Schema (draft 2020-12) checks the whole vocabulary — in your editor, in CI, or as an LLM response format.
  • Money survives. decimal and long cells are written as strings, so 500.00 never becomes a float.
  • Lossless both ways. CSV → JSON → CSV gives you the same file back. The text form stays canonical; only free-text ## comments drop.

sales.excsv.json

The plain example above, as JSON.

{
  "excsv": "0.6",
  "csv": { "delim": "comma", "header": true },
  "meta": { "grain": "one row per order" },
  "columns": [
    { "index": 0, "name": "id", "type": "int", "role": "id" },
    { "index": 1, "name": "customer", "type": "string" },
    { "index": 2, "name": "amount", "type": "decimal", "unit": "USD", "role": "measure" }
  ],
  "aggregates": { "sum": [null, null, "750.50"] },
  "rows": 2,
  "data": [
    [1, "Acme Corp", "500.00"],
    [2, "Globex Inc", "250.50"]
  ]
}

It's JSON, not CSV — no # comment trick, no awk on the rows, no ZIP-comment preview. Stay on plain when plain-CSV compatibility is the point. Try the browser converter.

More of the file language on the guide. Concrete files on Examples.