What is the TOON format? Syntax and examples

TOON, short for Token-Oriented Object Notation, is a text format for the same data JSON holds. It exists for one reason: AI models charge by the token, and JSON spends many of them on quotes, braces and field names repeated in every record. TOON writes a list of similar records as a table, with the field names once at the top.

Every example below was produced by the reference library, @toon-format/toon 4.4.0, and decoded back to check it gives the same JSON. Token counts are from the o200k_base tokenizer, the one GPT-4o uses.

A first example

Three users in JSON:

{"users":[{"id":1,"name":"Ada","role":"admin"},{"id":2,"name":"Linus","role":"user"},{"id":3,"name":"Grace","role":"user"}]}

The same data in TOON:

users[3]{id,name,role}:
  1,Ada,admin
  2,Linus,user
  3,Grace,user

The header says users is a list of 3 records with the fields id, name and role. Each indented line below it is one record. That's 32 tokens, against 42 for the compact JSON and 85 for the same JSON indented with two spaces.

Paste your own JSON into JSON to TOON to see its TOON and both token counts.

The four forms

The encoder looks at the shape of each value and picks one of four ways to write it. You don't choose; the data does.

Objects: key and value per line

An object becomes key: value lines, and a nested object is indented under its key, much like YAML:

name: gigajson
version: 3
private: false
owner:
  login: ada
  id: 42

Lists of plain values: one line

A list of strings, numbers or booleans goes on one line after its length:

tags[3]: json,toon,ai
scores[3]: 3,7,9
empty: []

Lists of records with the same fields: a table

This is the form from the first example, and where the savings come from. Nested objects that every record shares fold into the header. Here total{amount,currency} means each row's second and third values are total.amount and total.currency:

orders[2]{id,total{amount,currency},paid}:
  o1,12.5,EUR,true
  o2,40,USD,false

An object whose values are all records of the same shape (a config per environment, users by ID) gets a keyed table. The colon in [2:] marks it, and each row starts with its key:

environments[2:]{region,replicas}:
  production: eu-central-1,6
  staging: eu-central-1,2

Everything else: a list with dashes

Records that don't fit one row, because they hold a list or their fields differ, are written one item per - , with every field name repeated:

products[2]:
  - sku: A1
    name: Mug
    tags[2]: kitchen,gift
  - sku: B2
    name: Lamp
    tags[1]: home

This form is the one that costs you. For these two products it takes 43 tokens, more than the 37 of compact JSON. A single list field in your records is enough to switch to it.

Quoting: what keeps the types

TOON leaves most strings unquoted, so it has to quote the ones that would read back as something else. This record has a zip-code-like string, a comma, the text "true", an empty string, a null, a number and a number in a string:

rows[1]{code,note,flag,empty,missing,n,s,spaced}:
  "007","red, large","true","",null,7,"7"," padded "

"007" and "7" are quoted because unquoted they'd decode as numbers, "true" because it would decode as a boolean, and "red, large" because of the comma. null and 7 aren't quoted: they're the real null and the real number. That's how TOON stays lossless where CSV turns everything into text.

What [N] and {fields} are for

The length and the field list also let a decoder check what it reads. If a model, or a cut-off copy, drops a row or adds a value, decoding fails with the line number instead of returning wrong data. From the reference decoder:

• A header saying [3] with two rows below it: Expected 3 tabular rows, but got 2.
• A row with three values under a two-field header: Expected 2 tabular row values, but got 3.

Tabs instead of commas

Commas are the default, but the encoder can use tabs or pipes. The choice is written into the header so the decoder knows it: with tabs, the header reads users[3<TAB>]{id<TAB>name<TAB>role}:, where <TAB> stands for a tab character. For the three users above, tabs bring the count from 32 to 30 tokens. Values that contain commas then need no quotes either.

When TOON is smaller, and when it isn't

On small single objects TOON saves nothing. The name/owner object above is 27 tokens in TOON and 25 as compact JSON, and a mixed list such as [1, "two", {"three": 3}, null] is 23 against 14.

Tables are where it wins. In our measurements on 600 records, TOON used 32 to 42% fewer tokens than compact JSON for records that fit one row each, and 4 to 25% more for records with a list field, which push it into the dash form. Minifying the JSON came first in every case: removing indentation alone saved 25 to 38%.

So the order is: minify, check whether your records fit a table, and convert only if the count actually drops. The JSON Token Counter shows both counts side by side and lets you drop fields or rows you don't need. Our guide on how to reduce JSON tokens for AI prompts goes through a full example.

Converting JSON to TOON

In the browser: paste JSON into JSON to TOON. It shows the TOON with its token count, checks the result by decoding it back, and nothing is uploaded.

In JavaScript, with the reference library:

import { encode, decode } from '@toon-format/toon'

const toon = encode({ users: [{ id: 1, name: 'Ada' }, { id: 2, name: 'Linus' }] })
// users[2]{id,name}:
//   1,Ada
//   2,Linus

const data = decode(toon)   // the same object again

The project also has a command-line tool; its README gives npx @toon-format/cli data.json --stats to print the TOON with the tokens saved. Ports to other languages are listed at toonformat.dev.

One thing to keep in mind: an API or a parser that expects JSON still needs JSON. TOON is for the text you put in a prompt. When a model has to answer in a structured way, JSON with a schema is still the safer choice.