Skip to content

TOON API Reference

toon

TOON encoder/decoder — zero dependencies, stdlib only, Python 3.10+.

Part of zerodep: https://github.com/Oaklight/zerodep Copyright (c) 2026 Peng Ding. MIT License.

Implements the TOON 4.4 specification. The official conformance fixtures (564 cases: 160 encode, 404 decode) are vendored in toon/fixtures/spec/ and gated by toon/test_toon_conformance.py.

The encoder and decoder follow the TOON reference implementation, toon-format/toon-python <https://github.com/toon-format/toon-python>_ (MIT License, commit 91413d14), adapted to this module's single-file, standard-library-only form and public API.

TOON is a compact, human-readable serialization format designed for LLM contexts, achieving 30-60% token reduction vs JSON. It combines YAML-like indentation with CSV-like tabular arrays.

Example::

encode({"name": "Alice", "age": 30})
# 'name: Alice

age: 30' decode("name: Alice age: 30") # {'name': 'Alice', 'age': 30}

EncodeOptions

Bases: TypedDict

Options for TOON encoding.

Attributes:

Name Type Description
indent int

Number of spaces per indentation level (default: 2).

delimiter str

Delimiter character for arrays (default: ",").

The pre-4.0 lengthMarker option was removed by the specification; passing it raises :class:ValueError so callers notice the change.

Source code in toon/toon.py
class EncodeOptions(TypedDict, total=False):
    """Options for TOON encoding.

    Attributes:
        indent: Number of spaces per indentation level (default: 2).
        delimiter: Delimiter character for arrays (default: ",").

    The pre-4.0 ``lengthMarker`` option was removed by the specification;
    passing it raises :class:`ValueError` so callers notice the change.
    """

    indent: int
    delimiter: str

DecodeOptions

Bases: TypedDict

Options for TOON decoding.

Attributes:

Name Type Description
indent int

Number of spaces per indentation level (default: 2).

strict bool

Enforce every check of specification §14 (default: True).

Source code in toon/toon.py
class DecodeOptions(TypedDict, total=False):
    """Options for TOON decoding.

    Attributes:
        indent: Number of spaces per indentation level (default: 2).
        strict: Enforce every check of specification §14 (default: True).
    """

    indent: int
    strict: bool

ToonDecodeError

Bases: Exception

Error raised when TOON decoding fails.

Unlike :class:json.JSONDecodeError, this is not a :class:ValueError subclass: except ValueError does not catch decode failures, so catch ToonDecodeError instead. This is the module's public contract.

Attributes:

Name Type Description
msg str

The unformatted error message.

line int | None

1-based line number of the offending line, or None when the error is not tied to a line.

source str | None

The offending line as written, or None.

Source code in toon/toon.py
class ToonDecodeError(Exception):
    """Error raised when TOON decoding fails.

    Unlike :class:`json.JSONDecodeError`, this is not a :class:`ValueError`
    subclass: ``except ValueError`` does not catch decode failures, so catch
    ``ToonDecodeError`` instead.  This is the module's public contract.

    Attributes:
        msg: The unformatted error message.
        line: 1-based line number of the offending line, or ``None`` when the
            error is not tied to a line.
        source: The offending line as written, or ``None``.
    """

    msg: str
    line: int | None
    source: str | None

    def __init__(
        self, msg: str, line: int | None = None, source: str | None = None
    ) -> None:
        self.msg = msg
        self.line = line
        self.source = source
        super().__init__(f"line {line}: {msg}" if line is not None else msg)

    def __reduce__(
        self,
    ) -> tuple[type[ToonDecodeError], tuple[str, int | None, str | None]]:
        return type(self), (self.msg, self.line, self.source)

encode(value, options=None)

Serialize a Python value to a TOON document.

Args:
    value: The value to encode. Mappings, sequences, strings, numbers,
        booleans and ``None`` map onto the JSON data model; dates, UUIDs,
        paths, enums and dataclasses are converted following §3.
    options: Optional encoding options (``indent``, ``delimiter``).
        Unknown keys raise :class:`TypeError` rather than being ignored.

Returns:
    The TOON document, without a trailing newline.

Raises:
    TypeError: A value or mapping key cannot be encoded, or an unknown
        option was supplied.
    ValueError: An option is invalid, a string contains an unpaired
        surrogate, a mapping has colliding keys, or the value is nested
        too deeply or contains a reference cycle.

Examples::

    encode({"name": "Alice", "age": 30})
    # 'name: Alice

age: 30' encode([1, 2, 3], EncodeOptions(delimiter=" ")) # '[3 ]: 1 2 3'

Source code in toon/toon.py
def encode(value: Any, options: EncodeOptions | None = None) -> str:
    """Serialize a Python value to a TOON document.

    Args:
        value: The value to encode. Mappings, sequences, strings, numbers,
            booleans and ``None`` map onto the JSON data model; dates, UUIDs,
            paths, enums and dataclasses are converted following §3.
        options: Optional encoding options (``indent``, ``delimiter``).
            Unknown keys raise :class:`TypeError` rather than being ignored.

    Returns:
        The TOON document, without a trailing newline.

    Raises:
        TypeError: A value or mapping key cannot be encoded, or an unknown
            option was supplied.
        ValueError: An option is invalid, a string contains an unpaired
            surrogate, a mapping has colliding keys, or the value is nested
            too deeply or contains a reference cycle.

    Examples::

        encode({"name": "Alice", "age": 30})
        # 'name: Alice\nage: 30'
        encode([1, 2, 3], EncodeOptions(delimiter="\t"))
        # '[3\t]: 1\t2\t3'
    """
    opts: Any = options or {}
    if opts.get("lengthMarker", False):
        raise ValueError(
            "lengthMarker was removed in TOON 4.4; re-encode without the option"
        )
    unknown = set(opts) - {"indent", "delimiter", "lengthMarker"}
    if unknown:
        names = ", ".join(sorted(repr(name) for name in unknown))
        raise TypeError(f"unknown encode option(s): {names}")
    indent = opts.get("indent", 2)
    delimiter = opts.get("delimiter", ",")
    _check_indent_size(indent)
    if delimiter not in DELIMITERS:
        raise ValueError(f"delimiter must be ',', '\\t', or '|', got {delimiter!r}")
    return _encode_document(
        value,
        indent_size=indent,
        delimiter=delimiter,
        default=None,
        sort_keys=False,
    )

decode(text, options=None)

Deserialize a TOON document to a Python value.

Args:
    text: TOON-formatted string, or UTF-8 encoded bytes.
    options: Optional decoding options (``indent``, ``strict``).
        Unknown keys raise :class:`TypeError` rather than being ignored.

Returns:
    Decoded Python value.

Raises:
    ToonDecodeError: If the input is malformed or is not valid UTF-8.
    TypeError: If ``text`` is not str/bytes/bytearray, an option is
        unknown, or ``indent`` is not an int.
    ValueError: If ``indent`` is less than one.

Examples::

    decode("name: Alice

age: 30") # {'name': 'Alice', 'age': 30} decode("[3]: 1,2,3") # [1, 2, 3]

Source code in toon/toon.py
def decode(text: str | bytes | bytearray, options: DecodeOptions | None = None) -> Any:
    """Deserialize a TOON document to a Python value.

    Args:
        text: TOON-formatted string, or UTF-8 encoded bytes.
        options: Optional decoding options (``indent``, ``strict``).
            Unknown keys raise :class:`TypeError` rather than being ignored.

    Returns:
        Decoded Python value.

    Raises:
        ToonDecodeError: If the input is malformed or is not valid UTF-8.
        TypeError: If ``text`` is not str/bytes/bytearray, an option is
            unknown, or ``indent`` is not an int.
        ValueError: If ``indent`` is less than one.

    Examples::

        decode("name: Alice\nage: 30")
        # {'name': 'Alice', 'age': 30}
        decode("[3]: 1,2,3")
        # [1, 2, 3]
    """
    if isinstance(text, (bytes, bytearray)):
        try:
            document = bytes(text).decode("utf-8")
        except UnicodeDecodeError as exc:
            raise ToonDecodeError(
                f"Invalid UTF-8: {exc.reason} at byte {exc.start}"
            ) from None
    elif isinstance(text, str):
        document = text
    else:
        raise TypeError(
            f"text must be str, bytes or bytearray, not {type(text).__name__}"
        )
    opts: Any = options or {}
    unknown = set(opts) - {"indent", "strict"}
    if unknown:
        names = ", ".join(sorted(repr(name) for name in unknown))
        raise TypeError(f"unknown decode option(s): {names}")
    indent = opts.get("indent", 2)
    strict = opts.get("strict", True)
    _check_indent_size(indent)
    return _decode_document(
        document,
        indent_size=indent,
        strict=strict,
        parse_int=None,
        parse_float=None,
        object_hook=None,
        object_pairs_hook=None,
    )

is_number_token(token)

Return whether an unquoted token decodes as a number (§4).

Source code in toon/toon.py
def is_number_token(token: str) -> bool:
    """Return whether an unquoted token decodes as a number (§4)."""
    return _NUMBER.fullmatch(token) is not None

quote(value)

Return value as a quoted token, escaped per §7.1.

Source code in toon/toon.py
def quote(value: str) -> str:
    """Return ``value`` as a quoted token, escaped per §7.1."""
    return '"' + _NEEDS_ESCAPE.sub(_escape_char, value) + '"'

encode_string(value, delimiter)

Encode a string value, quoting it only when §7.2 requires it.

Source code in toon/toon.py
def encode_string(value: str, delimiter: str) -> str:
    """Encode a string value, quoting it only when §7.2 requires it."""
    return quote(value) if _needs_quotes(value, delimiter) else value

encode_key(key)

Encode an object key, entry key, or field name (§7.3).

Source code in toon/toon.py
def encode_key(key: str) -> str:
    """Encode an object key, entry key, or field name (§7.3)."""
    return key if _UNQUOTED_KEY.fullmatch(key) else quote(key)

format_decimal(value)

Format a finite decimal in the canonical number form of §2.

Magnitudes in [1e-6, 1e21) use plain decimal notation without trailing zeros. Other magnitudes use JSON exponent notation with a lowercase e and an explicit exponent sign.

Source code in toon/toon.py
def format_decimal(value: Decimal) -> str:
    """Format a finite decimal in the canonical number form of §2.

    Magnitudes in ``[1e-6, 1e21)`` use plain decimal notation without
    trailing zeros. Other magnitudes use JSON exponent notation with a
    lowercase ``e`` and an explicit exponent sign.
    """
    if not value:
        return "0"
    magnitude = abs(value)
    if _CANONICAL_MIN <= magnitude < _CANONICAL_MAX:
        text = f"{value:f}"
        if "." in text:
            text = text.rstrip("0").rstrip(".")
        return text
    # Decimal.normalize() would round to the context precision, so strip
    # trailing zeros by hand to keep every significant digit.
    sign, digits, exponent = value.as_tuple()
    assert isinstance(exponent, int)
    mantissa = "".join(map(str, digits)).rstrip("0")
    exponent += len(digits) - 1
    if len(mantissa) > 1:
        mantissa = f"{mantissa[0]}.{mantissa[1:]}"
    exponent_sign = "+" if exponent >= 0 else "-"
    return f"{'-' if sign else ''}{mantissa}e{exponent_sign}{abs(exponent)}"

format_float(value)

Format a finite float in the canonical number form of §2.

repr yields the shortest digit string that round-trips, so the output decodes back to the same float. Integral floats of 2**53 and beyond are written with their exact digits instead: they decode as int, and the shortest digits would not equal the float (float(2**60) == 2**60 but 1152921504606847000 != 2**60).

Source code in toon/toon.py
def format_float(value: float) -> str:
    """Format a finite float in the canonical number form of §2.

    ``repr`` yields the shortest digit string that round-trips, so the output
    decodes back to the same float. Integral floats of 2**53 and beyond are
    written with their exact digits instead: they decode as ``int``, and the
    shortest digits would not equal the float (``float(2**60) == 2**60`` but
    ``1152921504606847000 != 2**60``).
    """
    if value.is_integer() and abs(value) < 1e21:
        return str(int(value))
    return format_decimal(Decimal(repr(value)))