Parser safety
Every adapter in this repository is handed bytes somebody else produced. This page is the policy those parsers are held to. It is written for the formats Part 2 brings — AIXM and KML (XML), KMZ (a zip), GeoPackage (a zip and a SQLite database) — because those are the ones where a parser's default behaviour is dangerous, and it is enforced today on the thirty adapters that ship — since 2026-09-20/21 that roster includes the Part 2 formats it was written for: AIXM 5.1.1 and 5.2 and C2SIM (XML), GeoPackage (a SQLite database) and GeoJSON.
The convention is the specification's: MUST, SHOULD, MAY.
1. The five bounds, and their one home
ARCHITECTURE.md §3.5 puts the bounds inside capabilities, and the reason is that there must be
exactly one declaration of each: the conformance suite's resource-limits check and this policy have
to read the same number, and two declarations of one bound is the arrangement where a parser is
hardened against a figure the report does not print.
| bound | what it bounds |
|---|---|
max_input_bytes | the octets an adapter is handed for one payload |
max_depth | how deeply a document may nest |
max_objects | how many CDM objects one payload may produce |
max_decompressed_bytes | what an archive may expand to |
max_parse_seconds | wall time for one decode |
An adapter MUST declare every one of the five, either as a number or as absent with a reason.
Limitsrefuses a bareNone:absent_becausecarries the reason, keyed by field.
An adapter that declares a bound MUST record its basis.
declared_becausecarries, per field:kind(normativeorimplementation_cap), thesourcethe figure comes from, theenforced_atpoint, and thetestthat proves the refusal.
A bound MUST NOT be described as the format's normative maximum unless the specification actually says so. Twenty-five of the thirty adapters declare an implementation cap and say so in the enum and in the prose; the five ASTERIX categories declare the normative
LENfield.
Which bound runs before which allocation — size before the depth scan, the depth scan before
json.loads, the XML tree measured only after expat has built it — and what the conformance
worker and the hosting application each add on top is the subject of
docs/docs/security/deployment-envelope.mdx (audit finding F06). The boundary cases — a payload
at each bound and one octet or one level past it, in every octet form — are
tests/test_cdm_resource_envelope.py.
What is enforced today
max_input_bytes is declared and enforced on all thirty. max_depth is declared and
enforced on the five XML readers — tak and stanag4676 since 2026-09-16, and c2sim,
aixm511 and aixm52 since 2026-09-21, the last three inside expat's start-element handler
through secure_xml — on the five that decode JSON — adsb, ais, legion and pntmap
since 2026-09-17, geojson since 2026-09-20 — on geopackage (the parsed twin it also
accepts; the SQLite octets themselves have no nesting to measure, 2026-09-20), and on
tacticalapi (since 2026-10-06: the base class on the parsed twin, and the wire reader on the
decoded message before the object that would cross the bound is built), and on link16_gateway
(since 2026-10-10: the base class at 64 on the octets and on a dict, and inside the adapter the
gateway contract's own depth 32, refused as LIMIT_EXCEEDED), and on readsb_trace (since
2026-10-15: the base class at 64 on a dict and on JSON text, and inside the adapter on the text
its gzip member inflates to, before json.loads), and on nexrad2 (since 2026-10-15: the base
class at 16 on the parsed twin; the volume's octets have no nesting to measure), and on the three
station-text adapters ndbc_stdmet, coops_csv and airnow_hourly (since 2026-10-15: the base
class at 64 on a parsed twin, and for coops_csv on a JSON response's text before json.loads;
a file of lines has no nesting to measure), and on grib2 (since 2026-10-17: the base class at 16
on the parsed twin; a message's octets have no nesting to measure), cog (since 2026-10-17: the
base class at 16 on the parsed twin, and secure_xml at the same 16 inside expat's start-element
handler for a VRT index and for a GeoTIFF's GDAL_METADATA text) and iem_n0q (since 2026-10-17: the
base class at 16 on the parsed twin; a PNG's octets have no nesting to measure).
max_objects is declared and enforced on the five of the adapter expansion (geojson,
geopackage, c2sim, aixm511, aixm52), on tacticalapi (since 2026-10-06) and on nexrad2
(since 2026-10-15: 33, Table XVII-A's 32 elevation numbers and the site, held by refusing an
elevation number outside 1 to 32), on grib2 (since 2026-10-17: 16 fields a message, the
seventeenth Section 4 refused before it is read), on cog (since 2026-10-17: one object a file)
and on iem_n0q (since 2026-10-17: one Event a PNG), each before the objects it bounds are built, and declared absent with a reason on the other twenty.
max_decompressed_bytes is declared and enforced on readsb_trace (since 2026-10-15: 64 MiB,
held while zlib inflates the trace file's one gzip member, the first octet past it refused before
it is kept) and on nexrad2 (since 2026-10-15: 402,979,072 octets a volume, held while each LDM
record's bzip2 stream is inflated with max_length one octet past what the volume may still
produce, and each radial record also held to 2 MiB and the metadata record to its fixed 325,888)
and on grib2 (since 2026-10-17: 64 MiB of PNG rows a message, compared with the rows a PNG's
header implies before an octet is inflated, and each field's stream held to its own rows by
max_length) and on cog (since 2026-10-17: 1 MiB, one 512 x 512 tile of 32-bit samples, block 0's
decoded size compared with the bound before an octet is decompressed and LZW, Deflate and PackBits
each held to exactly the block's size) and on iem_n0q (since 2026-10-17: 80 MiB, above the 65,885,400
octets of rows the product's 12200 x 5400 grid inflates to; IHDR is held to that grid before an octet
is inflated, and the image data is inflated into a buffer of exactly its rows, the first octet past
them refused) and declared absent with a reason on the other twenty-five.
max_parse_seconds is declared absent with a reason on all thirty, and max_depth on the other nine; none of those absences is
enforced, and no page here says otherwise.
Enforcement of max_input_bytes is in the base class, at class-definition time:
Adapter.__init_subclass__ wraps each subclass's own to_cdm, so the bound is checked before any
adapter's decoder runs, and an adapter cannot forget it while remaining an Adapter. A payload
over the bound raises InputTooLarge, whose message names both numbers.
Enforcement of max_depth is in two places, and which one depends on where the danger is.
For JSON text and for a parsed document it is in the base class, beside the size bound: the
same wrapper calls enforce_depth_bound, which reads the nesting off the octets
(json_nesting_depth) or off the containers (container_depth) and raises InputTooDeep — a
ValueError naming both numbers — before any adapter's decoder runs. Before the decoder, because
on CPython 3.11 json.loads itself recurses once per container and raises RecursionError a
little under a thousand containers deep — 995 read from module level on 3.11.15 under the default
limit, fewer beneath the frames a test runner adds — and no interpreter in the matrix decodes an
arbitrary depth (3.12.13 raises it near ten thousand, 3.14.7 near a hundred and sixteen thousand;
the readings are of 2026-09-17, the day the 3.11 leg of the CI matrix first ran, and
adapter.InputTooDeep carries them). So a bound measured after the decode was a bound the decoder
could fail before. The text is read in one pass and decoded the way json.loads would decode it —
a byte-order mark or a UTF-16 document is measured, not walked past.
An adapter decoding JSON MUST declare
max_depth, and the text MUST be measured before the decoder sees it. Here the parser is what recurses — on CPython 3.11 — so a bound applied to the decoded document is applied too late.
For an XML tree it is in the two adapters themselves, immediately after ET.fromstring returns
and before anything walks the tree, and the reason it is there and not in the base class is that
only the adapter holds the tree. libexpat builds a tree of any depth without recursing — a
document nested fifty thousand elements deep parses — and everything that walks the tree afterwards
recurses once per level. So until 2026-09-16 a valid CoT event or NITS document nesting a thousand
empty elements — some 7 KB, against a 1 MiB max_input_bytes — made both adapters raise
RecursionError, which is one of the conformance suite's four crash classes and not a refusal.
Each XML adapter now measures the tree with a stack the moment it exists and refuses past 64
with its own ValueError naming both numbers.
The figure is 64 on all six and it is an implementation cap — each declared_because says
so: every CoT fixture in the package nests three elements deep, the deepest path AEDP-12's class
model admits is seven classes (eight elements at the deepest scalar), no JSON document shipped under
the four fixture directories, goldens included, nests more than seven containers (the deepest input
to_cdm is handed nests five), and sixty-four keeps every walker under two hundred
Python frames from any call depth beneath it. Each XML adapter also ships a malformed/ payload
nested one thousand deep, so check H records the refusal on every run.
max_input_bytesis on octets. Adict— the.parsed.jsontwin that ships beside a byte fixture — is not bounded by it, and this is a rule rather than an oversight: a caller holding a parsed document has already done the parse the bound exists to prevent. ADS-B's 14 wire octets serialise to over 400 as a decoded twin, so a bound applied to twins would refuse the format's own legal messages.max_depthis different in kind: the walk it protects happens after any parse, so the base class holds the parsed form of every adapter that declares the bound to the same number —stanag4676's included, although that reader never descends past what the class model names.
The thirty declarations
| adapter | max_input_bytes | kind | where the figure comes from |
|---|---|---|---|
cat021 | 65 535 | normative | ASTERIX LEN is a two-octet field over the whole data block |
cat023 | 65 535 | normative | as above |
cat034 | 65 535 | normative | as above |
cat048 | 65 535 | normative | as above |
cat062 | 65 535 | normative | as above |
adsb | 64 | implementation cap | the longest AVR line carrying a 112-bit squitter is 42 characters |
ais | 1 024 | implementation cap | NMEA 0183 caps a sentence at 82 characters and a message at 9 fragments — 738 — plus TAG blocks |
stanag4586 | 65 536 | implementation cap | one DLI message is bounded at 528 octets; the number of messages per datagram is not |
gmti | 1 048 576 | implementation cap | STANAG 4607's P2 is 32 bits, so its normative maximum is 4 GiB and is no safety bound |
stanag4609 | 1 048 576 | implementation cap | KLV BER long form can state a length no buffer could hold |
stanag4676 | 1 048 576 | implementation cap | XML; AEDP-12 states no maximum document size |
tak | 1 048 576 | implementation cap | XML; Cursor-on-Target states no maximum event size |
legion | 1 048 576 | implementation cap | JSON; the pinned API record states no maximum response size |
pntmap | 1 048 576 | implementation cap | JSON; no document in this tree states a maximum |
geojson | 8 388 608 | implementation cap | JSON; RFC 7946 states no maximum document size (geojson.GEOJSON_MAX_INPUT_BYTES) |
geopackage | 67 108 864 | implementation cap | a SQLite container; OGC 12-128r19 states no maximum file size (geopackage.GEOPACKAGE_MAX_INPUT_BYTES) |
c2sim | 4 194 304 | implementation cap | XML; SISO-STD-019-2020 states no maximum message size (c2sim.C2SIM_MAX_INPUT_BYTES) |
aixm511 | 16 777 216 | implementation cap | XML; AIXM 5.1.1 states no maximum document size (aixm511.AIXM511_MAX_INPUT_BYTES) |
aixm52 | 16 777 216 | implementation cap | XML; AIXM 5.2 states no maximum document size (aixm52.AIXM52_MAX_INPUT_BYTES) |
dis7 | 4 224 | implementation cap | one Entity State PDU of this subset is 144 octets plus at most 255 variable parameter records of 16 octets (dis7_codec.MAX_PDU_BYTES) |
tacticalapi | 4 195 328 | implementation cap | the contract states no maximum message size; 4 MiB is the default ceiling on a received message in the common gRPC implementations, plus 1 KiB for the Any envelope (tacticalapi_codec.MAX_INPUT_BYTES) |
link16_gateway | 1 048 576 | implementation cap | JSON; the gateway contract, an internal interface this repository defines, limits one report to 1 MiB of encoded UTF-8, and the adapter enforces it again for a dict and for a direct caller (link16_gateway.MAX_REPORT_BYTES, 2026-10-10) |
readsb_trace | 8 388 608 | implementation cap | gzip octets of one readsb trace file; no document states a maximum, and the design record's only figure is a secondary source's day mean of about 23 KB per file (readsb_trace.MAX_INPUT_BYTES, DECISIONS L-09, 2026-10-15) |
nexrad2 | 33 554 432 | implementation cap | one Archive II volume file; no ICD states a maximum, and the archive's listing for one site and day reads 3,601,535 to 8,908,516 octets a volume (nexrad2.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-15) |
ndbc_stdmet | 8 388 608 | implementation cap | one NDBC standard meteorological text file; no NDBC document states a maximum, and the design record's largest unit is a realtime file of the last 45 days at ten-minute rows (ndbc_stdmet.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-15) |
coops_csv | 8 388 608 | implementation cap | one CO-OPS data API response; the API bounds a request by its range (6-minute data one month, hourly one year) and states no size (coops_csv.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-15) |
airnow_hourly | 8 388 608 | implementation cap | one AirNow hourly file; no AirNow document states a maximum, and the design record's one HourlyData file holds 7878 lines for one hour (airnow_hourly.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-15) |
grib2 | 8 388 608 | implementation cap | one GRIB2 message; Section 0 states an eight-octet total length, so the format sets no usable maximum, and the largest message of the replay day across HRRR and RAP is 3,810,470 octets (grib2.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-17) |
cog | 268 435 456 | implementation cap | one GeoTIFF or VRT file; TIFF 6.0's 32-bit offsets reach 4 GiB, and the largest files the design record measured are a 3DEP 1/3 arc-second tile of 165,776,270 octets and coastal topobathy tiles of 150 to 217 MB (cog.MAX_INPUT_BYTES, DECISIONS U-07, 2026-10-17) |
iem_n0q | 8 388 608 | implementation cap | one IEM N0Q composite PNG; PNG states no file size beyond a chunk's 2^31 − 1 octets, and the archive's files are 1 to 4 MB, listed as 2M on 2026-09-11 (iem_n0q.MAX_INPUT_BYTES, DECISIONS L-09, 2026-10-17) |
2. XML
Part 2's AIXM and KML are XML, and so are two adapters that shipped before it — tak and
stanag4676, both through the standard library's xml.etree.ElementTree. Since 2026-09-21 three
Part 2 adapters parse XML as well — c2sim, aixm511 and aixm52 — through one shared reader,
secure_xml.parse (pyexpat driven directly: a DOCTYPE, an entity declaration, an external
entity and XInclude are refused in the handlers, the input is bounded before a parser exists, and
depth and element count are enforced in StartElementHandler; tests/test_cdm_secure_xml.py).
An XML parser MUST NOT resolve external entities.
An XML parser MUST NOT fetch an external DTD or schema, or make any other network request while parsing.
An XML document MUST be bounded by
max_input_bytesbefore it reaches the parser.
An adapter parsing XML MUST ship tests for a malformed document, an external entity reference and an entity-expansion bomb, and MUST record its parser choice as a limitation where that choice leaves a property to the runtime.
An adapter parsing XML MUST declare
max_depthand refuse a deeper document before anything walks the tree. The parser is not what recurses; the walk is, and a document insidemax_input_bytescan be deep enough to exhaust the interpreter's stack.
What the standard library actually does — measured, not recited
These are readings taken in round P5 on CPython 3.14.7 with libexpat 2.7.4, not claims from the documentation:
| case | xml.etree.ElementTree |
|---|---|
external entity (<!ENTITY xxe SYSTEM "file:///etc/passwd">) | refused — ParseError: undefined entity. The external subset is never read, so the entity was never declared. |
external DTD (<!DOCTYPE foo SYSTEM "http://…">) | not fetched — the document parses, the URL is never opened |
| small internal entity (3 levels, ×10) | expanded, silently, to 3 000 characters |
| entity bomb (6 levels, ×10) | refused — ParseError: limit on input amplification factor (from DTD and entities) breached |
| fifty thousand nested elements (read 2026-09-16) | parsed — the tree is built without recursing; what raised RecursionError was the adapter's own walk over it, which is what max_depth now bounds |
Two conclusions follow, and both matter more than the reassuring half.
The first is that the XXE case is stronger than "does not resolve": the reference is an error, not an empty string, so a document that depends on one cannot pass silently.
The second is that what stops an entity bomb is libexpat and not this package. That protection
was added in libexpat 2.4.0 and is on by default; a deployment linking an older expat loses it
without anything in this package changing. That is why it is recorded as a known limitation on
both XML adapters rather than presented as a property of the package, and why
tests/test_cdm_parser_safety.py reads pyexpat.version_info and fails if it is older than
2.4.0, rather than assuming the build it runs on.
defusedxml
defusedxml is not a dependency. It is deferred to Part 2, under a ruling that made the
deferral conditional on the four properties above being established by test rather than asserted —
which is what the table is. What it would add is the removal of the runtime dependency: it refuses
DTDs and entity declarations outright, in this package's own code, instead of relying on the expat
the deployment happens to link.
The declared version floor is Python 3.11. The readings above are from 3.14.7; the expat a 3.11 build links is a property of that build and not of the Python version, which is the whole reason the test reads it at runtime.
3. Archives — Part 2 (KMZ, GeoPackage)
No shipped adapter opens an archive today: a sweep of packages/cdm/synapse_cdm/adapters/ for
zipfile, gzip and zlib returns nothing. The rules are written now so that the first adapter
that needs one is held to them rather than setting the precedent.
An archive-reading adapter MUST refuse any member whose name is absolute, or contains
.., or resolves outside the extraction root — checked on the resolved path, not on the name as stored.
It MUST declare
max_decompressed_bytesand enforce it on the running total of what has been written, not on the header's declared size. A header is attacker-controlled.
It MUST enforce a decompression ratio bound as well as a total, and MUST decompress in bounded chunks rather than into one buffer.
It MUST bound the member count, and MUST NOT follow a symlink stored in an archive.
Added 2026-10-15. The first code in the package to open an archive is not an adapter: it is the
host command synapse-decode (decode_host, outside adapters/), which unwraps one gzip stream
or one named member of an uncompressed tar before an observation adapter sees the octets. It is
held to the four rules above as follows: nothing is extracted to disk, a member name that is
absolute, drive-qualified or carries a .. component is refused (the named one as a usage error,
any other in the archive as rejected data), and only a regular, non-sparse member is taken; the
running total of what is inflated or read is held to the hosted adapter's byte bound and, for gzip,
to 256 times the octets read, in steps of at most 65 536 octets; a tar is refused past 4 096
headers; and a link, a directory or a device member is never followed. A gzip stream is one member
or it is refused.
Added later on 2026-10-15 (after the checkpoint's hostile reading). The gzip input is fed to the
inflater in slices of 65 536 octets as well as taken out in steps of that size, so no step copies
more than one slice and the work stays linear in the file. And a tar header the standard library
cannot parse — a pax or GNU sparse record that is not the numbers it should be, an old GNU sparse
header cut short, a size past what an offset holds — is the corrupt-header refusal (exit 3)
whether tarfile raises its own TarError or a ValueError, IndexError or OverflowError
from below that net.
Added 2026-10-15, for nexrad2. A NEXRAD Level II volume is a sequence of compressed
records, not an archive of named members, so the member-name and member-count rules have no
subject in it; what it is held to is the total and the chunking. Every LDM record's bzip2 stream
is inflated by bz2.BZ2Decompressor with max_length one octet past the bound left, so the
running total is held to max_decompressed_bytes (402,979,072 octets) and each record to its own
bound (2 MiB a radial record, exactly 325,888 octets the metadata record) before an octet past
either is kept; the record's size comes from its control word and is checked against the file,
never trusted. No ratio bound is declared: the ICD's fixed, mostly empty metadata record
legitimately expands about 869 times, so a ratio a valid volume passes would admit every record
the per-record bound already admits, and no report figure supports another number.
Added 2026-10-15, for the three station-text adapters. ndbc_stdmet, coops_csv and
airnow_hourly open no container: a yearly NDBC file arrives as .txt.gz, and synapse-decode --container gzip unwraps it under the host's bounds above before the adapter is handed the text,
so each declares max_decompressed_bytes absent with that reason.
Added 2026-10-17, for grib2. A GRIB2 message opens no container, but a Template 5.41 field's
Section 7 is a PNG whose image data is a zlib stream. Its rows — Nj x (1 + Ni x octets per sample),
or one row of the data points under a bitmap — are compared with what max_decompressed_bytes
(64 MiB, MRMS's 49,003,500 octets of rows below it) leaves the message before any octet is inflated,
and the stream is inflated by zlib.decompressobj into a buffer of exactly the rows, at most 1 MiB
a step and with max_length never more than one octet past the rows, so an octet past them is
refused before it is kept. Every chunk's CRC is checked, only IHDR, IDAT and
IEND are read, and nothing may follow the stream or IEND. A .grib2.gz file is unwrapped by
synapse-decode --container gzip under the host's bounds above, then cut into its messages. The
file and its unwrapped octets are each held to grib2's max_input_bytes, 8 MiB, the bound of one
message, until a ruling sets one for files: a larger file is exit 3 before any message is cut.
The decode holds no field whole: each packing hands its codes on at most grib2.CHUNK (65,536) at
a time, so a 214-octet complex-packing message that describes 2^25 points is summarised without
2^25 values in memory; only a PNG's inflated rows, under the bound above, are held at once. Nor is
a record held per missing mark: complex packing first walks its codes, holding nothing, for the
present values its order of differencing needs, so a 1 MiB message of 2^23 alternating primary and
secondary marks is counted without a record per mark.
Added 2026-10-17, for cog. A GeoTIFF's blocks are compressed by TIFF's own codecs; cog reads
the header and every directory, checks that every block lies in the file, and decodes ONE block,
block 0 of the full-resolution image. Its decoded size — a whole tile, or the strip's rows — is
compared with max_decompressed_bytes (1 MiB, one 512 x 512 tile of 32-bit samples, the block of
the 3DEP and CUDEM COGs) before any octet is decompressed; Deflate is then inflated by
zlib.decompressobj with max_length one octet past the block, and LZW and PackBits are decoded
by adapters/raster_codec.py into exactly the block, so an octet past it is refused before it is
kept, and a stream that ends early or leaves octets after itself is refused too. The directory walk
is bounded before it reads: 64 directories, 128 entries a directory, 1 MiB for one value and 4 MiB
for every value together, because entries may point at the same octets; a chain that returns to a
directory already read is refused. BigTIFF and JPEG, LERC, ZSTD, LZMA, WebP and JPEG 2000
compression are refused by name. A VRT index and a GeoTIFF's GDAL_METADATA are XML and go through
secure_xml (no DTD, no entity, no external resource, the element and depth bounds); a VRT's
sources are listed and never opened, and a /vsicurl/ path, a URL, an s3:// or a DAP locator is
refused by name.
Added 2026-10-17, for iem_n0q. An IEM N0Q composite is one PNG of 12200 x 5400 palette
indices whose image data is one zlib stream. iem_n0q reads IHDR first and refuses any size, bit
depth or colour type other than the product's before a single octet is inflated, so the rows are
always exactly 65,885,400 octets, under the declared max_decompressed_bytes of 80 MiB; the shared
codec adapters/raster_codec.py then checks every chunk's CRC and order, refuses an unknown
critical chunk and an interlaced image by name, inflates the stream with zlib.decompressobj and
max_length into a buffer of exactly the rows, refusing the first octet past them and a stream
that ends early or leaves octets after itself, and undoes the five filter types in place. A tEXt
chunk is read under the PNG's keyword rules; every other ancillary chunk is kept as its octets and
read for nothing. Since 2026-10-18 (the 3.6.0 release) the ancillary chunks are bounded as well: at most
64 of them and at most 1 MiB of their data octets, the 65th chunk refused as too-many-ancillary-chunks
and the octet past the bound as ancillary-past-the-bound, each in the chunk walk before anything is
inflated (iem_n0q.ANCILLARY_BOUNDS_BASIS).
4. SQLite — Part 2 (GeoPackage)
Dated correction, 2026-09-20 (adapter expansion phase 2). The sentence below was true until
adapters/geopackage.py shipped; one adapter now opens a database, and how it meets each rule
follows the rules. The rules themselves are unchanged.
No shipped adapter opens a database: the same sweep for sqlite3 returns nothing. (superseded —
see the correction above)
A SQLite-reading adapter MUST open the file read-only through a URI —
file:…?mode=ro&immutable=1— and never by plain path.
It MUST call
enable_load_extension(False); an extension is arbitrary native code named by the file being read.
It MUST set a query timeout and a progress handler, and MUST bound the row count it will read.
It MUST NOT interpolate any value out of the file into SQL.
How geopackage meets them (adapters/geopackage.py, _Snapshot):
- Read-only, and stronger than
mode=ro: the adapter never has a path. The caller supplies the octets,sqlite3.Connection.deserializecopies them into an in-memory connection (sqlite3.connect(":memory:")is the onlyconnectin the module, held by a test),PRAGMA query_only = ONandPRAGMA trusted_schema = OFFare set before any schema-dependent statement, and an authorizer installed before the first read permits SELECT, READ onsqlite_master, the fourgpkg_*core tables and the SELECTED feature tables, four functions (count,length,max,typeof) and one pragma (table_xinfo), and denies everything else — ATTACH, every write, every view, every other table, every other function. A test hashes the caller's bytes before and after. WAL-mode octets are refused as WAL-dependent unless the caller asserts a checkpointed snapshot, in which case a COPY has its two header octets rewritten and every object records the assertion. enable_load_extension(False)is called explicitly on the connection (the module's default is already off; a build without loadable-extension support has no method and nothing to disable), andload_extensionis also denied by the authorizer's function allow-list.- Progress handler and bounds: a progress handler aborts any statement past
GEOPACKAGE_MAX_SQL_STEPS(20 000 000) virtual-machine steps — a step count rather than a clock, so the bound is deterministic;PRAGMA quick_check(1)runs once over the bounded input; the row count is bounded bycount(*)before any row is fetched (max_objects10 000), every column's largest value bymax(length(…))before any row is fetched (1 MiB), the coordinates across the package by a budget every WKB array debits before it is decoded (500 000), and the estimated output from the package block and the row count before any row is fetched (64 MiB).max_input_bytes(64 MiB) is the base class's, beforedeserialize. - No value from the file reaches SQL. The only identifiers in any statement are table and
column names read from
sqlite_masterandPRAGMA table_xinfo, validated (non-empty, no NUL, bounded length, no reserved prefix for a feature table) and double-quoted with"doubled; no caller-supplied SQL exists and no attribute value is ever interpolated. A generated or hidden column is refused rather than evaluated. Triggers never run because nothing is written; views are never selected from because the authorizer denies them.
5. Timeouts
A decode MUST be bounded in wall time once
max_parse_secondsis enforced, and an adapter MUST declare the bound or the reason it has none.
None of the twenty-three enforces one today, and each says so in its own absent_because. The
conformance suite's checks H and N hold the related property — that a malformed or
truncated payload is refused quickly rather than hanging — with a per-case deadline that is
enforced: since audit finding F03 (2026-09-19) each case runs in a spawned worker process
that is terminated and killed when the deadline passes, and the case is PARSER_TIMEOUT, so a
parser that goes quadratic or never returns on a bad input is caught, and the sweep finishes,
even where no wall-clock bound is declared. A memory or CPU envelope for that worker is
Linux-only and is refused where it cannot be enforced — docs/docs/security/deployment-envelope.mdx
§2 has the platform table.
6. What the conformance suite verifies
| check | what it asserts |
|---|---|
| H — malformed payloads | every malformed fixture is refused, without a crash class, inside an enforced deadline in a killable worker; a .json payload past the loader's depth bound is refused by the loader, and the layer is recorded |
| M — streaming | SKIP, declared inapplicable, for all thirty — never PASS: no adapter can declare a streaming capability, and the four properties one would consist of are printed as not implemented / not applicable |
| N — parser robustness | a truncated payload at many offsets is refused, inside the same enforced deadline |
| O — resource limits | one octet more than the declared max_input_bytes is refused; the payload at the bound is not refused by the size guard (tests/test_cdm_resource_envelope.py) |
O is required in CI for all thirty. It was a SKIP for all fourteen of the day's adapters until round P5, because no adapter declared a bound to check; it is now in the required set for every adapter in the sweep, which is what would notice a bound being quietly removed from a manifest.
7. The audit: every parser call in the thirty
Taken with
grep -rn "ET.fromstring\|xml\.\|zipfile\|sqlite3\|gzip\|zlib\|json.loads" packages/cdm/synapse_cdm/adapters/.
Line numbers are as at round P5 and move; the grep is the durable form.
| adapter | call | reached with | bounded before the call? |
|---|---|---|---|
tak | ET.fromstring (tak.py:637) | XML text | yes — 1 MiB, in the base class |
tak | json.loads (tak.py:592) | JSON text | yes — same bound; and max_depth 64, in the base class, since 2026-09-17 |
stanag4676 | ET.fromstring (stanag4676.py:748) | XML text | yes — 1 MiB |
stanag4676 | ET.fromstring (stanag4676.py:2447) | an egress fragment this adapter built | n/a — not attacker input |
stanag4676 | xml.dom.minidom.parseString (stanag4676.py:2473) | an egress document this adapter built, for pretty-printing | n/a — not attacker input |
legion | json.loads (legion.py:991) | JSON text | yes — 1 MiB; and max_depth 64, in the base class, since 2026-09-17 |
pntmap | json.loads (pntmap.py:375) | JSON text | yes — 1 MiB; and max_depth 64, in the base class, since 2026-09-17 |
ais | json.loads (ais.py:1144) | JSON text | yes — 1 024 octets; and max_depth 64, in the base class, since 2026-09-17 |
adsb | json.loads (adsb.py:1623) | JSON text | yes — 64 octets, which no text past max_depth 64 fits; the dict form is held to the depth bound in the base class, since 2026-09-17 |
geojson | json.loads (geojson.py:765, and geojson.py:787 in detect) | JSON text, with parse_constant refusing NaN/Infinity | yes — 8 MiB and max_depth 128, in the base class; max_objects 20 000 (the feature count) and a 500 000-position cap in the module, on the parsed document before any object is built (2026-09-20, adapter expansion phase 1) |
geopackage | sqlite3.connect(":memory:") + Connection.deserialize (geopackage.py, _Snapshot.__init__) | the octets of a .gpkg SQLite container, copied into memory | yes — 64 MiB in the base class before deserialize; then §4's snapshot discipline (query_only, trusted_schema off, authorizer, progress handler, quick_check), max_objects 10 000 rows by count(*), a 1 MiB per-value bound by max(length(…)), a 500 000-coordinate budget and a 64 MiB output estimate, all before the rows or the coordinates they bound are read (2026-09-20, adapter expansion phase 2). The WKB and GeoPackageBinary decoder is geopackage_codec.py, struct only, every count checked against the remaining octets before its array is read |
c2sim | secure_xml.parse → pyexpat (c2sim.py, _as_twin), json.loads (c2sim.py, _as_twin and detect, for a parsed twin) | XML octets; JSON text of a parsed twin | yes — 4 MiB in the base class before decode and again by secure_xml before a parser exists; max_depth 192 and 200 000 elements in expat's StartElementHandler, on the element that crosses the bound; a DOCTYPE, an entity, an external reference and XInclude refused there (§2's four properties, tests/test_cdm_secure_xml.py); max_objects 5 000 on the twin before any object is built; a JSON twin is held to max_depth by the base class (2026-09-21, adapter expansion phase 3) |
aixm511 | secure_xml.parse → pyexpat (aixm511.py, _as_twin), json.loads (aixm511.py, _as_twin, for a parsed twin) | XML octets; JSON text of a parsed twin | yes — 16 MiB in the base class before decode and again by secure_xml before a parser exists; max_depth 192 and 500 000 elements in expat's StartElementHandler, on the element that crosses the bound; a DOCTYPE, an entity, an external reference and XInclude refused there (§2's four properties, tests/test_cdm_secure_xml.py); max_objects 20 000 time slices on the twin before any object is built; a JSON twin is held to max_depth in the base class; no XLink is dereferenced — a reference is resolved in the document or in a caller's table, one hop, and a URL is classified and never fetched (2026-09-21, adapter expansion phase 4); the Digital NOTAM Event reading (phase 5) adds no parser — the same expat parse, the same bounds, the event:theEvent binding resolved the same one-hop way and never fetched; aixm_resolve parses nothing and reads no file, socket or clock |
aixm52 | secure_xml.parse → pyexpat and json.loads — both in aixm511.py's AixmAdapterBase._as_twin, the shared reader aixm52.py runs under its own profile (aixm52.py itself calls no parser) | XML octets; JSON text of a parsed twin | yes — the same enforcement as aixm511's row with this adapter's own declared figures (16 MiB, max_depth 192, 500 000 elements, 20 000 time slices, aixm52.XML_LIMITS), taken from Aixm52Adapter.PROFILE and Aixm52Adapter.metadata — the base class's wrapper is installed on Aixm52Adapter.to_cdm (it defines its own, so _bind_input_bound wraps it) and secure_xml.parse is handed the profile's limits; no XLink is dereferenced; no Digital NOTAM reading exists on 5.2, so no second parse path either (2026-09-21, adapter expansion phase 6) |
link16_gateway | json.loads (link16_gateway.py, parse_report, which detect also reaches) | the UTF-8 octets of one gateway report | yes — 1 MiB and max_depth 64 in the base class; then, before the call, a byte order mark and invalid UTF-8 are refused and the contract's depth 32 is read off the characters; the decoder refuses duplicate keys, non-finite literals and numbers no double holds, and a walk bounds the parsed tree at 10 000 nodes (added 2026-10-10) |
readsb_trace | zlib.decompressobj (readsb_trace.py, inflate), json.loads (readsb_trace.py, parse_document) | the gzip octets of one trace file; the text that member inflates to | yes — 8 MiB in the base class before the module runs; one gzip member only, its CRC and length checked by zlib, a truncated member, a second member and trailing octets refused; inflation held to max_decompressed_bytes 64 MiB, the output of each step limited to the octets still allowed; then strict UTF-8 and max_depth 64 read off the characters before the decoder, which refuses duplicate keys and NaN/Infinity; an integer no double holds and a string holding a lone surrogate are refused by name; a dict is held to the depth bound by the base class (added 2026-10-15); a trace of more than 100 000 points (MAX_POINTS) is refused too-many-points before any point is walked or any model built (added later the same day) |
nexrad2 | bz2.BZ2Decompressor (nexrad2.py, _inflate), struct (nexrad2.py, decode_volume, _radial) | the octets of one Archive II volume; each LDM record's bzip2 stream | yes — 32 MiB in the base class before the module runs; each record's control word (its absolute value) checked against the octets left; each stream must open with BZh, end at its end-of-stream marker with nothing after it inside its size, and pass its CRCs; inflation held to the volume's 402,979,072 octets and the record's own bound (2 MiB, or exactly 325,888 for the metadata record) with max_length; every message and data block is framed against the record and the radial before it is read, and every range ICD 2620002Y Tables XVII-A, -B, -E, -F and -H state is checked (Table XVII-I's data ranges are not rules, narrowed 2026-10-15); the gate codes are counted, never walked into objects; a twin is decoded from its own wire_hex and held to the depth bound by the base class (added 2026-10-15) |
ndbc_stdmet | str.split on decoded lines (ndbc_stdmet.py, parse_file; observation_contracts.text_lines) | the octets of one stdmet text file | yes — 8 MiB in the base class before the module runs; strict UTF-8 without a byte-order mark, no control character or tab, one line-end style, no blank line and a line end after the last line (a file cut inside a row is refused, never read as a shorter number); every header name read against a closed vocabulary, every row held to the header's width and every token to its column's grammar, and a number no binary double holds refused by the adapter's own code; a gzip yearly file is unwrapped by synapse-decode, never here; a twin is held to the depth bound by the base class (added 2026-10-15) |
coops_csv | json.loads (coops_csv.py, parse_response), csv.reader with strict=True (the same, one line at a time) | the octets of one data API response: a JSON object or a CSV header and rows | yes — 8 MiB and max_depth 64 in the base class before the module runs; JSON: strict UTF-8, duplicate keys, NaN/Infinity and an integer no double holds (more than 309 digits, before int() reads it) refused by the decoder's hooks, closed key sets, every value a string a text file could hold; CSV: the lines text_lines admits, the header compared whole with the request's product's, one CSV record a line, every field to its column's grammar, a value or a flag no double holds refused by the adapter's own code (added 2026-10-15) |
airnow_hourly | str.split on the pipe character and csv.reader with strict=True (airnow_hourly.py, parse_file) | the octets of one HourlyData or HourlyAQObs file | yes — 8 MiB in the base class before the module runs; the lines text_lines admits; nine fields a HourlyData line, the 34-name HourlyAQObs header compared whole and one CSV record a line; every field to its grammar, and a number no double holds refused by the adapter's own code; a twin is held to the depth bound by the base class (added 2026-10-15) |
grib2 | zlib.decompressobj (grib2.py, _png), zlib.crc32 (the same), struct (grib2.py, decode_message and the section readers) | the octets of one GRIB2 message; a Template 5.41 field's PNG and its zlib stream | yes — 8 MiB in the base class before the module runs; Section 0's total length must be the payload's and the message must end with 7777; every section's length is checked against the octets before Section 8 and every template's against its fixed layout; the grid points of a message are held to 2^25 and its fields to 16 before any value is read; a PNG's chunks are CRC-checked, only IHDR, IDAT and IEND are read, its rows are compared with what max_decompressed_bytes 64 MiB leaves before inflating and inflated into a buffer of exactly those rows, at most 1 MiB a step, with max_length never more than one octet past them; no field's values are held whole, each packing handing on at most 65,536 codes at a time; a twin is held to the depth bound by the base class (added 2026-10-17) |
cog, with its codec raster_codec | struct and zlib.decompressobj (raster_codec.py, read_tiff, block_layout, decode_block, inflate), the codec's own LZW and PackBits decoders (the same), secure_xml.parse (cog.py, decode_vrt, _gdal_metadata) | the octets of one GeoTIFF or VRT file; block 0's compressed octets; the VRT's and GDAL_METADATA's XML | yes — 256 MiB in the base class before the module runs; BigTIFF refused; 64 directories, 128 entries, 1 MiB a value and 4 MiB of values in all, each checked before it is read; every value and every block inside the file; one block decoded, its size compared with 1 MiB before decompression and every decoder held to exactly the block; XML through secure_xml with its bounds, a VRT source never opened and a remote one refused by name; a twin is held to the depth bound by the base class (added 2026-10-17) |
iem_n0q, with the codec's PNG reader | struct, zlib.crc32 (iem_n0q.py, _product_header), zlib.decompressobj and zlib.crc32 (raster_codec.py, read_png, _png_inflate) | the octets of one N0Q PNG; its IHDR; its image data's zlib stream; its tEXt chunks | yes — 8 MiB in the base class before the module runs; IHDR held to 12200 x 5400 x 8-bit indexed before anything is inflated; every chunk's CRC and order checked; the stream inflated into exactly its 65,885,400 octets of rows, at most 1 MiB a step, the first octet past them refused; an index past the palette refused; a twin is held to the depth bound by the base class (added 2026-10-17) |
| the rest of the roster | none | — | the ASTERIX, GMTI and KLV families decode octets with their own codecs and call no general-purpose parser |
| all fourteen | zipfile, sqlite3, gzip, zlib | — | no hit anywhere — a reading from before 2026-09-20; sqlite3 now hits in geopackage and its row is above |
Two rows are the ones worth reading twice. The two stanag4676 egress calls are not inbound
parsing: the adapter re-parses a fragment it has just serialised itself, so the input is its own
output. And the adapters with no row of their own are not unbounded — they are bounded by the same
max_input_bytes, and they simply reach no parser this audit's grep can find, which is a property
of hand-written binary codecs rather than a gap in the audit.
The DIS 7 codec and the host loader
dis7_codec is in adapters/ and reaches no parser the grep above finds; dis7_host is outside adapters/.
| module | call | reached with | bounded before the call? |
|---|---|---|---|
dis7_codec | no general-purpose parser; decode_pdu reads fixed offsets | the octets of one PDU | yes — 4 224 octets (MAX_PDU_BYTES), by the adapter's coded guard before the SDK's wrapper and again in decode_pdu; a parsed envelope is held to 16 levels and to acyclicity by the adapter before any field is read |
dis7_host | parse_json_text → the standard-library JSON decoder | the bytes of an envelope or replay document read by synapse-dis7 | yes — 65 536 bytes, then strict UTF-8 (a byte-order mark and NUL refused), then a lexical scan refusing more than 16 levels, all before the decoder; duplicate keys, NaN/Infinity and numbers that overflow are refused |
The observation host
decode_host is outside adapters/ (added 2026-10-15); it reaches the standard library's archive
readers and a small JSON loader before any adapter is called.
| module | call | reached with | bounded before the call? |
|---|---|---|---|
decode_host | zlib.decompressobj (gunzip), tarfile.open(mode="r:") (untar), json.loads (manifest row and station table) | the octets of the one --input file; the text of a --manifest-row or --station-table file | yes — --input is read to at most one octet past the hosted adapter's max_input_bytes and refused past it before it is digested or unwrapped; a remote locator is refused by name before anything is stat'ed; inflation and member reads run in 65 536-octet steps, the running total held to the adapter's bound and gzip to 256 times the octets read; 4 096 tar headers; the two context files to 1 MiB, strict UTF-8 without a byte-order mark and 16 levels before the decoder, which refuses repeated keys and NaN/Infinity |
The TacticalAPI wire reader
tacticalapi_codec is in adapters/ and reaches no parser the grep above finds: it reads the
protobuf wire format itself, against the contract's embedded field table (added 2026-10-06, when
tacticalapi landed).
| module | call | reached with | bounded before the call? |
|---|---|---|---|
tacticalapi_codec | no general-purpose parser; decode walks tags, varints and length-delimited fields itself, every declared length checked against the octets that remain before it is read | the octets of one google.protobuf.Any wrapping one blue-force response, or the parsed twin of one | yes — 4 195 328 octets in the base class before the reader runs and again in decode; max_depth 64, the base class on a twin and the reader on the object that would cross it; max_objects 10 000 blue forces counted before any element is decoded; 65 536 carried unknown fields and 16 777 216 characters of carried message-level data, refused before any object is built; a singular field twice, two members of one oneof, a named field with another wire type and an over-long varint are refused, never repaired |