The conformance suite.

Every check in the specifications is held to cases: 418 of them in 5 sections, each naming the diagnostics an implementation must report for one repository. They are published as data, for implementations that are not this one.

npm pnpm

npm install --save-dev @intentset/conformance-suite

pnpm add --save-dev @intentset/conformance-suite

The package holds the cases, the schemas and the worked example they are built from, and no code an implementation must use. A case's diagnostics are compared as a multiset of codes, errors and warnings alike; messages, locations and remediation are the implementation's own. The canonical copy is tests/ in the repository, where the reference implementation runs every case.

Sections

Section	Cases	Valid	Invalid	Codes
`core.json`	187	42	145	10
`vsa.json`	100	36	64	29
`evidence.json`	76	28	48	8
`export.json`	18	14	4	6
`publication.json`	37	12	25	7

Every code a case expects

The number of cases that expect each code at least once. The specifications define what each code means.

Code	Sections	Cases
`AMP001`	vsa	4
`AMP002`	vsa	2
`AMP004`	vsa	2
`AMP006`	vsa	2
`AMP007`	vsa	2
`AMP008`	vsa	2
`AMP009`	vsa	3
`AMP010`	vsa	1
`AMP011`	vsa	2
`CFG001`	core, vsa	7
`CFG002`	core	2
`COLUMNS_SINGLE`	core	1
`CORE001`	core, export, publication	88
`CORE002`	core, export	3
`CORE003`	core, export	26
`CORE004`	core	10
`CORE005`	core, publication	20
`CORE006`	core, vsa	5
`CORE007`	evidence, export	40
`CORE008`	publication	17
`CORE009`	core, vsa, export	14
`EVID001`	evidence	11
`EVID002`	evidence	2
`EVID003`	evidence	1
`EVID004`	evidence, export	2
`EVID005`	evidence	1
`EVID006`	evidence	1
`EVID007`	evidence	1
`PUB001`	publication	5
`PUB002`	publication	2
`PUB003`	publication	1
`PUB004`	publication	1
`REG001`	vsa	3
`TS001`	vsa	1
`TS002`	vsa	1
`TS003`	vsa	6
`TS004`	vsa	2
`TS006`	vsa	1
`TS007`	vsa	3
`TS008`	vsa	1
`VSA001`	vsa	2
`VSA002`	vsa	4
`VSA003`	vsa	10
`VSA004`	vsa	5
`VSA005`	vsa	6
`VSA006`	vsa	6
`VSA009`	vsa	10
`VSA013`	vsa	3
`VSA014`	vsa	1

Testing a consumer of the export

A tool that imports an export rather than producing one is checked against 23 envelopes for intentset/export/0.4: 10 it must accept and 13 it must reject. Every rejected envelope is an accepted one with one deliberate defect, and several are valid against the schema, so a schema validator alone is not a consumer. The export contract says what a consumer checks.