OLTREMATICA skills v2.1.0 GitHub
A Claude Code plugin

Your agent can't tell you when it's wrong.

It will say the work is done. It will say the tests pass. It will mean it. Ten skills that make the scaffolding auditable, the compliance evidence automatic and the team's own rules executable — and one hook that will not let it finish until it has proof.

claude — verification gate
$ composer test PASS 42 passed $ …agent edits app/Invoice.php… assistant: Done. Fixed the invoice bug. ⛔ BLOCKED — verification gate You claimed this work is done, but the tests are stale. A source file was modified after the last test run, so the last green result does not describe the code you are about to hand over. Run the test suite and report the actual output: composer test $ composer test
13
Skills
3
Tracks
9
Ecosystems
1111
Automated checks
891
Blind judgements
Install

Two commands.

This is the path that installs the verification gate along with the skills, the two slash commands and the two subagents. Requires Claude Code.

$ claude plugin marketplace add https://github.com/Oltrematica/oltrematica-skills.git
$ claude plugin install oltrematica-skills@oltrematica

Then, in any repository: say "audit our harness", or run /compliance-status, and read what comes back. Updating is marketplace update then plugin update, in that order — skipping the first updates nothing — and then restart Claude Code, because skills and hooks are read at session start.

Works with
PHP pest · phpunit Node vitest · jest Python pytest · tox · nox · uv Go go test Rust cargo Ruby rspec · rake Java/Kotlin maven · gradle .NET dotnet test Elixir mix

And, for the mandate gates, thirteen migration tools — Laravel, Rails, Django, Alembic, Ecto, EF Core, Knex, TypeORM, Doctrine, golang-migrate, Liquibase, and the two that have no rollback at all, which are reported as not applicable rather than as a gap.

Why

Three failures. None of them announces itself.

The harness fails through invisibility

Nothing tells you it broke.

A CLAUDE.md grows until the agent quietly stops obeying half of it. A skill's description drifts until it fires on everything, or on nothing. There is no error and no alert. The agent just gets worse — and the team concludes the model got worse.

Harness assurance →
Compliance fails through friction

Nobody regenerates an SBOM mid-sprint.

Not ignorance — tedium. The work is always less urgent than the feature, until it is suddenly a deadline. CRA vulnerability-reporting obligations begin 11 September 2026 for anyone shipping software into the EU.

The regulatory clock →
A mandate nobody verifies is a preference

Every team has rules it can't check.

Every endpoint has a Policy. Migrations are reversible. Every fix ships with a test. Checked, when at all, by whoever reviews the PR and happens to remember. They do not fail loudly; they erode.

The gates →
The catalogue

Thirteen skills, three tracks.

Each track carries one thesis and its own set of non-goals. A skill that needs a thesis widened to fit it belongs somewhere else — which is how a track stays a track instead of becoming a folder.

Track 01 · stack-agnostic

Harness

The harness fails through invisibility.
  • harness-auditEntry point. Inventories the ten surfaces a repo hands an agent and reports present / gap / not applicable.
  • claude-md-authoringWrites and repairs CLAUDE.md as policy, not procedure. Diagnoses "the agent keeps ignoring X".
  • subagent-authoringSkill, subagent, slash command or hook — decides which, then authors it.
  • harness-evalProves a skill's trigger fires. Or doesn't. Blind quorum, never a single judge.
  • model-routingWhy you keep hitting the usage limit, and which tier the task actually needed.
  • verification gateA Stop hook. Blocks a done-claim when a source file changed after the last passing test run.
Track 02 · EU regulatory

Compliance

Compliance fails through friction, not ignorance.
  • cra-evidenceSBOM, release-to-release diff, vulnerability scan and triage drafts, Annex I gap report, EAA/WCAG module.
  • gdpr-evidenceWhich columns are personal data, the Art. 6 and Art. 9 basis for each, retention — and the Art. 30 record rendered from that map rather than maintained beside it.
  • cra-incident-reportingThe Article 14 clock: 24-hour early warning, 72-hour notification, final report. Drafts each one; submits nothing.
  • adr-managementDrafts decision records proactively, whenever a decision with architectural weight is actually made.
Track 03 · stack-agnostic

Mandates

A mandate nobody verifies is a preference.
  • mandate-gatesThree checks over a real diff — a test with the change, a migration that rolls back, an authorization marker in the handler.
  • /compliance-statusDeterministic entry point for the compliance track. Inventories, reports, then routes — instead of hoping a router tells four similar skills apart.
  • /run-gatesRuns the gates on the current change, and reads the result — including the rule that "the gate could not see" is never a pass.
  • compliance-surveyor · migration-reviewerTwo subagents, for the read-heavy work that should never sit in the main conversation.
Mandate gates

A check may only block where its detection is exact.

Everything else warns. This is a concession to how these tools actually die: a gate that blocks on a heuristic gets switched off within a week, and it takes the checks that did work with it. Two of the three gates could have been written to look far more powerful than they are.

GateThe mandateSeverityWhy
tests-with-change Every change ships with a testEvery stack. Deleting a test never counts as shipping with one. block It is a fact about the diff, not an inference.
reversible-migrations Migrations roll backA down() that exists but is empty blocks too — it reads as reversible to a reviewer and rolls back nothing. block warn n/a Exact where the tool has an explicit reverse; a heuristic where the tool infers it; meaningless where the tool has no rollback at all.
authz-coverage Authenticated endpoints authorizeFinds handlers where nothing at all references authorization. Eight frameworks; elsewhere it says so. warn Whether a route is authenticated is a runtime property of a middleware stack. A gate that guessed it would be disabled within a week.
Not applicable is a real outcome

Flyway Community and Prisma have no rollback step. Reporting their migrations as irreversibility gaps would be a false finding — so the gate says the question does not apply here, and says why. A report that never says "not applicable" is a report that has started inventing findings to look useful.

A repo can lower a severity, never raise one

And a lowering is printed as a lowering. A team that switched a gate off must not be able to produce output that looks like a team that passed it. An advisory gate also stays advisory even if the config asks for block — otherwise a documented heuristic becomes a merge blocker, and then the tool gets the blame.

Evidence · the part most repos skip

We judged our own descriptions. One of them failed.

A skill's description: is the entire router surface — Claude sees it, and nothing else, when deciding whether to invoke a skill. So we test it the way the router meets it: three independent judges per prompt, each a fresh session started with --allowedTools "" — unable to call a tool rather than instructed not to — holding exactly one prompt, on one pinned model. 891 blind judgements so far, 120 of them in the round below.

"attackers got into our build server" FAIL — 0 of 3 → after the fix PASS — 3 of 3

It reached nothing. All three judges quoted the same clause back at us — the description said a severe incident "affects one", meaning a shipped product:

"A build server compromise is infrastructure security and does not establish that a shipped product or its vulnerabilities are involved."

A compromise of the build, signing or update path is the textbook CRA severe incident. The skill's body covered it in full. The router never sees the body. The fix was a few words of description — and the second finding of the round was stranger still: a 2-of-3 split on a phrase the description quotes verbatim, because an exclusion clause written to keep a neighbouring skill out was also excluding this skill's own second job.

Why three judges, not one

Triggering is probabilistic. One judge returns a single sample and calls it evidence — and on the row that was broken, a single judge would have returned a clean pass. A 2-of-3 split is recorded as FLAKY and never rounded up: the disagreement is the finding.

Two rows were left FLAKY on purpose

"how long do we keep this?" and "what is our reporting deadline?" split because the prompt has no antecedent, not because a clause reads two ways. Forcing 3-of-3 would mean claiming territory on the strength of a pronoun, and the cost would land on the neighbouring skills.

Why the compliance track exists now

The deadlines are the whole argument.

Nobody doubts an SBOM matters. Nobody regenerates one mid-sprint either, because nothing forces the moment — until something does.

2024-12-10
CRA enters into force
The clock starts on everything below.
2025-06-28
European Accessibility Act applies
EAA/WCAG obligations are live for in-scope products. cra-evidence carries the accessibility module.
2026-09-11
CRA Article 14 reporting obligations apply
An actively exploited vulnerability, or a severe incident: 24-hour early warning, 72-hour notification, final report. The two tracks anchor their final report on different events, which is the part teams get wrong — cra-incident-reporting refuses to compute a deadline from the wrong anchor and reports it as pending instead.
2027-12-11
Full CRA application
Annex I essential requirements, the technical file, the CE marking.

GDPR has no start date left to wait for. gdpr-evidence exists because the rule that a legal basis is decided before the column exists is unenforceable while nothing records what personal data a repository holds — so it records it, and then fails a build when a new personal-data column arrives without one.

How this repo tests itself

1111 automated checks, on every PR, on two operating systems.

This is the section a skeptical reader should check hardest, because it is also the one we are least entitled to get away with asserting. Every figure on this page is derived from its source and verified in CI — a script re-computes each one and fails the build when a published copy disagrees. It exists because we shipped three stale numbers in a single week.

$ scripts/run_suites.sh > results.tsv verify_gate.sh.test 137 test-command detection across 9 ecosystems stack_profile.py.test 78 stacks, frameworks, 13 migration tools check_published_facts.test 77 every case mutates a copy so a figure drifts record_activity.sh.test 59 what counts as an edit, what counts as a test run run_gates.py.test 94 five gates over real git repositories catalogue_eval.py.test 126 Mode 3: the permutation MUST, the arm diff a11y_audit.py.test 50 EAA: scope, routes, the statement's feedback path classify.py.test 47 AI Act detection — and it never emits a risk class release_trail.py.test 43 PLD: tag identity, the 10-year liability window check_description_drift.test 41 a description edited after the round that vouches inventory.sh.test 54 the harness surface inventory eval_run.py.test 54 the trigger-eval tooling itself scan_personal_data.py.test 49 Art. 9 detection, invalid bases, placeholders claims.py.test 46 1MB input, binary garbage, unicode — no crash install.sh.test 41 incl. running an installed script for real source_mutation.py.test 40 "did this command just write to a file" report_clock.py.test 35 two anchors, calendar months, naive timestamps verify_before_done.sh.test 19 the Stop hook's allow/block decisions state.sh.test 13 the hook's state store ———— 1111 across 19 suites $ scripts/check_published_facts.py --suite-results results.tsv OK: every published figure agrees with its source.
Evidence · the claim we were making about ourselves

We said ten to twelve skills was the ceiling. We had never measured it.

That sentence blocked three regulations we are exposed to — the AI Act, the PLD and a promoted EAA — and it was derived from nothing, in a repo whose CI fails when a published number drifts from its source. So we built the instrument and ran it: 600 selection judgements across 200 row-instances, four arms. Each judge sees the whole catalogue and picks one skill, or none, because a ceiling is not a property of any description — it is interference between them.

$ catalogue_eval.py --diff A.scored.json B.scored.json A → B our 3 new skills, alone 10 → 13 0 regressions 35 → 44 rows reaching their owner C → D our 3 new skills, in company 24 → 27 0 regressions 34 → 44 rows reaching their owner A → C the neighbours, alone 10 → 24 2 regressions 35 → 34 Adding three skills cost zero rows, twice. The catalogue routed BETTER.
The regression we found, and the edit that closed it

One row lost its owner: a draft description quoted a phrase in its own trigger list and ceded it to a neighbour two sentences later. The router reads the trigger list and never adjudicates the carve-out, so it won the prompt. Fixed by an edit, and the whole arm re-run — a description edit voids every earlier result for that skill.

The number nobody had published

A→C holds our catalogue fixed and adds only the neighbours. Both regressions are ordinary engineering work captured by a skill — "fix the invoice rounding bug" went to a debugging skill. Not a defect in any description: a boundary disagreement about who owns ordinary work. The usable conclusion is that a catalogue's interference is mostly not from its own growth, so measuring your skills in isolation measures the wrong arm.

A fixture that is deliberately broken

tests/harness/fixtures/bad-harness/ is a repository built to fail every surface the audit checks, on purpose, with five seeded defects. Its own README says outright: do not fix these — they are the test.

The ledger keeps its failures

Every trigger-validation round is recorded, including the discarded ones and the rounds that found a real gap. The clean final state is the least informative part of that file, so it is not the only part we keep.

Small N, and the ledger says so itself. Ten prompts per skill judged three ways is a smoke test for gross triggering failures, not a benchmark. It found two real defects in descriptions their author had read carefully several times, which is the entire argument for running it at all.

The contract

Claude drafts. Humans approve. Nothing is ever "fine".

Claude drafts, humans approve

No artifact — ADR, SBOM, gap report, harness audit, regulatory filing — is ever marked Accepted, Compliant or Validated by Claude on its own initiative. Everything ships Proposed or Draft until a named person says otherwise.

Evidence, never assertion

Every report states one of exactly three outcomes per item — present, gap, or not applicable — each with a pointer or a rationale. "Looks good" is not an output this plugin produces; where it appears, it is a bug.

Deadlines are drafted, never filed

Nothing here transmits to an authority, files, emails or posts to a reporting platform. It writes the text and computes the clock. Submission is an organizational act performed by a named human, and every draft says so on its first line.