Critical Gate: Repository-Aware Diff Integrity for AI-Generated Code

Neural Highlight Active

A deep technical guide to Critical Gate 2.3, a deterministic-first integrity gate that checks whether AI-generated diffs are scoped, justified, safe, calibrated, and repairable before merge.

AI coding agents are now good enough that the interesting question is no longer whether they can edit a repository.

They can.

They can search code, follow instructions, patch files, run tests, interpret failures, write documentation, and produce pull requests that look convincing. That is already changing software development.

The harder question is this:

When the agent says it is done, how do you know the final diff is acceptable for this repository?

That question is what led me to build Critical Gate, a repository-aware diff integrity gate for AI-assisted development.

Critical Gate is not a generic AI code reviewer. It is not trying to read the whole repository and produce broad comments about style, architecture, or possible improvements. The product is narrower and more enforceable:

It checks whether an agent-produced diff stayed inside the task, respected repository contracts, avoided suspicious side effects, and produced evidence that another developer or agent can repair.

The feature release this article now reflects is 2.3.0, with 2.3.1 as a follow-up package metadata patch. That matters because the project has moved from an initial detector bundle into a more realistic pre-stable tool: it now has agent onboarding, policy-as-code, public API snapshots, local hooks, framework-aware heuristics, explicit quality docs, evaluation artifacts, a dogfood evidence trail, and stricter boundaries around what the tool should and should not claim.

The core idea has not changed:

Do not trust an AI-generated diff just because the agent says it is done.
Check the evidence.

The problem is diff integrity, not code review

Most teams already have validation around code changes:

tests
linters
type checks
formatters
code review
branch protection
security scanning
CI workflows

Those tools matter, but they do not fully answer the AI-agent problem.

An AI-generated change can compile and still be dangerous. It can pass tests and still be wrong. It can solve the requested bug while quietly widening the blast radius.

Common examples:

A small UI task rewrites unrelated layout files.
A validation fix adds a production dependency.
A refactor removes or weakens assertions while keeping tests green.
A helper is created even though the repository already has one.
A public export changes without a changelog, changeset, migration note, or API snapshot update.
A runtime pin such as .node-version changes during an unrelated UI task.
A local path, internal host, or token-like value appears in the diff.

These are not always syntax failures. They are diff integrity failures.

That distinction is the whole product.

A generic AI reviewer asks:

What comments might be useful on this pull request?

Critical Gate asks:

Does this diff satisfy the task without taking unsafe or unjustified liberties?

That smaller question is much easier to enforce. It starts from the task, the git diff, and minimal repository context. It does not need to become a repo-wide LLM scanner. It does not need to review every line like a human reviewer. It should stay quiet until it has evidence.

What Critical Gate is in 2.3

Critical Gate is a TypeScript/Node.js CLI-first tool with multiple surfaces around one analysis core.

The current support is strongest for TypeScript and JavaScript repositories with Git history, package metadata, common source/test/config patterns, and Node-based workflows. It is useful today as a pre-review integrity gate for agent-produced TS/JS diffs, especially in repositories where agents are already making real changes.

It is also intentionally pre-stable. That is an important correction. The current official CLI distribution path is source-based: clone the repository, install dependencies, build, and run node dist/cli.js. The docs explicitly say not to assume an npm registry package, global install, or prebuilt GitHub Action artifact is available until the release policy changes.

The current surfaces are:

CLI: canonical local and scripted interface.
GitHub Action: source-based composite action or packaged action artifact after smoke testing.
Codex hook: repair-oriented Stop hook around the CLI.
VS Code extension: Activity Bar dashboard, Problems diagnostics, Analysis tree, output report, evidence navigation, and repair-copy actions.
Agent onboarding: init-agent writes a managed Critical Gate section into AGENTS.md while preserving existing instructions.

The CLI remains the source of truth. CI, Codex, and editor surfaces consume its output instead of reimplementing detector logic.

Why prompts are not controls

Prompts can guide an agent, but they cannot prove the agent followed the instruction.

You can tell an agent:

Only change the signup form and its tests.

The agent might obey. It might also touch a shared auth helper, update a lockfile, rewrite a styling module, and adjust a test expectation to make everything pass.

That does not mean the model is useless. It means a prompt is not a contract.

The contract is the final diff.

Critical Gate treats the diff as the artifact that matters. It asks:

What changed?
Which file roles changed?
How much churn happened?
Does the changed file set match the task?
Did the diff actually implement the requested category of work?
Were tests strengthened or weakened?
Did dependencies change?
Did public API surface change?
Did runtime or tooling config change?
Did the diff cross package boundaries?
Did it reinvent something the repository already has?
Is the finding strong enough to block, or should it stay observational?

That last question is central to 2.x. Critical Gate treats confidence, severity, and rollout mode as part of the decision contract, not just display metadata.

Deterministic-first by design

The most useful version of this tool is mostly deterministic.

Critical Gate starts with:

git diff parsing
changed file classification
synthetic hunks for untracked files
package manifest inspection
lockfile signals
test assertion heuristics
public export snapshots and entrypoint policy
config file classification
runtime/tooling pin classification
monorepo ownership context
repository history and co-change patterns
existing solution indexes
framework-specific companion expectations
evaluation cases and detector quality boundaries

Only after compact deterministic evidence exists should an LLM be used, and even then it should be optional. A model can explain ambiguous findings, summarize repair guidance, or improve prioritization, but it should not be the detector of record.

The hierarchy is:

diff evidence first
repository context second
optional model interpretation last

The 2.3 line hardens this boundary further. The LLM artifact is explicitly compact and redacted: task text, changed file metadata, diff totals, summary, and top findings. It excludes diff hunks, full file contents, repository-wide source trees, dependency trees, token indexes, history indexes, caches, and learned policy internals. Prompt-visible strings go through redaction for provider tokens, secret-like assignments, absolute local paths, internal URLs, localhost URLs, and email addresses.

That is the difference between using a model as an interpreter and using a model as an unbounded reviewer.

What changed after 2.1

The 2.1 version introduced the trust and calibration foundation: clean diff certificates, better task keyword handling, confidence counts, and less noisy companion findings.

The 2.2 and 2.3 releases make the tool more honest and more practical.

2.2: dogfood-backed detector improvements

The 2.2.0 release was based on an mv-ft real-repository dogfood pass against an Astro/TypeScript/SCSS project.

It added:

docs/dogfood-mv-ft-2026-06-19.md
eval cases for broad component rewrites
eval cases for skipped tests in newly added files
eval cases for runtime config drift
eval cases for framework contract export removals

It changed detector behavior in useful ways:

Source-like rewrites can be detected even when role metadata is stale or unknown.
Untracked file content is exposed as synthetic added-file hunks so detectors can inspect new files before staging.
Runtime and tooling pin files such as .node-version, .nvmrc, .tool-versions, .npmrc, and .pnpmrc are classified as configuration.
Task wording such as “without changing config” is treated as a config-change prohibition, not permission.
Framework contract exports such as Astro collections in content.config.ts can be protected before a repository commits a public API snapshot.

The dogfood run started with strong precision but weak recall: 10 labeled scenarios, 100% scenario precision, and 42.9% scenario recall. Four meaningful misses were turned into deterministic eval cases. After those fixes, the case harness baseline for those 10 dogfood-derived cases reported 100% case precision, 100% case recall, 100% finding precision, and 100% finding recall.

That is not a broad public benchmark. It is regression evidence from one real repository. The docs are careful about that distinction, and the blog post should be too.

2.3: onboarding and evaluation hardening

The 2.3.0 release adds a more complete adoption story:

critical-gate init-agent creates or updates a managed Critical Gate section in AGENTS.md while preserving existing repository instructions.
Evaluation metrics are grouped by source repository and case type.
The sanitized evaluation corpus expands to 22 cases across generic fixtures, mv-ft dogfood regressions, sindresorhus/ky, and withastro/docs.
Clean Markdown reports include more compact why-passed evidence.
LLM artifact redaction is hardened and the maximum model artifact shape is documented.
Release checklist guardrails now explicitly protect product non-goals.

The 2.3.1 patch is not a new feature release. It prepares a republish so npm package README metadata can be regenerated for the registry page. The meaningful feature story is 2.3.0.

The finding model

Every finding is designed to be evidence-backed and repairable.

A finding includes:

stable id
detector name
severity
confidence
title
concise message
evidence
repair guidance
tags such as scope, dependency, API, test, secret, or config

Severity levels are:

blocker: should fail the gate by default
high: should fail by default when confidence and rollout policy allow it
medium: should warn and appear in summaries
low: informational unless combined with other risk
info: context for humans and dashboards

Confidence is calibrated into bands:

very-high: 0.90 and above
high: 0.80 to 0.89
medium: 0.60 to 0.79
low: below 0.60

By default, only blocker and high findings can fail the gate. They must also clear the detector’s confidence threshold and active rollout policy.

Critical Gate reports calibration counts in JSON:

blockingEligibleCount: findings eligible to fail this run
observationModeCount: high-confidence findings kept observational by rollout policy
confidenceSuppressedCount: high or blocker severity findings below confidence threshold

Some detectors are naturally concrete. Dependency additions, skipped tests, removed assertions, secrets, rewrites, API removals, and high-confidence scope findings can block when evidence is strong.

Other detector families are intentionally observation-friendly by default: blast radius, expected companions, existing solutions, pattern violations, framework-pack hints, and repository intelligence. They are useful, but they should not become blockers until dogfood and policy prove they are precise enough for a given repository.

Detector families that matter

Critical Gate focuses on high-risk agent failure patterns rather than generic review comments.

Scope drift and unrelated file modifications

This detector asks whether the changed files fit the task intent.

If the task is:

Fix font weight in the profile heading.

then a small stylesheet change may be expected.

If the same diff also rewrites unrelated layout files, deletes a reset stylesheet, changes CI, or updates package metadata, the gate should raise the alarm.

Evidence can include:

task keywords and target nouns
changed file paths
file roles such as source, test, docs, config, generated, lockfile, or package manifest
repository token matches from paths, symbols, test names, Markdown headings, package names, and nearby folders
git co-change history
expected blast radius
churn metrics
Scope Expansion Score

The goal is not to punish normal companion files. Source plus tests can be normal. Component plus story can be normal. UI copy plus translation files can be normal. The goal is to catch surprising expansion beyond the task.

Intent coverage underimplementation

The gate does not only look for too much change. It also looks for too little meaningful implementation.

If the task says:

Add new section to the site to display works done.

and the diff only changes one typography value, that is probably not a complete implementation.

Critical Gate can emit an intent-coverage finding when a task uses strong implementation verbs such as add, create, implement, build, render, display, or show, but the diff only contains tiny indirect edits.

It avoids this for explicit style tasks like:

Adjust typography font weight for the works section.

That is the calibration spirit: catch likely underimplementation without harassing normal visual tuning.

Rewrite for small request

Agents sometimes solve small tasks by regenerating entire files.

That can delete edge handling, comments, exports, test cases, or local conventions while still producing plausible code.

Critical Gate tracks:

added and deleted lines
changed file count
rewrite percentage
structural similarity proxies
task complexity estimate
balanced churn
vague task text such as “update project”

The 2.2 dogfood fixes made this stronger for source-like files even when role metadata is stale or unknown. That matters because real repositories do not always classify cleanly on the first pass.

Dependency addition without justification

Adding a dependency changes more than code. It changes install behavior, security surface, transitive packages, lockfiles, bundle size, and long-term ownership.

Critical Gate parses manifest deltas and checks:

production dependency additions
dev dependency additions
lockfile companion changes
whether the task asked for a dependency
whether documentation or PR text justifies the addition
whether a local or native option is likely enough to ask for repair

For example:

Task: Add CSV export using the existing file writer utilities.

If the diff adds a CSV package, that is suspicious unless the diff explains why the existing utilities are insufficient.

Test weakening

Green tests do not always mean safe tests.

An agent can make tests pass by weakening what they prove:

- expect(result.error.message).toContain("email is required");
+ expect(result.error).toBeDefined();

Critical Gate looks for:

removed assertions
newly skipped tests
.only or todo
weaker matchers
assertion specificity drops
broad snapshot rewrites
behavioral UI assertions replaced by generic rendering presence checks

The untracked-file hunk work in 2.2 matters here. A newly added test file that contains .skip can be inspected before it is staged, which closes a real dogfood miss.

Public API and framework contract changes

Silent public API changes are dangerous, especially in libraries, CLIs, SDKs, shared packages, and framework config contracts.

Critical Gate can create a deterministic public API snapshot:

node dist/cli.js snapshot-api
git add .critical-gate/api-surface.json

The command infers entrypoints from package.json exports, main, module, types, browser, and bin, then falls back to common index files when package metadata does not declare a public surface. You can also pin entrypoints:

node dist/cli.js snapshot-api --entrypoint src/index.ts --entrypoint src/testing.ts

Durable policy can declare entrypoints too:

{
  "policy": {
    "publicApi": {
      "entrypoints": ["src/index.ts", "src/testing.ts"]
    }
  }
}

Once committed, normal checks load .critical-gate/api-surface.json.

If a diff removes a snapshotted export, changes a signature, adds an export to a snapshotted entrypoint, or updates the API snapshot without release evidence, the gate expects visible contract evidence:

snapshot update
changelog
changeset
migration note
explicit API release task intent

The 2.2 line also protects known framework contract exports, such as Astro collections in content.config.ts, before a repository has committed a public API snapshot.

Config and runtime drift

Configuration changes affect everyone.

Critical Gate classifies build, lint, test, bundler, Docker, CI, runtime, TypeScript, and package-manager config changes. Runtime and tooling pins such as .node-version, .nvmrc, .tool-versions, .npmrc, and .pnpmrc are now treated as configuration.

This matters because an agent may make a small UI change and silently alter the runtime. The task wording “without changing config” is now treated as a prohibition, not permission.

Secrets, paths, and environment leaks

AI agents can accidentally introduce machine-specific or sensitive details:

provider tokens
credentials
internal hosts
absolute paths
local usernames
private service URLs
.env-style assignments

Critical Gate includes lightweight diff-only checks. It is not a full vulnerability scanner, and the docs say so. It should not replace mature security tooling. Its job is to catch suspicious values introduced by the current diff and redact evidence so reports do not leak the secret.

Existing solution and utility reinvention

AI agents often create local helpers because they did not find the existing utility.

Critical Gate indexes utility-like modules and exported helper names, then compares new helpers against existing solutions using:

export name
parameter count
return shape
folder role
import count
nearby symbols

A useful finding can say:

This diff adds src/utils/dateFormatter.ts, but src/lib/date.ts already exports formatDate.

This is especially valuable in a Codex repair loop because the agent gets a concrete reuse target instead of a vague “avoid duplication” comment.

Pattern violations and framework packs

Repository convention is often implicit. Critical Gate builds local profiles from the touched area and can detect suspicious drift:

unusual symbol names
new abstraction styles
framework patterns inconsistent with the folder
rare file naming patterns
companion files missing for known framework workflows

Framework packs currently cover React, Next.js, Angular, Astro, Lit, Nest, Express, Vite, and Storybook. Packs are auto-detected from dependencies and common config files where possible, and they can also be forced in .critical-gate.json:

{
  "frameworkPacks": ["react", "storybook", "vite"]
}

These pack-driven hints use the expected-companions detector. They are evidence-backed and non-blocking by default.

Scores, reports, and clean diff certificates

Critical Gate reports more than pass or fail.

The main summary signals include:

Diff Cost Score
Scope Expansion Score
Diff Coherence Score

Diff Cost is a rough blast-radius and churn signal. It helps answer whether the amount of change fits the task.

Scope Expansion highlights changed clusters, unexpected files, missing companions, and unusual repository patterns.

Diff Coherence is a positive score from 0 to 100. A high score means the changed files, support files, churn, and findings appear to fit the task intent. A low score is not a separate detector. It summarizes evidence already shown in changed files and findings.

Clean Markdown reports now include compact why-passed evidence. That matters because “no findings” is not very useful in a PR or agent handoff. A clean report should explain that the gate checked changed file count and churn, dependency discipline, test integrity, public API surface, secret/path signals, and coherence.

PR-comment output is available when the audience is a pull request discussion:

node dist/cli.js check \
  --task "Add signup validation" \
  --base main \
  --format pr-comment \
  --output critical-gate-pr-comment.md

The PR-comment format groups the same evidence-backed data into blocking findings, observations, expected support changes, and strongest scope drivers.

Policy-as-code and repository learning

Real repositories have local rules. Some patterns are normal in one codebase and strange in another.

Critical Gate reads optional repository policy from .critical-gate.json. The policy file is reviewable policy-as-code. It should explain team-specific rollout choices and normal support-file relationships without hiding risky diffs globally.

Create a starter policy:

node dist/cli.js init-policy
git add .critical-gate.json

Policy can control defaults:

{
  "policy": {
    "failOn": "high",
    "detectorOverrides": [
      {
        "detector": "expected-companions",
        "mode": "observation",
        "reason": "Useful guidance during rollout, not blocking yet."
      }
    ]
  }
}

You can accept an exact finding when the team has reviewed it:

node dist/cli.js accept \
  --finding "scope:src/generated/client.ts" \
  --reason "Generated client file is expected for API schema refreshes."

You can teach a durable support-file relationship:

node dist/cli.js teach \
  --id "i18n-for-ui-copy" \
  --when-changed "src/features/**/*.tsx" \
  --allow "src/i18n/**/*.json,locales/**/*.json" \
  --reason "UI copy changes require translation updates."

Policy supports:

failOn
detector overrides
expected companions
allowed support files
public API entrypoints
framework packs
pattern aliases
feature roots
service roots
validator roots
exclude patterns for repository intelligence

Accepted findings are exact finding ids, not broad suppressions. excludePatterns tune repository intelligence, but they are not a security boundary.

Agent onboarding with init-agent

One of the most useful 2.3 additions is:

node dist/cli.js init-agent

This creates or updates a managed Critical Gate section in AGENTS.md while preserving existing repository instructions. Repeated runs replace only the managed Critical Gate block.

You can provide the command agents should use:

node dist/cli.js init-agent --cli "node ./node_modules/critical-gate/dist/cli.js"

The generated section tells agents how to run check, hook, snapshot-api, init-policy, and install-hooks. It also tells them what Critical Gate is not: it is not a generic reviewer, not a repo-wide LLM scan, and not an automatic fixer.

This matters because the best gate is not only a CI command. It is part of the repository’s instructions to agents. Agents should know before they edit that the final diff will be checked for evidence-backed scope, test, dependency, API, config, and repair integrity.

Normal change model and monorepo context

Critical Gate derives normal change patterns from git history.

It can identify typed relationships such as:

source/test
component/story
translation/UI
config/docs
package manifest/lockfile
source/docs

The model is local and deterministic. It uses co-change support and confidence from repository history, and it stays quiet when the repository does not have enough history to be reliable.

Monorepo support adds ownership context. Critical Gate detects:

pnpm-workspace.yaml
root package workspaces
turbo.json
nx.json
lerna.json
root TypeScript path aliases
changed package owners
package-local names

This is not a separate finding by itself. It gives downstream detectors better context. A change that stays inside one package is different from a change that crosses workspace boundaries without explanation.

CLI usage in the current release stage

The official CLI path for this release stage is source-based installation.

Requirements:

Node.js 22.13 or newer
pnpm 11.1.2
git history for baseline comparisons

Build from source:

git clone https://github.com/criticaldeveloper/critical-gate.git critical-gate
cd critical-gate
pnpm install --frozen-lockfile
pnpm build

Run a local Markdown report:

node dist/cli.js check \
  --task "Add email validation to signup form" \
  --base main \
  --format markdown

Write JSON for automation:

node dist/cli.js check \
  --task "Add email validation to signup form" \
  --base main \
  --format json \
  --output critical-gate.json

Write SARIF for code scanning:

node dist/cli.js check \
  --task "Add email validation to signup form" \
  --base main \
  --format sarif \
  --output critical-gate.sarif

Write compact repair guidance:

node dist/cli.js check \
  --task "Add email validation to signup form" \
  --base main \
  --format repair

Analyze staged changes:

node dist/cli.js check \
  --task "Prepare signup validation patch" \
  --staged \
  --format markdown

Control the failure threshold for a single run:

node dist/cli.js check \
  --task "Add signup validation" \
  --base main \
  --fail-on blocker

Current command surface:

critical-gate check --task <text> [--base <ref>] [--format json|markdown|sarif|repair|pr-comment] [--fail-on blocker|high|medium] [--staged] [--strict] [--output <path>]
critical-gate hook [--task <text>] [--base <ref>] [--output <path>]
critical-gate accept --finding <id> --reason <text>
critical-gate teach --id <id> --when-changed <glob> --allow <glob[,glob]> --reason <text>
critical-gate snapshot-api [--entrypoint <path>] [--output <path>]
critical-gate install-hooks [--hook pre-commit|pre-push|all] [--cli <command>] [--force]
critical-gate init-policy [--force]
critical-gate init-agent [--cli <command>]
critical-gate --version
critical-gate --help

Exit codes are:

0: gate passed
1: findings failed the configured threshold
2: usage or configuration error
3: internal error

Local hooks and Codex repair loops

Critical Gate can install reviewable local hooks:

node dist/cli.js install-hooks

That writes:

.git/hooks/pre-commit: checks staged changes with --fail-on blocker
.git/hooks/pre-push: checks the branch against ${CRITICAL_GATE_BASE:-origin/main} with --fail-on high

At runtime:

CRITICAL_GATE_TASK overrides task intent
CRITICAL_GATE_BASE overrides the pre-push base branch

For Codex, hook mode is designed for repair loops:

node dist/cli.js hook --base main

The loop is:

Codex works on a task.
Critical Gate runs over the resulting diff.
If the diff passes, the task can finish.
If the diff fails, the hook returns compact repair guidance.
Codex repairs only the relevant files and reruns the gate.

Each failing finding includes an agent repair contract:

instructions
allowed files
forbidden files
success criteria

That matters because “please fix the review comments” is too vague. A repair loop should tell the agent exactly which evidence-backed files are safe to touch and what must be true after the repair.

The hook should not mutate files directly. It reports evidence and lets Codex perform the scoped repair.

GitHub Action integration

Critical Gate can run as a GitHub Action or as plain CLI steps, but the current docs are explicit about distribution.

Until the action is published under a remote slug, use it from this repository checkout or from the same repository with:

uses: ./

Minimal workflow:

name: Critical Gate

on:
  pull_request:

permissions:
  contents: read
  pull-requests: read
  security-events: write

jobs:
  critical-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0

      - id: critical-gate
        uses: ./
        continue-on-error: true
        with:
          task: ${{ github.event.pull_request.title }}
          base: ${{ github.event.pull_request.base.sha }}
          format: sarif
          output: critical-gate.sarif

      - uses: github/codeql-action/upload-sarif@v4
        if: always() && hashFiles('critical-gate.sarif') != ''
        with:
          sarif_file: critical-gate.sarif

      - if: steps.critical-gate.outcome == 'failure'
        run: exit 1

For external repositories before publication, checkout Critical Gate as a secondary path:

- name: Checkout project
  uses: actions/checkout@v6
  with:
    fetch-depth: 0

- name: Checkout Critical Gate
  uses: actions/checkout@v6
  with:
    repository: criticaldeveloper/critical-gate
    path: .critical-gate

- name: Run Critical Gate
  uses: ./.critical-gate
  with:
    task: ${{ github.event.pull_request.title }}
    base: ${{ github.event.pull_request.base.sha }}

For release artifacts, the project now has:

pnpm package:action
pnpm smoke:action

That writes artifacts/action with action.yml, package metadata, and prebuilt dist/ output. Only use install: "false" and build: "false" with an artifact that has passed the smoke check.

VS Code surface

The VS Code extension consumes the same GateResult JSON as the CLI. It does not invent editor-only detectors.

It provides:

Critical Gate > Gate Runs
Critical Gate > Analysis
Problems diagnostics
output-channel report
latest decision and changed-file count
finding count
Scope Expansion Score
finding cards with evidence
changed clusters
missing companions
existing-solution signals
recent run history
status bar summaries

Quick actions include:

open evidence
copy repair prompt
open existing solution
open expected companion
open cluster report
accept blast-radius expansion locally

Build and package the VSIX:

pnpm package:vscode

Install locally:

code --install-extension artifacts/vscode/critical-gate-vscode.vsix --force

The extension is a local/editor surface over the CLI. It is not a separate analyzer.

Evaluation and quality evidence

The most important change in the 2.2 and 2.3 line is not just more detectors. It is more honesty about detector quality.

The project now has:

docs/detector-quality.md
docs/evaluation-strategy.md
docs/dogfood-mv-ft-2026-06-19.md
grouped evaluation metrics by source repository and case type
a sanitized evaluation corpus of 22 cases across generic fixtures, mv-ft dogfood regressions, sindresorhus/ky, and withastro/docs
CI artifacts for coverage and evaluation output

Run evaluation:

pnpm evaluate

The harness writes:

artifacts/evaluation/critical-gate-evaluation.json
artifacts/evaluation/critical-gate-evaluation.md

Run coverage:

pnpm coverage

CI uploads:

critical-gate-coverage
critical-gate-evaluation

The project is careful not to overclaim. The current corpus is useful regression evidence, not a broad external benchmark. That is the right posture for a pre-stable gate.

Rollout strategy

The right way to adopt a gate is gradual.

Start with default thresholds and high-confidence findings. Let medium and observation-mode findings teach the team before they block.

A practical rollout:

Run locally on AI-generated diffs with specific task text.
Add the VS Code extension for developer feedback where useful.
Add GitHub Action SARIF upload with default thresholds.
Review noisy findings and tune task text, repository docs, or .critical-gate.json policy.
Add init-agent so agents know how to use the gate.
Commit public API snapshots for packages with stable entrypoints.
Add local hooks for pre-commit or pre-push guardrails.
Add Codex hook enforcement where compact repair loops are useful.
Promote observation-friendly detector families only after dogfooding shows acceptable precision.

Detector promotion should not bypass confidence calibration. Even a promoted detector still needs enough confidence to fail the gate.

That is how Critical Gate stays low-noise.

What a clean run means

A clean run does not mean the code is perfect.

It means Critical Gate did not find evidence that the diff violated the task boundary, underimplemented the requested category of work, weakened tests, silently changed public contracts, added unjustified dependencies, leaked environment details, wandered through config, or drifted from known repository patterns above the configured confidence threshold.

That is a precise kind of assurance.

It sits beside other checks:

Linters enforce style and static rules.
Tests validate known behavior.
Security scanners detect known vulnerabilities and secrets.
Human reviewers judge design, product fit, and deeper correctness.
Critical Gate checks diff integrity for the task and repository.

This is why I do not frame it as an AI reviewer. It is adjacent to review, but it is a gate.

How it fits with Critical Templates

Critical Templates and Critical Gate are two sides of the same AI-native developer tooling idea.

Critical Templates says:

Do not make the agent guess how to create files. Encode the pattern.

Critical Gate says:

Do not trust the final diff because the agent says it is done. Check the evidence.

One is generation. The other is verification.

Together they create a healthier workflow:

templates encode how work should be created
task intent states what should change
agent instructions explain the rules before work begins
tests and type checks validate behavior
Critical Gate validates diff scope, contracts, and repairability

That is the shape serious AI-assisted development needs. Better models help, but better repository contracts matter just as much.

Where to get it

Critical Gate is open source:

criticaldeveloper/critical-gate on GitHub

The feature release covered here is 2.3.0; the current patch visible in the repo is 2.3.1, which is a metadata republish patch. The project includes the TypeScript CLI, deterministic detectors, policy-as-code learning controls, public API snapshots, agent onboarding, local git hooks, Codex hook integration, source-based GitHub Action integration, SARIF and PR-comment outputs, optional LLM explanation boundaries, evaluation artifacts, and a VS Code extension surface.

The fastest way to understand it is to run it against a real AI-generated diff:

node dist/cli.js check \
  --task "Describe the actual task the agent was asked to do" \
  --base main \
  --format markdown

Then inspect the report with one question in mind:

Did this diff stay inside the contract?

That is the heart of Critical Gate.

Not generic review.

Not AI judgment theater.

Not a repo-wide model scan.

A cheap, deterministic-first integrity gate for the moment when an AI-generated change is about to become part of the repository.