User Guide

Checks

Turn recurring AI mistakes into automated architecture fitness tests that run in CI/CD on every pull request. One mistake, one check, zero recurrence.

What Are Checks?

AI coding assistants (Claude, Copilot, Cursor, etc.) are powerful but they don't know your architecture. They'll import a banned library, use a retired auth pattern, or skip your team's error-handling convention — and do it again next time, because they have no memory of the correction.

Checks close that loop. When you spot an AI mistake, you describe it in SystemDox. The platform analyses the issue, generates a structured check entry, and produces an executable architecture fitness test — a shell script that detects the bad pattern in code. Publish that test to your CI/CD pipeline, and the mistake is blocked on every future pull request, across every repository that runs the shared test suite.

End-to-End Flow

How It Works

From spotting an AI mistake to blocking it in CI — in four steps.

Report the Mistake

Navigate to Standards → Checks in the SystemDox app. In the text area, describe the AI mistake in plain language. You can report one or multiple issues at once.

# Example input:

Claude keeps importing Sentry directly with
from '@sentry/react' instead of using our
shared-observability wrapper. It also uses
Sentry.captureException() instead of
captureError() from the shared library.

AI Analyses and Generates Checks

Click "Analyse & Generate". SystemDox sends your description to Claude, which extracts structured check entries. Each entry includes:

Title & category — e.g., "Ban direct Sentry imports" under imports

Severity — critical, high, medium, or low

Bad pattern — the code that should not appear

Correct fix — what should be used instead

Fitness test — a shell command (grep/ripgrep) that detects violations

Description — why this pattern is wrong and the impact

Review and Refine

Your checks appear in a knowledge base. For each entry you can:

Edit the title, description, pattern, fix, or fitness test command
Preview the script — see the exact check_*.sh that will be generated
Download or copy individual scripts for manual use
Delete entries that are no longer relevant

You can also use the check-aware chat to ask questions. The AI assistant respects all active checks when generating code or answering architecture questions.

Publish to CI/CD

Click "Publish to CI". SystemDox opens a single pull request on your target repository containing everything needed to run the checks: the generated check_*.sh scripts, a run_all.sh runner, and a GitHub Actions workflow that calls it.

Merge the PR and the checks run on every pull request from then on. There is nothing to wire up by hand and nothing to remember — publish once, enforce everywhere. The workflow is self-contained: it references no repository other than your own.

# Generated PR on your repository

+ .github/workflows/systemdox-checks.yml

+ scripts/run_all.sh

+ scripts/check_ban_direct_sentry_imports.sh

+ scripts/check_use_capture_error.sh

# Every generated file carries a managed-by marker:

# managed-by: systemdox guardrail abc123

Under the Hood

Architecture Fitness Tests

Every check becomes an executable script — a simple shell script that scans your codebase for violations, and lands as a check run on the pull request.

Example: check_ban_direct_sentry_imports.sh

#!/usr/bin/env bash
# managed-by: systemdox guardrail sentry-imports-001
# scope: frontend
# severity: critical
# description: Detect direct @sentry/react imports outside wrapper files

set -euo pipefail

TARGET="$1"
VIOLATIONS=$(grep -rn "from '@sentry/react'" "$TARGET" \
  --include="*.ts" --include="*.tsx" \
  | grep -v "sentry.ts" \
  | grep -v "node_modules" || true)

if [ -n "$VIOLATIONS" ]; then
  echo "FAIL: Direct @sentry/react imports found."
  echo "Use captureError() from shared-observability instead."
  echo "$VIOLATIONS"
  exit 1
fi
echo "PASS: No direct Sentry imports found."

Scope Header

Each script declares its scope (frontend, backend, or repo). The test runner reads this header and only runs the script against matching directories.

Managed-By Marker

Every file SystemDox generates — scripts, runner, and workflow — carries a # managed-by: comment. The publish action only creates, updates, or removes files with this marker — hand-written CI is never touched.

Auto-Discovery

The committed workflow runs run_all.sh, which auto-discovers every check_*.sh next to it. Adding a guardrail never changes your CI configuration.

Continuous Improvement

The Closed Loop

Checks don't just block mistakes — they feed back into AI context so the mistake is less likely to happen in the first place.

Prevention Layer

Active checks are included in CLAUDE.md generation. When an AI assistant reads your project context, it sees the check and avoids the mistake before writing code.

The check-aware chat in SystemDox also enforces all active rules. Ask it to generate code and it will follow your checks automatically.

Detection Layer

Even if the AI ignores the check (or the developer uses a different tool), the architecture fitness test catches the violation in CI before it reaches the main branch.

Two layers of defence: AI context prevents most mistakes; CI/CD catches the rest.

The Loop in Practice

AI makes mistake

Report in SystemDox

Publish to CI

Blocked forever

Getting Started

Setup & Requirements

1. Connect Your GitHub Organisation

In Settings → Integrations, install the SystemDox GitHub App on your organisation and pick the repository the checks should live in. Grant it Contents, Pull requests, and Workflows — the last one is what lets SystemDox commit .github/workflows/systemdox-checks.yml for you. Without it the scripts are still published, and the PR tells you what to do.

2. There Is No Step 2

The workflow arrives in the publish PR, already wired to the scripts beside it. Nothing to copy, no reusable workflow to reference, no repository outside your own to depend on:

# .github/workflows/systemdox-checks.yml — committed by SystemDox
# managed-by: systemdox workflow
name: SystemDox Checks

on:
  pull_request:

jobs:
  systemdox-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: bash scripts/run_all.sh .

3. Create Your First Check

Go to Standards → Checks in the SystemDox app, describe an AI mistake, click "Analyse & Generate", review the result, and hit "Publish to CI". A pull request appears on your repository within seconds. Merge it, and the next PR you open runs the check.

What Can You Guard Against?

Checks are categorised automatically. Here are the most common categories teams use.

Imports

Banned libraries, direct SDK imports that should use wrappers

Auth

Retired auth patterns, missing authoriser config, legacy claim extraction

Error Handling

Raw error.message shown to users, missing error boundaries

Architecture

Wrong service boundaries, direct DB access from wrong layer

Security

Unsafe CSP directives, hardcoded secrets, missing input validation

Conventions

Wrong naming patterns, missing file headers, incorrect folder structure

Testing

Mocking what should be integration-tested, missing assertion patterns

Data Access

Wrong DynamoDB key patterns, missing tenant isolation, N+1 queries

Frequently Asked Questions

Can I edit a generated fitness test before publishing?

Yes. Each check entry is fully editable. You can modify the fitness test command, the description, the pattern, and the fix before publishing. You can also download the script and test it locally first.

What happens if I update a check after publishing?

Publishing again creates a new PR that updates the existing script. The managed-by marker links the script back to the check ID, so SystemDox knows which file to update.

Will publishing overwrite hand-written fitness tests?

No. The publish action only touches scripts that contain the # managed-by: systemdox marker. Hand-written tests (without the marker) are never modified or deleted.

Can I "eject" a managed test into manual maintenance?

Yes. Remove the # managed-by: line from the script in your repository. SystemDox will no longer update or delete it. The test continues to run normally.

What repositories are fitness tests enforced on?

Every repository whose CI pipeline calls the shared architecture fitness test workflow. Adding a new repo to the enforcement list is one line in its CI config — no changes to the tests themselves.

Ready to Transform Your Documentation?

Start capturing architecture decisions, meeting notes, and technical designs with your voice.

Start free Request a Demo