User Guide

AI Guardrails

Turn recurring AI mistakes into automated architecture fitness tests that run in CI/CD on every pull request. One mistake, one guardrail, zero recurrence.

What Are AI Guardrails?

AI coding assistants (Claude, Copilot, Cursor, etc.) are powerful but they don't know your architecture. They'll import a banned library, use a retired auth pattern, or skip your team's error-handling convention — and do it again next time, because they have no memory of the correction.

AI Guardrails close that loop. When you spot an AI mistake, you describe it in SystemDox. The platform analyses the issue, generates a structured guardrail entry, and produces an executable architecture fitness test — a shell script that detects the bad pattern in code. Publish that test to your CI/CD pipeline, and the mistake is blocked on every future pull request, across every repository that runs the shared test suite.

End-to-End Flow

How It Works

From spotting an AI mistake to blocking it in CI — in four steps.

1

Report the Mistake

Navigate to Define → Guardrails in the SystemDox app. In the text area, describe the AI mistake in plain language. You can report one or multiple issues at once.

# Example input:

Claude keeps importing Sentry directly with
from '@sentry/react' instead of using our
shared-observability wrapper. It also uses
Sentry.captureException() instead of
captureError() from the shared library.

2

AI Analyses and Generates Guardrails

Click "Analyse & Generate". SystemDox sends your description to Claude, which extracts structured guardrail entries. Each entry includes:

Title & category — e.g., "Ban direct Sentry imports" under imports
Severity — critical, high, medium, or low
Bad pattern — the code that should not appear
Correct fix — what should be used instead
Fitness test — a shell command (grep/ripgrep) that detects violations
Description — why this pattern is wrong and the impact
3

Review and Refine

Your guardrails appear in a knowledge base. For each entry you can:

  • Edit the title, description, pattern, fix, or fitness test command
  • Preview the script — see the exact check_*.sh that will be generated
  • Download or copy individual scripts for manual use
  • Delete entries that are no longer relevant

You can also use the guardrail-aware chat to ask questions. The AI assistant respects all active guardrails when generating code or answering architecture questions.

4

Publish to CI/CD

Click "Publish to CI". SystemDox creates a pull request on your organisation's shared architecture fitness tests repository (e.g., shared-arch-fitness-tests) containing the generated check_*.sh scripts.

Once the PR is merged, the fitness tests run automatically on every pull request across every repository that uses the shared CI workflow. No manual script copying, no forgetting to add the test — publish once, enforce everywhere.

# Generated PR on shared-arch-fitness-tests

+ scripts/check_ban_direct_sentry_imports.sh

+ scripts/check_use_capture_error.sh

# Each script includes a managed-by marker:

# managed-by: systemdox guardrail abc123

Under the Hood

Architecture Fitness Tests

Every guardrail becomes an executable check — a simple shell script that scans your codebase for violations.

Example: check_ban_direct_sentry_imports.sh 

#!/usr/bin/env bash
# managed-by: systemdox guardrail sentry-imports-001
# scope: frontend
# severity: critical
# description: Detect direct @sentry/react imports outside wrapper files

set -euo pipefail

TARGET="$1"
VIOLATIONS=$(grep -rn "from '@sentry/react'" "$TARGET" \
  --include="*.ts" --include="*.tsx" \
  | grep -v "sentry.ts" \
  | grep -v "node_modules" || true)

if [ -n "$VIOLATIONS" ]; then
  echo "FAIL: Direct @sentry/react imports found."
  echo "Use captureError() from shared-observability instead."
  echo "$VIOLATIONS"
  exit 1
fi
echo "PASS: No direct Sentry imports found."

Scope Header

Each script declares its scope (frontend, backend, or repo). The test runner reads this header and only runs the script against matching directories.

Managed-By Marker

Scripts generated by SystemDox include a # managed-by: comment. The publish action only creates, updates, or removes scripts with this marker — hand-written tests are never touched.

Auto-Discovery

The shared CI workflow runs run_all.sh which auto-discovers all check_*.sh scripts. No CI configuration changes needed when new tests are added.

Continuous Improvement

The Closed Loop

Guardrails don't just block mistakes — they feed back into AI context so the mistake is less likely to happen in the first place.

Prevention Layer

Active guardrails are included in CLAUDE.md generation. When an AI assistant reads your project context, it sees the guardrail and avoids the mistake before writing code.

The guardrail-aware chat in SystemDox also enforces all active rules. Ask it to generate code and it will follow your guardrails automatically.

Detection Layer

Even if the AI ignores the guardrail (or the developer uses a different tool), the architecture fitness test catches the violation in CI before it reaches the main branch.

Two layers of defence: AI context prevents most mistakes; CI/CD catches the rest.

The Loop in Practice

AI makes mistake
Report in SystemDox
Publish to CI
Blocked forever

Getting Started

Setup & Requirements

1. Connect Your GitHub Organisation

SystemDox needs permission to create pull requests on your architecture fitness tests repository. In Settings → Integrations, connect your GitHub organisation and select the target repository (e.g., shared-arch-fitness-tests).

2. Add the Shared CI Workflow

Each repository that should enforce guardrails needs to call the shared architecture fitness test workflow in its CI pipeline: 

# .github/workflows/ci.yml
architecture-fitness:
  uses: your-org/shared-cicd/.github/workflows/architecture-fitness.yml@main
  with:
    fitness-tests-repo: your-org/shared-arch-fitness-tests

3. Create Your First Guardrail

Go to Define → Guardrails in the SystemDox app, describe an AI mistake, click "Analyse & Generate", review the result, and hit "Publish to CI". A pull request appears on your fitness tests repo within seconds.

Categories

What Can You Guard Against?

Guardrails are categorised automatically. Here are the most common categories teams use.

Imports

Banned libraries, direct SDK imports that should use wrappers

Auth

Retired auth patterns, missing authoriser config, legacy claim extraction

Error Handling

Raw error.message shown to users, missing error boundaries

Architecture

Wrong service boundaries, direct DB access from wrong layer

Security

Unsafe CSP directives, hardcoded secrets, missing input validation

Conventions

Wrong naming patterns, missing file headers, incorrect folder structure

Testing

Mocking what should be integration-tested, missing assertion patterns

Data Access

Wrong DynamoDB key patterns, missing tenant isolation, N+1 queries

Frequently Asked Questions

Can I edit a generated fitness test before publishing?
Yes. Each guardrail entry is fully editable. You can modify the fitness test command, the description, the pattern, and the fix before publishing. You can also download the script and test it locally first.
What happens if I update a guardrail after publishing?
Publishing again creates a new PR that updates the existing script. The managed-by marker links the script back to the guardrail ID, so SystemDox knows which file to update.
Will publishing overwrite hand-written fitness tests?
No. The publish action only touches scripts that contain the # managed-by: systemdox marker. Hand-written tests (without the marker) are never modified or deleted.
Can I "eject" a managed test into manual maintenance?
Yes. Remove the # managed-by: line from the script in your repository. SystemDox will no longer update or delete it. The test continues to run normally.
What repositories are fitness tests enforced on?
Every repository whose CI pipeline calls the shared architecture fitness test workflow. Adding a new repo to the enforcement list is one line in its CI config — no changes to the tests themselves.

Ready to Transform Your Documentation?

Start capturing architecture decisions, meeting notes, and technical designs with your voice. Free to get started.

Start Free Trial Request a Demo