Metrics & Analytics

Overview

AI Craftsman Superpowers records quality data in a local SQLite database, metrics.db. Each host and session keeps its own store. It holds the violations the gate found, what happened to them, session summaries, the semantic layer’s own runs, and the instincts the correction loop proposes.

Viewing Metrics

/craftsman:metrics

The skill reads the database rather than estimating: violations by rule and severity over 7 days, a 14-day trend, sessions, correction outcomes over 30 days, the semantic layer’s runs and fixed rate, whether each rule earns its severity, churn-by-complexity hotspots, and the instinct candidates waiting for your decision. --dashboard renders the aggregated view across your repositories.

Database Schema

The core tables, as hooks/lib/metrics-db.sh creates them:

CREATE TABLE violations (
    id INTEGER PRIMARY KEY,
    timestamp TEXT NOT NULL DEFAULT (datetime('now')),
    project_hash TEXT NOT NULL,
    rule TEXT NOT NULL,                 -- e.g. "PHP001"
    file_pattern TEXT NOT NULL,         -- directory bucket, e.g. "src/Domain/**/*.php"
    severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
    blocked BOOLEAN NOT NULL DEFAULT 0,
    ignored BOOLEAN NOT NULL DEFAULT 0
    -- added by migration: source TEXT, file_path TEXT (project-relative)
);

CREATE TABLE sessions (
    id INTEGER PRIMARY KEY,
    timestamp TEXT NOT NULL DEFAULT (datetime('now')),
    project_hash TEXT NOT NULL,
    duration_seconds INTEGER,
    skills_used TEXT,
    agents_spawned TEXT,
    violations_blocked INTEGER DEFAULT 0,
    violations_warned INTEGER DEFAULT 0,
    writes_count INTEGER DEFAULT 0
);

CREATE TABLE corrections (
    id INTEGER PRIMARY KEY,
    timestamp TEXT NOT NULL DEFAULT (datetime('now')),
    project_hash TEXT NOT NULL,
    rule TEXT NOT NULL,
    file_pattern TEXT NOT NULL,
    action TEXT NOT NULL CHECK (action IN ('fixed', 'ignored', 'overridden', 'scoped', 'open')),
    context TEXT
    -- added by migration: source TEXT, file_path TEXT (project-relative)
);

Three more tables sit beside them: haiku_runs (every semantic verification, clean ones included, so a rate has a denominator), verdicts (a developer’s own right or wrong on a finding, from the reason written into an ignore marker or from a review in /craftsman:metrics), and instincts (the candidates of the correction loop, below).

SQL Queries (Direct Access)

Find the metrics database path for the current host and session:

METRICS_DB=$(bash "$(craftsman-path bin/craftsman-runtime)" metrics)
sqlite3 "$METRICS_DB"

Or query directly:

# Violations in the last 7 days
sqlite3 "$(bash "$(craftsman-path bin/craftsman-runtime)" metrics)" \
  "SELECT rule, severity, COUNT(*) AS total, SUM(blocked) AS blocked
   FROM violations WHERE timestamp > datetime('now', '-7 days')
   GROUP BY rule, severity ORDER BY total DESC;"

# What happened to the findings, by rule, over 30 days
sqlite3 "$(bash "$(craftsman-path bin/craftsman-runtime)" metrics)" \
  "SELECT rule, action, COUNT(*) AS count FROM corrections
   WHERE timestamp > datetime('now', '-30 days')
   GROUP BY rule, action ORDER BY count DESC;"

Correction Learning System

Every fix you make to a finding is recorded. The loop turns a fix that recurs into a lesson, and only you decide whether it becomes one:

1. Recorded: a finding and what was done with it (fixed, ignored, …)
2. Candidate: a rule fixed 3 or more times across 3 or more files becomes a candidate instinct, weighed against how often it was rejected
3. Reviewed: /craftsman:metrics lists the candidates; you approve or reject each one
4. Codified: an approved instinct is written as a project skill, with its provenance

Nothing is promoted automatically: detection is automatic, codification stays human-gated.

Resetting Metrics

To reset metrics for the current host and session:

# Find the database path
METRICS_DB=$(bash "$(craftsman-path bin/craftsman-runtime)" metrics)

# Reset all metrics (irreversible)
rm "$METRICS_DB"

# Remove only the last week of violations
sqlite3 "$METRICS_DB" \
  "DELETE FROM violations WHERE timestamp > datetime('now', '-7 days')"

Each host and session keeps its own database, so resetting one does not affect the others.