> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcanonic.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Instrumentation & evaluation

> The local event log, the accuracy harness, and the model baseline: how canonic measures itself.

canonic backs its trustworthiness claims with a local, inspectable log rather than asserting them: every served answer and every reconciliation decision is recorded, and two harnesses turn that record into tracked numbers.

## The event log

An append-only, structured log under `.canonic/` (local, git-ignored). It's the single substrate every producer writes to. [`canonic status`](/cli-reference/status-report#canonic-status) and [`canonic audit`](/cli-reference/status-report#canonic-audit) read from it.

**No warehouse content, ever.** The semantic query and compiled SQL are recorded as `sha256` hashes, never as text, so filter literals (dates, ids) never land on disk. Resolved bindings, guardrail ids, and counts are metadata and are stored as-is. Events are immutable once written. Corrections are new events, never edits.

One record per served answer:

```yaml theme={null}
kind: served_answer
query_hash: "sha256:…"              # the semantic query, hashed, never stored verbatim
compiled_sql_hash: "sha256:…"       # compiled SQL, hashed
resolved: { metrics: { revenue: orders.total_revenue } }
guardrails_fired: [revenue-excludes-refunds]
finality: { final_rows: 6, provisional_rows: 1 }
freshness: [{ source: orders, stale: false, age_days: 2 }]
latency_ms: 142
bytes_scanned: 10485760
error: null
trust_score: trusted                # the E14 tier: caution | provisional | trusted
# reserved, present now, populated once their producing feature lands:
cache_hit: null
over_limit_blocked: null
```

Under an active [tenancy/role policy](/concepts/tenancy-and-access-control), a served-answer event also carries `tenant` (`str | null`), `roles` (`list[str] | null`, sorted), and `tenancy_exempt` (`bool`, default `false`), populated for every `query()` and `run_sql` call, not just `query()`. `tenant` is the one exception to this page's "no content, ever" rule above: it's a verified identifier, not warehouse content, so it's stored verbatim rather than hashed, which is what makes per-tenant attribution, cost accounting, and breach forensics possible from the log at all. Every `tenancy_exempt: true` event is a cross-tenant read and should be treated as alertable. All three are `null`/`false` when no tenancy or role policy is configured.

A `reconcile_decision` event (add/edit/prune/contradiction/no-op, with tier, confidence, and anchored evidence) shares the same substrate: one store backs both serving and ingest traceability.

### Outcome capture: `answer_outcome`

The ground-truth feed: an analyst, CI verdict, or agent self-report marking a served answer correct or incorrect, recorded via [`canonic outcome mark`](/cli-reference/status-report#canonic-outcome-mark) and linked back to the originating `served_answer` event by `query_hash`.

```yaml theme={null}
kind: answer_outcome
ref: "sha256:…"                     # the AnswerEvent.query_hash this is about
verdict: incorrect                  # correct | incorrect
reason_code: wrong_definition       # only set when verdict is incorrect
correction: "sha256:…"              # optional: corrected SQL hash or definition ref
marked_by: analyst                  # analyst | agent | ci
```

The `reason_code` is an attribution safeguard: acting on the wrong cause corrupts good context. `wrong_definition` is the only code that implicates the binding used: it's the one eligible to become contradiction evidence downstream. `wrong_data` (the warehouse data is wrong) and `wrong_interpretation` (the wrong metric was picked) are recorded but never flag the binding. `unspecified`, the default when a verdict is marked incorrect without an explicit reason, carries the lowest weight. canonic records the code faithfully. It does not act on it.

## Local inspection, no telemetry required

`canonic status` and `canonic audit` read the log directly: counts, error-code distribution, latency percentiles, bytes scanned, freshness and guardrail-hit coverage, the onboarding funnel (setup started → connection added → bootstrap completed → first answer served → first curated review completed), trust-tier calibration, and correction recurrence. None of this requires telemetry to be enabled. The log is not a black box on day one.

## Opt-in telemetry

`canonic.yaml`'s `telemetry.enabled` flag defaults to `false`, and is forced off whenever `runtime.air_gapped: true` is set. In a freshly scaffolded project, or any project that hasn't explicitly configured telemetry, **nothing leaves the machine**: the transport exists in the code, but it refuses to send anything unless every one of the following is explicitly set in `canonic.yaml`:

* `telemetry.enabled: true`
* `telemetry.endpoint`: the destination URL. There is no default, canonic never guesses one
* `telemetry.transport_acknowledged: true`: an attestation that *your project* has reviewed the exact aggregate payload below and is comfortable sending it. canonic cannot verify that a review happened. This is a project-level policy decision, not a technical guarantee.

Missing any of these (or being air-gapped) fails the send closed with a structured error: it never silently no-ops and never partially sends.

`canonic audit --telemetry-preview` prints precisely the payload that would be sent, without sending it, regardless of configuration: counts, distributions, latency and bytes-scanned aggregates, the trust-calibration buckets, and (when available) accuracy/baseline numbers. It **never** includes `query_hash`, resolved bindings, SQL, `compiled_sql_hash`, freshness source names, or any other field that could re-identify a query or reveal warehouse/schema content, the same content-safety rule as the local log itself, applied one notch stricter since this payload is meant to leave the machine. Use `canonic audit --telemetry-send` to actually send that same payload once `enabled`, `endpoint`, and `transport_acknowledged` are all set.

## Accuracy harness: `canonic assert`

The mechanism that turns ">90% accuracy" from aspirational into measured. Every executable assertion in `contracts/assertions/` is compiled, executed read-only, and compared to its expected value within tolerance. The harness reports `accuracy = passed / total` and gates on a floor (default `1.0`, every assertion must hold). See [`canonic assert`](/cli-reference/query-sql-assert#canonic-assert) for the CLI, and [Assertions](/concepts/contracts-and-guardrails#assertions) for how an assertion is declared.

`canonic assert --baseline` also runs the same assertions against a schema-only resolver, raw physical schema, no curated bindings/aliases/guardrails, so the accuracy *lift* the canonical context layer provides is a measured delta, not an asserted one.

## Model baseline harness: `canonic eval baseline`

A separate harness for a separate question: not "is the compiled SQL correct" (that's `canonic assert`, and the compiler is deterministic regardless of model), but **which LLM is good enough to draft with**. `canonic eval baseline` drives the real production drafter (not a re-implemented prompt) over a labeled dataset of grain-inference and contradiction-resolution cases, for each candidate model in a supplied list. It scores structured-output adherence (does the model actually return parseable JSON), accuracy against the labeled answer, and latency, then recommends a candidate that clears the adherence floor. See [`canonic eval baseline`](/cli-reference/eval#eval-baseline).

Re-run this before tagging a release so the recommended local model tracks reality as models churn, rather than being asserted once.