Audit Log & Tamper Verification¶

TDB writes a tamper-evident, hash-chained audit log for every query, every refused attempt and every query that fails. Every request — REST or MCP — produces one entry. The log is the primary compliance artifact for SOC 2, HIPAA, and EU AI Act readiness.

Log format¶

The log is NDJSON (newline-delimited JSON), one JSON object per line. Default path: tdb_audit.jsonl. Override with TDB_LOG_FILE.

There are two entry types, distinguished by event: query for a query that ran, and denied for an attempt that was refused. Both are signed into the same chain. Enterprise also records events that are neither, such as sign-ins and, from 0.32.0, query approvals: approval_requested and approval_decided.

event: "query" — a query that executed:

{
  "event": "query",
  "ts": "2026-05-22T09:01:23.456789+00:00",
  "source_id": "f47ac10b-...",
  "sql": "SELECT COUNT(*) FROM orders",
  "rows_returned": 1,
  "key_hint": "tdbk_a...",
  "seq": 42,
  "prev_hash": "a3f1...",
  "hash": "9c82..."
}

Field	Description
`event`	`"query"` for an executed query, `"denied"` for a refused attempt; `"query_started"` precedes each statement on the wire gateway (enterprise, from 0.14.0) and each stream (from 0.31.0)
`ts`	ISO 8601 timestamp (UTC) of when the query executed
`source_id`	UUID of the registered source that was queried
`sql`	The SQL statement executed (or `<view:name>` for named views). With `TDB_AUDIT_SQL=redacted`, the statement with its values replaced by `?`
`sql_redacted`, `sql_sha256`	From community 0.16.0 / enterprise 0.27.0, with `TDB_AUDIT_SQL=redacted` only: `true`, and the SHA-256 of the redacted text. Runs of one statement with different values share it
`rows_returned`	Number of rows included in the response (`query` entries only)
`key_hint`	First 6 characters of the key that made the request, then `...`. The raw key is never written to the log
`user`, `user_id`, `user_inactive`	From 0.24.0, entries made with a JWT: `user` is the person (the full value, so two people cannot look alike). SSO entries add `user_id`, and `user_inactive` when a removed person is refused. API key entries do not have these
`seq`	Monotonically increasing sequence number, starting from 1
`prev_hash`	SHA-256 hash of the previous entry (genesis entry uses `000...0`)
`hash`	SHA-256 hash of this entry (computed over all fields except `hash` itself)
`mcp_client`, `mcp_protocol`	MCP entries only (from community 0.12.0 / enterprise 0.18.0): the client's self-reported name, `name/version` from its `clientInfo` or else its `User-Agent`, at most 200 characters; and the protocol version it was served at. See the MCP audit notes
`lean`	MCP entries served in Lean Mode only (from community 0.13.0 / enterprise 0.21.0): the settings applied, e.g. `{"format": "table"}`
`policy_sha256`	Enterprise, from 0.22.0, while a policy file is loaded: on every query and denial entry, the SHA-256 of the file in force
`approval_id`	Enterprise, from 0.32.0: a query run under a query approval, and a refusal that waits on or was denied by one
`policy_rule`, `policy_masked`, `policy_withheld`, `policy_row_filtered`	A query run under a policy rule: the rule's name, `{table: [masked columns]}`, `{table: [withheld columns]}` and the tables a row filter applied to. Present (possibly empty) on every query under a rule; a denial under a rule carries `policy_rule`
`page`, `cursor_id`, `rows_so_far`, `result_capped`	Enterprise, from 0.31.0, a page: its number, an ID linking the chain's pages (not the cursor, which is never logged), the rows sent across the chain, and `true` on the page that reached `TDB_MAX_RESULT_ROWS`
`stream_id`, `stream_format`, `stream_outcome`	Enterprise, from 0.31.0, a stream: `query_started` is written before it runs, then `query` with `rows_returned` and how it ended. `stream_id` is also the `X-TDB-Stream-Id` response header. A `query_started` with no `query` means the process died during the stream
`params_count`	Wire gateway entries with `TDB_AUDIT_SQL=redacted`: how many bound values the statement had, in place of `params`
`transport`, `client_user`, `statement_kind`, `params`, `outcome`, `sqlstate`	Wire gateway entries only — see the gateway's audit trail

Keeping values out of the log¶

From community 0.16.0 / enterprise 0.27.0

By default the log stores each statement exactly as submitted. That makes it good evidence, and it also means a value written into the SQL (WHERE email = 'ann@example.com') is stored in the log, so the log needs the same protection as the data. Set TDB_AUDIT_SQL=redacted to keep those values out:

{
  "event": "query",
  "sql": "SELECT name FROM customers WHERE email = ?",
  "sql_redacted": true,
  "sql_sha256": "1a8f6d9d...",
  "rows_returned": 1
}

What is hidden: string literals in every form the supported engines accept, numbers and comments. A string that is never closed is hidden to the end of the statement.
What stays: keywords, table names and column names, which can themselves be sensitive.
Application log: engine errors in the application log are scrubbed the same way, and tracebacks are withheld.
Wire gateway (enterprise): bound values are not stored, only params_count.
Not rewritten: entries already in the log, and the query response, which still repeats the SQL to the caller who sent it.

The cost: an investigation can see what ran, but not with which values, so choose it deliberately if you rely on the log to reconstruct exactly what someone asked. A value other than full or redacted stops TDB at startup, so a typo cannot silently leave values in the log. Entries from before and after a change of mode sit in one log, and in enterprise the hash chain verifies across it.

From the same releases, a new audit file is created with mode 0600, readable only by the user TDB runs as. An existing file keeps its mode, so check it with ls -l. A log shipper that runs as another user needs access granted to a new file, for example by running it as the same user.

Denied attempts¶

A trail that records only what succeeded cannot answer the question an auditor actually asks: who tried what, and was turned away? TDB records refusals as first-class audit events, signed into the same chain, so a deleted denial breaks verification exactly as a deleted query does.

A query that was accepted and then failed is recorded the same way, under a reason saying why. From community 0.17.0 / enterprise 0.28.0 that covers every failure; before them, only a timeout or a busy source on REST. Agents often send a query that fails, so an alert on event: "denied" should filter on reason.

event: "denied" — an attempt that was refused:

{
  "event": "denied",
  "action": "query",
  "reason": "sql_validation_failed",
  "ts": "2026-05-22T09:01:24.881204+00:00",
  "source_id": "f47ac10b-...",
  "sql": "DROP TABLE orders",
  "key_hint": "tdbk_a...",
  "seq": 43,
  "prev_hash": "9c82...",
  "hash": "4d10..."
}

Two fields describe the refusal:

Field	Description
`action`	What was attempted: `auth`, `authorize`, `query`, `register`, `ratelimit`, `mcp_auth`, `mcp_query`, `mcp_call`, `schema` (enterprise from 0.12.0, community from 0.18.0), and — enterprise — `view_list` or `view_run`
`reason`	Machine-readable cause — see the table below

`reason`	Raised when
`missing_credentials`	No `Authorization` header was sent
`invalid_api_key`	The key is not recognised
`invalid_or_expired_token`	A JWT failed signature or expiry validation
`insufficient_role_required_<role>_have_<role>`	RBAC refused the request (403)
`rate_limit_exceeded`	The key's per-minute budget was exhausted (429)
`limit_exceeds_max`	A request's `limit` was above the deployment's `TDB_MAX_ROWS` (enterprise): `POST /v1/query` from 0.2.0; MCP tools and view runs from 0.16.0
`sql_validation_failed`	The statement was not a permitted read-only `SELECT` — including, from community 0.7.2 / enterprise 0.12.0, SQL that some supported engine would read as more than one statement
`prompt_injection_detected`	The prompt-injection filter rejected the input
`source_not_found`	The referenced source does not exist
`path_outside_allowed_dir`	A CSV path resolved outside `TDB_ALLOWED_DATA_DIR`
`sql_file_access`	The SQL itself named a file other than the source's own (from community 0.15.0 / enterprise 0.26.0; outside its data directory from 0.7.0 / 0.11.0)
`file_unreadable`	The CSV backing a source is missing or unreadable
`unsupported_source_type`	The `source_type` is not a registered connector
`registry_conflict`	The source name is already taken
`invalid_origin`	An MCP request came from a browser page on another origin (403; from community 0.10.0 / enterprise 0.15.0). See MCP: browser origins
`tool_not_permitted_<tool>`	The key's `allowed_tools` scope excludes this MCP tool — or, from enterprise 0.12.0, the REST route equivalent to it (see RBAC)
`wire_requires_read_role`	Wire gateway: the key or token is not `read`-role, or is a static admin key
`row_cap_exceeded`	Wire gateway: a result exceeded `TDB_PG_MAX_ROWS`; the connection was closed
`statement_after_transaction_end`	Wire gateway: a statement followed `COMMIT`/`ROLLBACK` in the same batch
`privileged_db_role`	Enterprise, from 0.12.0: the source connects as a database role that can reach the database server's files or programs (see connectors)
`missing_api_key`	An MCP request carried no bearer token
`invalid_connection`	Registration: the `connection` block is not valid for the `source_type`
`query_failed`	The engine rejected the query, for example a column that does not exist (from community 0.17.0 / enterprise 0.28.0). The engine's error text is in the application log only, because it can quote data
`source_unavailable`	The source's file was missing or unreadable when the query ran (from community 0.17.0 / enterprise 0.28.0)
`internal_error`	The query failed for any other reason, including a driver error from a source that cannot be reached; see the application log (from community 0.17.0 / enterprise 0.28.0)
`invalid_arguments`	An enterprise MCP tool or a view run refused its arguments: a column or function it does not know, or a view parameter that does not fit (from 0.28.0)
`view_not_found`	A view run named a view that does not exist (enterprise, from 0.28.0)
`query_timeout`	The query ran past `TDB_QUERY_TIMEOUT` and was cancelled at the source (enterprise)
`source_busy`	Another process had the DuckDB file open for writing (enterprise, from 0.23.0). See DuckDB
`approval_pending`, `approval_denied`, `approval_expired`, `approval_void`, `approval_unresolved`, `approval_requires_identity`, `approval_limit`, `approval_not_found`, `approval_not_approver`, `approval_self`	A query approval held, refused or could not be decided (enterprise, Mid-Market, from 0.32.0). See Errors & audit reasons
`insufficient_role_required_<role>_have_<role>`	The key's role is below what the endpoint needs (enterprise)
`tool_not_permitted_<tool>`	The key's `allowed_tools` does not include the tool (enterprise)

What the caller sees for each reason, and what to do: Errors & audit reasons.

To review refusals only:

cat tdb_audit.jsonl | jq 'select(.event == "denied")'

To find keys probing for access:

cat tdb_audit.jsonl | jq -r 'select(.action == "auth") | .key_hint' | sort | uniq -c | sort -rn

Community edition

tdb-community records the same denied events with the same action and reason fields. The hash chain (seq, prev_hash, hash) and GET /v1/audit/verify are enterprise features — community entries are unsigned NDJSON.

Tenants¶

With tenants on (Enterprise, from 0.33.0), each tenant has its own log in its directory, verified on its own, and every entry carries "tenant". What belongs to no tenant (refusals before a tenant is known, the operator's actions, each start's tenant list) is in the deployment's own chain, TDB_TENANTS_DIR/.deployment/audit.jsonl. The operator verifies or exports any of them with ?tenant=<name> or ?tenant=_deployment.

Hash chain¶

Each entry's hash is a SHA-256 digest of the entry's canonical JSON (all fields sorted, no whitespace, hash field excluded). The next entry includes the previous hash as prev_hash.

This creates a chain: to forge or delete any entry, an attacker would need to recompute every subsequent hash — and TDB detects the break during verification.

Entry 1: seq=1, prev_hash=000...0, hash=H1
Entry 2: seq=2, prev_hash=H1,      hash=H2
Entry 3: seq=3, prev_hash=H2,      hash=H3

Deleting entry 2 would make entry 3's prev_hash point to a non-existent hash. Adding a forged entry would produce the wrong hash value. Both are detected by GET /v1/audit/verify.

Verifying the log¶

GET /v1/audit/verify

Requires admin role. Reads the entire log file and walks the hash chain.

Response — valid log:

{
  "valid": true,
  "entries": 2847
}

Response — tampered log:

{
  "valid": false,
  "error": "hash mismatch at seq=42",
  "at_seq": 42
}

curl -H "Authorization: Bearer <ADMIN_KEY>" http://localhost:8000/v1/audit/verify

Run this check as part of your daily compliance job, or trigger it after any log rotation.

Exporting to a SIEM¶

Use POST /v1/audit/export?destination=splunk|s3 to ship the log onward. Exports are incremental — each destination resumes where it left off, so a scheduled export does not re-send history. GET /v1/audit/export/status shows how far each has consumed.

Synchronous or backgrounded¶

By default the request waits for the export to finish and returns what it shipped, which is what a cron job checking errors needs:

{"exported": 2847, "skipped": 0, "errors": [], "disabled": false}

A first export of a large backlog can take a while. Pass wait=false to get 202 straight away and let it run in the background:

curl -fsS -X POST "http://localhost:8000/v1/audit/export?destination=s3&wait=false" \
  -H "Authorization: Bearer $ADMIN_KEY"

{"accepted": true, "destination": "s3", "detail": "Export started. Read the outcome from GET /v1/audit/export/status."}

Nothing is waiting on that run, so its outcome is recorded instead of returned. GET /v1/audit/export/status reports the last run for every destination — including synchronous ones, so the field means the same thing either way:

{
  "destinations": [
    {
      "name": "s3",
      "configured": true,
      "last_exported_seq": 4210,
      "last_run": {
        "state": "succeeded",
        "started_at": "2026-08-12T04:10:02+00:00",
        "finished_at": "2026-08-12T04:10:39+00:00",
        "exported": 120,
        "skipped": 0,
        "errors": []
      }
    }
  ]
}

state is running, succeeded or failed; last_run is null for a destination that has never exported.

One run per destination at a time

A second export to a destination that is already running returns 409. Both runs would read the same cursor and ship the same entries, so overlapping them duplicates events in your SIEM.

This matters most with wait=false: a cron firing every five minutes against an export that takes ten will collide. Either widen the interval past the slowest run, or treat 409 as "the previous run is still going" and skip.

A run whose process is killed is treated as abandoned after one hour, so a crash cannot block a destination permanently.

Configuration¶

Variable	Default	Description
`TDB_LOG_FILE`	`tdb_audit.jsonl`	Path to the NDJSON audit log
`TDB_LOG_LEVEL`	`INFO`	Log verbosity (`DEBUG`, `INFO`, `WARNING`, `ERROR`)
`TDB_AUDIT_SQL`	`full`	`redacted` keeps statement values out of the logs
`TDB_AUDIT_ROTATE_MAX_BYTES`	`0` (off)	Seal the log into a segment once it exceeds this size
`TDB_AUDIT_ROTATE_MAX_DAYS`	`0` (off)	Seal the log once its oldest entry is older than this

Retention: sealed segments¶

Enterprise, from 0.4.0

Before 0.4.0 the log grew without bound and rotation was entirely manual.

TDB never deletes an audit entry. Retention here decides when the live log is sealed, not when anything is dropped.

When a threshold is crossed, TDB closes the live log with a final rotated entry, renames it to a timestamped segment, and starts a fresh log:

tdb_audit.jsonl                         ← live chain
tdb_audit-20260801T031500Z-seq48210.jsonl   ← sealed segment
tdb_audit-20260715T014500Z-seq22119.jsonl   ← sealed segment

Each segment keeps its chain intact and verifies on its own. The new live log opens with a chain_anchor entry naming the segment it follows and that segment's final hash, so the segments form a verifiable sequence rather than disconnected files.

Rotation is evaluated when an entry is written and once at startup. Set either variable, both, or neither; whichever threshold is crossed first seals the log. You can also seal on demand with POST /v1/audit/rotate (admin).

Why you should enable this

Not for speed — that argument expired in 0.8.0. Every audit write needs the previous entry's hash. 0.4.0 made that independent of history for a running server, and 0.8.0 made the first write after a restart independent of it too, by reading the chain head backwards from the end of the file (measured at 500,000 entries: 4.3 s before, under a millisecond after, and flat as the log grows). A large live log no longer slows startup or the first request.

Enable rotation for the reasons that remain: a multi-gigabyte single file is awkward to back up, copy and retain, and sealed segments are what let you prune old history while keeping every remaining segment independently verifiable. Size the live log to whatever your backup story wants.

Verifying across segments¶

GET /v1/audit/verify proves the live log has not been altered. It cannot prove a whole segment was not removed, because every segment legitimately restarts its chain from genesis.

GET /v1/audit/history walks the sealed segments, the live log, and the anchors linking them:

{ "valid": true, "segments": 4, "entries": 128340,
  "segment_files": ["tdb_audit-20260715T014500Z-seq22119.jsonl", "..."] }

Removing a segment from the middle of the sequence is reported as invalid. Pruning the oldest segments is a supported retention action and stays valid — that is how you actually delete old audit data.

Community edition: rotate with logrotate¶

tdb-community has no built-in rotation; its log grows until you rotate it. It opens the file for every entry, so an ordinary logrotate rule that renames the file is safe: the next entry starts a new file and nothing is lost.

/srv/tdb/logs/tdb_audit.jsonl {
  daily
  rotate 30
  compress
  delaycompress
  missingok
  notifempty
}

Two rules:

Mount the directory, not the file. Run with -e TDB_LOG_FILE=/logs/tdb_audit.jsonl -v /srv/tdb/logs:/logs. If you bind-mount the file itself, the container keeps writing to the renamed file and the new one stays empty.
Do not use copytruncate. Entries written between the copy and the truncate are lost.

Backup¶

Back up sealed segments to immutable object storage (S3, GCS). They never change again, which makes them well suited to write-once buckets.
Run GET /v1/audit/history before pruning anything, to confirm the sequence is intact first.

Do not edit the log file

Any modification — including deleting lines or reordering entries — will break the hash chain and fail verification. Treat the log as write-once. Use POST /v1/audit/rotate rather than truncating the file yourself: rotation is recorded in the chain, hand-truncation is indistinguishable from tampering.

Legacy entries¶

Entries written by TDB versions before hash-chaining was introduced do not contain seq or hash fields. The verifier skips these entries and starts chain validation from the first signed entry it encounters. Legacy and signed entries can coexist in the same file.