Skip to content

TDB Enterprise — IT Administrator Deployment Guide

TDB Enterprise

This guide is written for the IT or platform team that will deploy and operate the commercial enterprise edition on your own infrastructure. For a faster first run, start with Installation and the Quickstart. For the free, open-source CSV edition, see the Community Edition quickstart.


0. What you are deploying

TDB Enterprise is a single, self-contained Docker image. It runs one FastAPI/uvicorn process that exposes:

  • a REST query API and source registry (/v1/...),
  • an MCP server for AI tools (/v1/mcp),
  • OpenAPI/Swagger docs (/docs),
  • operational endpoints: /health, /metrics (Prometheus), / (version banner).

It is read-only middleware in front of your data sources (CSV files and, in enterprise, PostgreSQL / MySQL / SQL Server / Snowflake, Parquet files and DuckDB database files). It does not copy or warehouse your data — it proxies governed SELECT-only queries and writes an audit record for each one. Access is gated by a signed license (see the build/licensing playbook).

There is one container; there is no separate database server to run. State lives in two files on a mounted volume (registry DB + audit log).


1. Pre-installation checklist

1.1 Operating system & runtime

Requirement Supported / recommended
Host OS Any 64-bit Linux that runs a current container engine (Ubuntu 22.04/24.04 LTS, RHEL/Rocky 8–9, Debian 12 verified in test). macOS/Windows via Docker Desktop work for evaluation.
CPU arch Production / commercial image: linux/amd64 (x86-64). Trial and evaluation images are multi-arch (linux/amd64 + linux/arm64), so they run natively on Apple Silicon Macs — docker pull selects your architecture automatically. If you need arm64 for a production deployment (e.g. AWS Graviton), tell us: it is a build-pipeline change, not a code change.
Container engine Docker Engine ≥ 20.10 (BuildKit era) or any OCI-compliant runtime (Podman ≥ 4 works).
Compose (optional) Docker Compose v2 if you deploy via compose file.
Kubernetes (optional) Any conformant cluster ≥ 1.25. We ship no Helm chart or manifests today — a reference Deployment is in §4.3.

The image is based on python:3.12-slim. The only system library baked in is unixodbc (so the SQL Server connector can load). To actually connect to SQL Server you additionally need the Microsoft ODBC Driver 18, which is not in the image — see §5.4.

1.2 Hardware sizing (starting point — not an SLA)

Starting points, not an SLA. These are conservative recommendations derived from the deployment footprint — they are not benchmarked figures, and we do not publish throughput numbers we have not measured. Size against your own workload: §8 gives a repeatable method.

Profile vCPU RAM Disk
Evaluation / single user 1 1 GB 5 GB
Small team (default) 2 2–4 GB 20 GB
Heavier CSV / concurrent 4 8 GB 50 GB+

Notes that drive sizing: - The image is ~520 MB on disk / ~130 MB compressed. - CSV queries run in-process via DuckDB, which is memory- and CPU-hungry on large files (it reads the file per query — see §6). RAM should comfortably exceed your largest CSV working set. Database connectors push the query to the remote engine and, from 0.2.1, read only the rows they return — so the TDB host stays light regardless of how large the result set would have been. On 0.2.0 and earlier they read the whole result set first: size those hosts for the largest result a permitted query can return (see §8). - Disk grows mainly with the audit log (one NDJSON line per query, refusal or failure, append-only). Plan rotation/retention — see §5.3.

1.3 Network & ports

  • The container listens on TCP 8000 (HTTP) inside the container. Publish/remap it with -p (e.g. -p 8000:8000). The server speaks plain HTTP — terminate TLS in front of it (see §7).
  • Egress the host must allow: to each registered data source (e.g. Postgres 5432, MySQL 3306, SQL Server 1433, Snowflake 443); to your private registry if pulling the image; and to your SIEM endpoint if you enable the Splunk exporter (HEC over 443/8088). If you enable Client ID Metadata Documents (TDB_OAUTH_CIMD_HOSTS, off by default, 0.18.0 and later), allow HTTPS (443) to the hosts you list there. TDB connects to them directly and does not use an egress proxy. If you set an approval webhook (TDB_APPROVAL_WEBHOOK_URL, 0.32.0 and later), allow HTTPS to that one host.
  • No inbound connections are required other than to port 8000 from your API/MCP clients (and the reverse proxy).

1.4 Getting the image

Two channels — see Installation for the step-by-step version:

A. Customer-scoped private registry (default). You receive a pull token scoped to your own repo path:

docker login registry.tdb.asvanlabs.com        # use the token we provide
docker pull registry.tdb.asvanlabs.com/tdb-customers/<your-slug>/tdb-enterprise:<edition>-<expiry>-<build_sha>

B. Air-gapped tarball. For locked-down environments we deliver a .tar.gz:

docker load < tdb-enterprise-<edition>-<expiry>-<build_sha>.tar.gz
docker images tdb-enterprise        # confirm the tag is now present

License: trial images have a signed, expiring license baked in — nothing to configure. Commercial deployments may instead receive a token-less image and supply the license at runtime via -e TDB_LICENSE=.... See §5.5.


2. Quick deploy (single container)

The minimum viable run. Generate real secrets — do not copy the placeholders.

docker run -d --name tdb -p 8000:8000 \
  -e TDB_API_KEYS="$(python3 -c 'import secrets;print(secrets.token_urlsafe(32))')" \
  -e TDB_JWT_SECRET="$(python3 -c 'import secrets;print(secrets.token_hex(32))')" \
  -e TDB_ADMIN_USER=admin \
  -e TDB_ADMIN_PASSWORD='<choose-a-strong-password>' \
  -e TDB_LOG_FILE=/app/logs/tdb_audit.jsonl \
  -v "$(pwd)/tdb-data:/app/data" \
  -v "$(pwd)/tdb-logs:/app/logs" \
  -v "$(pwd)/csv:/data:ro" \
  <image>

What each mount is for: - /app/data — persistent state (SQLite registry DB + API-key store). Must be writable. Do not mount it read-only: the registry DB is created here at startup, and a read-only mount makes the container exit at migration (unable to open database file). - /app/logs — the audit log (if you point TDB_LOG_FILE here). Persist it. - /data (read-only) — where you place CSV, Parquet and DuckDB files. The image confines registered file paths to this directory by default (TDB_ALLOWED_DATA_DIR=/data). Mounting it :ro is correct and recommended — it's a separate path from /app/data.

Confirm it came up:

docker logs tdb | grep license_ok      # INFO: license_ok customer=... edition=... tier=... days_left=...
curl -fsS http://localhost:8000/health  # {"status":"ok",...}

The container runs as a non-root user (tdb). Keep it that way; do not add --user root.


3. Configuration reference (environment variables)

All configuration is via environment variables — there is no config file to edit inside the image. Defaults in parentheses.

3.1 Core / required

Variable Default Purpose
TDB_API_KEYS (required) Comma-separated static API keys (Authorization: Bearer <key>), each with admin rights. From 0.25.0 there is no default, and TDB will not start with an example value, or when unset unless another login exists.
TDB_JWT_SECRET (empty) HMAC secret for JWT auth. Required for JWT/OAuth login flows.
TDB_ADMIN_USER / TDB_ADMIN_PASSWORD (empty) Admin credentials for the login/OAuth consent flow.
TDB_LICENSE (empty) Runtime license token. Takes precedence over the baked file (commercial/renewals).
TDB_LICENSE_FILE /app/license.jwt Path to the baked license token.

3.2 Data, storage & logging

Variable Default Purpose
TDB_REGISTRY_DB data/tdb_registry.db SQLite path for the source registry + API keys.
TDB_ALLOWED_DATA_DIR /data (set in image) Confines registered CSV, Parquet and DuckDB file paths to this dir (symlinks/.. resolved). Empty = no restriction. DB connectors unaffected.
TDB_LOG_FILE tdb_audit.jsonl Audit log path (NDJSON). Point at a persisted volume, e.g. /app/logs/....
TDB_LOG_LEVEL INFO DEBUG/INFO/WARNING/ERROR.
TDB_AUDIT_SQL full From 0.27.0. redacted keeps values in statements out of the audit and application logs (§5.3). Any other value stops startup.
TDB_VIEWS_DIR (unset) Directory of YAML named-view definitions. Unset = views disabled.
TDB_POLICIES_FILE (unset) YAML column and row policy file (from 0.22.0, Mid-Market). Unset = no policies. An invalid file, or an SMB licence, stops startup.
TDB_SCHEMA_CACHE_TTL 300 Schema cache TTL (seconds). 0 disables caching.
TDB_MAX_ROWS 1000 Ceiling on rows in a single query response, on every path: POST /v1/query, the MCP tools, and view runs (the MCP tools and view runs from 0.16.0; before that they clamped at a fixed 1,000). A request asking for more is rejected with 400; a result larger than the request's limit is cut and flagged truncated. Raising it trades host RAM for bigger responses — size with §1.2. From 0.2.1 it caps both the response and the rows TDB reads from the source to produce it, so raising it raises both; on 0.2.0 and earlier it capped the response only (see §8). A bad value falls back to 1000 rather than uncapping.
TDB_MAX_RESULT_ROWS 100000 From 0.31.0. Rows one result may reach across a chain of pages, or in one stream.
TDB_CURSOR_TTL 3600 From 0.31.0. Seconds a page cursor stays valid.
TDB_STREAM_TIMEOUT 600 From 0.31.0. Seconds one stream may run. It cannot be turned off.
TDB_MAX_STREAMS 4 From 0.31.0. Streams one process serves at once; each holds its own source connection, so count them in your connection budget.

3.3 Access, auth & API behaviour

Variable Default Purpose
TDB_JWT_EXPIRE_MINUTES 60 JWT lifetime.
TDB_DEFAULT_RATE_LIMIT 60 Default requests/min per API key (overridable per key).
TDB_CORS_ORIGINS (empty = off) Comma-separated allowed origins. * = all (dev only).
TDB_CORS_ALLOW_CREDENTIALS false Access-Control-Allow-Credentials. Don't combine with *.
TDB_SERVER_URL (derived) Public base URL TDB advertises as the OAuth issuer. Set to your external HTTPS URL behind a proxy. From 0.18.0 it is also the base of the absolute resource_metadata URL in the MCP 401 challenge.
TDB_OAUTH_CIMD_HOSTS (empty: off) Comma-separated hosts whose OAuth Client ID Metadata Documents TDB may fetch (0.18.0 and later). Enabling it makes TDB fetch HTTPS URLs that clients choose, from inside your network; every fetch is guarded (public addresses only, no redirects, size and time limits). Leave empty unless a client needs it. Not usable air-gapped.
TDB_APPROVAL_PENDING_TTL, TDB_APPROVAL_TTL, TDB_APPROVAL_MAX_PENDING 86400, 3600, 20 From 0.32.0, Mid-Market. How long a query approval request waits, how long an approval can be run, and how many requests one requester may have waiting.
TDB_APPROVAL_WEBHOOK_URL, TDB_APPROVAL_WEBHOOK_SECRET, TDB_APPROVAL_WEBHOOK_SQL (unset: off), —, redacted From 0.32.0. Announce new approval requests with a signed POST. https only (http to localhost); a URL without a secret of at least 32 characters stops startup.
TDB_OIDC_ISSUER (unset: off) From 0.24.0, Mid-Market. Turns on single sign-on and SCIM, whose page lists the settings that go with it.
TDB_LEAN_MODE off Default Lean Mode for MCP results (0.21.0 and later): off, on, or a JSON object such as {"lean": true, "max_cell_chars": 500}. Smaller results for AI agents; a key's setting or a call's own arguments override it. MCP only — REST is unchanged. An invalid value refuses every MCP tool call until fixed, naming this variable, rather than being ignored.

3.4 SIEM export (optional — Splunk)

Variable Default Purpose
TDB_SPLUNK_HEC_URL (unset = exporter off) Splunk HEC endpoint, e.g. https://splunk.corp/services/collector/event.
TDB_SPLUNK_HEC_TOKEN (empty) HEC token (a secret — see §6.3).
TDB_SPLUNK_INDEX (unset) Target index.
TDB_SPLUNK_SOURCETYPE tdb:audit Event sourcetype.
TDB_SPLUNK_VERIFY_TLS true Set false only for self-signed lab Splunk.

3.5 Advanced

Variable Default Purpose
TDB_LICENSE_PUBKEY_FILE (baked key) Override the license verification public key path. Normally leave unset.
TDB_POOL_SIZE 0 (off) Reuse PostgreSQL connections instead of opening one per query. See the note below before enabling.

Connection pooling and your database's connection budget. With TDB_POOL_SIZE unset, TDB opens a connection per query and closes it, and holds nothing open between requests — so it costs your database server nothing while idle. Setting a positive value trades that for speed: on a link where the handshake costs more than the query, most of a short query's database time is the connection setup.

Before enabling it, count the ceiling against your server's max_connections:

TDB_POOL_SIZE  ×  registered PostgreSQL sources  ×  TDB processes

Each registered source pools separately — they authenticate as different users against different databases and cannot share sessions. Idle connections are closed after 5 minutes, so a burst does not hold its peak indefinitely, and pooled sessions remain read-only across reuse exactly as fresh ones are.

PostgreSQL only in this release. MySQL, SQL Server and Snowflake ignore the variable and continue to connect per query.

Sizing it by measurement rather than by guess (0.6.0 and later). A pool that is too small does not fail — it queues, and the wait is invisible in query latency because the query has not started yet. tdb_pool_checkout_duration_seconds is that wait. A healthy pool checks out in microseconds; a p95 climbing into milliseconds means requests are queueing for a connection before they reach PostgreSQL, and the pool wants to be larger (budget permitting). tdb_pool_checkout_failures_total{reason="pool_timeout"} is the same condition after it has become an error. Neither metric records anything while pooling is off, so an empty series is expected rather than a fault.


4. Deployment topologies

# docker-compose.yml
services:
  tdb:
    image: registry.tdb.asvanlabs.com/tdb-customers/<slug>/tdb-enterprise:<edition>-<expiry>-<build_sha>
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      TDB_API_KEYS: ${TDB_API_KEYS}
      TDB_JWT_SECRET: ${TDB_JWT_SECRET}
      TDB_ADMIN_USER: ${TDB_ADMIN_USER}
      TDB_ADMIN_PASSWORD: ${TDB_ADMIN_PASSWORD}
      TDB_LOG_FILE: /app/logs/tdb_audit.jsonl
      TDB_SERVER_URL: https://tdb.yourcompany.com
    volumes:
      - tdb-data:/app/data
      - tdb-logs:/app/logs
      - ./csv:/data:ro
volumes:
  tdb-data:
  tdb-logs:

Keep secrets in a .env file (referenced as ${...}) with chmod 600, or use your platform's secret store — see §6.3.

4.2 Single docker run

See §2. Add --restart unless-stopped for production. The image already declares a HEALTHCHECK against /health, so docker ps / orchestrators see container health automatically.

4.3 Kubernetes (reference only)

Reference only — not a supported artifact. No official Helm chart or manifests ship today. The manifest below is a minimal starting point to adapt to your cluster conventions; we do not support it as a deployment artifact. The supported single-host path is Docker Compose (§4.1).

apiVersion: apps/v1
kind: Deployment
metadata: { name: tdb-enterprise }
spec:
  replicas: 1                       # see scaling caveat below
  selector: { matchLabels: { app: tdb } }
  template:
    metadata: { labels: { app: tdb } }
    spec:
      securityContext: { runAsNonRoot: true }
      containers:
        - name: tdb
          image: <image>
          ports: [ { containerPort: 8000 } ]
          envFrom: [ { secretRef: { name: tdb-secrets } } ]
          volumeMounts:
            - { name: data, mountPath: /app/data }
            - { name: logs, mountPath: /app/logs }
          readinessProbe: { httpGet: { path: /health, port: 8000 } }
          livenessProbe:  { httpGet: { path: /health, port: 8000 } }
      volumes:
        - { name: data, persistentVolumeClaim: { claimName: tdb-data } }
        - { name: logs, persistentVolumeClaim: { claimName: tdb-logs } }

Scaling caveat: state (registry DB + audit log) is local SQLite + a local file. Do not run multiple replicas against the same volume — SQLite and the append-only hash-chained log assume a single writer. Run one replica with a Recreate strategy and an RWO volume. Horizontal scale-out is not supported today — this is a property of the state layer, not of the packaging, so no chart or manifest change would lift it.


5. Custom configuration tasks

5.1 Registering a data source

Sources are registered at runtime over the REST API (admin key), not in a config file. Example (PostgreSQL):

curl -fsS -X POST http://localhost:8000/v1/sources \
  -H "Authorization: Bearer $ADMIN_KEY" -H 'Content-Type: application/json' \
  -d '{
        "name": "orders_pg",
        "source_type": "postgres",
        "connection": {
          "host": "db.internal", "port": 5432, "dbname": "sales",
          "user": "tdb_readonly", "password": "***",
          "schema": "public", "table": "orders"
        }
      }'
  • One registered source maps to one table (matches the CSV model). Need several tables? Register several sources, or use named views.
  • Read-only is enforced at the engine level, not just by SQL parsing: Postgres sets connection.read_only = True; MySQL uses a read-only transaction; SQL Server always rolls back (autocommit off) and is fronted by the SELECT-only validator.
  • Best practice: give TDB a least-privilege, read-only DB login (e.g. Postgres role with SELECT only; SQL Server db_datareader). This is your real guarantee — TDB's enforcement is defence-in-depth on top of it.
  • From 0.12.0 TDB enforces part of this: a source connecting as a role that can reach the database server's files or programs — a PostgreSQL superuser, a MySQL user with FILE, SQL Server sa / sysadmin / bulkadmin — is refused on every query (403, audited privileged_db_role). A read-only session does not stop such a role reading host files, so every API key allowed to query the source would inherit that reach. Plan the role before upgrading; TDB_ALLOW_PRIVILEGED_DB_ROLE=true overrides.
  • The PostgreSQL wire gateway (from 0.14.0) is off unless TDB_PG_PORT is set. When on: open that port only to the clients that need it, give TDB a certificate they trust (TDB_PG_TLS_CERT/TDB_PG_TLS_KEY), run a single worker process, and budget one source-database session per connected client (TDB_PG_MAX_CLIENTS). See PostgreSQL wire gateway.
  • For PostgreSQL over a network you do not control, set "sslmode": "verify-full" and mount the CA as sslrootcert. From 0.12.0 TDB never sends the password in cleartext to a server whose identity it has not verified.

Security: the connection block (including the password) is persisted, encrypted at rest — read §6.2 for the key model before registering anything against production.

5.2 Named views (optional)

Set TDB_VIEWS_DIR=/app/views and mount a directory of YAML view definitions to expose curated, reusable queries. Leave unset to disable the feature.

5.2.1 Column and row policies (optional, Mid-Market)

From 0.22.0. Mount a policy file read-only and set TDB_POLICIES_FILE:

-v /srv/tdb/policies.yml:/config/policies.yml:ro -e TDB_POLICIES_FILE=/config/policies.yml
  • Read once at startup: restart to load a change. The log line policies_loaded sources=… rules=… sha256=… names the file's hash, which every audit entry carries as policy_sha256.
  • TDB refuses to start when the file is missing, invalid, or the licence's tier is smb (policies_tier_refused, with the reason on the console). It never serves a source without the rules written for it. Plan the restart like any other.
  • From 0.30.0, a source the file does not name refuses every caller, admins included (policy_source_unlisted). List every source, or set unlisted_sources: allow at the top of the file to leave unlisted sources open. See Sources the file does not name. Before upgrading, compare the file with GET /v1/sources. After a restart, look for policy_names_no_source and policy_source_unlisted in the log.
  • Check a file against the live schema before relying on it: GET /v1/policies?validate=true (admin). From 0.30.0 it compares source names exactly, as enforcement does, and lists sources the file does not name.
  • Row filters read key attributes (PUT /v1/auth/keys/{id}/attributes) or a JWT's attributes claim. Set them before handing out the key: a key missing one is refused.
  • A draft for your sources: docker exec tdb python -m tdb.cli policy suggest <source>.

Full reference: Column and row policies.

Query approvals (from 0.32.0). A rule's approval: {tables, columns} block makes a query that reads those columns wait for one of the source's approvers. Before turning it on:

  • Give each approver their own database key or SSO login, and name them (or their group) under the source's approvers. A static TDB_API_KEYS key names no one: it can neither request nor decide. TDB refuses to start if a rule needs approval and its source has no approvers.
  • Approvers work from the CLI: docker exec -e TDB_API_KEYS=<their key> tdb python -m tdb.cli approvals list, then approve or deny with --reason.
  • Editing the policy file and restarting voids every open request (approvals_voided in the log).
  • Optional: a signed webhook to tell approvers a request is waiting.

Full reference: Query approvals.

5.2.2 Tenants (optional, Enterprise)

From 0.33.0, with an Enterprise licence. One deployment serves several teams or customers, each in a directory of its own under TDB_TENANTS_DIR:

-v /srv/tdb/tenants:/tenants -e TDB_TENANTS_DIR=/tenants
docker exec tdb python -m tdb.cli tenants init acme   # prints acme's admin key, once
docker restart tdb

Before turning it on:

  • Mint a database key for default (the deployment as it was). With tenants on, static TDB_API_KEYS keys become the operator: they manage tenants and read no tenant's data.
  • Keep default's data directory (TDB_ALLOWED_DATA_DIR) outside TDB_TENANTS_DIR, or TDB will not start.
  • With single sign-on, add tenant: to every group in TDB_SSO_FILE.

Back up TDB_TENANTS_DIR whole: each tenant's registry (with its -wal and -shm files, from 0.35.1) and audit chain is in it, and the deployment's own chain is in .deployment/. Give each tenant a database user that reads only its data; TDB keeps tenants from naming each other's sources, not from sharing a password.

Full reference: Tenants.

5.3 Logging & audit

  • The audit log is append-only NDJSON at TDB_LOG_FILE, hash-chained (each entry carries seq + prev_hash + SHA-256 hash) so tampering is detectable.
  • Both outcomes are recorded. A successful query writes event: "query"; a refusal writes event: "denied" with the attempted action and a machine-readable reason (e.g. invalid_api_key, sql_validation_failed). Rejected access attempts are therefore first-class audit evidence, not just application-log noise — and because denials are signed into the same chain, deleting an inconvenient one breaks /v1/audit/verify.
  • A query that fails is recorded too (0.28.0+): event: "denied" with query_failed, source_unavailable or internal_error. Agents often send a query that fails, so an alert on denied should filter on reason.
  • The log holds what callers typed. By default each statement is stored as written, including any values in it, so protect the log like the data. TDB_AUDIT_SQL=redacted (0.27.0+) stores it with values replaced by ?, at the cost of not seeing which values were asked for. See Keeping values out of the log.
  • New audit files are created 0600 (0.27.0+), readable only by the user TDB runs as. An existing file keeps its mode. A log shipper running as another user needs access granted.
  • Verify integrity any time (admin role):
    curl -fsS -H "Authorization: Bearer $ADMIN_KEY" http://localhost:8000/v1/audit/verify
    # {"valid": true, "entries": N}
    
  • Retention (0.4.0+): set a rotation threshold. Both default to off, which lets the log grow without bound:
    TDB_AUDIT_ROTATE_MAX_BYTES=268435456   # seal at 256 MB
    TDB_AUDIT_ROTATE_MAX_DAYS=30           # or seal monthly
    
    TDB seals the live log into a timestamped segment and starts a fresh chain. It never deletes an entry — pruning old segments is your decision. Each segment verifies on its own, and GET /v1/audit/history verifies the whole sequence including the links between segments, so a segment removed from the middle is detectable. Removing the oldest segments is a supported way to age data out and keeps the history valid. Seal on demand with POST /v1/audit/rotate. Prefer these over truncating the file yourself: rotation is recorded in the chain, hand-truncation looks like tampering. Optionally stream to a SIEM via the Splunk exporter (§3.4).

    Denial auditing applies to both editions. Entry shapes and the verification endpoint are documented in full at Audit Log.

5.4 SQL Server driver

The image ships unixodbc but not the Microsoft ODBC Driver 18. If you register a SQL Server source you must provide the driver. A source names it by its odbcinst.ini name (the default is ODBC Driver 18 for SQL Server); from 0.35.0 a path to the library is refused, because ODBC would load whatever library a path names. The cleanest path is to install it in a thin layer on top of our image:

FROM <tdb-enterprise-image>
USER root
RUN curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft.gpg \
 && curl -fsSL https://packages.microsoft.com/config/debian/12/prod.list > /etc/apt/sources.list.d/mssql.list \
 && apt-get update && ACCEPT_EULA=Y apt-get install -y --no-install-recommends msodbcsql18 \
 && rm -rf /var/lib/apt/lists/*
USER tdb

The connector defaults to Encrypt=yes. For self-signed SQL Server certs set "trust_server_certificate": true in the source's connection block (lab only).

5.5 License

  • Trial: baked in and expiring. On expiry the container keeps /health, /, and /metrics green but returns HTTP 403 on every data route. Renew by re-pulling a new image.
  • Commercial: supply/rotate the token at runtime with -e TDB_LICENSE=<token> and restart — no rebuild. See Installation → License and the TDB_LICENSE / TDB_LICENSE_FILE entries in the environment variable reference.
  • Tier (from 0.22.0): the startup line license_ok … tier=smb|midmarket|enterprise shows it. Column and row policies, single sign-on and approvals need midmarket or above; tenants need enterprise (from 0.33.0). A licence issued before 0.22.0 counts as Mid-Market. An image before 0.33.0 reads an enterprise licence as malformed and serves nothing: upgrade the image first.
  • To extend a trial or renew a commercial license, contact hello@tdb.asvanlabs.com.

5.6 Snowflake sources

Tested against a real Snowflake account from 0.35.2 (queries, paging, streaming, the timeout, the MCP tools, policies, views and approvals); run 0.36.1 or later. Three things differ from the other databases:

  • The role is the only write boundary. Snowflake has no read-only session, so beyond TDB's own read-only check only the registered role stops a write. Give it USAGE on the warehouse, database and schema and SELECT on the tables, nothing more (minimum permissions).
  • The service user must be TYPE = LEGACY_SERVICE. A PERSON user with only a password signs in and is then refused its role (error 390186).
  • A role that cannot use the database or warehouse still signs in. From 0.35.2 registration checks the session's current database, schema and warehouse, and refuses the source as not accessible (503) instead of failing on every query later.

For any database source, from 0.36.0 a 503 "not accessible" leaves a source_unreachable WARNING line in the application log with the driver's reason, password masked (Snowflake troubleshooting).


6. Security model — what an admin must know

6.1 What TDB stores, and where

Data Location At rest
Source registry (incl. DB connection + password) SQLite TDB_REGISTRY_DB (/app/data/tdb_registry.db) Encrypted (Fernet, enc:v1: prefix) — whole connection blob, no plaintext mode. Key from TDB_ENCRYPTION_KEY or an auto-generated 0600 file beside the DB — see §6.2
API keys same SQLite DB (api_keys table) SHA-256 hashed (only hash + 6-char prefix); raw key shown once at creation
OAuth clients / authorization codes same SQLite DB codes are short-lived
Query approval requests (0.32.0 and later) same SQLite DB (approvals table) The statement is encrypted with the credential key and deleted when the request is denied, expires or is voided; the rest (who, which source, which columns, decision) is kept as a record
Audit log (query text, source id, row count, key hint, timestamp) TDB_LOG_FILE NDJSON plaintext, hash-chained
License token /app/license.jwt (or TDB_LICENSE env) signed JWT; baked tokens are recoverable via docker history
Tenants (0.33.0 and later, Enterprise) TDB_TENANTS_DIR/<tenant>/: its own registry.db (sources, keys, approvals), audit.jsonl, data/, optional policies.yml and views/; the deployment's chain in .deployment/audit.jsonl. default stays at the paths above, and people who sign in stay in TDB_REGISTRY_DB As the rows above, per tenant; one encryption key for the deployment

6.2 Where source credentials live

When you register a database source, the entire connection block — including the password — is encrypted before it is written to the sources.connection column of the SQLite registry DB (Fernet: AES-128-CBC + HMAC-SHA256, values prefixed enc:v1:). The whole blob is encrypted, not just password-like fields, so host and username are covered too. There is no plaintext mode — you cannot turn this off, and a database from an older version is re-encrypted in place at first startup.

The encryption key comes from one of two places:

Key source How What it protects against
TDB_ENCRYPTION_KEY (recommended) You supply it from your secrets manager Theft of the database file and theft of the whole data volume — the key never lands on disk beside the data
Auto-generated (default) Created as .tdb_secret_key (mode 0600) beside the registry DB on first start Theft of the database file alone — e.g. a stray backup or a copied .db. Not theft of the whole data directory, since the key travels with it

Be precise about that distinction in any security questionnaire: with the default key, "credentials are encrypted at rest" is true and the key sits next to the data. Setting TDB_ENCRYPTION_KEY is what makes the claim unconditional.

Still applies regardless:

  • Protect the /app/data volume: restrictive filesystem permissions, and put it on an encrypted volume / encrypted-at-rest storage class.
  • Use a dedicated least-privilege, read-only DB account per source so the blast radius of a leaked credential is "read this one table," not "own the database."
  • Restrict who holds an admin API key — registering/reading sources is an admin operation.
  • Back the key up separately from the database. Losing it means re-registering every source; there is no offline recovery and no online re-key command today.

Registration responses and GET /v1/sources mask secret fields (password, token, …) as ***, so credentials are not echoed back over the API either.

Full detail: Credentials at Rest and the TDB_ENCRYPTION_KEY entry in the environment variable reference.

6.3 Where runtime secrets live

TDB_JWT_SECRET, TDB_ADMIN_PASSWORD, TDB_SPLUNK_HEC_TOKEN, and a runtime TDB_LICENSE are read from environment variables. They are not written to the registry DB, but standard env-secret caveats apply: they are visible via docker inspect, the container's /proc/1/environ, and orchestrator manifests. Inject them from your secret manager (Docker/K8s secrets, Vault) rather than committing them. A baked license token is additionally recoverable from image metadata (docker history) — acceptable because it only verifies against our public key and grants nothing beyond what was issued.

6.4 DuckDB / query engine access

  • CSV and Parquet sources are queried in-process by DuckDB; there is no DuckDB server, and TDB keeps no DuckDB database of its own. DuckDB reads the file directly per query.
  • A DuckDB source (enterprise, from 0.23.0) is a DuckDB database file you provide. TDB opens it read-only, in an engine of its own that can open no other file, and only while a query runs, so the process that maintains the file can write between queries. While that process holds the file, queries return 503 (source_busy in the audit log). A :ro mount works, including after a writer crash that left a .wal file.
  • CSV file access is confined to TDB_ALLOWED_DATA_DIR (default /data); symlinks and .. are resolved and paths outside are rejected. Keep that mount read-only. From 0.11.0 this also bounds the SQL: DuckDB refuses any file outside that directory named inside a query, extensions cannot be loaded, and a query cannot change the engine's settings. A query may contain only one statement.
  • Queries are SELECT-only, enforced by the SQL validator before execution; the CSV path is passed through DuckDB's read_csv Python API (never string-interpolated), which prevents path/SQL injection.

6.5 Authentication & directory integration

TDB authenticates callers via static API keys, JWT (HMAC, TDB_JWT_SECRET), and OAuth 2.1 + PKCE (for Claude/ChatGPT/Cursor MCP), plus a single TDB_ADMIN_USER/TDB_ADMIN_PASSWORD for the consent/login flow. Authorization is RBAC (read / readwrite / admin) per API key, with optional MCP tool-level scoping. With tenants on (0.33.0, Enterprise), the static keys and the admin login are the operator, which manages tenants and reads none, and a request carrying two Authorization headers is refused (400, with tenants off too). From 0.18.0, every audit entry written for an MCP request also records which AI client made it and at what protocol version (mcp_client, mcp_protocol). Clients report these themselves, so treat them as a label, not proof of identity.

Directory / AD integration is not supported today: - Single sign-on is OIDC only, from 0.24.0 on Mid-Market (setup). SAML and direct LDAP login are not supported; Active Directory works through an IdP that speaks OIDC, such as Entra. Okta and Entra have not yet been run against TDB. - No Kerberos / keytab / Windows-integrated auth to data sources. All DB connectors use SQL username/password auth only (SQL Server uses UID/PWD, not Trusted_Connection). If your security policy mandates AD/Kerberos service accounts for database access, that requirement is not met today — the supported pattern is a dedicated, least-privilege SQL login per source. If AD-backed database auth is a hard requirement in your environment, raise it with us before you evaluate.


7. TLS / SSL

TDB serves plain HTTP on port 8000 and does not terminate TLS itself. For any non-localhost deployment, put it behind a TLS-terminating reverse proxy (nginx, Caddy, Traefik) or an ingress controller / cloud load balancer.

Minimal nginx example:

server {
    listen 443 ssl;
    server_name tdb.yourcompany.com;
    ssl_certificate     /etc/ssl/certs/tdb.crt;
    ssl_certificate_key /etc/ssl/private/tdb.key;

    location / {
        proxy_pass         http://127.0.0.1:8000;
        proxy_set_header   Host              $host;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto $scheme;
    }
}

Then set TDB_SERVER_URL=https://tdb.yourcompany.com so TDB advertises the correct external HTTPS URL as its OAuth issuer. Note the OAuth flow requires https:// redirect URIs (only http://localhost is exempted) — OAuth will not work over plain HTTP to a non-local host, which is by design. Restrict the container's port 8000 to the proxy (bind to 127.0.0.1 or a private network), not the public interface.


8. Performance & capacity

No published benchmark figures. We deliberately do not quote throughput or latency numbers, because we have not run a benchmark we would be willing to hold ourselves to. What follows is a repeatable method for measuring TDB on your own hardware and data, plus the qualitative behaviour that drives the result.

How to benchmark in your environment: 1. Register a representative source (your real CSV size / DB table). 2. Drive load with a tool like k6/hey/wrk against POST /v1/query with a representative SELECT, ramping concurrency. 3. Watch container CPU/RAM (docker stats) and the /metrics endpoint. On 0.6.0 and later the useful signals are tdb_source_query_duration_seconds (time in the data source, labelled by source_type), tdb_request_latency_seconds (the whole request), and tdb_audit_append_duration_seconds. The gap between the first two is TDB's own overhead; if they track each other, the time is in your source and tuning TDB will not help. On earlier versions only tdb_requests_total, tdb_request_latency_seconds and the schema-cache counters exist. 4. Ramp concurrency, do not just repeat a single query. Both performance defects found in this product to date were invisible at concurrency 1 and only appeared under parallel load — one of them made throughput fall as load rose. A single-query timing tells you very little about behaviour under a real workload.

What to expect qualitatively: - CSV/DuckDB is CPU/RAM-bound and re-reads the file per query — large CSVs benefit most from more RAM and from the schema cache (TDB_SCHEMA_CACHE_TTL). Re-reading the file is deliberate: it is what lets a CSV source pick up appended rows and changed columns without re-registration. On 0.5.1 and later, concurrent CSV queries share one query engine. Earlier versions built a private engine per query, each claiming a thread per CPU core, so CSV throughput fell as concurrency rose — if you are sizing a CSV-backed deployment on measurements from 0.5.0 or earlier, re-measure. - DB connectors push work to the remote engine; TDB mostly marshals rows, so the TDB host stays light and the source DB is the bottleneck. - Every response is capped, and the cap is enforced after fetching — a LIMIT in your own SQL cannot raise it. The ceiling is TDB_MAX_ROWS (default 1,000; community is fixed at 1,000). When rows are dropped the response carries truncated: true; treat that flag as "this is not the whole answer." From 0.31.0 a caller can read further with a cursor or a stream, up to TDB_MAX_RESULT_ROWS (100,000). A stream holds a read session on the source for up to TDB_STREAM_TIMEOUT (10 minutes), which on PostgreSQL stops vacuum, and takes its own connection, up to TDB_MAX_STREAMS per process. Watch tdb_streams_active. - TDB reads at most TDB_MAX_ROWS + 1 rows from the source (0.2.1 and later), whatever your SQL says. TDB appends LIMIT <n> only when the statement does not already contain the word LIMIT, so a query carrying its own LIMIT 100000 — or one that merely mentions the word, such as a credit_limit column — reaches the source uncapped. The fetch is therefore what enforces the ceiling: a server-side cursor on PostgreSQL, an unbuffered cursor on MySQL, a bounded fetch on SQL Server, Snowflake, CSV, Parquet and DuckDB. A two-million-row query returning ten rows costs the TDB host under a megabyte. On 0.2.0 and earlier this was not true — the full result set was read before being cut to the ceiling, so size those hosts for the largest result a permitted query can produce. - A query cannot run forever (0.3.0 and later). TDB_QUERY_TIMEOUT (default 30 seconds) is applied to every database source, and it is enforced by the engine — statement_timeout, max_execution_time, the ODBC query timeout, STATEMENT_TIMEOUT_IN_SECONDS — so the database abandons the work rather than TDB merely abandoning the wait. A cancelled query returns 504 and is recorded in the audit log as a query_timeout denial. Set TDB_QUERY_TIMEOUT=0 to disable it, which is what a deployment with genuinely long-running analytical queries will want. On 0.2.2 and earlier there was no timeout at all. - Two limits this does not remove. CSV, Parquet and DuckDB sources are not interruptible — they run in TDB's own DuckDB engine with no server to ask, so a long query on them still runs to completion. And a timeout bounds duration, not cost: a heavy join still consumes the source for as long as it is allowed to run. Giving TDB its own read-only database role with a statement timeout and memory limits (PostgreSQL statement_timeout and work_mem, MySQL max_execution_time, SQL Server the query governor, Snowflake plus warehouse sizing) remains worthwhile as defence in depth — it is enforced whatever TDB is configured to do, including by an operator who sets TDB_QUERY_TIMEOUT=0. - Single process/single writer — see the scaling caveat in §4.3.


9. Post-setup verification

Run these after deploying.

  1. Liveness: curl -fsS http://localhost:8000/health → {"status":"ok"}. The version banner is GET /.
  2. License accepted: docker logs tdb | grep license_ok shows customer=... edition=... tier=... days_left=N. (If you see 403s on data routes with reason: missing|expired, the license isn't being picked up.)
  3. Auth enforced: a query without a bearer key → 401; with a valid key → 200.
  4. Register + query a source (§5.1) → rows returned.
  5. Read-only enforced: send a non-SELECT (e.g. UPDATE ...) → rejected (400), and nothing is written to the source.
  6. Audit working: confirm a new line appended to TDB_LOG_FILE after a successful query, then GET /v1/audit/verify → {"valid": true, ...}.
  7. Metrics: curl -fsS http://localhost:8000/metrics returns Prometheus text; wire it into your scraper. After step 4's query, confirm it was measured (0.6.0 and later):
curl -fsS http://localhost:8000/metrics | grep tdb_source_query_duration_seconds_count

You should see a sample carrying the source_type of the source you queried. 8. MCP (if used): point your AI client at https://<host>/v1/mcp and confirm the initialize handshake + a query_source call succeed.


10. Day-2 operations

  • Backups: snapshot the /app/data volume (registry DB + API keys) and archive the audit log. These two volumes are the entire stateful surface. From 0.35.1 the registry runs in SQLite's WAL mode: a registry file X.db has X.db-wal and X.db-shm beside it, and recent changes may be only in X.db-wal. Snapshot the whole directory at once, or stop TDB first; copying X.db alone can lose them. Keep the volume on a local disk or block device: WAL does not work on a network filesystem such as NFS.
  • Upgrades: pull the new image tag, stop the old container, start the new one against the same /app/data volume. Migrations are idempotent and run on startup.
  • Key rotation: add and rotate API keys with the /v1/auth/keys admin API (API keys; the raw key is shown once); rotate TDB_JWT_SECRET (invalidates outstanding JWTs) on a schedule.
  • License renewal: see §5.5.
  • A source answers 503 "not accessible": from 0.36.0 the application log has a source_unreachable line naming the source type and the driver's reason (wrong password, host unreachable, a role that cannot use the database). Before 0.36.0 it said nothing.
  • Monitoring: scrape /metrics; alert on /health and on audit/verify returning valid:false. On 0.6.0 and later three more signals are worth alerting on: tdb_source_query_duration_seconds p95 per source_type (a single slow source is visible without correlating against your database's own monitoring); tdb_audit_append_duration_seconds p95, which should be sub-millisecond and flat as the log grows — if it climbs with log size, turn on rotation (TDB_AUDIT_ROTATE_MAX_BYTES / _MAX_DAYS); and, if you pool, tdb_pool_checkout_duration_seconds. See Prometheus metrics for the queries.

Questions about a deployment we have not covered here? hello@tdb.asvanlabs.com