Module trase.tools.ducklake.connection

Open a DuckDB connection with the Trase DuckLake catalog attached.

Reads settings.ducklake.*, fetches the role's Postgres password from AWS Secrets Manager, and runs trase/data_pipeline/tools/ducklake_init.sql

Password caching:

  • RO is cached on disk under ${XDG_RUNTIME_DIR:-/tmp}/trase-ducklake/ro_pw_<uid> for 12 h to share hits across connect("ro"), edr report, and other read-only workflows. The cache directory is 0700, the file is 0600, written atomically (tmp + os.replace), and validated on read (owner uid + permissions). Disable by setting TRASE_DUCKLAKE_PASSWORD_CACHE_ENABLED=false.
  • RW is never cached. ./dbt --target production and other RW callers fetch from Secrets Manager on every invocation (~2 s).

Global variables

var DUCKLAKE_READ_ONLY

READ_ONLY for the catalog attach, per role. Read by ducklake_init.sql.

Functions

def attach_metadata_postgres(con: duckdb.DuckDBPyConnection, alias: str = 'pg_meta', role: Role = 'ro') ‑> str

ATTACH the DuckLake metadata Postgres as a plain READ-ONLY postgres DB.

For diagnostics the DuckLake catalog attach doesn't surface — server GUCs (idle_in_transaction_session_timeout, statement_timeout) and pg_stat_activity. Query it via postgres_query('<alias>', $$ ... $$). Idempotent: returns alias without re-attaching (and without a redundant Secrets Manager fetch) if it is already attached. Returns the attach alias.

def build_exports(role: Role) ‑> dict[str, str]

Every value ducklake_init.sql reads through getenv(…).

The one description of the DuckLake environment: printed as export lines by python -m trase.tools.ducklake.env_export (for the dbt wrapper, the Elementary report generator and the duckdb CLI wrapper) and handed straight to :func:connect()'s getenv for the in-process path.

Contains the role's Postgres password — do not log it.

def clear_ro_cache() ‑> bool

Delete the RO password cache file (and dir if empty). Returns True if a file was removed.

def connect(role: Role = 'ro') ‑> _duckdb.DuckDBPyConnection

Open a DuckDB connection with the Trase DuckLake catalog attached.

role="ro" (default) attaches READ-ONLY with the readonly Postgres user. role="rw" attaches read-write with the rw user; the RW password is fetched from Secrets Manager on every call (never cached).

The catalog is attached as trase_ducklake and made the current catalog (USE trase_ducklake), so callers can query main_elementary.elementary_test_results directly.

The attach itself is trase/data_pipeline/tools/ducklake_init.sql — the very script trase/data_pipeline/tools/ducklake_shell.sh reads for an interactive duckdb CLI session, so the two cannot drift apart. Change extension loads, pool settings, the secret definitions or the ATTACH options there, not here.

def ro_cache_path() ‑> pathlib.Path

Path of the on-disk RO password cache file (may not exist).

def run_ducklake_init_sql(con: duckdb.DuckDBPyConnection, environment: dict[str, str]) ‑> None