Module trase.tools.aws.athena

Helpers to register parquet datasets in the AWS Glue Data Catalog (so they are queryable from Athena).

The Glue database defaults to trase.config.settings.athena.database (set in trase.env via TRASE_ATHENA__DATABASE, default s3_big_data).

Schemas are inferred from the parquet footer by awswrangler, so nested/complex types (arrays, structs, arrays-of-structs) are mapped correctly and the registration self-corrects if a dataset's schema changes between versions.

This logic is wrapped as a DuckDB UDF (plugins/udfs/register_glue_table.py) so it can be used as a dbt post-hook via the register_glue_table macro, but it is equally usable from plain Python.

For reading from Athena, see trase.tools.aws.aws_helpers.read_athena_*.

Functions

def register_parquet_table(table: str, s3_path: str, database: Optional[str] = None) ‑> str

Register (or refresh) a parquet dataset as an external Glue/Athena table.

s3_path may point at the dataset folder or a single .parquet file inside it. Re-running refreshes the schema of this table only – other tables in the database are untouched.

Layout constraint (Athena scans everything under the table's LOCATION): each Athena table needs its own S3 folder. Passing a single parquet that shares its folder with unrelated parquets raises RuntimeError.