Module trase.tools.data_release_package.validate.markers

Markers that categorise validation functions.

Most validations only read the local package folder, so they are fast and need no credentials. A few may need to reach the actual data in S3 – following a *.data.txt pointer to check a parquet file's schema, say. Those are slow and require AWS access, so they are marked with @accesses_s3 and can be left out with --skip-s3.

The test suite always skips them, so tests never touch the network.

Functions

def accesses_s3(function: ~F) ‑> ~F

Mark a validation function as one that reads data from S3.

Validations that only read the local package folder need no marker.

def validation_accesses_s3(function: Callable) ‑> bool

True if function was marked with @accesses_s3.