Module trase.tools.pcs.connect

Functions

def create_sqlalchemy_engine(*args, cnx=None, **kwargs)

Example usage with Pandas:

import pandas as pd
from sqlalchemy import text

from trase.tools.pcs.connect import create_sqlalchemy_engine

engine = create_sqlalchemy_engine()
with engine.connect() as connection:
    df = pd.read_sql(
        text("select * from views.commodities limit 1"),
        connection,
    )
def parallel_enabled(func)

Decorator that creates a new connection for a function that takes supply_chain as an argument and tears it down after execution

def run_parallel(process_function, supply_chain, *args, chunk_size=1000, n_threads=None, chunks_by=None, **kwargs)

Run process_function on supply_chain by chunks. process_function must be decorated with parallel_enabled(). A first chunk is run with a single thread to avoid issues with insertion of new node roles in multiple threads when running supply_chain.to_db.

def sqlalchemy_uri(cnx=None)
def uses_database(func)

A decorator for all functions that need to access the database.

The decorator recognises up to three named arguments:

  • cur=None: database cursor
  • cnx=None: database connection
  • autocommit=False: whether a commit should be performed after the function returns

If the values are not provided during a function call, a default connection and cursor will be created. The connection is lazy and also cached: it is only made once per Python session.

There is a full list of behaviours at trase/tools/pcs/test/test_uses_database.py.

A function decorated with uses_database() should always pass cur and cnx into database functions that it calls. This ensures that the same cursor is used in both the parent and the child functions:

```python
@uses_database
def my_database_function(cur=None, cnx=None):
    another_database_function()                  # bad! a new cur will be created
    another_database_function(cnx=cnx, cur=cur)  # good :)
```

It is also recommended that functions decorated with uses_database() should set autocommit to False when they themselves call database functions. This means that the autocommit will be done at the highest function that was called:

```python
@uses_database
def writing_function(cur=None, cnx=None, autocommit=True):
    another_writing_function(cnx=cnx, cur=cur)                    # bad!
    another_writing_function(autocommit=False, cnx=cnx, cur=cur)  # good :)
```

Note that the cnx and cur objects that this decorator may inject are of type loggingConnection and loggingCursor respectively.

Classes

class loggingConnection (cnx)

Modified psycopg2.extensions.connection with a simple representation.

The name is given by analogy to loggingCursor but it does not have any relation to logging.

Methods

def commit(self)

Commit embedded connection.

class loggingCursor (cur)

Modified psycopg2.extensions.cursor with a counter for queries and a simple representation.

Methods

def execute(self, *args, **kwargs)

Execute query and output query to log.

def update_attributes(self)

Update values of attributes with new values from embedded cursor.