Module trase.tools.pcs.connect
Functions
def create_sqlalchemy_engine(*args, cnx=None, **kwargs)-
Example usage with Pandas:
import pandas as pd from sqlalchemy import text from trase.tools.pcs.connect import create_sqlalchemy_engine engine = create_sqlalchemy_engine() with engine.connect() as connection: df = pd.read_sql( text("select * from views.commodities limit 1"), connection, ) def parallel_enabled(func)-
Decorator that creates a new connection for a function that takes
supply_chainas an argument and tears it down after execution def run_parallel(process_function, supply_chain, *args, chunk_size=1000, n_threads=None, chunks_by=None, **kwargs)-
Run
process_functiononsupply_chainby chunks.process_functionmust be decorated withparallel_enabled(). A first chunk is run with a single thread to avoid issues with insertion of new node roles in multiple threads when runningsupply_chain.to_db. def sqlalchemy_uri(cnx=None)def uses_database(func)-
A decorator for all functions that need to access the database.
The decorator recognises up to three named arguments:
cur=None: database cursorcnx=None: database connectionautocommit=False: whether a commit should be performed after the function returns
If the values are not provided during a function call, a default connection and cursor will be created. The connection is lazy and also cached: it is only made once per Python session.
There is a full list of behaviours at trase/tools/pcs/test/test_uses_database.py.
A function decorated with
uses_database()should always passcurandcnxinto database functions that it calls. This ensures that the same cursor is used in both the parent and the child functions:```python @uses_database def my_database_function(cur=None, cnx=None): another_database_function() # bad! a new cur will be created another_database_function(cnx=cnx, cur=cur) # good :) ```It is also recommended that functions decorated with
uses_database()should set autocommit toFalsewhen they themselves call database functions. This means that the autocommit will be done at the highest function that was called:```python @uses_database def writing_function(cur=None, cnx=None, autocommit=True): another_writing_function(cnx=cnx, cur=cur) # bad! another_writing_function(autocommit=False, cnx=cnx, cur=cur) # good :) ```Note that the
cnxandcurobjects that this decorator may inject are of typeloggingConnectionandloggingCursorrespectively.
Classes
class loggingConnection (cnx)-
Modified
psycopg2.extensions.connectionwith a simple representation.The name is given by analogy to
loggingCursorbut it does not have any relation to logging.Methods
def commit(self)-
Commit embedded connection.
class loggingCursor (cur)-
Modified
psycopg2.extensions.cursorwith a counter for queries and a simple representation.Methods
def execute(self, *args, **kwargs)-
Execute query and output query to log.
def update_attributes(self)-
Update values of attributes with new values from embedded cursor.