main_db_class module

This module contains utility functions for working with SQLAlchemy engine connections.

Class: - SQLAlchemyEngine: Represents a class to connect with any database using SQLAlchemy.

Methods: - __init__: Initializes the class with database connection details. - test: Tests the connection to the database. - get_metadata: Retrieves schema metadata from the connection. - execute_query: Executes a SQL query against the connection. - get_metadata_df: Retrieves schema metadata in a dataframe format.

class DB

Bases: DatabaseUtils

A class to connect with any database using SQLAlchemy.

authentication_details = {AuthType.BASIC: {'database': '', 'hostname': '', 'password': '', 'port': 0, 'username': ''}}
close_session()

Close the session and set the session close flag.

Return type:

None

connection_type = 'database'
create_engine(engine, hostname, username, password, port, database, connection_name=None, connection_type=None)
execute_query(query)

Execute query against the connection

Parameters:

query (string) – Valid SQL query

Returns:

Pandas dataframe of your query results

Return type:

Dataframe

get_metadata(*args, **kwargs)

Returns the metadata for the API.

Returns:

A dictionary containing the metadata for the API.

Return type:

dict

install_missing_libraries()

Checks if there are any missing libraries required for the function to run. If there are missing libraries, it calls the install_libraries function to install them.

Return type:

bool

Returns:

True if the libraries are installed successfully, False otherwise.

read_table(table_name, schema_name='public', page_size=10000)

Read data from a table in etl_batches.

Parameters:

  • table_name (str) – The name of the table to read from.

  • page_size (int, optional) – The number of rows to read per page. Defaults to 10000.

Returns:

The DataFrame containing the data from the table.

Return type:

pandas.DataFrame

required_libs = []
test_connection()

Test connection to database

Returns:

True if connected else False

Return type:

Boolean

write_data(data, table_name, if_exists='append', schema='public')

Writes data to a table in etl_batches.

Parameters:

  • df (DataFrame) – The DataFrame to write to the table.

  • table_name (str) – The name of the table to write to.

Return type:

bool