Python Interface

Dates

class epyk.core.py.PyDates.PyDates(src: Optional[PageModel] = None)

Common module for managing dates.

This module is a light wrapper on top of datetime in order to perform basic operations on dates. This will also standardise the date format to YYYY-MM-DD in the Python layer to simplify the conversion to the Javascript

All the tests in this module are using doctest.

property cob

Returns the last close of business date.

In this property the parameter weekdays is forced to True.

Usage:

page.py.dates.cob
Returns:

A string date

date_from_alias(alias: str, from_date: Optional[str] = None)

Return the date corresponding to an alias code like T, T-N, M…

Usage:

>>> PyDates().date_from_alias("T", "2019-08-08")
'2019-08-07'
Parameters:

  • alias – The alias of the operation (T-3, M-2….)

  • from_date – Optional. The start date from which the time operation is applied. Today by default

Returns:

The converted date or a list of dates.

static date_from_excel(xl_date: int)

Convert an Excel date to a standard date format YYYY-MM-DD.

Usage:

>>> PyDates().date_from_excel(39448)
'2008-01-01'

Related Pages:

Parameters:

xl_date – A date in the Excel format.

Returns:

The date as a String in the common format YYYY-MM-DD

static delta(from_dt: str, to_dt: str, format_dt: str = '%Y-%m-%d')
Parameters:

  • from_dt –

  • to_dt –

  • format_dt –

static elapsed(delta_time, with_time: bool = False)

Get the time between two dates. This function will only format the result of a delta time object.

# TODO: Fix this method

Parameters:

  • delta_time – delta_time. The delta time between two python dates.

  • with_time (bool) – Optional. A flag to mention if the time should be computed.

static from_timestamp(timestamp: int, offset: int = 0, reference: int = 60, factor: int = 1000)

The default value will be given considering the GMT time.

Usage:

timestamp_s = page.py.dates.from_timestamp(1573074335010, 0)
Parameters:

  • timestamp – The timestamp in milliseconds.

  • offset – Optional. The time zone.

  • reference – Optional. The reference shift in minutes.

  • factor –

Returns:

The server timestamp string

property month_end

Returns the last month end date.

In this property the parameter weekdays is forced to True.

UUsage:

page.py.dates.month_end
Returns:

A string date

month_ends(from_dt: str, to_dt: str, weekdays: bool = True)

Return the list of end of month dates between two dates.

UUsage:

>>> PyDates().month_ends("2019-01-01", "2019-06-05", False)
['2019-01-31', '2019-02-28', '2019-03-31', '2019-04-30', '2019-05-31']
Parameters:

  • from_dt – The start date in format YYYY-MM-DD.

  • to_dt – The end date in format YYYY-MM-DD.

  • weekdays – Optional. remove the weekends from the potential dates (take the day before). Default True.

Returns:

A list of dates.

property months

Returns the list of month end dates from the beginning of the year.

In this property the parameter weekdays is forced to True.

Usage:

page.py.dates.months
Returns:

A list of String dates

property now

dd.

Usage:

PyDates().now

Related Pages:

Returns:

Return a string timestamp

Type:

Return the current timestamp in a format YYYY-MM-DD HH

Type:

mm

static path(with_time: bool = False)

Return a predefined format for date in a file path. Using this method will ensure a consistency in the naming convention of the various files in the project.

Parameters:

with_time (bool) – Optional. Specify if the time should be added to the path.

property quarters

Return the list of quarter dates since the beginning of the year.

In this property the parameter weekdays is forced to True.

Usage:

page.py.dates.quarters
Returns:

A list of String dates

range_dates(to_dt: str, from_dt: Optional[str] = None, weekdays: bool = True)

Get the list of dates between two dates.

The date should be two string dates in the format YYYY-MM-DD. The resulting range of date will always be increasing

Usage:

>>> PyDates().range_dates("2019-01-01", "2019-01-11")
['2019-01-11', '2019-01-10', '2019-01-09', '2019-01-08', '2019-01-07', '2019-01-04', '2019-01-03', '2019-01-02', '2019-01-01']
Parameters:

  • from_dt – The start date in format YYYY-MM-DD.

  • to_dt – Optional. The end date in format YYYY-MM-DD.

  • weekdays – Optional. Remove the weekends from the potential dates (take the day before). Default True

Returns:

A list of dates.

static to_server_time(timestamp: str, offset: int = 0, reference: int = 60)

Return the converted timestamp to be stored in the database. This conversion will be based on the offset coming from the UI to convert to common time.

Usage:

>>> PyDates().to_server_time("2019-08-20 20:04:10", 2)
'2019-08-20 21:06:10'
Parameters:

  • timestamp – The client timestamp.

  • offset – Optional. The client offset time to be applied before storage in hour.

  • reference – Optional. The reference time used on the server side.

Returns:

The server timestamp string

static to_user_time(timestamp: str, offset: int, reference: int = 60)

Return the converted timestamp to be returned to the user. This is converting a stored timestamp to a user one.

Usage:

>>> PyDates().to_user_time('2019-08-20 21:06:10', 2)
'2019-08-20 20:04:10'
Parameters:

  • timestamp – The server timestamp.

  • offset – The client offset time to be applied before storage.

  • reference – Optional. The reference time used on the server side (default 20).

Returns:

The client timestamp string

property today

Return a String date in a format YYYY-MM-DD.

Even if within the property python date object are used, this function will always return a string date in a specific format to guarantee and simplify the compatibility between languages within the components.

Usage:

PyDates().today

Related Pages:

Returns:

A string date in the format YYYY-MM-DD

Mails

class epyk.core.py.PyMails.Email(sender, recipients, subject, content, attachments=None, headers=None, mimetype='text/plain', charset='utf8')
to_mime()

Return a MIME representation of this object - used internally

Markdown

class epyk.core.py.PyMarkdown.MarkDown(page: PageModel)
all(data: str)

TODO: Improve this function.

Parameters:

data – The data to be parsed.

resolve(data: str, css_attrs: Optional[dict] = None)

Convert a string to a markdown file.

Parameters:

  • data – Data to be converted.

  • css_attrs – Optional. The CSS Style to be applied to the component.

classmethod translate(data: str)
Parameters:

data –

NPM

class epyk.core.py.PyNpm.Packages
classmethod descriptions(verbose: bool = True)

Get all the packages and the short description from NPM.

Parameters:

verbose – Optional. Display version details (default True).

classmethod repositories(verbose: bool = True)

Get the repositories used to retrieve the external packages.

This is a collaborative framework so do not hesitate to contact the author of those packages with ideas or even things to fix. It is important to encourage this open source community and to be part of modules improvements.

Usage:

repos = PyNpm.Packages.repositories()
Parameters:

verbose – Optional. Display version details (default True).

classmethod versions(verbose: bool = True)

Get the current latest version of all the package in NPM. This could help on maintaining the internal framework up to date with the improvements.

It is important to align with the new version in order to benefit from the community hard work !

Parameters:

verbose – Optional. Display version details (default True).

ReST

class epyk.core.py.PyRest.PyRest(page: Optional[PageModel] = None)
static csv(url: str, delimiter: str = ',', encoding: str = 'utf-8', with_header: bool = True, store_location: Optional[str] = None, quotechar: str = '"')

Retrieve tabular data from an external REST service.

Parameters:

  • url – The url with the data to request.

  • delimiter – Optional. The line delimiter.

  • encoding – Optional. The encoding format.

  • with_header – Optional. A flag to mention if the header is available. (it will be used for the keys)

  • store_location – Optional. The temp folder to cache the data locally. False will cancel the temps data retrieval

  • quotechar – Optional.

get(url: str, data=None, encoding: str = 'utf-8', headers: Optional[dict] = None, proxy: Optional[dict] = None)

Run an external REST call using the GET method.

This should be used to retrieve data from external services. If data should be extracted using an existing internal service the method query is better as it will embedded the security aspects

Usage:

page.py.requests.get("https://api.cdnjs.com/libraries")
pyrest.get(r"https://jsonplaceholder.typicode.com/todos/1")

Related Pages:

Parameters:

  • url – Should be a string containing a valid URL

  • data – Optional. Must be an object specifying additional data to send to the server, or None if no such data is needed

  • encoding – Optional. The encoding of this request (defaults to ‘utf-8’). This encoding will be used to percent-encode the URL and to convert the body to str (if given as unicode)

  • headers – Optional. Should be a dictionary, and will be treated as if add_header() was called with each key and value as arguments

  • proxy – Optional.

Returns:

Return a Python object by default

http_server(port: int = 5000, service_name: str = '')

Start a local server for all the services. This should be at the end of the script in order to allow the services debug.

Parameters:

  • port –

  • service_name –

json(url: str, encoding: str = 'utf-8', store_location: Optional[str] = None)

Retrieve Json data from an external REST service.

Parameters:

  • url – The url with the data to request.

  • encoding – Optional. The encoding format.

  • store_location – Optional. String. The temp folder to cache the data locally.

post(url: str, data=None, encoding: str = 'utf-8', headers: Optional[dict] = None, proxy: Optional[dict] = None)

Run a external REST call using the POST method.

This should be used to retrieve data from external services. If data should be extracted using an existing internal service the method query is better as it will embedded the security aspects

Usage:

page.py.requests.post("https://jsonplaceholder.typicode.com/todos/1")

Related Pages:

Parameters:

  • url – The external URL to the REST service

  • data – Optional. Must be an object specifying additional data to send to the server, or None if no such data is needed

  • encoding – Optional. The encoding of this request (defaults to ‘utf-8’). This encoding will be used to percent-encode the URL and to convert the body to str (if given as unicode)

  • headers – Optional. Should be a dictionary, and will be treated as if add_header() was called with each key and value as arguments

  • proxy – Optional.

Returns:

The content of the REST call as a String

proxy(username: str, password: str, proxy_host: str, proxy_port: int, protocols: Optional[list] = None)

Set the proxy connexions for the Python requests.

Parameters:

  • username – The username.

  • password – The user password.

  • proxy_host – The proxy server hostname.

  • proxy_port – The proxy server port.

  • protocols – Protocols for the proxy. Default [(‘http’, ‘http://’), (‘https’, ‘https://’)]

static request(url: str, data=None, method: Optional[str] = None, encoding: str = 'utf-8', headers: Optional[dict] = None, unverifiable: bool = False, proxy: Optional[dict] = None)

Run a external REST call using a specific method (PUT, DELETE, OPTIONS, HEAD, PUT, PATCH…).

This should be used to retrieve data from external services. If data should be extracted using an existing internal service the method query is better as it will embedded the security aspects

Usage:

json.loads(PyRest().request(r"https://jsonplaceholder.typicode.com/todos/1", method="GET"))

Related Pages:

Parameters:

  • url – Should be a string containing a valid URL

  • method – Optional. Must be an object specifying additional data to send to the server, or None if no such data is needed

  • data – Optional. Must be an object specifying additional data to send to the server, or None if no such data is needed

  • encoding – String. Optional. the encoding of this request (defaults to ‘utf-8’). This encoding will be used to percent-encode the URL and to convert the body to str (if given as unicode)

  • headers – Optional. Should be a dictionary, and will be treated as if add_header() was called with each key and value as arguments

  • unverifiable – Optional. Should indicate whether the request is unverifiable, as defined by RFC 2965

  • proxy – Optional.

Returns:

The content of the REST call as a String

static webscrapping(url: str, data=None, encoding: str = 'utf-8', headers: Optional[dict] = None, unverifiable: bool = False, proxy: Optional[dict] = None)

Create a REST request with the appropriate header to mimic a browser GET request.

Usage:

PyRest().webscrapping(r"https://doc.scrapy.org/en/latest/topics/request-response.html")
Parameters:

  • url – Should be a string containing a valid URL

  • data – Optional. Must be an object specifying additional data to send to the server, or None if no such data is needed

  • encoding – Optional. the encoding of this request (defaults to ‘utf-8’). This encoding will be used to percent-encode the URL and to convert the body to str (if given as unicode)

  • headers – Optional. Should be a dictionary, and will be treated as if add_header() was called with each key and value as arguments

  • unverifiable – Optional. Should indicate whether the request is unverifiable, as defined by RFC 2965

  • proxy – Optional.

Returns:

The HTML content of the REST call as a String

Database

class epyk.core.py.PySql.SqlConn(family, database=None, filename=None, model_path=None, reset=False, migrate=True, tables_scope=None, **kwargs)
column(table_name, column_name)

Return a sqlAlchemy column object. This can be useful in the where clauses

Usage:

select('table').where([db.column("table", 'column') == 'X')
Parameters:

  • table_name – The database table name

  • column_name – The column name

Returns:

Python column object

columns(table_name)

Return the list of columns defined in the selected database.

Usage:

page.db().columns("table_name")
Parameters:

table_name –

Returns:

A python object with the list of tables

commit()

Commit the current transaction.

THis will save the results in the database

Returns:

The SQL object

property count

Return the number of records.

Usage:

print(db.select("table").count)
Returns:

The number of records.

data(limit=None)

Returns the results of the select statement previously instantiated in a pandas dataframe.

Usage:

rptObj.db().getData()
Parameters:

limit – Optional. The number of records to be returned.

Returns:

A pandas dataframe

delete(table_name)

Create a SQL delete SQL statement.

Usage:

page.db().delete('table1')

Related Pages:

Parameters:

table_name – Optional. The database table name.

Returns:

self

distinct(columns=None)
Parameters:

columns – the list of columns.

Returns:

self for the chaining.

drop(table_name, validate=True)

Delete the table from the database. The pre check can be disabled and the table will be automatically created again when the report will be triggered again. No extra function to create a table in the framework this is done by the SQL framework itself

Usage:

page.db().drop('test')
page.db().drop('test', withCheck=False)

Related Pages:

Parameters:

  • table_name – The database table name

  • validate – Boolean, Validation check before dropping the table

Returns:

Python column object

execute()

Execute the current SQL query.

Related Pages:

Returns:

The SQL Result proxy

first(items=False)

Return only the first items from the SQL query.

Usage:

print(db.first())
print(db.first(items=True))
Parameters:

items – Return a dictionary or a list of data.

Returns:

None or the first record.

force_create()

Force the creation of the database in the given project.

Returns:

The python Sql object

get_last_id(table_name)

Return the table last primary key ID. This will return an error if the table does not have a primary key defined in its schema.

Usage:

db.get_last_id("table")
Parameters:

table_name – The table name.

Returns:

Return the last row ID or -1

insert(table_name, records, commit=False, col_user_name=None, clean_rec=False, getIdCol=False)

insert a list of records to a table.

Usage:

db.insert('table1',[{'name': 'test'}], commit=True)
db.insert("table", {"user_name": "Test", "data": "test"}, commit=True)

Related Pages:

Parameters:

  • table_name – The database table name.

  • records – The list of dictionaries with the data to inserts.

  • commit – Optional. Boolean to commit the insert. Set to False by default.

  • col_user_name –

  • clean_rec – Optional. Remove the key in the dictionaries which are not related to the table. Set to False.

  • getIdCol –

Returns:

The python object itself

limit(n)

Limit the number of records returned.

Parameters:

n – Integer, the number of records.

Returns:

The SQL Query object.

load_data_file(filename, path, reset=False, new_tables=None)

Load a python sql file to the local database. This will only add records and then commit the changes.

Those data should not be sensitive ones if they are store and committed to the folder.

Parameters:

  • filename –

  • path –

  • reset –

  • new_tables –

Returns:

The Python SQL object.

load_schema(filename=None, model_path=None, reset=False)

Function that takes care of initialising the DB. Please note that some column names are prohibited such as lst_mod_dt.

Parameters:

  • filename – Optional. The python module used to get the database schema.

  • model_path – Optional. The python path with the model.

  • reset – Optional. Flag to reset the database. THis will emtpy the tables.

property records

Return the records.

Usage:

for rec in db.select("table").records:
  print(rec)
Returns:

A iterator for the SQL result

select(table_name=None, columns=None)

Create a SQL statement.

Usage:

page.db().select(["worldcup_teams"])

Documentation

Parameters:

  • table_name – String. Optional. The database table name.

  • columns – String. Optional. The list of columns.

Returns:

self

table(table_name)

Return a sqlAlchemy table object. This can be useful in the where clauses.

Usage:

db.table('table1')
Parameters:

table_name – The table name.

Returns:

Python table object

table_clone(old_table, new_table, mapping=None)

Helps to migrate between two tables. The mapping argument is used in case the column names differ between the two tables.

Parameters:

  • old_table – A SQLAlchemy table class in the current database model

  • new_table – A SQLAlchemy table class

  • mapping – Optional. A dictionary for the column names.

Returns:

self for the chaining.

table_create(table_name, table_def, reset=False)

Create a table in the database.

Usage:

db = rptObj.db(database=r"newTest.db")
tableDef = [sqlalchemy.Column('environment', sqlalchemy.String, nullable=False),
            sqlalchemy.Column('report', sqlalchemy.String, nullable=False),]
db.createTable('newTable', tableDef)
Parameters:

  • table_name – The table name.

  • table_def –

  • reset –

table_create_from_file(filename, table_name, records=None, path=None, reset=False, commit=True)

Usage:

df = page.py.file(htmlCode=r"IBRD_Balance_Sheet__FY2010.csv").read()
db = page.py.db(database=r"newTest.db").forceCreate()
dbObj.createTable('myschema.py', 'mytable', records=df)
Parameters:

  • filename –

  • table_name –

  • records –

  • path –

  • reset –

  • commit –

Returns:

The Python SQL object.

table_empty(table_name)

This function will empty an existing table.

Usage:

db.emptyTable('test')
Parameters:

table_name – A string with the datable name.

Returns:

self

table_migrate(from_table, to_table)

Copy data from one table to another.

Parameters:

  • from_table – The table name.

  • to_table – The destination table name.

Returns:

The Python SQL object

property tables

Return the list of tables defined in the selected database

Usage:

page.db().tables()
Returns:

A python object with the list of tables

update(table_name, values)

Create a delete SQL statement.

Usage:

page.db().update('table1', {db.column('test', 'name'): 'user'})

Documentation

Parameters:

  • table_name – The table name.

  • values –

Returns:

self for the chaining.

where(stmts)

Add a where clause to the SqlAlchemy query.

Usage:

db.select().where([db.column("table", 'column') == 'X')
Parameters:

stmts – The SQL where statement.

Returns:

The python object itself

class epyk.core.py.PySql.SqlConnOdbc(database=None, **kwargs)

Connector to Access databases. This connector will allow you to create, store and retrieve data from any MS Access Database. This will return the SQL database object. It will be possible to reuse the same syntax to then interact with it.

This would need the ODBC driver available here: https://www.microsoft.com/en-us/download/confirmation.aspx?id=13255

property tables

return:

class epyk.core.py.PySql.SqlConnNeo4j(host=None, port=None, usr=None, pwd=None)
alias(aliases)

Defines a set of aliases that will appear as WITH a, b, c, d as count(id) The aliases argument will be defined as follows: [‘a’, ‘b’, ‘c’, {‘d’: ‘count(id)’}]

clear()

Clears all nodes and edges from the Database

compose(query)

Simply joins the query clauses all together

foreach(conditions)

Adds the edge definition to the query

node(name='', labels=None, attr=None)

Adds the node pattern to the query

Cryptography

class epyk.core.py.PyCrypto.PyCrypto(src: Optional[PageModel] = None)
classmethod b64encode(text: str, salt: Optional[str] = None)

Usage:

PyCrypto.b64encode("Test")

Related Pages:

Parameters:

  • text (str) – The text to be encrypted.

  • salt (Optional[str]) – Optional. The salt used for the encryption (default None).

classmethod cryptKeyPairs(msg, key1, key2)

Usage:

:param msg:
:param key1:
:param key2:
decrypt(encrypted: str, token: Optional[str] = None, salt: Optional[str] = None, label: str = '')

This function will use the two keys in order to decrypt the data. In case of failure this function will raise an exception.

Usage:

PyCrypto().decrypt(encrypted)
Parameters:

  • encrypted (str) – The encrypted data.

  • token (Optional[str]) – Optional. The token used to encrypt the data.

  • salt (Optional[str]) – Optional. The salt id.

  • label (str) – Optional. A label used to store the reference in the log file.

Returns:

A string with the decrypted data.

classmethod decryptKeyPairs(encrypted, key1, key2)

Usage:

:param encrypted: The encrypted data
:param key1:
:param key2:
encrypt(data: str, token: Optional[str] = None, salt: Optional[str] = None)

This function will use the cryptography to ensure a secured encryption of the different credential and private data.

This can be also used to protect data from the report. In order to ensure the right privacy please do not store the token and the salt in the framework.

Parameters:

  • data (str) – The data to be encrypted.

  • token (Optional[str]) – Optional. The token used to encrypt the data.

  • salt (Optional[str]) – Optional. The salt id.

Returns:

The encrypted data with the salt used.

property getId

Return a unique token.

Usage:

PyCrypto.getId

Related Pages:

Returns:

A unique ID (based on the timestamp

property key

Return a Fernet key.

Usage:

PyCrypto().key

Related Pages:

Extension

class epyk.core.py.PyExt.PyExt(page: PageModel)
property crypto: PyCrypto

Property to the internal cryptography module.

This will rely on the package cryptography. This should be added to the python environment before using it. This package can be installed using the usual pip install function.

Related Pages:

Return type:

PyCrypto.PyCrypto

property dates: PyDates

This is a simple wrapper to the datetime Python module.

No external package is required to use this interface.

Usage:

page.py.dates.today()
Returns:

A PyDate object

Return type:

PyDates.PyDates

static encode_html(text: str, encoding: str = 'utf-8')
Parameters:

  • text – a test to encode with HTML special symbols.

  • encoding – Optional. teh encoding type.

static format_money(text: float, digits: int = 0, thousand_sep: str = ',', decimal_sep: str = '.', symbol: str = '£', format: str = '%s%v')
Parameters:

  • text –

  • digits – Optional.

  • thousand_sep – Optional.

  • decimal_sep – Optional.

  • symbol – Optional.

  • format – Optional.

static format_number(value: float, digits: int = 0, thousand_sep: str = ',', decimal_sep: str = '.')
Parameters:

  • value –

  • digits –

  • thousand_sep –

  • decimal_sep –

property geo: PyGeo

Property to some predefined Geolocation functions.

Return type:

PyGeo.PyGeo

import_lib(lib_name: str, folder: str = 'libs', report_name: Optional[str] = None, path: Optional[str] = None)

Import dynamically a python module.

Usage:

page.py.import_lib("test.py", folder="tables", path=r"filePath")
Parameters:

  • lib_name – The python module name.

  • folder – Optional. The internal folder with the libraries to be imported.

  • report_name – Optional. the report name in which the library is defined. Default current folder.

  • path – Optional. the path to be added to the classpath.

Returns:

The imported Python module.

import_package(package: str, sub_module: Optional[str] = None)

Install the external Python package. This can automatically install it from the Python Index online repository is missing.

Usage:

>>> PyExt().import_package("sqlalchemy").__name__
'sqlalchemy'
Parameters:

  • package – The Python Package Name.

  • sub_module – Optional. The sub module or class within the package.

Returns:

The installed Python module.

property markdown: MarkDown

Property to the Markdown String conversion.

Return type:

PyMarkdown.MarkDown

property requests: PyRest

This is a simple wrapper to the internal Python modules to run REST calls.

No external package is required to use this interface.

Returns:

A PyRest object.

Return type:

PyRest.PyRest

Geo

class epyk.core.py.PyGeo.PyGeo(page: Optional[PageModel] = None)
static distance(lat1: float, lon1: float, lat2: float, lon2: float, unit: str = 'km')

Calculate the great circle distance between two points on the earth (specified in decimal degrees) 3956

Related Pages:

Parameters:

  • lat1 – Float.

  • lon1 – Float.

  • lat2 – Float.

  • lon2 – Float.

  • unit – String. mi / km. Default km

Hash

class epyk.core.py.PyHash.SipHash(c: float = 2, d: float = 4)

Generate a unique hash ID from the given string. This is supposed to be unique with a minimum expectation of collisions. This module is only in charge of producing the hash ID and the potential collisions should be monitored in the environment by the users.

Related Pages:

hashId(text: str)

Produce a unique ID for a given string. This can be used to replace the internal numbers.

Usage:

>>> SipHash().hashId("Test")
3169529217224722230

Related Pages:

Parameters:

text – String. The String to be hashed.