util module

util.get_user_json_config(filename)

Read a json file and return as dictionary

Return type:: dict
Parameters:: filename (str) –
Return type:: dict

util.get_user_data()

Get the user metadata.

Return type:: dict
Returns:: user
Return type:: dict

util.sqs_wait(*args, **kwargs)

Deprecated.

Return type:: str

util.get_calibration_data(folder_name, s3_bucket_name)

Download to local file system all data in a folder of an s3 bucket.

Return type:

list[str]

Parameters:

folder_name (str) – The s3 bucket key
s3_bucket_name (str) –

Returns:

List of all the files downloaded.

Return type:

list[str]

util.setup_uncertainty_propagation_db(config, delete_existing_table=False, delete_existing_db=False)

Setup an SQL database with several tables needed to store the results of the uncertainty progation from twinstat.

Return type:: None

The schema will use the following column names:

batch int eval_id int

The config input dict will be reviewed and all keys that contain the words ‘input’ or ‘result’ will be used as column names with the double datatype.

Requires proper IAM role that enables modification of the SQL databased in the ‘mysql_db_endpoint’

Tables are produced:

Example:

config = {
    'input_0': "sprocket_1",
    'input_1': "sprocket_2",
    'output_0': "flowrate_12"
    'output_1': "flowrate_14"
    }

‘uncertainty_propagation_samples’,

The config input dict will be reviewed and all keys that contain the words ‘input’ or ‘result’ will be used as column names with the double datatype.
Example Schema:
[
    'batch int',
    'eval_id int',
    'sprocket_1 double',
    'sprocket_2 double',
    'flowrate_12 double'
    'flowrate_14 double'
]

‘uncertainty_propagation_pdf’,

Example Schema:

[
    'flowrate_12_xtest double',
    'flowrate_12_pdf double',

    'flowrate_14_xtest double',
    'flowrate_14_pdf double'
]

‘uncertainty_propagation_cdf’,

Example Schema:

[
    'flowrate_12_xtest double',
    'flowrate_12_cdf double',

    'flowrate_14_xtest double',
    'flowrate_14_cdf double'
]

‘uncertainty_propagation_sensitivity’

Example Schema:

[
    'output_variable tinytext',
    'sprocket_1 double',
    'sprocket_2 double',
]

Parameters:

config (dictionary) – Inputs provided from the user json file.
delete_existing_table (bool, optional) – If the table exists, delete it from the database. The default is False.
delete_existing_db (bool, optional) – If the database exists attempt to delete, although this may not delete if there are other active connections to the database. A warning will be thrown if this occurs. The default is False.

Return type:

None.

util.setup_sql_db(config, table_name, delete_existing_table=False, delete_existing_db=False)

Setup an SQL database.

Return type:: None

The schema will use the following column names:

batch int eval_id int

The config input dict will be reviewed and all keys that contain the words ‘input’ or ‘result’ will be used as column names with the double datatype.

Example

Schema:

[
    'batch int',
    'eval_id int',
    'sprocket_1 double',
    'flowrate_14 double'
]

Requires proper IAM role that enables modification of the SQL databased in the ‘mysql_db_endpoint’

Parameters:

config (dictionary) – Inputs provided from the user json file.
table_name (str) –
delete_existing_table (bool, optional) – If the table exists, delete it from the database. The default is False.
delete_existing_db (bool, optional) – If the database exists attempt to delete, although this may not delete if there are other active connections to the database. A warning will be thrown if this occurs. The default is False.

Return type:

None.

util.is_steady_state(input_signal, last_n_values=100, method='relative', tolerance=0.0)

Check if the incoming signal has stopped moving and reached a constant value.

Determination made by regressing the last ‘last_n_values’ on time and calculating the probability the linear coefficient is different from ‘signal_magnitude_relevance’. If ‘signal_magnitude_relevance’ is not provided, zero is assumed.

If it is different from zero, the signal has not reached steady state and False is returned.

Example: If we have very noisy temperature it may be difficult to determine exactly when it is steady state, but we also maybe only care about temperature increases above 1deg. Thus, use ‘signal_magnitude_relevance’=1.0 and steady will only be True if the signal is changing at a rate of 1deg/s or less.

Parameters:

input_signal (np.array) – Signal to check for steady state.
last_n_values (int, optional) – Only include this number of data points in the regression. The default is 100.
method (str, optional) –
‘statistical’ or ‘relative’ The default is ‘relative.

’statistical’ uses the moving linear regression to compensate for noise

’relative’ uses the average relative change in the signal
tolerance (float, optional) – Size of statistical coefficient needed to be relevant. Or the average relative change. The default is 0.0.

Returns:

If True, the signal is steady state.

Return type:

bool