{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# **Amazon Lookout for Equipment** - Demonstration on an anonymized compressor dataset\n", "*Part 5: Scheduling regular inference calls*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialization\n", "---\n", "In this notebook, we will update the repository structure to add an inference directory in the data folder:\n", "```\n", "/lookout-equipment-demo/getting_started/\n", "|\n", "├── data/\n", "| |\n", "| ├── inference-data/\n", "| | ├── input/\n", "| | └── output/\n", "| |\n", "| ├── labelled-data/\n", "| | └── labels.csv\n", "| |\n", "| └── training-data/\n", "| └── expander/\n", "| ├── subsystem-01\n", "| | └── subsystem-01.csv\n", "| |\n", "| ├── subsystem-02\n", "| | └── subsystem-02.csv\n", "| |\n", "| ├── ...\n", "| |\n", "| └── subsystem-24\n", "| └── subsystem-24.csv\n", "|\n", "├── dataset/ <<< Original dataset <<<\n", "| ├── labels.csv\n", "| ├── tags_description.csv\n", "| ├── timeranges.txt\n", "| └── timeseries.zip\n", "|\n", "├── notebooks/\n", "| ├── 1_data_preparation.ipynb\n", "| ├── 2_dataset_creation.ipynb\n", "| ├── 3_model_training.ipynb\n", "| ├── 4_model_evaluation.ipynb\n", "| ├── 5_inference_scheduling.ipynb <<< This notebook <<<\n", "| └── config.py\n", "|\n", "└── utils/\n", " ├── aws_matplotlib_light.py\n", " └── lookout_equipment_utils.py\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Notebook configuration update\n", "Amazon Lookout for Equipment being a very recent service, we need to make sure that we have access to the latest version of the AWS Python packages. If you see a `pip` dependency error, check that the `boto3` version is ok: if it's greater than 1.17.48 (the first version that includes the `lookoutequipment` API), you can discard this error and move forward with the next cell:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install --quiet --upgrade tqdm sagemaker" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Imports\n", "**Note:** Update the content of the **config.py** file **before** running the following cell" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import boto3\n", "import config\n", "import datetime\n", "import os\n", "import pandas as pd\n", "import pprint\n", "import pyarrow as pa\n", "import pyarrow.parquet as pq\n", "import sagemaker\n", "import s3fs\n", "import sys\n", "import time\n", "import uuid\n", "\n", "# Helper functions for managing Lookout for Equipment API calls:\n", "sys.path.append('../utils')\n", "import lookout_equipment_utils as lookout" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Parameters" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "DATA = os.path.join('..', 'data')\n", "RAW_DATA = os.path.join('..', 'dataset')\n", "INFER_DATA = os.path.join(DATA, 'inference-data')\n", "\n", "os.makedirs(os.path.join(INFER_DATA, 'input'), exist_ok=True)\n", "os.makedirs(os.path.join(INFER_DATA, 'output'), exist_ok=True)\n", "\n", "ROLE_ARN = sagemaker.get_execution_role()\n", "REGION_NAME = boto3.session.Session().region_name\n", "DATASET_NAME = config.DATASET_NAME\n", "BUCKET = config.BUCKET\n", "PREFIX_TRAINING = config.PREFIX_TRAINING\n", "PREFIX_LABEL = config.PREFIX_LABEL\n", "PREFIX_INFERENCE = config.PREFIX_INFERENCE\n", "MODEL_NAME = config.MODEL_NAME" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create an inference scheduler\n", "---\n", "While navigating to the model details part of the console, you will see that you have no inference scheduled yet:\n", "\n", "![Schedule Starting point](../assets/schedule_start.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Scheduler configuration\n", "Let's create a new inference schedule: some parameters are mandatory, while others offer some added flexibility.\n", "\n", "#### Parameters\n", "\n", "* Set `DATA_UPLOAD_FREQUENCY` at which the data will be uploaded for inference. Allowed values are `PT5M`, `PT10M`, `PT15M`, `PT30M` and `PT1H`.\n", " * This is both the frequency of the inference scheduler and how often data are uploaded to the source bucket.\n", " * **Note**: ***the upload frequency must be compatible with the sampling rate selected at training time.*** *For example, if a model was trained with a 30 minutes resampling, asking for 5 minutes won't work and you need to select either PT30M and PT1H for this parameter at inference time.*\n", "* Set `INFERENCE_DATA_SOURCE_BUCKET` to the S3 bucket of your inference data\n", "* Set `INFERENCE_DATA_SOURCE_PREFIX` to the S3 prefix of your inference data\n", "* Set `INFERENCE_DATA_OUTPUT_BUCKET` to the S3 bucket where you want inference results\n", "* Set `INFERENCE_DATA_OUTPUT_PREFIX` to the S3 prefix where you want inference results\n", "* Set `ROLE_ARN_FOR_INFERENCE` to the role to be used to **read** data to infer on and **write** inference output" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Name of the model on which you want to create this inference scheduler\n", "MODEL_NAME_FOR_CREATING_INFERENCE_SCHEDULER = MODEL_NAME\n", "\n", "# Name of the inference scheduler you want to create\n", "INFERENCE_SCHEDULER_NAME = config.INFERENCE_SCHEDULER_NAME\n", "\n", "# Mandatory parameters:\n", "INFERENCE_DATA_SOURCE_BUCKET = BUCKET\n", "INFERENCE_DATA_SOURCE_PREFIX = f'{PREFIX_INFERENCE}/input/'\n", "INFERENCE_DATA_OUTPUT_BUCKET = BUCKET\n", "INFERENCE_DATA_OUTPUT_PREFIX = f'{PREFIX_INFERENCE}/output/'\n", "ROLE_ARN_FOR_INFERENCE = ROLE_ARN\n", "DATA_UPLOAD_FREQUENCY = 'PT5M'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Optional parameters\n", "\n", "* Set `DATA_DELAY_OFFSET_IN_MINUTES` to the number of minutes you expect the data to be delayed to upload. It's a time buffer to upload data.\n", "* Set ``INPUT_TIMEZONE_OFFSET``. The allow values are : +00:00, +00:30, -01:00, ... +11:30, +12:00, -00:00, -00:30, -01:00, ... -11:30, -12:00\n", "* Set `TIMESTAMP_FORMAT`. The allowed values `EPOCH`, `yyyy-MM-dd-HH-mm-ss` or `yyyyMMddHHmmss`. This is the format of timestamp which is the suffix of the input data file name. This is used by Lookout Equipment to understand which files to run inference on (so that you don't need to remove previous files to let the scheduler finds which one to run on).\n", "* Set `COMPONENT_TIMESTAMP_DELIMITER`. The allowed values `-`, `_` or ` `. This is the delimiter character that is used to separate the component from the timestamp in the input filename." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "DATA_DELAY_OFFSET_IN_MINUTES = None\n", "INPUT_TIMEZONE_OFFSET = '+00:00'\n", "COMPONENT_TIMESTAMP_DELIMITER = '_'\n", "TIMESTAMP_FORMAT = 'yyyyMMddHHmmss'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create the inference scheduler\n", "The CreateInferenceScheduler API creates a scheduler **and** starts it: this means that this starts costing you right away. However, you can stop and start an existing scheduler at will (see at the end of this notebook):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "scheduler = lookout.LookoutEquipmentScheduler(\n", " scheduler_name=INFERENCE_SCHEDULER_NAME,\n", " model_name=MODEL_NAME_FOR_CREATING_INFERENCE_SCHEDULER,\n", " region_name=REGION_NAME\n", ")\n", "\n", "scheduler_params = {\n", " 'input_bucket': INFERENCE_DATA_SOURCE_BUCKET,\n", " 'input_prefix': INFERENCE_DATA_SOURCE_PREFIX,\n", " 'output_bucket': INFERENCE_DATA_OUTPUT_BUCKET,\n", " 'output_prefix': INFERENCE_DATA_OUTPUT_PREFIX,\n", " 'role_arn': ROLE_ARN_FOR_INFERENCE,\n", " 'upload_frequency': DATA_UPLOAD_FREQUENCY,\n", " 'delay_offset': DATA_DELAY_OFFSET_IN_MINUTES,\n", " 'timezone_offset': INPUT_TIMEZONE_OFFSET,\n", " 'component_delimiter': COMPONENT_TIMESTAMP_DELIMITER,\n", " 'timestamp_format': TIMESTAMP_FORMAT\n", "}\n", "\n", "scheduler.set_parameters(**scheduler_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prepare the inference data\n", "---\n", "Let's prepare and send some data in the S3 input location our scheduler will monitor:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Let's load all our original signals:\n", "all_tags_fname = os.path.join(DATA, 'training-data', 'expander.parquet')\n", "table = pq.read_table(all_tags_fname)\n", "all_tags_df = table.to_pandas()\n", "del table\n", "all_tags_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's load the tags description: this dataset comes with a tag description file including:\n", "\n", "* `Tag`: the tag name as it is recorded by the customer in his historian system (for instance the [Honeywell process history database](https://www.honeywellprocess.com/en-US/explore/products/advanced-applications/uniformance/Pages/uniformance-phd.aspx))\n", "* `UOM`: the unit of measure for the recorded signal\n", "* `Subsystem`: an ID linked to the part of the asset this sensor is attached to\n", "\n", "From there, we can collect the list of components (subsystem column):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "tags_description_fname = os.path.join(RAW_DATA, 'tags_description.csv')\n", "tags_description_df = pd.read_csv(tags_description_fname)\n", "components = tags_description_df['Subsystem'].unique()\n", "tags_description_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To build our sample inference dataset, we will extract a sample from the original evaluation period of the time series where we know something odd is happening (around **November 21st, 2015 at 4am**):" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "start = pd.to_datetime('2015-11-21 04:00:00')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# How many sequences do we want to extract:\n", "num_sequences = 12\n", "\n", "# The scheduling frequency in minutes: this **MUST** match the\n", "# resampling rate used to train the model:\n", "frequency = 5\n", "\n", "# Loops through each sequence:\n", "for i in range(num_sequences):\n", " end = start + datetime.timedelta(minutes=+frequency - 1)\n", " \n", " # Rounding time to the previous 5 minutes:\n", " tm = datetime.datetime.now()\n", " tm = tm - datetime.timedelta(\n", " minutes=tm.minute % frequency,\n", " seconds=tm.second,\n", " microseconds=tm.microsecond\n", " )\n", " tm = tm + datetime.timedelta(minutes=+frequency * (i))\n", " current_timestamp = (tm).strftime(format='%Y%m%d%H%M%S')\n", "\n", " # For each sequence, we need to loop through all components:\n", " print(f'Extracting data from {start} to {end}')\n", " new_index = None\n", " for component in components:\n", " # Extracting the dataframe for this component and this particular time range:\n", " signals = list(tags_description_df.loc[(tags_description_df['Subsystem'] == component), 'Tag'])\n", " signals_df = all_tags_df.loc[start:end, signals]\n", " \n", " # We need to reset the index to match the time \n", " # at which the scheduler will run inference:\n", " if new_index is None:\n", " new_index = pd.date_range(\n", " start=tm,\n", " periods=signals_df.shape[0], \n", " freq='1min'\n", " )\n", " signals_df.index = new_index\n", " signals_df.index.name = 'Timestamp'\n", " signals_df = signals_df.reset_index()\n", "\n", " # Export this file in CSV format:\n", " component_fname = os.path.join(INFER_DATA, 'input', f'{component}_{current_timestamp}.csv')\n", " signals_df.to_csv(component_fname, index=None)\n", " \n", " start = start + datetime.timedelta(minutes=+frequency)\n", " \n", "# Upload the whole folder to S3, in the input location:\n", "INFERENCE_INPUT = os.path.join(INFER_DATA, 'input')\n", "!aws s3 cp --recursive --quiet $INFERENCE_INPUT s3://$BUCKET/$PREFIX_INFERENCE/input" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Now that we've prepared the data, create the scheduler by running:\n", "create_scheduler_response = scheduler.create()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our scheduler is now running and its inference history is currently empty:\n", "\n", "![Scheduler created](../assets/schedule_created.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get inference results\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### List inference executions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Let's now wait for 5-15 minutes to give some time to the scheduler to run its first inferences.** Once the wait is over, we can use the ListInferenceExecution API for our current inference scheduler. The only mandatory parameter is the scheduler name.\n", "\n", "You can also choose a time period for which you want to query inference executions for. If you don't specify it, then all executions for an inference scheduler will be listed. If you want to specify the time range, you can do this:\n", "\n", "```python\n", "START_TIME_FOR_INFERENCE_EXECUTIONS = datetime.datetime(2010,1,3,0,0,0)\n", "END_TIME_FOR_INFERENCE_EXECUTIONS = datetime.datetime(2010,1,5,0,0,0)\n", "```\n", "\n", "Which means the executions after `2010-01-03 00:00:00` and before `2010-01-05 00:00:00` will be listed.\n", "\n", "You can also choose to query for executions in particular status, the allowed status are `IN_PROGRESS`, `SUCCESS` and `FAILED`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "START_TIME_FOR_INFERENCE_EXECUTIONS = None\n", "END_TIME_FOR_INFERENCE_EXECUTIONS = None\n", "EXECUTION_STATUS = None\n", "\n", "execution_summaries = []\n", "\n", "while len(execution_summaries) == 0:\n", " execution_summaries = scheduler.list_inference_executions(\n", " start_time=START_TIME_FOR_INFERENCE_EXECUTIONS,\n", " end_time=END_TIME_FOR_INFERENCE_EXECUTIONS,\n", " execution_status=EXECUTION_STATUS\n", " )\n", " if len(execution_summaries) == 0:\n", " print('WAITING FOR THE FIRST INFERENCE EXECUTION')\n", " time.sleep(60)\n", " \n", " else:\n", " print('FIRST INFERENCE EXECUTED\\n')\n", " break\n", " \n", "execution_summaries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have configured this scheduler to run every five minutes. After a few minutes we can also see the history in the console populated with its first executions: if you leave the scheduler to run beyond 15 minutes, any execution after the 3rd one will fail as we only generated 3 sequences above.\n", "\n", "![Inference history](../assets/schedule_inference_history.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When the scheduler starts (for example `datetime.datetime(2021, 1, 27, 9, 15)`, it looks for **a single** CSV file located in the input location with a filename that contains a timestamp set to the previous step. For example, a file named:\n", "\n", "* subsystem-01_2021012709**10**00.csv will be found and ingested\n", "* subsystem-01_2021012708**15**00.csv will **not be** ingested (it will be ingested at the next inference execution)\n", "\n", "In addition, when opening the file `subsystem-01_20210127091000.csv`, it will look for any row with a date that is between the DataStartTime and the DataEndTime of the inference execution. **If it doesn't find such a row, an internal exception will be thrown.**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Download inference results\n", "Let's have a look at the content now available in the scheduler output location: each inference execution creates a subfolder in the output directory. The subfolder name is the timestamp (GMT) at which the inference was executed and it contains a single [JSON lines](https://jsonlines.org/) file named `results.jsonl`:\n", "\n", "![Inference input](../assets/schedule_inference_output_data.png)\n", "\n", "Each execution summary is a JSON document that has the following format:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Fetch the list of execution summaries in case all executions were not captured yet:\n", "execution_summaries = scheduler.list_inference_executions()\n", "execution_summaries[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When the `Status` key from the previous JSON result is set to `SUCCESS`, you can collect the results location in the `CustomerResultObject` field. We are now going to loop through each execution result and download each JSON lines files generated by the scheduler. Then we will insert their results into an overall dataframe for further analysis:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Loops through the executions summaries:\n", "results_json = []\n", "for execution_summary in scheduler.execution_summaries:\n", " print('.', end='')\n", " \n", " # We only get an output if the inference execution is a sucess:\n", " status = execution_summary['Status']\n", " if status == 'SUCCESS':\n", " # Download the JSON-line file locally:\n", " bucket = execution_summary['CustomerResultObject']['Bucket']\n", " key = execution_summary['CustomerResultObject']['Key']\n", " current_timestamp = key.split('/')[-2]\n", " local_fname = os.path.join(INFER_DATA, 'output', f'centrifugal-pump_{current_timestamp}.jsonl')\n", " s3_fname = f's3://{bucket}/{key}'\n", " \n", " !aws s3 cp --quiet $s3_fname $local_fname\n", " \n", " # Opens the file and concatenate the results into a dataframe:\n", " with open(local_fname, 'r') as f:\n", " content = [eval(line) for line in f.readlines()]\n", " results_json = results_json + content\n", " \n", "# Build the final dataframes with all the results:\n", "results_df = pd.DataFrame(results_json)\n", "results_df['timestamp'] = pd.to_datetime(results_df['timestamp'])\n", "results_df = results_df.set_index('timestamp')\n", "results_df = results_df.sort_index()\n", "results_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The content of each JSON lines file follows this format:\n", " \n", "```json\n", "[\n", " {\n", " 'timestamp': '2021-04-17T13:25:00.000000',\n", " 'prediction': 1,\n", " 'prediction_reason': 'ANOMALY_DETECTED',\n", " 'diagnostics': [\n", " {'name': 'subsystem-19\\\\signal-067', 'value': 0.12},\n", " {'name': 'subsystem-18\\\\signal-099', 'value': 0.0},\n", " {'name': 'subsystem-09\\\\signal-016', 'value': 0.0},\n", " .\n", " .\n", " .\n", " {'name': 'subsystem-06\\\\signal-119', 'value': 0.08},\n", " {'name': 'subsystem-10\\\\signal-071', 'value': 0.02},\n", " {'name': 'subsystem-20\\\\signal-076', 'value': 0.02}\n", " ]\n", " }\n", " ...\n", "]\n", "```\n", "Each timestamp found in the file is associated to a prediction: 1 when an anomaly is detected an 0 otherwise. When the `prediction` field is 1 (an anomaly is detected), the `diagnostics` field contains each sensor (with the format `component`\\\\`tag`) and an associated percentage. This percentage corresponds to the magnitude of impact of a given sensor to the detected anomaly. For instance, in the example above, the tag `signal-067` located on the `subsystem-19` component has an estimated 12% magnitude of impact to the anomaly detected at 8pm on April 7th 2021. This dataset has 122 sensors: if each sensor contributed the same way to this event, the impact of each of them would be `100 / 122 = 0.82%`, so 12% is indeed statistically significant." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visualizing the inference results" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import matplotlib.ticker as mtick\n", "import numpy as np\n", "\n", "%matplotlib inline\n", "\n", "# Load style sheet:\n", "plt.style.use('../utils/aws_matplotlib_light.py')\n", "\n", "# Get colors from custom AWS palette:\n", "prop_cycle = plt.rcParams['axes.prop_cycle']\n", "colors = prop_cycle.by_key()['color']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Single inference analysis\n", "Let's first expand the results to expose the content of the **diagnostics** column above into different dataframe columns:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "expanded_results = []\n", "for index, row in results_df.iterrows():\n", " new_row = dict()\n", " new_row.update({'timestamp': index})\n", " new_row.update({'prediction': row['prediction']})\n", " \n", " if row['prediction'] == 1:\n", " diagnostics = pd.DataFrame(row['diagnostics'])\n", " diagnostics = dict(zip(diagnostics['name'], diagnostics['value']))\n", " new_row = {**new_row, **diagnostics}\n", " \n", " expanded_results.append(new_row)\n", " \n", "expanded_results = pd.DataFrame(expanded_results)\n", "expanded_results['timestamp'] = pd.to_datetime(expanded_results['timestamp'])\n", "expanded_results = expanded_results.set_index('timestamp')\n", "expanded_results.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each detected event have some detailed diagnostics. Let's unpack the details for the first event and plot a similar bar chart than what the console provides when it evaluates a trained model:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "event_details = pd.DataFrame(expanded_results.iloc[0, 1:]).reset_index()\n", "event_details.columns = ['name', 'value']\n", "event_details = event_details.sort_values(by='value')\n", "event_details_limited = event_details.tail(10)\n", "\n", "# We can then plot a horizontal bar chart:\n", "y_pos = np.arange(event_details_limited.shape[0])\n", "values = list(event_details_limited['value'])\n", "\n", "fig = plt.figure(figsize=(12,5))\n", "ax = plt.subplot(1,1,1)\n", "ax.barh(y_pos, event_details_limited['value'], align='center')\n", "ax.set_yticks(y_pos)\n", "ax.set_yticklabels(event_details_limited['name'])\n", "ax.xaxis.set_major_formatter(mtick.PercentFormatter(1.0))\n", "\n", "# Add the values in each bar:\n", "for i, v in enumerate(values):\n", " if v == 0:\n", " ax.text(0.0005, i, f'{v*100:.2f}%', color='#000000', verticalalignment='center')\n", " else:\n", " ax.text(0.0005, i, f'{v*100:.2f}%', color='#FFFFFF', fontweight='bold', verticalalignment='center')\n", " \n", "plt.title(f'Event detected at {expanded_results.index[0]}', fontsize=12, fontweight='bold')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As we did in the previous notebook, the above bar chart is already of great help to pinpoint what might be going wrong with your asset. Let's load the initial tags description file we prepared in the first notebook and match the sensors with our initial components to group sensors by component:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Agregate event diagnostics at the component level:\n", "event_details[['asset', 'sensor']] = event_details['name'].str.split('\\\\', expand=True)\n", "component_diagnostics = event_details.groupby(by='asset').sum().sort_values(by='value')\n", "component_diagnostics = component_diagnostics[component_diagnostics['value'] > 0.0]\n", "\n", "# Prepare Y position and values for bar chart:\n", "y_pos = np.arange(component_diagnostics.shape[0])\n", "values = list(component_diagnostics['value'])\n", "\n", "# Plot the bar chart:\n", "fig = plt.figure(figsize=(12,6))\n", "ax = plt.subplot(1,1,1)\n", "ax.barh(y_pos, component_diagnostics['value'], align='center')\n", "ax.set_yticks(y_pos)\n", "ax.set_yticklabels(list(component_diagnostics.index))\n", "ax.xaxis.set_major_formatter(mtick.PercentFormatter(1.0))\n", "\n", "# Add the values in each bar:\n", "for i, v in enumerate(values):\n", " ax.text(0.005, i, f'{v*100:.2f}%', color='#FFFFFF', fontweight='bold', verticalalignment='center')\n", " \n", "# Show the final plot:\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Inference scheduler operations\n", "---\n", "### Stop inference scheduler\n", "**Be frugal**, running the scheduler is the main cost driver of Amazon Lookout for Equipment. Use the following API to stop an already running inference scheduler. This will stop the periodic inference executions:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "scheduler.stop()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Start an inference scheduler\n", "You can restart any `STOPPED` inference scheduler using this API:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "scheduler.start()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Delete an inference scheduler\n", "You can delete a **stopped** scheduler you have no more use of: you can only have one scheduler per model." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "scheduler.stop()\n", "scheduler.delete()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Conclusion\n", "---\n", "In this notebook, we used the model created in part 3 of this notebook, configured a scheduler and extracted the predictions obtained after it executed a few inferences." ] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.13" } }, "nbformat": 4, "nbformat_minor": 4 }