{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Multi-Region IoT with ACM PCA\n", "\n", "These Jupyter notebooks contain some code snippets for the re:Invent 2019 chalk talk (IOT335) about multi-region IoT setups. The setup assumes that you have a master and a slave region where you want to connect your devices to.\n", "\n", "AWS infrastructure like [AWS IoT topic rules](https://docs.aws.amazon.com/iot/latest/developerguide/iot-rules.html), [Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html) tables, [AWS Step Function](https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html) and [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) functions will be created by [AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html). There is one CloudFormation template for the slave region and one for the master region. Please refer to the README to launch the templates.\n", "\n", "The series of Jupyter notebooks let you create an AWS IoT multi-region setup based on the resources launched by CloudFormation. An example to connect devices is also provided as well as some examples to persist data across regions. You'll find the instructions in the Jupyter notebooks.\n", "\n", "To create a multi-region setup for AWS IoT you must bring your own CA. It must be registered in every region which applies to the multi-region setup. The CA will be created with the AWS Certificate Manager's [Private Certificate Authority](https://docs.aws.amazon.com/acm-pca/latest/userguide/PcaWelcome.html) (PCA).\n", "\n", "AWS Certificate Manager Private Certificate Authority is a managed private CA service with which you can easily and securely manage your certificate authority infrastructure and your private certificates.\n", "\n", "The devices certificates issued by the PCA will be automatically regiseterd using [Just-in-Time registration](https://aws.amazon.com/blogs/iot/just-in-time-registration-of-device-certificates-on-aws-iot/).\n", "\n", "If you run the Jupyter notebooks on Amazon EC2 or an Amazon SageMaker notebook instance you need to attach the following permissions to the instance.\n", "\n", "```\n", "{\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Action\": [\n", " \"acm-pca:*\",\n", " \"dynamodb:CreateGlobalTable\",\n", " \"dynamodb:DescribeGlobalTable\",\n", " \"iot:*\",\n", " \"iam:CreateRole\",\n", " \"iam:AttachRolePolicy\",\n", " \"iam:PassRole\",\n", " \"iam:GetRole\",\n", " \"lambda:AddPermission\",\n", " \"lambda:GetPolicy\",\n", " \"lambda:RemovePermission\",\n", " \"sns:*\",\n", " \"sqs:*\"\n", " ],\n", " \"Resource\": \"*\"\n", " }\n", " ]\n", "}\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Library" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import exists, join" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Shared Variables\n", "Variables which will be used in other notebooks.\n", "\n", "Feel free to modify the variable to you needs.\n", "\n", "* **aws_region_pca** is the region where you set up the private CA\n", "* **aws_region_master** is the AWS IoT master region\n", "* **aws_region_slave** is the AWS IoT slave region" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "CA_subject = {\"C\": \"DE\", \"O\": \"AWS\", \"OU\": \"IoT\", \"ST\": \"Berlin\", \"L\": \"Berlin\", \"CN\": \"Multi Region CA\"}\n", "CA_directory = 'CA_{}'.format(CA_subject['CN'])\n", "CA_key = 'ca.key.pem'\n", "CA_cert = 'ca.crt.pem'\n", "\n", "PCA_directory = join(CA_directory, 'PCA')\n", "\n", "config = {}\n", "config['aws_region_pca'] = \"eu-west-1\"\n", "config['aws_region_master'] = \"us-west-2\"\n", "config['aws_region_slave'] = \"us-east-1\"\n", "config['CA_directory'] = CA_directory\n", "config['CA_key'] = CA_key\n", "config['CA_cert'] = CA_cert\n", "config['PCA_directory'] = PCA_directory\n", "config['CA_subject'] = CA_subject\n", "config['Sub_CN'] = 'Subordinated IoT Device CA for multi region'\n", "config['dynamo_provisioning_table'] = 'IoTMRProvisioning'\n", "%store config" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Launch CloudFormation Stacks\n", "\n", "Before you continue with the next notebook launch two CloudFormation stacks:\n", "\n", "* [master-region.json](../cfn/master-region.json)\n", "* [slave-region.json](../cfn/slave-region.json)\n", "\n", "### Preparation\n", "The CloudFormation stacks will create Lambda functions. You can find the code in the folder `../lambda`. The CloudFormation template requires to have the Lambda code as zip files in an S3 bucket. The Lambda function for Just-in-Time registration makes use of the pyOpenSSL library which is by default not installed in the Lambda environment. Before you create the zip files the pyOpenSSL library must be installed locally into the directory where the Lambda function is stored. Use the package installer [pip](https://pypi.org/project/pip/) to install the pyOpenSSL library.\n", "\n", "**Please note: The Lambda functions require Python 3.7. Make sure that the pip version you use is for Python 3.7. This can be verified with the command:** `pip --version`\n", "\n", "The following commands will create the required zip files. Execute the commands from within the `lambda` directory:\n", "\n", "```\n", "cd iot-mr-jitr\n", "pip install pyOpenSSL -t .\n", "zip ../iot-mr-jitr.zip -r .\n", "cd ..\n", "\n", "cd iot-mr-cross-region\n", "zip ../iot-mr-cross-region.zip lambda_function.py\n", "cd ..\n", "\n", "cd sfn-iot-mr-dynamo-trigger\n", "zip ../sfn-iot-mr-dynamo-trigger.zip lambda_function.py \n", "cd ..\n", "\n", "cd sfn-iot-mr-thing-crud\n", "zip ../sfn-iot-mr-thing-crud.zip lambda_function.py \n", "cd ..\n", "\n", "cd sfn-iot-mr-thing-type-crud\n", "zip ../sfn-iot-mr-thing-type-crud.zip lambda_function.py \n", "cd ..\n", "\n", "```\n", "\n", "* If you don't have a bucket in your master and slave region create one in each region\n", "* Master region:\n", " * Copy the file `iot-mr-jitr.zip` to your S3 bucket\n", "* Slave region:\n", " * Copy all zip files from the `lambda` folder to the S3 bucket\n", "\n", "### Launch stacks\n", "When launching each CloudFormation stack you need to provide the S3 bucket name where you have stored the code for the Lambda functions.\n", "\n", "* Launch the stack **master-region.json** in the region that you have defined as master region above.\n", "\n", "* Launch the stack **slave-region.json** in the region that you have defined as slave region above.\n", "\n", "The CloudFormation stacks will create several AWS resources that are required to implement the IoT master-slave setup in the architecture diagram below. \n", "\n", "In the subsequent Jupyter notebooks you will do some hands-on exercises for an IoT multi-region setup.\n", "\n", "## Architecture\n", "\n", "![iot-mr-provisioning.png](iot-mr-provisioning.png)" ] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 4 }