Chainlink Node on the AWS Cloud
Quick Start Reference Deployment
DRAFT DOCUMENT / UNOFFICIAL GUIDANCE
This portion of the deployment guide is located at docs/partner_editable/__settings.adoc_
September 2021 |
| Visit our GitHub repository for source files and to post feedback, report bugs, or submit feature ideas for this Quick Start. |
This Quick Start was created by Chainlink Labs in collaboration with Amazon Web Services (AWS). Quick Starts are automated reference deployments that use AWS CloudFormation templates to deploy key technologies on AWS, following AWS best practices.
Overview
This portion of the deployment guide is located at docs/partner_editable/overview_target_and_usage.adoc
This guide provides instructions for deploying the Chainlink node Quick Start reference architecture on the AWS Cloud. This Quick Start deploys highly available Chainlink nodes with default parameters and a blockchain to the AWS Cloud. It is for enterprise users who want to run a secure Chainlink node to provide external resources such as external APIs, tamper-proof price data, and verifiable randomness directly to smart contracts stored on the blockchain. |
| Amazon may share user-deployment information with the AWS Partner that collaborated with AWS on the Quick Start. |
Chainlink Node on AWS
This portion of the deployment guide is located at docs/partner_editable/product_description.adoc
Chainlink is a data oracle that enables smart contracts on any blockchain to take advantage of extensive off-chain resources, such as tamper-proof price data, verifiable randomness, and external APIs. When deployed to a blockchain, a smart contract is a set of instructions that can be run without intervention from third parties. The code of a smart contract determines how it responds to input, just like the code of any other computer program. The Chainlink node is middleware that operates between the blockchain and external data. Node operators are able to run their Chainlink nodes with oracle job specifications, or specs, which contain the sequential tasks that the node must perform to produce a final result. The Chainlink node provides external data directly to smart contracts. |
AWS costs
You are responsible for the cost of the AWS services and any third-party licenses used while running this Quick Start. There is no additional cost for using the Quick Start.
The AWS CloudFormation templates for Quick Starts include configuration parameters that you can customize. Some of the settings, such as the instance type, affect the cost of deployment. For cost estimates, see the pricing pages for each AWS service you use. Prices are subject to change.
| After you deploy the Quick Start, create AWS Cost and Usage Reports to deliver billing metrics to an Amazon Simple Storage Service (Amazon S3) bucket in your account. These reports provide cost estimates based on usage throughout each month and aggregate the data at the end of the month. For more information, see What are AWS Cost and Usage Reports? |
Software licenses
This portion of the deployment guide is located at docs/partner_editable/licenses.adoc
No licenses are required to deploy this Quick Start. All AWS service resources consumed during the launch of the Quick Start incur AWS service usage costs. |
Architecture
This portion of the deployment guide is located at docs/partner_editable/architecture.adoc
Deploying this Quick Start for a new virtual private cloud (VPC) with default parameters builds the following Chainlink node environment in the AWS Cloud. 
Figure 1. Quick Start architecture for Chainlink node on AWS
As shown in Figure 1, the Quick Start sets up the following:
* The template that deploys the Quick Start into an existing VPC skips the components marked by asterisks and prompts you for your existing VPC configuration. |
Planning the deployment
Specialized knowledge
This deployment requires a moderate level of familiarity with AWS services. If you’re new to AWS, see Getting Started Resource Center and AWS Training and Certification. These sites provide materials for learning how to design, deploy, and operate your infrastructure and applications on the AWS Cloud.
This portion of the deployment guide is located at docs/partner_editable/specialized_knowledge.adoc
This Quick Start assumes familiarity with networking, DevOps, and Chainlink nodes. It also requires a moderate level of familiarity with the following AWS services: If you’re new to AWS, see the Getting Started Resource Center and the AWS Training and Certification site for materials and programs that can help you develop the skills to design, deploy, and operate your infrastructure and applications on the AWS Cloud. |
AWS account
If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.
Your AWS account is automatically signed up for all AWS services. You are charged only for the services you use.
Technical requirements
Before you launch the Quick Start, review the following information and ensure that your account is properly configured. Otherwise, deployment might fail.
Resource quotas
If necessary, request service quota increases for the following resources. You might need to request increases if your existing deployment currently uses these resources and if this Quick Start deployment could result in exceeding the default quotas. The Service Quotas console displays your usage and quotas for some aspects of some services. For more information, see What is Service Quotas? and AWS service quotas.
This portion of the deployment guide is located at docs/partner_editable/service_limits.adoc
| Resource | This deployment uses |
|---|---|
VPCs |
1 |
Elastic IP addresses |
3 |
Security groups |
2 |
AWS Identity and Access Management (IAM) roles |
2 |
Auto Scaling groups |
2 |
Application Load Balancers |
1 |
t2.micro instances |
1 |
t3.small instances |
2 |
db.t3.medium instances |
3 |
Supported AWS Regions
For any Quick Start to work in a Region other than its default Region, all the services it deploys must be supported in that Region. You can launch a Quick Start in any Region and see if it works. If you get an error such as “Unrecognized resource type,” the Quick Start is not supported in that Region.
For an up-to-date list of AWS Regions and the AWS services they support, see AWS Regional Services.
| Certain Regions are available on an opt-in basis. For more information, see Managing AWS Regions. |
IAM permissions
Before launching the Quick Start, you must sign in to the AWS Management Console with IAM permissions for the resources that the templates deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions. For more information, see AWS managed policies for job functions.
Deployment options
This portion of the deployment guide is located at docs/partner_editable/deployment_options.adoc
This Quick Start provides two deployment options:
The Quick Start provides separate templates for these options. It also lets you configure Classless Inter-Domain Routing (CIDR) blocks, instance types, and Chainlink node settings, as discussed later in this guide. |
This portion of the deployment guide is located at docs/partner_editable/pre-reqs.adoc
Prepare your AWS account
|
Deployment steps
This portion of the deployment guide is located at docs/partner_editable/deploy_steps.adoc
Confirm your AWS account configuration
-
Sign in to your AWS account at https://aws.amazon.com with an IAM user role that has the necessary permissions. For details, see Planning the deployment earlier in this guide.
-
Make sure that your AWS account is configured correctly, as discussed in the Technical requirements section.
-
Use the region selector in the navigation bar to choose the AWS Region where you want to deploy your Chainlink node.
Launch the Quick Start
| If you’re deploying Chainlink node into an existing VPC, make sure that your VPC has two private subnets in different Availability Zones. These subnets require NAT gateways in their route tables to allow instances to communicate with other Chainlink nodes without exposing them to the internet. |
Each deployment takes about 45 minutes to complete.
-
Sign in to your AWS account, and choose one of the following options to launch the AWS CloudFormation template. For help with choosing an option, see Deployment options earlier in this guide.
-
Check the AWS Region that’s displayed in the upper-right corner of the navigation bar, and change it if necessary. This Region is where you build the network infrastructure. The template is launched in the us-east-1 Region by default. For other choices, see Supported Regions earlier in this guide.
-
On the Create stack page, keep the default setting for the template URL, and then choose Next.
-
On the Specify stack details page, change the stack name if needed. Review the parameters for the template. Provide values for the parameters that require input. For all other parameters, review the default settings and customize them as necessary. For details on each parameter, see the Parameter reference section of this guide. When you finish reviewing and customizing the parameters, choose Next.
+
+ . On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you finish, choose Next. . On the Review page, review and confirm the template settings. Under Capabilities, select the two check boxes to acknowledge that the template creates IAM resources and might require the ability to automatically expand macros. . Choose Create stack to deploy the stack. . Monitor the status of the stack. When the status is CREATE_COMPLETE, the Chainlink node deployment is ready. . To view the created resources, see the values displayed in the Outputs tab for the stack.
This portion of the deployment guide is located at docs/partner_editable/additional_info.adoc
Post-deployment steps
Access your Chainlink node
You can access the Chainlink node user interface (UI) using one of the following options:
Option 1: If using a domain and SSL certificate created through ACM, you can access the Chainlink node UI through the Application Load Balancer endpoint or through your domain by adding the load balancer endpoint to your domain’s DNS record.
Option 2: Allow TCP port forwarding on your bastion host to port forward the Chainlink node UI to your machine. When accessing the Chainlink node through a bastion host, use SSH agent forwarding. Then connect to your bastion host with an SSH agent and TCP port forwarding. Access the Chainlink node UI at http://localhost:6688/.
ssh-add <your_ssh_key> ssh ec2-user@<bastion_host_public_ip> -A -L 6688:localhost:6688 ssh ec2-user@<chainlink_node_internal_ip> -L 6688:localhost:6688
Fulfill requests with the Chainlink node
After the Quick Start is deployed, your Chainlink node is ready to interact with smart contracts and fulfill requests. For more information, see Fulfilling Requests.
Work with external adapters
Chainlink enables integration of custom computations and specialized APIs using external adapters. The Chainlink node communicates with external adapter services via an API using a JSON specification. To get started, see External Adapters Introduction.
Manually create files for Chainlink node instances
If the Chainlink node instances are stopped, then you must create new .env, .password, and .api files to start the Chainlink node.
Generate the .env file:
cd /home/ec2-user/.chainlink/ && ./create-env.sh \
${chainNetwork} \
${blockchainNodeUrl} \
${psqlUser} \
$(aws secretsmanager get-secret-value --secret-id DBSecret --query "SecretString" --output text) \
${psqlHostname} \
${psqlPort} \
${psqlDb}
Generate the .password file:
cd /home/ec2-user/.chainlink/ && ./create-password.sh \ $(aws secretsmanager get-secret-value --secret-id WalletSecret --query "SecretString" --output text)
Generate the .api file:
cd /home/ec2-user/.chainlink/ && ./create-api.sh \
${apiUser} \
$(aws secretsmanager get-secret-value --secret-id ApiSecret --query "SecretString" --output text)
Start Chainlink node instances
To run the latest image of the Chainlink node Docker instance, run this command:
latestimage=$(curl -s -S "https://registry.hub.docker.com/v2/repositories/smartcontract/chainlink/tags/" | jq -r '."results"[]["name"]' | head -n 1) cd /home/ec2-user/.chainlink && docker run -d \ --log-driver=awslogs \ --log-opt awslogs-group=ChainlinkLogs \ --restart unless-stopped \ -u 1000:1000 \ --name chainlink \ -p 6688:6688 \ -v /home/ec2-user/.chainlink:/chainlink \ --env-file=/home/ec2-user/.chainlink/.env smartcontract/chainlink:$latestimage local n \ -p /chainlink/.password \ -a /chainlink/.api
Best practices for using Chainlink node on AWS
Failover capabilities
To reduce downtime, failover capabilities are required on both the Chainlink and Ethereum clients so that if one server fails, the service is still online. The Amazon EC2 Auto Scaling group has two Chainlink nodes: one active Chainlink node and one standby Chainlink node. Data from both the Chainlink and Ethereum clients are stored in a highly available PostgreSQL database.
Disaster recovery
If problems occur, the right processes must be in place to reduce downtime. The most common reason for downtime (in the context of Chainlink node operators) is a corrupted Ethereum node that requires a resynchronization.
To recover an Ethereum client and reduce downtime, follow these recommendations:
-
Take daily snapshots of the Ethereum chain on a separate server than the one connected to the Chainlink node.
-
Use an Ethereum client start-up process that pulls down the latest template of the chain and syncs it to the latest height.
Active monitoring
To detect issues before or when they occur, use active monitoring in the following areas:
-
(Minimum required) Ethereum of the wallet address assigned to the node.
-
Errored job runs.
-
Operator UI port to be open and responsive (usually 6688).
-
Ethereum HTTP and web socket ports to be open and responsive (usually 8545 and 8546).
-
Ethereum client disk, RAM, and CPU usage.
You can use Amazon CloudWatch to monitor Chainlink nodes.
Security
The Chainlink node Docker instance requires the .env, .password, and .api file that contains plaintext passwords. Remove the .env, .password, and .api files after the Chainlink node instance is running to prevent potential exposure of sensitive passwords.
This portion of the deployment guide is located at docs/partner_editable/faq_troubleshooting.adoc
FAQ
Q. I encountered a CREATE_FAILED error when I launched the Quick Start.
A. If AWS CloudFormation fails to create the stack, relaunch the template with Rollback on failure set to Disabled. This setting is under Advanced in the AWS CloudFormation console on the Configure stack options page. With this setting, the stack’s state is retained, and the instance keeps running so that you can troubleshoot the issue. (For Windows, look at the log files in %ProgramFiles%\Amazon\EC2ConfigService and C:\cfn\log.)
| When you set Rollback on failure to Disabled, you continue to incur AWS charges for this stack. Delete the stack when you finish troubleshooting. |
For more information, see Troubleshooting AWS CloudFormation.
Q. I encountered a size-limitation error when I deployed the AWS CloudFormation templates.
A. Launch the Quick Start templates from the links in this guide or from another S3 bucket. If you deploy the templates from a local copy on your computer or from a location other than an S3 bucket, you might encounter template-size limitations. For more information, see AWS CloudFormation quotas.
Q. Are there other environment variables I can add to my Chainlink node?
A. Yes. To see a list of environment variables and their explanations, see Configuration Variables.
Q. I encountered an Error reading password: open /chainlink/.password: no such file or directory error message when starting my Chainlink node.
A. Make sure that the .password files exist inside the Chainlink root directory. If the .password file does not exist, run the following command to create the .password file:
cd /home/ec2-user/.chainlink/ && ./create-password.sh \ $(aws secretsmanager get-secret-value --secret-id WalletSecret --query "SecretString" --output text)
Q. I encountered a Creating application: parse "XXXXXX": invalid URI for request error message when starting my Chainlink node.
A. This error typically happens when a parsing error exists within the PostgreSQL connection string (postgresql://) in the DATABASE_URL variable. Double-check the .env file for any errors in format or spelling. You can manually edit the connection string or run the following command to create a new .env file:
cd /home/ec2-user/.chainlink/ && ./create-env.sh \
${chainNetwork} \
${blockchainNodeUrl} \
${psqlUser} \
$(aws secretsmanager get-secret-value --secret-id DBSecret --query "SecretString" --output text) \
${psqlHostname} \
${psqlPort} \
${psqlDb}
Q. Where can I find more information about the Chainlink node deployment?
A. For more information, see Best Practices for Deploying Nodes on AWS.
Customer responsibility
After you successfully deploy this Quick Start, confirm that your resources and services are updated and configured — including any required patches — to meet your security and other needs. For more information, see the AWS Shared Responsibility Model.
Parameter reference
| Unless you are customizing the Quick Start templates for your own deployment projects, keep the default settings for the parameters labeled Quick Start S3 bucket name, Quick Start S3 bucket Region, and Quick Start S3 key prefix. Changing these parameter settings automatically updates code references to point to a new Quick Start location. For more information, see the AWS Quick Start Contributor’s Guide. |
Send us feedback
To post feedback, submit feature ideas, or report bugs, use the Issues section of the GitHub repository for this Quick Start. To submit code, see the Quick Start Contributor’s Guide.
Quick Start reference deployments
See the AWS Quick Start home page.
GitHub repository
Visit our GitHub repository to download the templates and scripts for this Quick Start, to post your comments, and to share your customizations with others.
Notices
This document is provided for informational purposes only. It represents AWS’s current product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS’s products or services, each of which is provided “as is” without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.
The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied. See the License for specific language governing permissions and limitations.