AWSTemplateFormatVersion: "2010-09-09" Description: QnABot on AWS LLM Plugin for Anthropic - v0.1.0 Parameters: EmbeddingsModelId: Type: String Default: amazon.titan-e1t-medium AllowedValues: - amazon.titan-e1t-medium Description: Bedrock Embeddings ModelId (Bedrock preview access only) LLMModelId: Type: String Default: amazon.titan-tg1-large AllowedValues: - amazon.titan-tg1-large - ai21.j2-grande-instruct - ai21.j2-jumbo-instruct - anthropic.claude-v1 - anthropic.claude-instant-v1 - anthropic.claude-v2 Description: Bedrock LLM ModelId (Bedrock preview access only) BedrockPreviewSdkUrl: Type: String Default: https://preview.documentation.bedrock.aws.dev/Documentation/SDK/bedrock-python-sdk.zip Description: URL for the Bedrock SDK zip file (Bedrock preview access only) Resources: BedrockBoto3Bucket: Type: AWS::S3::Bucket BedrockBoto3ZipFunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole Policies: - PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - 's3:PutObject' - 's3:DeleteObject' - 's3:ListBucket' Resource: !Sub 'arn:aws:s3:::${BedrockBoto3Bucket}*' PolicyName: S3Policy BedrockBoto3ZipFunction: Type: AWS::Lambda::Function Properties: Handler: index.handler Runtime: python3.10 Role: !GetAtt 'BedrockBoto3ZipFunctionRole.Arn' Timeout: 60 MemorySize: 512 Environment: Variables: SDK_DOWNLOAD_URL: !Ref BedrockPreviewSdkUrl BOTO3_BUCKET: !Ref BedrockBoto3Bucket Code: ZipFile: | import os import sys import re import shutil import subprocess import boto3 import zipfile import urllib3 from datetime import datetime import cfnresponse bedrock_sdk_url = os.environ['SDK_DOWNLOAD_URL'] boto3_bucket = os.environ['BOTO3_BUCKET'] def download_file_from_url(url, local_path): """Download a file from a URL to a local save path.""" http = urllib3.PoolManager() response = http.request('GET', url) if response.status == 200: with open(local_path, 'wb') as file: file.write(response.data) print("File downloaded successfully.") else: print("Failed to download the file.", response) def upload_file_to_s3(file_path, bucket, key): s3 = boto3.client('s3') s3.upload_file(file_path, bucket, key) print(f"Upload successful. {file_path} uploaded to {bucket}/{key}") def extract_file_from_zip(zip_file_path, file_name): with zipfile.ZipFile(zip_file_path, 'r') as zip_ref: zip_ref.extract(file_name) print(f"Successfully extracted {file_name} from {zip_file_path}") def find_boto_wheels(zipname): zipf = zipfile.ZipFile(zipname, 'r') zip_files = zipf.namelist() b = re.compile('boto3(.*)\.whl') bc = re.compile('botocore(.*)\.whl') boto3_whl_file = [ s for s in zip_files if b.match(s) ][0] botocore_whl_file = [ s for s in zip_files if bc.match(s) ][0] return boto3_whl_file, botocore_whl_file def make_zip_filename(): now = datetime.now() timestamp = now.strftime('%Y%m%d_%H%M%S') filename = f'BedrockBoto3SDK_{timestamp}.zip' return filename def zipdir(path, zipname): zipf = zipfile.ZipFile(zipname, 'w', zipfile.ZIP_DEFLATED) for root, dirs, files in os.walk(path): for file in files: zipf.write(os.path.join(root, file), os.path.relpath(os.path.join(root, file), os.path.join(path, '..'))) zipf.close() def empty_bucket(bucket_name): s3_client = boto3.client('s3') response = s3_client.list_objects_v2(Bucket=bucket_name) if 'Contents' in response: keys = [{'Key': obj['Key']} for obj in response['Contents']] s3_client.delete_objects(Bucket=bucket_name, Delete={'Objects': keys}) return def handler(event, context): print("Event: ", event) responseData={} reason="" status = cfnresponse.SUCCESS try: if event['RequestType'] != 'Delete': os.chdir('/tmp') # download Bedrock SDK zip_file_name='bedrock-python-sdk.zip' print(f"downloading from {bedrock_sdk_url} to {zip_file_name}") download_file_from_url(bedrock_sdk_url, zip_file_name) boto3_whl_file, botocore_whl_file = find_boto_wheels(zip_file_name) extract_file_from_zip(zip_file_name, botocore_whl_file) extract_file_from_zip(zip_file_name, boto3_whl_file) if os.path.exists("python"): shutil.rmtree("python") os.mkdir("python") print(f"running pip install botocore") subprocess.check_call([sys.executable, "-m", "pip", "install", botocore_whl_file, "-t", "python" ]) print(f"running pip install boto3") subprocess.check_call([sys.executable, "-m", "pip", "install", boto3_whl_file, "-t", "python" ]) boto3_zip_name = make_zip_filename() zipdir("python",boto3_zip_name) print(f"uploading {boto3_zip_name} to s3 bucket {boto3_bucket}") upload_file_to_s3(boto3_zip_name, boto3_bucket, boto3_zip_name) responseData = {"Bucket": boto3_bucket, "Key": boto3_zip_name} else: # delete - empty the bucket so it can be deleted by the stack. empty_bucket(boto3_bucket) except Exception as e: print(e) status = cfnresponse.FAILED reason = f"Exception thrown: {e}" cfnresponse.send(event, context, status, responseData, reason=reason) BedrockBoto3Zip: Type: Custom::BedrockBoto3Zip Properties: ServiceToken: !GetAtt BedrockBoto3ZipFunction.Arn # Rerun BedrockBoto3ZipFunction if any of the following parameters change SDK_DOWNLOAD_URL: !Ref BedrockPreviewSdkUrl BOTO3_BUCKET: !Ref BedrockBoto3Bucket BedrockBoto3Layer: Type: "AWS::Lambda::LayerVersion" Properties: Content: S3Bucket: !GetAtt BedrockBoto3Zip.Bucket S3Key: !GetAtt BedrockBoto3Zip.Key CompatibleRuntimes: - python3.10 LambdaFunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole Policies: - PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - "bedrock:*" Resource: "*" PolicyName: BedrockPolicy EmbeddingsLambdaFunction: Type: AWS::Lambda::Function Properties: Handler: embeddings.lambda_handler Role: !GetAtt 'LambdaFunctionRole.Arn' Runtime: python3.10 Layers: - !Ref BedrockBoto3Layer Timeout: 60 MemorySize: 128 Environment: Variables: DEFAULT_MODEL_ID: !Ref EmbeddingsModelId EMBEDDING_MAX_WORDS: 300 Code: ./src LLMLambdaFunction: Type: AWS::Lambda::Function Properties: Handler: llm.lambda_handler Role: !GetAtt 'LambdaFunctionRole.Arn' Runtime: python3.10 Layers: - !Ref BedrockBoto3Layer Timeout: 60 MemorySize: 128 Code: ./src OutputSettingsFunctionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: lambda.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole OutputSettingsFunction: Type: AWS::Lambda::Function Properties: Handler: settings.lambda_handler Role: !GetAtt 'OutputSettingsFunctionRole.Arn' Runtime: python3.10 Timeout: 10 MemorySize: 128 Code: ./src OutputSettings: Type: Custom::OutputSettings Properties: ServiceToken: !GetAtt OutputSettingsFunction.Arn EmbeddingsModelId: !Ref EmbeddingsModelId LLMModelId: !Ref LLMModelId Outputs: BedrockBoto3Layer: Description: Lambda layer for Boto3 Bedrock SDK extensions Value: !Ref BedrockBoto3Layer EmbeddingsLambdaArn: Description: Lambda function for LLM (use for QnABot param "EmbeddingsLambdaArn") Value: !GetAtt 'EmbeddingsLambdaFunction.Arn' LLMLambdaArn: Description: Lambda function for LLM (use for QnABot param "LLMLambdaArn") Value: !GetAtt LLMLambdaFunction.Arn QnABotSettingEmbeddingsScoreThreshold: Description: QnABot Designer Setting "EMBEDDINGS_SCORE_THRESHOLD" Value: !GetAtt OutputSettings.EMBEDDINGS_SCORE_THRESHOLD QnABotSettingEmbeddingsScoreAnswerThreshold: Description: QnABot Designer Setting "EMBEDDINGS_SCORE_ANSWER_THRESHOLD" Value: !GetAtt OutputSettings.EMBEDDINGS_SCORE_ANSWER_THRESHOLD QnABotSettingEmbeddingsTextPassageScoreThreshold: Description: QnABot Designer Setting "EMBEDDINGS_TEXT_PASSAGE_SCORE_THRESHOLD" Value: !GetAtt OutputSettings.EMBEDDINGS_TEXT_PASSAGE_SCORE_THRESHOLD QnABotSettingGenerateQueryPromptTemplate: Description: QnABot Designer Setting "LLM_GENERATE_QUERY_PROMPT_TEMPLATE" Value: !GetAtt OutputSettings.LLM_GENERATE_QUERY_PROMPT_TEMPLATE QnABotSettingGenerateQueryModelParams: Description: QnABot Designer Setting "LLM_GENERATE_QUERY_MODEL_PARAMS" Value: !GetAtt OutputSettings.LLM_GENERATE_QUERY_MODEL_PARAMS QnABotSettingQAPromptTemplate: Description: QnABot Designer Setting "LLM_QA_PROMPT_TEMPLATE" Value: !GetAtt OutputSettings.LLM_QA_PROMPT_TEMPLATE QnABotSettingQAModelParams: Description: QnABot Designer Setting "LLM_QA_MODEL_PARAMS" Value: !GetAtt OutputSettings.LLM_QA_MODEL_PARAMS