{ "cells": [ { "cell_type": "markdown", "id": "67a9dea9", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "### Dreambooth 模型微调\n", "DreamBooth 是一种深度学习生成模型，用于微调现有的文本到图像模型，由 Google Research 和波士顿大学的研究人员于 2022 年开发。最初使用 Google 自己的 Imagen 文本到图像模型开发，DreamBooth 的实现可以应用到其他文本到图像模型，它可以让模型通过的三到五张图像对一个主题进行训练后生成更精细和个性化的输出。\n", "\n", "![](../../images/dreambooth.png)\n", "\n", "接下来我们将使在笔记本中docker中使用DreamBooth 来微调我们的 stable diffusion 模型.\n", "\n", "#### 微调步骤\n", "1. 设置train.sh,微调参数\n", "2. 准备微调图片\n", "3. 构建 dreambooth fine-tuning 镜像\n", "4. 使用docker 运行微调任务\n", "5. 查看微调结果\n", "6. 部署微调模型到SageMaker Endpoint" ] }, { "cell_type": "code", "execution_count": null, "id": "6edda7b6", "metadata": {}, "outputs": [], "source": [ "import time\n", "import boto3\n", "import sagemaker\n", "account_id = boto3.client('sts').get_caller_identity().get('Account')\n", "region_name = boto3.session.Session().region_name\n", "\n", "sagemaker_session = sagemaker.Session()\n", "bucket = sagemaker_session.default_bucket()\n", "\n", "print(bucket)" ] }, { "cell_type": "markdown", "id": "0f98de66", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 1. 参数设置\n", "所有参数都在train.sh中\n", "\n", "关键参数\n", " * --class_prompt \"photo of a man\"\n", " * --instance_prompt \"photo of a zwx man\"\n", " * --max_train_steps 300\n", " * --learning_rate 2e-06\n", "\n", "可以根据需要进行修改" ] }, { "cell_type": "code", "execution_count": null, "id": "728e0360", "metadata": { "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [], "source": [ "!cat train.sh" ] }, { "cell_type": "markdown", "id": "03e2eb93", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 2. 准备微调的图片\n", "请下载预先准备的图片" ] }, { "cell_type": "code", "execution_count": null, "id": "7a5e988b", "metadata": {}, "outputs": [], "source": [ "#请以现场指导人员提供的图片地址为准\n", "!curl https://dfjcgkift2mhn.cloudfront.net/aigc/images.tgz | tar -xz \n", "!ls images/" ] }, { "cell_type": "markdown", "id": "7fe7b20b", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 3. 构建 dreambooth fine-tuning 镜像\n", " 使用预先定义的镜像，将修改过的train.sh和image 复制到镜像中" ] }, { "cell_type": "code", "execution_count": null, "id": "29c0c2d4", "metadata": { "pycharm": { "name": "#%%\n" }, "tags": [] }, "outputs": [], "source": [ "!./build_push.sh" ] }, { "cell_type": "markdown", "id": "1bb8eacd", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 4. 模型微调\n", "\n", " 打开终端运行\n", " \n", " ```bash\n", " docker run --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -ti sd-dreambooth-finetuning-v2 train.sh\n", " ```\n", " * 如果训练中间出现了空间不足，可以使用以下代码将/var/lib/docker 迁移到/home/ec2-user/SageMaker目录\n", " ```bash\n", " sudo systemctl stop docker\n", " sudo systemctl stop docker.socket \n", " sudo mv /var/lib/docker /home/ec2-user/SageMaker \n", " sudo ln -s /home/ec2-user/SageMaker/docker /var/lib/docker \n", " sudo systemctl start docker.socket\n", " sudo systemctl start docker\n", " ```\n", " " ] }, { "cell_type": "markdown", "id": "e77924a0", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 5. 查看微调效果测试\n", " " ] }, { "cell_type": "code", "execution_count": null, "id": "58585ed1", "metadata": {}, "outputs": [], "source": [ "import json\n", "import io\n", "from PIL import Image\n", "import traceback\n", "import time\n", "from sagemaker.async_inference.waiter_config import WaiterConfig\n", "\n", "\n", "s3_resource = boto3.resource('s3')\n", "\n", "def get_bucket_and_key(s3uri):\n", " pos = s3uri.find('/', 5)\n", " bucket = s3uri[5 : pos]\n", " key = s3uri[pos + 1 : ]\n", " return bucket, key\n", "\n", "def draw_image(response):\n", " try:\n", " bucket, key = get_bucket_and_key(response)\n", " print(bucket,key)\n", " obj = s3_resource.Object(bucket, key)\n", " bytes = obj.get()['Body'].read()\n", " image = Image.open(io.BytesIO(bytes))\n", " image.show()\n", " except Exception as e:\n", " traceback.print_exc()\n", " print(e)\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "1e69c9d7", "metadata": {}, "outputs": [], "source": [ "#请修改model_floder\n", "model_floder=\"dreambooth/model/a21e22fe-b625-45ab-bde5-e080f1ff48c2\"\n", "for count in range(1,6):\n", " if model_floder!=\"\":\n", " print(f's3://{bucket}/{model_floder}/samples/sample-{count}.jpg')\n", " draw_image(f's3://{bucket}/{model_floder}/samples/sample-{count}.jpg')" ] }, { "cell_type": "markdown", "id": "aa65cd08", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "#### 6. 部署微调模型到SageMaker Endpoint\n", " " ] }, { "cell_type": "code", "execution_count": null, "id": "5547f606", "metadata": { "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "#回到4.1 部署Stable Diffusion 模型 ,将andite/anything-v4.0替换为你fine tuning后的模型\n", "framework_version = '1.10'\n", "py_version = 'py38'\n", "\n", "model_environment = {\n", " 'SAGEMAKER_MODEL_SERVER_TIMEOUT':'600', \n", " 'SAGEMAKER_MODEL_SERVER_WORKERS': '1', \n", " #'model_name':'andite/anything-v4.0',\n", " 'model_name':'s3://bucket/',\n", " 's3_bucket':bucket\n", "}\n", "\n", "print(f's3://{bucket}/{model_floder}/samples/sample-{count}.jpg')" ] }, { "cell_type": "code", "execution_count": null, "id": "6351ac05", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "instance_type": "ml.m5.large", "kernelspec": { "display_name": "conda_pytorch_p39", "language": "python", "name": "conda_pytorch_p39" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.15" } }, "nbformat": 4, "nbformat_minor": 5 }