{ "cells": [ { "cell_type": "markdown", "id": "11ff828c", "metadata": {}, "source": [ "# Data preparation\n", "\n", "\n", "In real business scenarios adapting tracking solution, data labeling on video file is required. [The MOTChallenge datasets](https://motchallenge.net/) are the datasets for the task of multiple object tracking, it includes datasets such as MOT15, MOT16, MOT17. In this notebook, we download a video from MOT16 test set to simulate the custom data, and split it into several clips for manual data labeling." ] }, { "cell_type": "markdown", "id": "2af6e1d8", "metadata": {}, "source": [ "## 1. Prepare raw data\n", "\n", "Download MOT16 test data as the raw data. The original data is the test data for [motchallenge MOT16](https://motchallenge.net/sequenceVideos/MOT16-03-raw.webm). To process the video in the following section, we download the `mp4` format data from [FairMOT repo](https://github.com/ifzhang/FairMOT)." ] }, { "cell_type": "code", "execution_count": null, "id": "4ae33cff", "metadata": {}, "outputs": [], "source": [ "!mkdir datasets\n", "!wget https://raw.githubusercontent.com/ifzhang/FairMOT/master/videos/MOT16-03.mp4 -O datasets/MOT16-03.mp4" ] }, { "cell_type": "markdown", "id": "1b627ed1", "metadata": {}, "source": [ "### Split video into clips\n", "\n", "Install opencv to run data processing." ] }, { "cell_type": "code", "execution_count": null, "id": "e0061191-1bfd-49a7-9ed8-14f91ebababf", "metadata": { "tags": [] }, "outputs": [], "source": [ "!pip install opencv-python" ] }, { "cell_type": "markdown", "id": "0869c6de-5f2e-44ea-9229-751039ec1b03", "metadata": {}, "source": [ "Read a video file and split the video file by each 200 frames. You can change the number of clips by adjusting `clip_interval`, but note that the number of clips cannot be less than 5 as we will separate these clips into train/validation/test dataset for model training." ] }, { "cell_type": "code", "execution_count": null, "id": "783e39a2", "metadata": {}, "outputs": [], "source": [ "import os\n", "import cv2\n", "\n", "clip_interval = 200\n", "\n", "def split_video(video_path, clip_dir=\"./datasets/clips\"):\n", " cap = cv2.VideoCapture(video_path)\n", " \n", " if not os.path.exists(clip_dir):\n", " os.makedirs(clip_dir)\n", " \n", " if (cap.isOpened()== False): \n", " print(\"Error opening video stream or file\")\n", " \n", " frame_width = int(cap.get(3))\n", " frame_height = int(cap.get(4))\n", " fps = int(cap.get(cv2.CAP_PROP_FPS))\n", "\n", " fourcc = cv2.VideoWriter_fourcc('m', 'p', '4', 'v')\n", " \n", " frame_cnt = 0\n", " clip_cnt = 0\n", " while(cap.isOpened()):\n", " ret, frame = cap.read()\n", " if ret == True:\n", " if frame_cnt % clip_interval == 0:\n", " if frame_cnt > 0:\n", " out.release()\n", " \n", " clip_path = f\"{clip_dir}/sample_{clip_cnt}.mp4\"\n", " out = cv2.VideoWriter(clip_path, fourcc, fps, (frame_width, frame_height))\n", " print(f\"save clip: {clip_path}\")\n", " clip_cnt += 1\n", " out.write(frame)\n", " frame_cnt += 1\n", " else:\n", " break\n", " out.release()\n", "\n", "clip_dir = \"./datasets/clips\"\n", "split_video('./datasets/MOT16-03.mp4', clip_dir=clip_dir)" ] }, { "cell_type": "markdown", "id": "507e3b0d-c366-41d6-a605-c316f4be9a24", "metadata": {}, "source": [ "Upload the clips to S3 Bucket." ] }, { "cell_type": "code", "execution_count": null, "id": "9c09a67b", "metadata": {}, "outputs": [], "source": [ "bucket_name = \n", "prefix = \n", "sample_data_s3uri = f\"s3://{bucket_name}/{prefix}/sample-data\"" ] }, { "cell_type": "code", "execution_count": null, "id": "6dd35512", "metadata": {}, "outputs": [], "source": [ "!aws s3 cp --recursive $clip_dir $sample_data_s3uri" ] }, { "cell_type": "markdown", "id": "ef844646", "metadata": {}, "source": [ "## 2. Label raw data" ] }, { "cell_type": "markdown", "id": "a4df4dd0", "metadata": {}, "source": [ "You can follow the [SageMaker Ground Truth guide](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-getting-started.html) to run the below tasks:\n", "- Step-1: [create a Private workforce(team)](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-workforce-create-private-cognito.html)\n", "- Step-2: Add a worker into the private team you created, you will receive an email with the title like `You're invited by aws to work on a labeling project.` which includes `User name`, `Temporary password` and the link for login\n", "- Step-3: Create a labeling job with as the input datasets in SageMaker Ground Truth Console.\n", " - Choose `Automated data setup` as **Input data setup**\n", " - Set `sample_data_s3uri` to **S3 location for input datasets**\n", " - In **S3 location for output datasets**, choose **Specify a new location** and set the s3 bucket as name such as `s3://{bucket_name}/{prefix}/sample-data-gt`\n", " - In **IAM Role**, choose a IAM role or create a new one which can access S3 bucket\n", " - Run **Complete data setup** to complete your input data setup. It will take minutes in this step. You will see `Input data connection successful.` once this step is done.\n", " - In **Data type**, Choose **Video->Video files**\n", " - In **Frame extraction**, choose **Use every 5 frame from a video to create a labeling task.**\n", " - In **Task type->Task category**, choose **Video - Object tracking** and select **Bounding box**\n", " - Click `next` to go to **Select workers and configure tool**, and choose `Private` in **Worker types**\n", " - Choose the private team you created before in **Private teams**\n", " - Leave default values for **Task timeout**\n", " - In **Video object tracking**, fill in **Task description** with description such as `this is a labelling task video tracking`\n", " - In **Label values**, add `person` and `car` as label and create labeling job.\n", " - You will the the status of the labelling job you created is `In progress`\n", " - Go to **Ground Truth->Labeling workforces** and choose `Private`, you will see `https://xxxxx.labeling.us-east-1.sagemaker.aws` under **Labeling portal sign-in URL** in **Private workforce summary**, and open this link.\n", " - You can the labeling job you just created before which title is `Track objects across video frames: this is a labelling task video tracking`\n", " - By clicking **Start working** button, you can start labeling job.\n", " \n", "- Step-4: Label data by following [the guide](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-video-object-tracking.html). To accelarate the labeling, you can use the **Predict** function to predict the boxes in the current frame.\n", "\n", "\n", "\n", "\n", "Once finishing a labeling task, you can get the following annotation directory in the defined S3 path.\n", "\n", "
\n", " \n", "
Ground Truth Structure
\n", "
\n", "\n", "Under manifest directory, there should be an `out` folder created if we finish labeling all files.\n", "
\n", " \n", "
Manifest in Ground Truth Structure
\n", "
\n", "\n", "You will see a file `output.manifest` like this:\n", "
\n", " \n", "
output.manifest
\n", "
" ] }, { "cell_type": "markdown", "id": "264b60b0", "metadata": {}, "source": [ "Refer to [Use Amazon SageMaker Ground Truth to Label Data](https://docs.aws.amazon.com/sagemaker/latest/dg/sms.html) for guide of labeling data. You can choose either video files or frame files to label data. " ] }, { "cell_type": "code", "execution_count": null, "id": "358f03eb", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "conda_python3", "language": "python", "name": "conda_python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" } }, "nbformat": 4, "nbformat_minor": 5 }