# Create Workload Guide

This guide explores how users can use the `create-workload` subcommand in OpenSearch Benchmark to create a workload based on pre-existing data in a cluster.

### Create a Workload from Pre-Existing Indices in a Cluster

**Prerequisites:**
* OpenSearch cluster with data ingested into it in an index. Ensure that index has 1000+ docs. If not, a workload will be created but users cannot run the workload with `--test-mode`.
* Ensure that your cluster is permissive.

Create a workload with the following command:
```
$ opensearch-benchmark create-workload \
--workload="<WORKLOAD NAME>" \
--target-hosts="<CLUSTER ENDPOINT>" \
--client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" \
--indices="<INDICES TO GENERATE WORKLOAD FROM>" \
--output-path="<LOCAL DIRECTORY PATH TO STORE WORKLOAD>"
```
Note that:
* `--indices` can be 1+ indices specified in a comma-separated list.
* If the cluster uses basic authentication and has TLS enabled, users will need to provide them through `--client-options`.

The following is an example output of when a user creates a workload from an index called movies that contains 2000 docs.

```
   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].

Extracting documents for index [movies] for test mode...      1000/1000 docs [100.0% done]
Extracting documents for index [movies]...                    2000/2000 docs [100.0% done]

[INFO] Workload movies has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies

-------------------------------
[INFO] SUCCESS (took 2 seconds)
-------------------------------
```

By default, workloads created will come with the following operations run in the following order:
* **delete-index**: Deletes any pre-existing indices with the same name(s) as the indices provided in `--indices`
* **create-index**: Creates the index with the same name(s) as the indices provided in `--indices`
* **cluster-health**: Verifies that cluster health is green before proceeding with the ingestion
* **bulk**: Ingests documents collected from the indices specified in `--indices`
* **default**: Runs a match-all query on the index for a number of iterations

To invoke the newly created workload, run the following:
```
$ opensearch-benchmark execute_test \
--pipeline="benchmark-only" \
--workload-path="<PATH OUTPUTTED IN THE OUTPUT OF THE CREATE-WORKLOAD COMMAND>" \
--target-host="<CLUSTER ENDPOINT>" \
--client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'"
```

Users have the options to specify a subset of documents from the index or override the default match_all query. See the following sections for more information on how.

### Adding Custom Queries
Add `--custom-queries` to the `create-workload` command. This parameter takes in a JSON filepath. This overrides the default match_all query with the queries present in the input file.

Requirements:
* Ensure that queries are properly formatted and adhere to JSON schema
* Ensure that all queries are contained within a list. Exception: If providing only a single query, it does not have to be in a list.

Adding to the previous example, a user wants to override default query with the following two custom queries in a JSON file.
```
[
  {
    "name": "default",
    "operation-type": "search",
    "body": {
      "query": {
        "match_all": {}
      }
    }
  },
  {
    "name": "term",
    "operation-type": "search",
    "body": {
      "query": {
        "term": {
          "director": "Ian"
        }
      }
    }
  }
]
```

To do this, the user can provide the JSON filepath to `--custom-queries` parameter:
```
$ opensearch-benchmark create-workload \
--workload="<WORKLOAD NAME>" \
--target-hosts="<CLUSTER ENDPOINT>" \
--client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" \
--indices="<INDICES TO GENERATE WORKLOAD FROM>" \
--output-path="<LOCAL DIRECTORY PATH TO STORE WORKLOAD>" \
--custom-queries="<JSON filepath containing queries>"
```

### Common Errors
When adding custom queries, users might experience the following error will occur if the queries do not adhere to JSON schema standards or are not in a list.
```
[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[ERROR] Cannot create-workload. Ensure JSON schema is valid and queries are contained in a list: Extra data: line 9 column 2 (char 113)
```