# Development Getting Started ## Introduction The following describes the steps involved to initialize a CDF development environment from scratch, to build, run and test a project, then finally on how to commit modifications to the source code. Due to the scripts used as part of both the build and deployment steps, only linux type environments (including macOS) are officially supported. ## Configuring the development environment The following is a one-time setup to configure the CDF development environment. + Ensure that you have all [development prerequisites](prerequisites.md) installed. + Clone the project: ```shell > git clone https://github.com/aws/aws-connected-device-framework.git > cd aws-connected-device-framework/source ``` + Optionally switch to a specific release tag: ```shell aws-connected-device-framework/source> git switch -c tags/ ``` ## Build The CDF monorepo is managed by [rush](https://rushjs.io) which under the covers is configured to use [pnpm](http://pnpm.js.org) as its package manager. The following is a brief introduction of how to use _rush_: ```sh # If this is your first time at using Rush for this project, remove any node_modules # that may have been installed as part of a non-Rush (npm/pnpm) release: aws-connected-device-framework/source> rm -rf node_modules # One time setup only, install pinned dependencies based on shrinkwrapped aws-connected-device-framework/source> rush install # When running the `clean`, `build`, `lint` or `test` commands you have the option to # run globally (for all packages), or for a specific package. To run for all packages # run as follows: # # rush # # To run for a specific package you can either provide a target filter as follows: # # rush -t # # or alternatively run the following within the package's directory (which is a shortcut # for `rush -t .`): # # rushx # # Taking the above comments into consideration, to build run the following. Note that the first build # may take time, but subsequent builds will be quicker delta builds: aws-connected-device-framework/source> rush build # To lint: aws-connected-device-framework/source> rush lint # And to run unit tests: aws-connected-device-framework/source> rush test # If you experience issues and need to reset everything you have the following 2 commands available: # To remove all build artifacts: aws-connected-device-framework/source> rush purge # to purge all node_modules: aws-connected-device-framework/source> rush clean # perform a deep clean aws-connected-device-framework/source> rush install # refresh dependencies again ``` ## Running a module locally Each module uses [dotenv-flow](https://github.com/kerimdzhanov/dotenv-flow) to manage its application configuration. When a module is deployed, the [installer module](../packages/installer/README.md) takes care of injecting the configuration into the environment. But when running locally, this configration needs to be provided. The easiest way to generate this configuration is to use the `config-to-env` command of the [installer module](../packages/installer/README.md), If you need to manually generate the configuration file, or understand the available configuration, refer to the modules own `docs/configuration.md` help file. Once you have the configuration file, you can start a module as follows: ```shell aws-connected-device-framework> cd source/packages/services/ aws-connected-device-framework/source/packages/services/> export CONFIG_LOCATION=; npm run start ``` ## Making changes to an existing module We adhere to what is known as a [GitHub flow](https://guides.github.com/introduction/flow/) as far as our approach to branching is concerned. Basically this boils down to: + The `main` branch always represents a working version of the code, including latest (maybe unofficially released) updates, that may be deployed to a production environment + Under no circumstances ever commit directly to `main`! + When starting a new feature or fixing a bug, create a new branch from `main`. Name the branch `feat_***` for new features or `fix_***` for hotfixes: ```sh aws-connected-device-framework> git switch -c Switched to a new branch '' ``` + At suitable points, commit your work by running the following, and following the prompts to describe your commit. Note that you must run `rush commit` inside the `source/` directory whereas you can run the `git` commands anywhere within the repo. ```sh aws-connected-device-framework> git add -A aws-connected-device-framework> cd source aws-connected-device-framework/source> rush commit ``` + When you have finished with your implementation, and ensured that all existing unit tests pass as well as creating any new tests, the following steps are required: + Merge changes with the `main` branch: ```sh # pull in main into your branch aws-connected-device-framework> git merge origin/main # once any conflicts have been resolved, test aws-connected-device-framework> cd source aws-connected-device-framework/source> rush test # commit changes aws-connected-device-framework/source> git add -A aws-connected-device-framework/source> rush commit ``` + + Generate release notes. the `rush change` command will analyze all commits on the branch, filter out the projects that changed, then prompt you to enter release notes for the updated project: ```sh # generate release notes aws-connected-device-framework/source> rush change # commit release notes aws-connected-device-framework/source> git add -A aws-connected-device-framework/source> rush commit ``` + + Push the branch to the git repo ```sh aws-connected-device-framework> git push ``` + + Create a pull request + Once your pull request has been reviewed, and any issues addressed, merge your implementation back into the main code branch. ## Changing package dependencies If your code changes include changes to the dependencies listed in a `package.json` file. you need to update the repo-wide "shrinkwrap" file. This file contains the resolved and pinned dependencies for all CDF packages: ```sh rush update git add source/common/config/rush/pnpm-lock.yaml git add source/common/config/rush/repo-state.json ``` Note that these two files cannot be merged. If your branch `feat_branch` has a merge conflict with `main`, the steps to resolve are: 1. Merge `main` into `feat_branch`. 2. Resolve all other merge conflicts, discard all changes in `pnpm-lock.yaml` and `repo-state.json`. 3. Run `rush update` again. 4. Commit the resulting `pnpm-lock.yaml` and `repo-state.json` to your branch. ## Understanding the directory structure Directory | Description ---|--- source/cicd/ | The CloudFormation template to deploy the cicd pipeline, along with the related CodeBuild scripts source/common/ | All build and package manager related files source/docs/ | CDF core related documentation source/infrastructure/ | The main deployment script for deploying the CDF core modules, along with CloudFormation templates that are not specific to any module source/packages/integration-tests/ | BDD related automated integration tests source/packages/libraries/ | All internal libraries, as well as CDF client libraries source/packages/services/ | Deployable modules, such as the Asset Library ## FAQ + "Why do I have to use `rush commit` to commit my work instead of the usual `git commit`?" `rush commit` is configured to run the [commitizen command line utility](https://github.com/commitizen/cz-cli) which forces you to describe commits using a specific format. This is vitally important, as there are steps in our CI/CD pipeline that analyze all git commits since the last release, use these specially formatted messages to intelligently increment the version numbers (e.g. understand if there's a breaking change), and finally auto generates a change log for the release. + "What is the need for using ([`rush`](https://rushjs.io)) and ([`pnpm`](https://pnpm.js.org) package manager)? What's wrong with just `npm`?" The `aws-connected-device-framework` git repo is what is known as a monorepo, a large single repository that contains many different projects. The decision to migrate CDF to a monorepo was made to: + simplify the development environment by removing the need for an npm private repo (e.g. verdaccio) + simplify the dependency management across projects (reduced the development environment footprint from >6GB to 300MB) + allow for atomic commits spanning multiple projects, simplifying branching, merging and code reviews `pnmp` in conjunction with `rush` have features that allow us to efficiently work with monorepos, while still being able to bundle individual modules in the way required by AWS Lambda.