= Kubernetes Concepts
:toc:
:icons:
:linkattrs:
:imagesdir: ../../resources/images

== Introduction

Now that we have a cluster up and running we can start exploring the Kubernetes CLI via the `kubectl` (pronounced "cube control") command.

`kubectl` interacts with the Kubernetes API Server, which runs on the master nodes in the cluster.

Kubernetes as a platform has a number of abstractions that map to API objects. These Kubernetes API Objects can be used to describe your cluster's desired state - including information such as applications and workloads running, container images, networking resources, and more. This section explains the most-used Kubernetes API concepts and how to interact with them via `kubectl`.

== Prerequisites

This chapter uses an EKS cluster with worker nodes as described link:../102-your-first-cluster[here].

[NOTE]
This lab and future labs can be completed using the EKS cluster or the multi-master kops cluster. Some of the outputs may vary slightly if using the kops cluster.

All configuration files for this chapter are in the `01-path-basics/103-kubernetes-concepts/templates` directory.
Please be sure to `cd` into that directory before running the commands below.

    $ cd ~/environment/aws-workshop-for-kubernetes/01-path-basics/103-kubernetes-concepts/templates

== Display Nodes

This command will show all the nodes available in your kubernetes cluster:

    $ kubectl get nodes

It will show an output similar to:

    NAME                                            STATUS    ROLES     AGE       VERSION
    ip-192-168-160-85.us-west-2.compute.internal    Ready     <none>    10m       v1.10.3
    ip-192-168-229-150.us-west-2.compute.internal   Ready     <none>    10m       v1.10.3
    ip-192-168-79-105.us-west-2.compute.internal    Ready     <none>    10m       v1.10.3

If you do not see this output, or receive an error, please ensure that you've followed the steps link:../102-your-first-cluster[here] and have a validated cluster.

=== Create your first Pod

This command instantiates an nginx container into your cluster, inside a pod:

    $ kubectl run nginx --image=nginx
    deployment "nginx" created

Get the list of deployments:

    $ kubectl get deployments
    NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
    nginx     1         1         1            0           41s

Get the list of running pods:

    $ kubectl get pods
    NAME                     READY     STATUS    RESTARTS   AGE
    nginx-4217019353-pmkzb   1/1       Running   0          1m

Get additional details for the pod by using the `<pod-name>` from the above output:

```
$ kubectl describe pod nginx-65899c769f-nd8qk
Name:           nginx-65899c769f-nd8qk
Namespace:      default
Node:           ip-192-168-229-150.us-west-2.compute.internal/192.168.229.150
Start Time:     Wed, 06 Jun 2018 11:55:08 +0000
Labels:         pod-template-hash=2145573259
                run=nginx
Annotations:    <none>
Status:         Running
IP:             192.168.217.248
Controlled By:  ReplicaSet/nginx-65899c769f
Containers:
  nginx:
    Container ID:   docker://fae3055b8829a6395d60a4f78cc38e6d4d7439dfb9b13e3e2c7bea2c550bee68
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:6be552b9a3c4762ae34962ddcb0e1aed3dfd0513860a846777182030b1a1bd0c
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 06 Jun 2018 11:55:14 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-prvj5 (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          True
  PodScheduled   True
Volumes:
  default-token-prvj5:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-prvj5
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason                 Age   From                                                    Message
  ----    ------                 ----  ----                                                    -------
  Normal  Scheduled              1m    default-scheduler                                       Successfully assigned nginx-65899c769f-nd8qk to ip-192-168-229-150.us-west-2.compute.internal
  Normal  SuccessfulMountVolume  1m    kubelet, ip-192-168-229-150.us-west-2.compute.internal  MountVolume.SetUp succeeded for volume "default-token-prvj5"
  Normal  Pulling                1m    kubelet, ip-192-168-229-150.us-west-2.compute.internal  pulling image "nginx"
  Normal  Pulled                 1m    kubelet, ip-192-168-229-150.us-west-2.compute.internal  Successfully pulled image "nginx"
  Normal  Created                1m    kubelet, ip-192-168-229-150.us-west-2.compute.internal  Created container
  Normal  Started                1m    kubelet, ip-192-168-229-150.us-west-2.compute.internal  Started container
```

By default, pods are created in a `default` namespace. In addition, a `kube-system` namespace is also reserved for Kubernetes system pods. A list of all the pods in `kube-system` namespace can be displayed as shown:

```
$ kubectl get pods --namespace=kube-system
NAME                       READY     STATUS    RESTARTS   AGE
aws-node-76w4v             1/1       Running   0          16m
aws-node-m55x9             1/1       Running   1          16m
aws-node-wxd2z             1/1       Running   0          16m
kube-dns-7cc87d595-d95l5   3/3       Running   0          1h
kube-proxy-dq4jx           1/1       Running   0          16m
kube-proxy-kq5f9           1/1       Running   0          16m
kube-proxy-rgxn7           1/1       Running   0          16m
```

Again, the exact output may vary but your results should look similar to these.

=== Get logs from the pod

Logs from the pod can be obtained (a fresh nginx does not have logs - check again later once you have accessed the service):

    $ kubectl logs <pod-name> --namespace <namespace-name>

=== Execute a shell on the running pod

This command will open a TTY to a shell in your pod:

    $ kubectl get pods
    $ kubectl exec <pod-name> -it /bin/bash

This opens a bash shell and allows you to look around the filesystem of the container.

=== Clean up

Delete all the Kubernetes resources created so far:

    $ kubectl delete deployment/nginx

In the next sections, we will go into more detail about Pods, Deployments, and other commonly used Kubernetes objects.

== Pods

A Pod is the smallest deployable unit that can be created, scheduled, and managed. It’s a logical collection of containers that belong to an application. Pods are created in a namespace. All containers in a pod share the namespace, volumes and networking stack. This allows containers in the pod to "`find`" each other and communicate using `localhost`.

=== Create a Pod

Each resource in Kubernetes can be defined using a configuration file. For example, an NGINX pod can be defined with configuration file shown in below:

	  $ cat pod.yaml
	  apiVersion: v1
	  kind: Pod
	  metadata:
	    name: nginx-pod
	    labels:
	      name: nginx-pod
	  spec:
	    containers:
	    - name: nginx
	      image: nginx:latest
	      ports:
	      - containerPort: 80

Create the pod as shown below:

	$ kubectl apply -f pod.yaml
	pod "nginx-pod" created

Get the list of pod:

	$ kubectl get pods
	NAME        READY     STATUS    RESTARTS   AGE
	nginx-pod   1/1       Running   0          22s

Verify that the pod came up fine (ensure nothing else is running on port 8080):

	$ kubectl -n default port-forward $(kubectl -n default get pod -l name=nginx-pod -o jsonpath='{.items[0].metadata.name}') 8080:80

In your Cloud9 IDE, click **Preview** and **Preview Running Application**. This opens up a preview tab and shows the NGINX main page:

image::nginx-pod-default-page.png[]

If the containers in the pod generate logs, then they can be seen using the command shown:

	$ kubectl logs nginx-pod
	127.0.0.1 - - [03/Nov/2017:17:33:30 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" "-"
	127.0.0.1 - - [03/Nov/2017:17:33:32 +0000] "GET /favicon.ico HTTP/1.1" 404 571 "http://localhost:8080/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" "-"
	2017/11/03 17:33:32 [error] 5#5: *2 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 127.0.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "localhost:8080", referrer: "http://localhost:8080/"

=== Memory and CPU resource request

A Container in a Pod can be assigned memory and CPU _request_ and _limit_. Request is the minimum amount of memory/CPU that Kubernetes will give to the container. Limit is the maximum amount of memory/CPU that a container will be allowed to use. The memory/CPU request/limit for the Pod is the sum of the memory/CPU requests/limits for all the Containers in the Pod. Request defaults to limit if not specified. Default value of the limit is the node capacity.

A Pod can be scheduled on a node if the Pod's memory and CPU request can be met. Memory and CPU limits are not taken into consideration for scheduling.

Pod can continue to operate on the node if Containers in the Pod does not exceed the memory request. If Containers in the Pod exceeds the memory request then they become target of eviction whenever the node runs out of memory. If Containers in the Pod exceeds the memory limit then they are terminated. If the Pod can be restarted, then kubelet will restart it, just like any other type of runtime failure. A Container might or might not be allowed to exceed its CPU limit for extended periods of time. However, it will not be killed for excessive usage.

Memory and CPU request/limit can be specified using the following:

[options="header", width="75%", cols="1,3"]
|====
| Type | Field
| Memory request | `spec.containers[].resources.requests.memory`
| Memory limit | `spec.containers[].resources.limits.memory`
| CPU request | `spec.containers[].resources.requests.cpu`
| CPU limit | `spec.containers[].resources.limits.cpu`
|====

Memory resources are requested in bytes. You can specify them in integer or decimals with one of the suffixes `E`, `P`, `T`, `G`, `M`, `K`. It can also be expressed with power-of-two equivalents `Ei`, `Pi`, `Ti`, `Gi`, `Mi`, `Ki`.

CPU can be requested in _cpu units_. 1 cpu unit is equivalent 1 AWS vCPU. It can also be requested in fractional units, such as 0.5 or in _millicpu_ such as 500m.

===== Default memory and CPU

By default, a container in a pod is not allocated any requests or limits. This can be verified using the previously started pod:

$ kubectl get pod/nginx-pod -o jsonpath={.spec.containers[].resources}
map[]

===== Assign memory and CPU

Let's assign a memory request and limit to a Pod using the configuration file shown:

	$ cat pod-resources.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod2
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    resources:
	      limits:
	        memory: "200Mi"
	        cpu: 2
	      requests:
	        memory: "100Mi"
	        cpu: 1
	    ports:
	    - containerPort: 80

The only change in this configuration file is the addition of `spec.containers[].resources` section. The limits are specified in the `limits` section and the requests are specified in the `requests` section.

Create the pod:

	$ kubectl apply -f pod-resources.yaml
	pod "nginx-pod2" created

Get more details about the requests and limits:

	$ kubectl get pod/nginx-pod2 -o jsonpath={.spec.containers[].resources}
	map[limits:map[memory:200Mi cpu:2] requests:map[cpu:1 memory:100Mi]]

NGINX container requires fairly low memory and CPU. And so these request and limit numbers would work well, and the pod is started correctly. Now, let's try to start a WildFly pod using similar numbers. The configuration file for the same is shown:

	$ cat pod-resources1.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: wildfly-pod
	  labels:
	    name: wildfly-pod
	spec:
	  containers:
	  - name: wildfly
	    image: jboss/wildfly:11.0.0.Final
	    resources:
	      limits:
	        memory: "200Mi"
	        cpu: 2
	      requests:
	        memory: "100Mi"
	        cpu: 1
	    ports:
	    - containerPort: 8080

The max amount of memory allocated for the WildFly container in this pod is restricted to 200MB. Let's create this Pod:

	$ kubectl apply -f pod-resources1.yaml
	pod "wildfly-pod" created

Watch the status of the Pod:

	$ kubectl get pods -w
	NAME          READY     STATUS              RESTARTS   AGE
	wildfly-pod   0/1       ContainerCreating   0          5s
	wildfly-pod   1/1       Running   0         26s
	wildfly-pod   0/1       OOMKilled   0         29s
	wildfly-pod   1/1       Running   1         31s
	wildfly-pod   0/1       OOMKilled   1         34s
	wildfly-pod   0/1       CrashLoopBackOff   1         45s
	wildfly-pod   1/1       Running   2         46s
	wildfly-pod   0/1       OOMKilled   2         49s
	wildfly-pod   0/1       CrashLoopBackOff   2         1m
	wildfly-pod   1/1       Running   3         1m
	wildfly-pod   0/1       OOMKilled   3         1m

`OOMKilled` shows that the container was terminated because it ran out of memory.

To correct this, we'll need to re-create the pod with higher memory limits.  

Although it may be instinctive to simply adjust the memory limit in the existing pod definition and re-apply it, Kubernetes does not currently support changing resource limits on running pods, so we'll need to first delete the existing pod, then recreate it.

In `pod-resources2.yaml`, confirm that the value of `spec.containers[].resources.limits.memory` is `300Mi`. Delete the existing Pod, and create a new one:

	$ kubectl delete -f pod-resources1.yaml
	pod "wildfly-pod" deleted
	$ kubectl apply -f pod-resources2.yaml
	pod "wildfly-pod" created
	$ kubectl get -w pod/wildfly-pod
	NAME          READY     STATUS              RESTARTS   AGE
	wildfly-pod   0/1       ContainerCreating   0          3s
	wildfly-pod   1/1       Running   0         25s

Now, the Pod successfully starts.

Get more details about the resources allocated to the Pod:

	$ kubectl get pod/wildfly-pod -o jsonpath={.spec.containers[].resources}
	map[limits:map[cpu:2 memory:300Mi] requests:map[cpu:1 memory:100Mi]]

=== Quality of service

Kubernetes opportunistically scavenges the difference between request and limit if they are not used by the Containers. This allows Kubernetes to oversubscribe nodes, which increases utilization, while at the same time maintaining resource guarantees for the containers that need guarantees.

Kubernetes assigns one of the QoS classes to the Pod:

. `Guaranteed`
. `Burstable`
. `BestEffort`

QoS class is used by Kubernetes for scheduling and evicting Pods.

When every Container in a Pod is given a memory and CPU limit, and optionally non-zero request, and they exactly match, then a Pod is scheduled with `Guaranteed` QoS. This is the highest priority.

A Pod is given `Burstable` QoS class if the Pod does not meet the `Guaranteed` QoS and at least one Container has a memory or CPU request. This is intermediate priority.

When no memory and CPU request or limit is assigned to any Container in the Pod, then a Pod is scheduled with `BestEffort` QoS. This the lowest and the default priority.

Pods that need to stay up can request `Guaranteed` QoS. Pods with less stringent requirement can use a weaker or no QoS.

==== Guaranteed

Here is an example of Pod with `Guaranteed` QoS:

	$ cat pod-guaranteed.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod-guaranteed
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    resources:
	      limits:
	        memory: "200Mi"
	        cpu: 1
	    ports:
	    - containerPort: 80

Note that no request values are specified here, and will default to limit.

Create this Pod:

	$ kubectl apply -f pod-guaranteed.yaml
	pod "nginx-pod-guaranteed" created

Check the resources:

	$ kubectl get pod/nginx-pod-guaranteed -o jsonpath={.spec.containers[].resources}
	map[limits:map[cpu:1 memory:200Mi] requests:map[cpu:1 memory:200Mi]]

Check the QoS:

	$ kubectl get pod/nginx-pod-guaranteed -o jsonpath={.status.qosClass}
	Guaranteed

Another Pod with explicit value for limit and request is shown:

	$ cat pod-guaranteed2.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod-guaranteed2
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    resources:
	      limits:
	        memory: "200Mi"
	        cpu: 1
	      requests:
	        memory: "200Mi"
	        cpu: 1
	    ports:
	    - containerPort: 80

Create this Pod:

	$ kubectl apply -f pod-guaranteed2.yaml
	pod "nginx-pod-guaranteed2" created

Check the resources:

	$ kubectl get pod/nginx-pod-guaranteed2 -o jsonpath={.spec.containers[].resources}
	map[limits:map[cpu:1 memory:200Mi] requests:map[cpu:1 memory:200Mi]]

Check the QoS:

	$ kubectl get pod/nginx-pod-guaranteed2 -o jsonpath={.status.qosClass}
	Guaranteed

==== Burstable

Here is an example of Pod with `Burstable` QoS:

	$ cat pod-burstable.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod-burstable
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    resources:
	      limits:
	        memory: "200Mi"
	        cpu: 1
	      requests:
	        memory: "100Mi"
	        cpu: 1
	    ports:
	    - containerPort: 80

Note that both request and limit values are specified here.

Create this Pod:

	$ kubectl apply -f pod-burstable.yaml
	pod "nginx-pod-burstable" created

Check the resources:

	$ kubectl get pod/nginx-pod-burstable -o jsonpath={.spec.containers[].resources}
	map[limits:map[cpu:1 memory:200Mi] requests:map[cpu:1 memory:100Mi]]

Check the QoS:

	$ kubectl get pod/nginx-pod-burstable -o jsonpath={.status.qosClass}
	Burstable

==== BestEffort

Check the resources:

	$ kubectl get pod/nginx-pod -o jsonpath={.spec.containers[].resources}
	map[requests:map[cpu:100m]]

Check the QoS:

	$ kubectl get pod/nginx-pod -o jsonpath={.status.qosClass}
	Burstable

This should be `BestEffort` and filed as https://github.com/kubernetes/kubernetes/issues/55278[kubernetes#55278].

=== Delete a Pod

Get all the Pods that are running:

	$ kubectl get pods
	NAME                    READY     STATUS    RESTARTS   AGE
	nginx-pod               1/1       Running   0          6m
	nginx-pod-burstable     1/1       Running   0          9m
	nginx-pod-guaranteed    1/1       Running   0          23m
	nginx-pod-guaranteed2   1/1       Running   0          12m
	nginx-pod2              1/1       Running   0          6m
	wildfly-pod             1/1       Running   0          6m

Delete the Pods as shown below:

	$ kubectl delete $(kubectl get pods -o=name)
	pod "nginx-pod" deleted
	pod "nginx-pod-burstable" deleted
	pod "nginx-pod-guaranteed" deleted
	pod "nginx-pod-guaranteed2" deleted
	pod "nginx-pod2" deleted
	pod "wildfly-pod" deleted

== Deployment

A "`desired state`", such as 4 replicas of a pod, can be described in a Deployment object. The Deployment controller in Kubernetes cluster then ensures the desired and the actual state are matching. Deployment ensures the recreation of a pod when the worker node fails or reboots. If a pod dies, then a new pod is started to ensure the desired vs actual matches. It also allows both up- and down-scaling the number of replicas. This is achieved using ReplicaSet. The Deployment manages the ReplicaSets and provides updates to those pods.

=== Create a Deployment

The folowing example will create a Deployment with 3 replicas of NGINX base image. Let's begin with the template:

  $ cat deployment.yaml
	apiVersion: apps/v1
	kind: Deployment # kubernetes object type
	metadata:
	  name: nginx-deployment # deployment name
	spec:
	  replicas: 3 # number of replicas
	  selector:
	    matchLabels:
	      app: nginx
	  template:
	    metadata:
	      labels:
	        app: nginx # pod labels
	    spec:
	      containers:
	      - name: nginx # container name
	        image: nginx:1.12.1 # nginx image
	        imagePullPolicy: IfNotPresent # if exists, will not pull new image
	        ports: # container and host port assignments
	        - containerPort: 80
	        - containerPort: 443

This deployment will create 3 instances of NGINX image.

Run the following command to create Deployment:

	$ kubectl create -f deployment.yaml --record
	deployment "nginx-deployment" created

The `--record` flag will track changes made through each revision.

To monitor deployment rollout status:

	$ kubectl rollout status deployment/nginx-deployment
	deployment "nginx-deployment" successfully rolled out

A Deployment creates a ReplicaSet to manage the number of replicas. Let's take a look at existing deployments and replica set.

Get the deployments:

	$ kubectl get deployments
	NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
	nginx-deployment   3         3         3            3           25s

Get the replica set for the deployment:

	$ kubectl get replicaset
	NAME                          DESIRED   CURRENT   READY     AGE
	nginx-deployment-3441592026   3         3         3         1m

Get the list of running pods:

	$ kubectl get pods
	NAME                                READY     STATUS    RESTARTS   AGE
	nginx-deployment-3441592026-ddpf0   1/1       Running   0          2m
	nginx-deployment-3441592026-kkp8h   1/1       Running   0          2m
	nginx-deployment-3441592026-lx304   1/1       Running   0          2m

=== Scaling a Deployment

Number of replicas for a Deployment can be scaled using the following command:

	$ kubectl scale --replicas=5 deployment/nginx-deployment
	deployment "nginx-deployment" scaled

Verify the deployment:

	$ kubectl get deployments
	NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
	nginx-deployment   5         5         5            5           2m

Verify the pods in the deployment:

	$ kubectl get pods
	NAME                                READY     STATUS    RESTARTS   AGE
	nginx-deployment-3441592026-36957   1/1       Running   0          44s
	nginx-deployment-3441592026-8wch5   1/1       Running   0          44s
	nginx-deployment-3441592026-ddpf0   1/1       Running   0          3m
	nginx-deployment-3441592026-kkp8h   1/1       Running   0          3m
	nginx-deployment-3441592026-lx304   1/1       Running   0          3m

=== Update a Deployment

A more general update to Deployment can be made by making edits to the pod spec. In this example, let's change to the latest nginx image.

First, type the following to open up a text editor:

	$ kubectl edit deployment/nginx-deployment

Next, change the image from `nginx:1.12.1` to `nginx:latest`.

This should perform a rolling update of the deployment. To track the deployment details such as revision, image version, and ports - type in the following:

```
$ kubectl describe deployments
Name:                   nginx-deployment
Namespace:              default
CreationTimestamp:      Mon, 23 Oct 2017 09:14:36 -0400
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision=2
                        kubernetes.io/change-cause=kubectl edit deployment/nginx-deployment
Selector:               app=nginx
Replicas:               5 desired | 5 updated | 5 total | 5 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:latest
    Ports:        80/TCP, 443/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-deployment-886641336 (5/5 replicas created)
Events:
  Type    Reason             Age                From                   Message
  ----    ------             ----               ----                   -------
  Normal  ScalingReplicaSet  4m                 deployment-controller  Scaled up replica set nginx-deployment-3441592026 to 3
  Normal  ScalingReplicaSet  1m                 deployment-controller  Scaled up replica set nginx-deployment-3441592026 to 5
  Normal  ScalingReplicaSet  32s                deployment-controller  Scaled up replica set nginx-deployment-886641336 to 1
  Normal  ScalingReplicaSet  32s                deployment-controller  Scaled down replica set nginx-deployment-3441592026 to 4
  Normal  ScalingReplicaSet  32s                deployment-controller  Scaled up replica set nginx-deployment-886641336 to 2
  Normal  ScalingReplicaSet  29s                deployment-controller  Scaled down replica set nginx-deployment-3441592026 to 3
  Normal  ScalingReplicaSet  29s                deployment-controller  Scaled up replica set nginx-deployment-886641336 to 3
  Normal  ScalingReplicaSet  28s                deployment-controller  Scaled down replica set nginx-deployment-3441592026 to 2
  Normal  ScalingReplicaSet  28s                deployment-controller  Scaled up replica set nginx-deployment-886641336 to 4
  Normal  ScalingReplicaSet  25s (x3 over 26s)  deployment-controller  (combined from similar events): Scaled down replica set nginx-deployment-3441592026 to 0
```

=== Rollback a Deployment

To rollback to a previous version, first check the revision history:

	$ kubectl rollout history deployment/nginx-deployment
	deployments "nginx-deployment"
	REVISION  CHANGE-CAUSE
	1         kubectl scale deployment/nginx-deployment --replicas=5
	2         kubectl edit deployment/nginx-deployment

If you only want to rollback to the previous revision, enter the following command:

	$ kubectl rollout undo deployment/nginx-deployment
	deployment "nginx-deployment" rolled back

In our case, the deployment will rollback to use the `nginx:1.12.1` image. Check the image name:

	$ kubectl describe deployments | grep Image
    Image:        nginx:1.12.1

If rolling back to a specific revision then enter:

	$ kubectl rollout undo deployment/nginx-deployment --to-revision=<version>

=== Delete a Deployment

Run the following command to delete the Deployment:

	$ kubectl delete -f deployment.yaml
	deployment "nginx-deployment" deleted

== Service

A pod is ephemeral. Each pod is assigned a unique IP address. If a pod that belongs to a replication controller dies, then it is recreated and may be given a different IP address. Further, additional pods may be created using Deployment or Replica Set. This makes it difficult for an application server, such as WildFly, to access a database, such as MySQL, using its IP address.

A Service is an abstraction that defines a logical set of pods and a policy by which to access them. The IP address assigned to a service does not change over time, and thus can be relied upon by other pods. Typically, the pods belonging to a service are defined by a label selector. This is similar mechanism to how pods belong to a replica set.

This abstraction of selecting pods using labels enables a loose coupling. The number of pods in the deployment may scale up or down but the application server can continue to access the database using the service.

A Kubernetes service defines a logical set of pods and enables them to be accessed through microservices.

=== Create a Deployment for Service

Pods belong to a service by using a loosely-coupled model where labels are attached to a pod and a service picks the pods by using those labels.

Let's create a Deployment first that will create 3 replicas of a pod:

  $ cat echo-deployment.yaml
	apiVersion: apps/v1
	kind: Deployment
	metadata:
	  name: echo-deployment
	spec:
	  replicas: 3
	  selector:
	    matchLabels:
	      app: echo-pod
	  template:
	    metadata:
	      labels:
	        app: echo-pod
	    spec:
	      containers:
	      - name: echoheaders
	        image: k8s.gcr.io/echoserver:1.10
	        imagePullPolicy: IfNotPresent
	        ports:
	        - containerPort: 8080


This example creates an echo app that responds with HTTP headers from an Elastic Load Balancer.

Type the following to create the deployment:

	$ kubectl create -f echo-deployment.yaml --record

Use the `kubectl describe deployment` command to confirm `echo-app` has been deployed:

```
$ kubectl describe deployment
Name:                   echo-deployment
Namespace:              default
CreationTimestamp:      Mon, 23 Oct 2017 10:07:47 -0400
Labels:                 app=echo-pod
Annotations:            deployment.kubernetes.io/revision=1
                        kubernetes.io/change-cause=kubectl create --filename=templates/echo-deployment.yaml --record=true
Selector:               app=echo-pod
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:  app=echo-pod
  Containers:
   echoheaders:
    Image:        k8s.gcr.io/echoserver:1.10
    Port:         8080/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   echo-deployment-3396249933 (3/3 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  10s   deployment-controller  Scaled up replica set echo-deployment-3396249933 to 3
```

Get the list of pods:

```
$ kubectl get pods
NAME                               READY     STATUS    RESTARTS   AGE
echo-deployment-3396249933-8slzp   1/1       Running   0          1m
echo-deployment-3396249933-bjwqj   1/1       Running   0          1m
echo-deployment-3396249933-r05nr   1/1       Running   0          1m
```

Check the label for a pod:

```
$ kubectl describe pods/echo-deployment-3396249933-8slzp | grep Label
Labels:         app=echo-pod
```

Each pod in this deployment has `app=echo-pod` label attached to it.

=== Create a Service

In the following example, we create a service `echo-service`:

	$ cat service.yaml
	apiVersion: v1
	kind: Service
	metadata:
	  name: echo-service
	spec:
	  selector:
	    app: echo-pod
	  ports:
	  - name: http
	    protocol: TCP
	    port: 80
	    targetPort: 8080
	  type: LoadBalancer

The set of pods targeted by the service are determined by the label `app: echo-pod` attached to them. It also defines an inbound port 80 to the target port of 8080 on the container.

Kubernetes supports both TCP and UDP protocols.

=== Publish a Service

A service can be published to an external IP using the `type` attribute. This attribute can take one of the following values:

. `ClusterIP`: Service exposed on an IP address inside the cluster. This is the default behavior.
. `NodePort`: Service exposed on each Node's IP address at a defined port.
. `LoadBalancer`: If deployed in the cloud, exposed externally using a cloud-specific load balancer.
. `ExternalName`: Service is attached to the `externalName` field. It is mapped to a CNAME with the value.

Let's publish the service load balancer and expose your services, add a `type` field of `LoadBalancer`.

This template will expose `echo-app` service on an Elastic Load Balancer (ELB):

	$ cat service.yaml
	apiVersion: v1
	kind: Service
	metadata:
	  name: echo-service
	spec:
	  selector:
	    app: echo-pod
	  ports:
	  - name: http
	    protocol: TCP
	    port: 80
	    targetPort: 8080
	  type: LoadBalancer

Run the following command to create the service:

	$ kubectl create -f service.yaml --record

Get more details about the service:

```
$ kubectl get service
NAME           TYPE           CLUSTER-IP       EXTERNAL-IP        PORT(S)        AGE
echo-service   LoadBalancer   10.100.114.136   aefc2ab9c6985...   80:30730/TCP   12m
kubernetes     ClusterIP      10.100.0.1       <none>             443/TCP        1h
$ kubectl describe service echo-service
Name:                     echo-service
Namespace:                default
Labels:                   <none>
Annotations:              kubernetes.io/change-cause=kubectl create --filename=service.yaml --record=true
Selector:                 app=echo-pod
Type:                     LoadBalancer
IP:                       10.100.114.136
LoadBalancer Ingress:     aefc2ab9c698511e88f6106a0b8e1215-513633583.us-west-2.elb.amazonaws.com
Port:                     http  80/TCP
TargetPort:               8080/TCP
NodePort:                 http  30730/TCP
Endpoints:                192.168.165.191:8080,192.168.193.104:8080,192.168.97.237:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason                Age   From                Message
  ----    ------                ----  ----                -------
  Normal  EnsuringLoadBalancer  16s   service-controller  Ensuring load balancer
  Normal  EnsuredLoadBalancer   14s   service-controller  Ensured load balancer
```

The output shows `LoadBalancer Ingress` as the address of an Elastic Load Balancer (ELB). It takes about 2-3 minutes for the ELB to be provisioned and be available. Wait for a couple of minutes, and then access the service:

```
curl aefc2ab9c698511e88f6106a0b8e1215-513633583.us-west-2.elb.amazonaws.com


Hostname: echo-deployment-7bcf9557cc-bfdls

Pod Information:
        -no pod information available-

Server values:
        server_version=nginx: 1.13.3 - lua: 10008

Request Information:
        client_address=192.168.79.105
        method=GET
        real path=/
        query=
        request_version=1.1
        request_scheme=http
        request_uri=http://aefc2ab9c698511e88f6106a0b8e1215-513633583.us-west-2.elb.amazonaws.com:8080/

Request Headers:
        accept=*/*
        host=aefc2ab9c698511e88f6106a0b8e1215-513633583.us-west-2.elb.amazonaws.com
        user-agent=curl/7.53.1

Request Body:
        -no body in request-

```

Note the `client_address` value shown in the output. This is the IP address of the pod serving the request. Multiple invocations of this command will show different values for this attribute.

Now, the number of pods in the deployment can be scaled up and down. Or the pods may terminate and restart on a different host. But the service will still be able to target those pods because of the labels attached to the pod and used by the service.

=== Delete a Service

Run the following command to delete the Service:

    $ kubectl delete -f service.yaml

The backend Deployment needs to be explicitly deleted as well:

    $ kubectl delete -f echo-deployment.yaml

== Daemon Set

Daemon Set ensure that a copy of the pod runs on a selected set of nodes. By default, all nodes in the cluster are selected. A selection critieria may be specified to select a limited number of nodes.

As new nodes are added to the cluster, pods are started on them. As nodes are removed, pods are removed through garbage collection.

=== Create a DaemonSet

The following is an example DaemonSet that runs a Prometheus container. Let's begin with the template:

	$ cat daemonset.yaml
	apiVersion: extensions/v1beta1
	kind: DaemonSet
	metadata:
	  name: prometheus-daemonset
	spec:
	  selector:
	    matchLabels:
	      tier: monitoring
	      name: prometheus-exporter
	  template:
	    metadata:
	      labels:
	        tier: monitoring
	        name: prometheus-exporter
	    spec:
	      containers:
	      - name: prometheus
	        image: prom/node-exporter
	        ports:
	        - containerPort: 80

Run the following command to create the ReplicaSet and pods:

	$ kubectl create -f daemonset.yaml --record

The `--record` flag will track changes made through each revision.

Get basic details about the DaemonSet:

	$ kubectl get daemonsets/prometheus-daemonset
	NAME                   DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
	prometheus-daemonset   3         3         3         3            3           <none>          7s

Get more details about the DaemonSet:

```
$ kubectl describe daemonset/prometheus-daemonset
Name:           prometheus-daemonset
Selector:       name=prometheus-exporter,tier=monitoring
Node-Selector:  <none>
Labels:         name=prometheus-exporter
                tier=monitoring
Annotations:    kubernetes.io/change-cause=kubectl create --filename=templates/daemonset.yaml --record=true
Desired Number of Nodes Scheduled: 3
Current Number of Nodes Scheduled: 3
Number of Nodes Scheduled with Up-to-date Pods: 3
Number of Nodes Scheduled with Available Pods: 3
Number of Nodes Misscheduled: 0
Pods Status:  3 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  name=prometheus-exporter
           tier=monitoring
  Containers:
   prometheus:
    Image:        prom/node-exporter
    Port:         80/TCP
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age   From        Message
  ----    ------            ----  ----        -------
  Normal  SuccessfulCreate  28s   daemon-set  Created pod: prometheus-daemonset-pzfl8
  Normal  SuccessfulCreate  28s   daemon-set  Created pod: prometheus-daemonset-sjcgh
  Normal  SuccessfulCreate  28s   daemon-set  Created pod: prometheus-daemonset-ctrg4
```

Get pods in the DaemonSet:

```
$ kubectl get pods -lname=prometheus-exporter
NAME                         READY     STATUS    RESTARTS   AGE
prometheus-daemonset-ctrg4   1/1       Running   0          57s
prometheus-daemonset-pzfl8   1/1       Running   0          57s
prometheus-daemonset-sjcgh   1/1       Running   0          57s
```

=== Limit DaemonSets to specific nodes

Verify that the Prometheus pod was successfully deployed to the cluster nodes:

	$ kubectl get pods -o wide
	NAME                         READY     STATUS    RESTARTS   AGE       IP            NODE
	prometheus-daemonset-sjcgh   1/1       Running   0          1m        100.96.7.10   ip-172-20-52-200.ec2.internal
	prometheus-daemonset-ctrg4   1/1       Running   0          1m        100.96.6.10   ip-172-20-64-152.ec2.internal
	prometheus-daemonset-pzfl8   1/1       Running   0          1m        100.96.5.10   ip-172-20-125-181.ec2.internal

Rename one of the node labels as follows:

	$ kubectl label node ip-172-20-52-200.ec2.internal app=prometheus-node
	node "ip-172-20-52-200.ec2.internal" labeled

Next, edit the DaemonSet template using the command shown:

	$ kubectl edit ds/prometheus-daemonset

Change the `spec.template.spec` to include a `nodeSelector` that matches the changed label:
```
      nodeSelector:
        app: prometheus-node
```

After the update is performed, we have now configured Prometheus to run on a specific node:

	$ kubectl get ds/prometheus-daemonset
	NAME                   DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR         AGE
	prometheus-daemonset   1         1         1         0            1           app=prometheus-node   2m

=== Delete a DaemonSet

Run the following command to delete the DaemonSet:

	$ kubectl delete -f daemonset.yaml

== Job

A Job creates one or more pods and ensures that a specified number of them successfully complete. A job keeps track of successful completion of a pod. When the specified number of pods have successfully completed, the job itself is complete. The job will start a new pod if the pod fails or is deleted due to hardware failure. A successful completion of the specified number of pods means the job is complete.

This is different from a replica set or a deployment which ensures that a certain number of pods are always running. So if a pod in a replica set or deployment terminates, then it is restarted again. This makes replica set or deployment as long-running processes. This is well suited for a web server, such as NGINX. But a job is completed if the specified number of pods successfully completes. This is well suited for tasks that need to run only once. For example, a job may convert an image format from one to another. Restarting this pod in replication controller would not only cause redundant work but may be harmful in certain cases.

Jobs are complementary to Replica Set. A Replica Set manages pods which are not expected to terminate (e.g. web servers), and a Job manages pods that are expected to terminate (e.g. batch jobs).

Job is only appropriate for pods with `RestartPolicy` equal to `OnFailure` or `Never`.

=== Non-parallel Job

Only one pod per job is started, unless the pod fails. Job is complete as soon as the pod terminates successfully.

Here is the job specification:

  $ cat job.yaml
	apiVersion: batch/v1
	kind: Job
	metadata:
	  name: wait
	spec:
	  template:
	    metadata:
	      name: wait
	    spec:
	      containers:
	      - name: wait
	        image: ubuntu
	        command: ["sleep",  "20"]
	      restartPolicy: Never

It creates an Ubuntu container, sleeps for 20 seconds and that's it!

Create a job using the command:

	$ kubectl apply -f job.yaml
	job "wait" created

Look at the job:

	$ kubectl get jobs
	NAME      DESIRED   SUCCESSFUL   AGE
	wait      1         0            0s

The output shows that the job is not successful yet. Watch the pod status to confirm:

	$ kubectl get -w pods
	NAME         READY     STATUS    RESTARTS   AGE
	wait-lk49x   1/1       Running   0          7s
	wait-lk49x   0/1       Completed   0         24s

To begin with, it shows that the pod for the job is running. The pod successfully exits after a few seconds and shows the `Completed` status.

Now, watch the job status again:

	$ kubectl get jobs
	NAME      DESIRED   SUCCESSFUL   AGE
	wait      1         1            1m

The output shows that the job was successfully executed.

To delete the job, you can run this command

	$ kubectl delete -f job.yaml

=== Parallel Job

Non-parallel jobs run only one pod per job. This API is used to run multiple pods in parallel for the job. The number of pods to complete is defined by `.spec.completions` attribute in the configuration file. The number of pods to run in parallel is defined by `.spec.parallelism` attribute in the configuration file. The default value for both of these attributes is 1.

The job is complete when there is one successful pod for each value in the range in 1 to `.spec.completions`. For that reason, it is also called as _fixed completion count_ job.

Here is a job specification:

  $ cat job-parallel.yaml
	apiVersion: batch/v1
	kind: Job
	metadata:
	  name: wait
	spec:
	  completions: 6
	  parallelism: 2
	  template:
	    metadata:
	      name: wait
	    spec:
	      containers:
	      - name: wait
	        image: ubuntu
	        command: ["sleep",  "20"]
	      restartPolicy: Never

This job specification is similar to the non-parallel job specification. It has two new attributes added: `.spec.completions` and `.spec.parallelism`. This means the job will be complete when six pods have successfully completed. A maximum of two pods will run in parallel at a given time.

Create a parallel job using the command:

	$ kubectl apply -f job-parallel.yaml

Watch the status of the job as shown:

	$ kubectl get -w jobs
	NAME      DESIRED   SUCCESSFUL   AGE
	wait      6         0            2s
	wait      6         1         22s
	wait      6         2         22s
	wait      6         3         43s
	wait      6         4         43s
	wait      6         5         1m
	wait      6         6         1m

The output shows that 2 pods are created about every 20 seconds.

In another terminal window, watch the status of pods created:

	$ kubectl get -w pods -l job-name=wait
	NAME         READY     STATUS    RESTARTS   AGE
	wait-f7kgb   1/1       Running   0          5s
	wait-smp4t   1/1       Running   0          5s
	wait-smp4t   0/1       Completed   0         22s
	wait-jbdp7   0/1       Pending   0         0s
	wait-jbdp7   0/1       Pending   0         0s
	wait-jbdp7   0/1       ContainerCreating   0         0s
	wait-f7kgb   0/1       Completed   0         22s
	wait-r5v8n   0/1       Pending   0         0s
	wait-r5v8n   0/1       Pending   0         0s
	wait-r5v8n   0/1       ContainerCreating   0         0s
	wait-r5v8n   1/1       Running   0         1s
	wait-jbdp7   1/1       Running   0         1s
	wait-r5v8n   0/1       Completed   0         21s
	wait-ngrgl   0/1       Pending   0         0s
	wait-ngrgl   0/1       Pending   0         0s
	wait-ngrgl   0/1       ContainerCreating   0         0s
	wait-jbdp7   0/1       Completed   0         21s
	wait-6l22s   0/1       Pending   0         0s
	wait-6l22s   0/1       Pending   0         0s
	wait-6l22s   0/1       ContainerCreating   0         0s
	wait-ngrgl   1/1       Running   0         1s
	wait-6l22s   1/1       Running   0         1s
	wait-ngrgl   0/1       Completed   0         21s
	wait-6l22s   0/1       Completed   0         21s

`kubectl get jobs` shows the status of the job after it has completed:

	$ kubectl get jobs
	NAME      DESIRED   SUCCESSFUL   AGE
	wait      6         6            3m

Deleting a job deletes all the pods as well. Delete the job as:

	$ kubectl delete -f job-parallel.yaml

== Cron Job

=== Prerequisites

For Kubernetes cluster versions < 1.8, Cron Job can be created with API version `batch/v2alpha1`. You need to explicitly enable API version `batch/v2alpha1` in Kubernetes cluster and perform a rolling-update.

If you use *Amazon EKS* for provisioning your Kubernetes cluster, your version should be >= v1.10 and you can proceed without any changes. You can check the cluster version using this command,

    $ kubectl version
    Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:16:17Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
    Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-28T20:13:43Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

=== Create Cron Job

A Cron Job is a job that runs on a given schedule, written in Cron format. There are two primary use cases:

. Run jobs once at a specified point in time
. Repeatedly at a specified point in time

Here is the job specification:

  $ cat cronjob.yaml
	apiVersion: batch/v1beta1
	kind: CronJob
	metadata:
	  name: hello
	spec:
	  schedule: "*/1 * * * *"
	  jobTemplate:
	    spec:
	      template:
	        metadata:
	          labels:
	            app: hello-cronpod
	        spec:
	          containers:
	          - name: hello
	            image: busybox
	            args:
	            - /bin/sh
	            - -c
	            - date; echo Hello World!
	          restartPolicy: OnFailure

This job prints the current timestamp and the message "`Hello World`" every minute.

Create the Cron Job as shown in the command:

	$ kubectl create -f cronjob.yaml

Watch the status of the job as shown:

	$ kubectl get -w cronjobs
	NAME      SCHEDULE      SUSPEND   ACTIVE    LAST SCHEDULE   AGE
	hello     */1 * * * *   False     0         <none>
	hello     */1 * * * *   False     0         <none>
	hello     */1 * * * *   False     1         Tue, 24 Oct 2017 15:41:00 -0700
	hello     */1 * * * *   False     0         Tue, 24 Oct 2017 15:41:00 -0700
	hello     */1 * * * *   False     1         Tue, 24 Oct 2017 15:42:00 -0700
	hello     */1 * * * *   False     0         Tue, 24 Oct 2017 15:42:00 -0700

In another terminal window, watch the status of pods created:

	$ kubectl get -w pods -l app=hello-cronpod
	NAME                     READY     STATUS    RESTARTS   AGE
	hello-1508884860-cq004   0/1       Pending   0          0s
	hello-1508884860-cq004   0/1       Pending   0         0s
	hello-1508884860-cq004   0/1       ContainerCreating   0         0s
	hello-1508884860-cq004   0/1       Completed   0         1s
	hello-1508884920-wl5bx   0/1       Pending   0         0s
	hello-1508884920-wl5bx   0/1       Pending   0         0s
	hello-1508884920-wl5bx   0/1       ContainerCreating   0         0s
	hello-1508884920-wl5bx   0/1       Completed   0         2s
	hello-1508884980-45ktd   0/1       Pending   0         0s
	hello-1508884980-45ktd   0/1       Pending   0         0s
	hello-1508884980-45ktd   0/1       ContainerCreating   0         0s
	hello-1508884980-45ktd   0/1       Completed   0         2s

Get logs from one of the pods:

  $ kubectl logs hello-1508884860-cq004
	Tue Oct 24 22:41:02 UTC 2017
	Hello World!

=== Delete Cron Job

Delete the Cron Job as shown in the following command:

	$ kubectl delete -f cronjob.yaml
	cronjob "hello" deleted

== Namespaces

Namespaces allows a physical cluster to be shared by multiple teams. A namespace allows to partition created resources into a logically named group. Each namespace provides:

. a *unique scope* for resources to avoid name collisions
. *policies* to ensure appropriate authority to trusted users
. ability to specify *constraints for resource consumption*

This allows a Kubernetes cluster to share resources by multiple groups and provide different levels of QoS each group. Resources created in one namespace are hidden from other namespaces. Multiple namespaces can be created, each potentially with different constraints.

=== Default namespace

The list of namespaces can be displayed using the command:

	$ kubectl get namespace
	NAME          STATUS    AGE
	default       Active    2m
	kube-public   Active    2m
	kube-system   Active    2m

By default, all resources in Kubernetes cluster are created in a `default` namespace.

`kube-public` is the namespace that is readable by all users, even those not authenticated. Any clusters booted with `kubeadm` will have a `cluster-info` ConfigMap. The clusters in this workshop are created using kops and so this ConfigMap will not exist.

`kube-system` is the namespace for objects created by the Kubernetes system.

Let's create a Deployment:

	$ kubectl apply -f deployment.yaml
	deployment "nginx-deployment" created

 Check its namespace:

	$ kubectl get deployment -o jsonpath={.items[].metadata.namespace}
	default

=== Custom namespace

A new namespace can be created using a configuration file or `kubectl`.

. The following configuration file can be used to create Namespace:

	$ cat namespace.yaml
	kind: Namespace
	apiVersion: v1
	metadata:
	  name: dev
	  labels:
	    name: dev

. Create a new Namespace:

	$ kubectl apply -f namespace.yaml
	namespace "dev" created

. Get the list of Namespaces:

	$ kubectl get ns
	NAME          STATUS    AGE
	default       Active    3h
	dev           Active    12s
	kube-public   Active    3h
	kube-system   Active    3h

. Get more details about the Namespace:
+
```
$ kubectl describe ns/dev
Name:         dev
Labels:       name=dev
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Namespace","metadata":{"annotations":{},"labels":{"name":"dev"},"name":"dev","namespace":""}}

Status:  Active

No resource quota.

No resource limits.
```
+
. Create a Deployment in this new Namespace using a configuration file:
+
  $ cat deployment-namespace.yaml
	apiVersion: extensions/v1beta1
	kind: Deployment
	metadata:
	  name: nginx-deployment-ns
	  namespace: dev
	spec:
	  replicas: 3
	  selector:
	    matchLabels:
	      app: nginx
	  template:
	    metadata:
	      labels:
	        app: nginx
	    spec:
	      containers:
	      - name: nginx
	        image: nginx:1.12.1
	        ports:
	        - containerPort: 80
	        - containerPort: 443
+
The main change is the addition of `namespace: dev`.
+
. Create the Deployment:

	$ kubectl apply -f deployment-namespace.yaml
	deployment "nginx-deployment-ns" created

. Deployment in a Namespace can be queried by providing an additional switch `-n` as shown:

	$ kubectl get deployments -n dev
	NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
	nginx-deployment-ns   3         3         3            3           1m

. Query the Namespace for this Deployment:

	$ kubectl get deployments/nginx-deployment-ns -n dev -o jsonpath={.metadata.namespace}
	dev

Alternatively, a namespace can be created using `kubectl` as well.

. Create a Namespace:

	$ kubectl create ns dev2
	namespace "dev2" created

. Create a Deployment:

	$ kubectl -n dev2 apply -f deployment.yaml
	deployment "nginx-deployment" created

. Get Deployments in the newly created Namespace:

	$ kubectl get deployments -n dev2
	NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
	nginx-deployment   3         3         3            3           1m

. Get Deployments in all Namespaces:

	$ kubectl get deployments --all-namespaces
	NAMESPACE     NAME                  DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
	default       nginx-deployment      3         3         3            3           1h
	dev           nginx-deployment-ns   3         3         3            3           1h
	dev2          nginx-deployment      3         3         3            3           1m
	kube-system   dns-controller        1         1         1            1           5h
	kube-system   kube-dns              2         2         2            2           5h
	kube-system   kube-dns-autoscaler   1         1         1            1           5h

=== Quota and Limits

Each namespace can be assigned resource quota. Specifying quota allows to restrict how much of cluster resources can be consumed across all resources in a namespace. Resource quota can be defined by a ResourceQuota object. A presence of ResourceQuota object in a namespace ensures that resource quotas are enforced. There can be at most one ResourceQuota object in a namespace. Currently, multiple ResourceQuota objects are allowed. This is filed as https://github.com/kubernetes/kubernetes/issues/55430[kubernetes#55430].

A quota can be specified for compute resources such as CPU and memory, storage resources such as PersistentVolume and PersistentVolumeClaim and number of objects of a given type. A complete list of resources that can be restricted using ResourceQuota are listed at https://kubernetes.io/docs/concepts/policy/resource-quotas/.

==== Create ResourceQuota

A ResourceQuota can be created using a configuration file or `kubectl`.

. The following configuration file can be used to create ResourceQuota:

	$ cat resource-quota.yaml
	apiVersion: v1
	kind: ResourceQuota
	metadata:
	  name: quota
	spec:
	  hard:
	    cpu: "4"
	    memory: 6G
	    pods: "10"
	    replicationcontrollers: "3"
	    services: "5"
	    configmaps: "5"
+
This configuration file places the following requirements on the namespace:
+
.. Every new Container created must have a memory and CPU limit
.. Total number of Pods in this namespace cannot exceed 10
.. Total number of ReplicationController in this namespace cannot exceed 3
.. Total number of Service in this namespace cannot exceed 5
.. Total number of ConfigMap in this namespace cannot exceed 5
+
. Create a new ResourceQuota:

	$ kubectl apply -f resource-quota.yaml
	resourcequota "quota" created
+
Alternatively, a ResourceQuota may be created using the `kubectl` CLI:
+
	kubectl create resourcequota quota2 --hard=cpu=10,memory=6G,pods=10,services=5,replicationcontrollers=3
+
In either this case, these restrictions would be placed on the `default` namespace in this case. An alternate namespace can be specified either in the configuration file or using the `--namespace` option on the `kubectl` CLI.
+
. Get the list of ResourceQuota:

	$ kubectl get quota
	NAME      AGE
	quota     25s

. Get more details about the ResourceQuota:

	$ kubectl describe quota/quota
	Name:                   quota
	Namespace:              default
	Resource                Used  Hard
	--------                ----  ----
	configmaps              0     5
	cpu                     300m  4
	memory                  0     6G
	pods                    3     10
	replicationcontrollers  0     3
	services                1     5
+
The output shows that three Pods and one Service already exists in the `default` namespace.

==== Scale resources with ResourceQuota

Now that the ResourceQuota has been created, let's see how this impacts the new resources that are created or existing resources that are scaled.

We already have a Deployment `nginx-deployment`. Let's scale the number of replicas to exceed the assigned quota and see what happens.

. Scale the number of replicas for the Deployment:

	$ kubectl scale --replicas=12 deployment/nginx-deployment
	deployment "nginx-deployment" scaled
+
The command output says that the Deployment is scaled.
+
. Let's check if all the replicas are available:
+
	$ kubectl get deployment/nginx-deployment -o jsonpath={.status.availableReplicas}
	3
+
It shows only three replicas are available.
+
. More details can be found:
+
	$ kubectl describe deployment nginx-deployment
        ...
        Conditions:
          Type             Status  Reason
          ----             ------  ------
          Progressing      True    NewReplicaSetAvailable
          Available        False   MinimumReplicasUnavailable
          ReplicaFailure   True    FailedCreate
+
The current reason is displayed in the output.

==== Create resources with ResourceQuota

Let's create a Pod with the following configuration file:

	$ cat pod.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    ports:
	    - containerPort: 80

You may have to remove a previously running Pod or Deployment before attempting to create this Pod.

	$ kubectl apply -f pod.yaml
	Error from server (Forbidden): error when creating "pod.yaml": pods "nginx-pod" is forbidden: failed quota: quota: must specify memory

The error message indicates that a ResourceQuota is in effect, and that the Pod must explicitly specify memory resources.

Update the configuration file to:

	$ cat pod-cpu-memory.yaml
	apiVersion: v1
	kind: Pod
	metadata:
	  name: nginx-pod
	  labels:
	    name: nginx-pod
	spec:
	  containers:
	  - name: nginx
	    image: nginx:latest
	    resources:
	      requests:
	        memory: "100m"
	    ports:
	    - containerPort: 80

There is an explicity memory resource defined here. Now, try to create the pod:

	$ kubectl apply -f pod-cpu-memory.yaml
	pod "nginx-pod" created

The Pod is successfully created.

Get more details about the Pod:

	$ kubectl get pod/nginx-pod -o jsonpath={.spec.containers[].resources}
	map[requests:map[cpu:1 memory:100m]

Get more details about the ResourceQuota:

	$ kubectl describe quota/quota
	Name:                   quota
	Namespace:              default
	Resource                Used  Hard
	--------                ----  ----
	configmaps              0     5
	cpu                     400m  4
	memory                  100m  6G
	pods                    4     12
	replicationcontrollers  0     3
	services                1     5

Note, how CPU and memory resources have incremented values.

https://github.com/kubernetes/kubernetes/issues/55433[kubernetes#55433] provide more details on how an explicit CPU resource is not needed to create a Pod with ResourceQuota.

	$ kubectl delete quota/quota
	$ kubectl delete quota/quota2

You are now ready to continue on with the workshop!

:frame: none
:grid: none
:valign: top

[align="center", cols="3", grid="none", frame="none"]
|=====
|image:button-continue-standard.png[link=../../02-path-working-with-clusters/201-cluster-monitoring]
|image:button-continue-developer.png[link=../../03-path-application-development/301-local-development]
|image:button-continue-operations.png[link=../../02-path-working-with-clusters/201-cluster-monitoring]
|link:../../standard-path.adoc[Go to Standard Index]
|link:../../developer-path.adoc[Go to Developer Index]
|link:../../operations-path.adoc[Go to Operations Index]
|=====
