# Vsock Communication between 2 enclaves Sample

A hello-world example for Nitro Enclaves vsock server and client communication
between two enclaves.

## Prerequisites

1. The `server`, `client` and `proxy` applications from the `py` folder.
The client sends 3 different "Hello" messages to the server.
The server receives the message, adds an ACK and sends it back.
The proxy handles the communications between enclaves.
All logs and messages are printed to the standard output.

2. The sample is written and tested in Python 3.7.9. Any Python version
should work as long as it has `AF_VSOCK` socket support.

3. An instance with enough vCPUs to run 2 enclaves. 6 vCPUs are needed if hyperthreading
is enabled, otherwise 3 vCPUs are enough (i.e. ARM instances).

## How to run the sample

1. Build the Enclave Image File (EIF) starting from the `Dockerfile.server` file
in this directory. We chose to start from the *python-alpine* Docker image to keep
the enclave image as small as possible, but you can also use other optimized
Docker images to further decrease the final image size.

__Note__: You can use any other port number besides 5001 by modifying the command
inside the `Dockerfile.server` file.

```
cd multiple-enclaves/server
docker build -t multi-sample-server -f Dockerfile .
nitro-cli build-enclave --docker-uri multi-sample-server --output-file multi_server.eif
```

2. Repeat step 1 for the client enclave using the `Dockerfile.client` file

```
cd multiple-enclaves/client
docker build -t multi-sample-client -f Dockerfile .
nitro-cli build-enclave --docker-uri multi-sample-client --output-file multi_client.eif
```

3. Configure the pool of memory and vCPUs (the `nitro-cli-config` script can be used)
```
// 4 vCPUs and 512 MiB memory
nitro-cli-config -t 4 -m 512
```
4. Prepare 3 terminals to run the following commands.
	- Terminal 1: Run the server enclave connected to the console
	```
	SERVER_CID=10001
	nitro-cli run-enclave --eif-path multi_server.eif --cpu-count 2 --memory 256 --enclave-cid $SERVER_CID --debug-mode --attach-console
	```
	
	- Terminal 2: Run the proxy server:
	```
	SERVER_CID=10001
	sudo python3 proxy.py 5001 5001 $SERVER_CID
	```

	- Terminal 3: Run the client enclave connected to the console:
	```
	nitro-cli run-enclave --eif-path multi_client.eif --cpu-count 2 --memory 256 --debug-mode --attach-console
	```

__Note__: For `$SERVER_CID` other values can be used or use the autogenerated one

5. And the output for all terminal will be:
	- Terminal 1 (Enclave server):
	```
	Starting server on port 5001

	Recv: Hello 1!
	Sent: ACK(Hello 1!)

	Recv: Hello 2!
	Sent: ACK(Hello 2!)

	Recv: Hello 3!
	Sent: ACK(Hello 3!)
	Exiting server
	```
	- Terminal 2 (Host): 
	```
	Starting proxy on port 5001 to exchange data with (10001, 5001)
        Client enclave: (24, 3356208998)
        Server enclave: (10001, 5001)
                Message | Source enclave                --> Destination enclave
               Hello 1! | {cid:    24, port:3356208998} --> {cid: 10001, port:      5001}
          ACK(Hello 1!) | {cid: 10001, port:      5001} --> {cid:    24, port:3356208998}
               Hello 2! | {cid:    24, port:3356208998} --> {cid: 10001, port:      5001}
          ACK(Hello 2!) | {cid: 10001, port:      5001} --> {cid:    24, port:3356208998}
               Hello 3! | {cid:    24, port:3356208998} --> {cid: 10001, port:      5001}
          ACK(Hello 3!) | {cid: 10001, port:      5001} --> {cid:    24, port:3356208998}
	Connection closed
        Exiting proxy
	```
	- Terminal 3 (Enclave client):
	```
	Starting client

	Sent: Hello 1!
	Recv: ACK(Hello 1!)

	Sent: Hello 2!
	Recv: ACK(Hello 2!)

	Sent: Hello 3!
	Recv: ACK(Hello 3!)
	Exiting client
	```