Merge pull request #10 from eSailors/ssl

Add support for SSL

Author: mqasimsarfraz

.gitignore

@@ -0,0 +1,2 @@
+/dev
+/*.retry

Dockerfile

@@ -13,8 +13,10 @@ COPY files/ecr.ini /etc/supervisor.d/ecr.ini
 COPY files/root /etc/crontabs/root
 
 COPY files/nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
+COPY files/ssl.conf /usr/local/openresty/nginx/conf/ssl.conf
 
 ENV PORT 5000
+RUN chmod a+x /startup.sh /renew_token.sh
 
 ENTRYPOINT ["/startup.sh"]
 CMD ["/usr/bin/supervisord", "-c", "/etc/supervisord.conf"]

README.md

@@ -1,38 +1,64 @@
-aws-ecr-http-proxy
-===========
+<p align="left">
+    <a href="https://hub.docker.com/r/esailors/aws-ecr-http-proxy/builds" alt="Build">
+        <img src="https://img.shields.io/docker/build/esailors/aws-ecr-http-proxy" /></a>
+    <a href="https://hub.docker.com/r/esailors/aws-ecr-http-proxy" alt="Pulls">
+        <img src="https://img.shields.io/docker/pulls/esailors/aws-ecr-http-proxy" /></a>
+    <a href="https://www.esailors.de" alt="Maintained">
+        <img src="https://img.shields.io/maintenance/yes/2019.svg" /></a>
 
-A very simple nginx proxy that forwards requests to AWS ECR and caches the responses locally.
+</p>
 
-Run it like this, replace UPSTREAM with your target address with following required params:
-- `AWS_REGION`
-- `AWS_ACCESS_KEY_ID`
-- `AWS_SECRET_ACCESS_KEY`
+# aws-ecr-http-proxy
 
-It is also possible to define `CACHE_MAX_SIZE` env to limit maximum cache size on provided volume
+A very simple nginx push/pull proxy that forwards requests to AWS ECR and caches the responses locally.
 
-For example:
+### Configuration:
+The proxy is packaged in a docker container and can be configured with following environment variables:
+
+| Environment Variable                | Description                                    | Status                            | Default    |
+| :---------------------------------: | :--------------------------------------------: | :-------------------------------: | :--------: |
+| `AWS_REGION`                        | AWS Region for AWS ECR                         | Required                          |            |
+| `AWS_ACCESS_KEY_ID`                 | AWS Account Access Key ID                      | Required                          |            |
+| `AWS_SECRET_ACCESS_KEY`             | AWS Account Secret Access Key                  | Required                          |            |
+| `UPSTREAM`                          | URL for AWS ECR                                | Required                          |            |
+| `RESOLVER`                          | DNS server to be used by proxy                 | Required                          |            |
+| `PORT`                              | Port on which proxy listens                    | Required                          |            |
+| `CACHE_MAX_SIZE`                    | Maximum size for cache volume                  | Optional                          |  `75g`     |
+| `CACHE_KEY`                         | Cache key used for the content by nginx        | Optional                          |  `$uri`    |
+| `ENABLE_SSL`                        | Used to enable SSL/TLS for proxy               | Optional                          | `false`    |
+| `REGISTRY_HTTP_TLS_KEY`             | Path to TLS key in the container               | Required with TLS                 |            |
+| `REGISTRY_HTTP_TLS_CERTIFICATE`     | Path to TLS cert in the container              | Required with TLS                 |            |
+
+### Example:
 
 ```sh
 docker run --rm --name docker-registry-proxy --net=host \
-  -v /local-storage/cache:/cache \
+  -v /registry/local-storage/cache:/cache \
+  -v /registry/certificate.pem:/opt/ssl/certificate.pem
+  -v /registry/key.pem:/opt/ssl/key.pem
   -e PORT=5000 \
   -e RESOLVER=8.8.8.8 \
   -e UPSTREAM=https://XXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com \
   -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
   -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
   -e AWS_REGION=${AWS_DEFAULT_REGION} \
   -e CACHE_MAX_SIZE=100g \
+  -e ENABLE_SSL=true \
+  -e REGISTRY_HTTP_TLS_KEY=/opt/ssl/key.pem \
+  -e REGISTRY_HTTP_TLS_CERTIFICATE=/opt/ssl/certificate.pem
   esailors/aws-ecr-http-proxy:latest
 ```
 
 If you ran this command on "registry-proxy.example.com" you can now get your images using `docker pull registry-proxy.example.com:5000/repo/image`.
 
 ### Deploying the proxy
-Modify the ansible role variables according to your need and run the playbook as follow:
+Modify the ansible role [variables](https://github.com/eSailors/aws-ecr-http-proxy/tree/master/roles/docker-registry-proxy/defaults) according to your need and run the playbook as follow:
 ```sh
 ansible-playbook -i hosts playbook-docker-registry-proxy.yaml
 ```
-The docker registry for project is available [here](https://hub.docker.com/r/esailors/aws-ecr-http-proxy)
+In case you want to enable SSL/TLS please replace the SSL certificates with the valid ones in [roles/docker-registry-proxy/files/*.pem](https://github.com/eSailors/aws-ecr-http-proxy/tree/master/roles/docker-registry-proxy/files)
 
-### Note
-The proxy has `HTTP` endpoint so in order to avoid docker client complaining about it either mark the registry host as insecure in your [deamon config](https://docs.docker.com/registry/insecure/) or add [SSL/TLS termination](https://docs.docker.com/registry/recipes/nginx)
+### Note on SSL/TLS
+The proxy is using `HTTP` (plain text) as default protocol for now. So in order to avoid docker client complaining either:
+ - (**Recommended**) Enable SSL/TLS using `ENABLE_SSL` configuration. For that you will have to mount your **valid** certificate/key in the container and pass the paths using  `REGISTRY_HTTP_TLS_*` variables.
+ - Mark the registry host as insecure in your client [deamon config](https://docs.docker.com/registry/insecure/).

files/nginx.conf

@@ -21,8 +21,15 @@ http {
   # will run before forking out nginx worker processes
   init_by_lua_block { require "cjson" }
 
+  #https://docs.docker.com/registry/recipes/nginx/#setting-things-up
+  map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
+  '' 'registry/2.0';
+  }
+
   server {
-    listen PORT default_server;
+    listen PORT SSL_LISTEN default_server;
+
+    SSL_INCLUDE
 
     # Cache
     add_header X-Cache-Status   $upstream_cache_status;
@@ -36,20 +43,21 @@ http {
     chunked_transfer_encoding on;
 
     # increases timeouts to avoid HTTP 504
-    proxy_connect_timeout  300s;
+    proxy_connect_timeout  3s;
     proxy_read_timeout     300s;
     proxy_send_timeout     300s;
     send_timeout           300s;
 
     # disable proxy request buffering
     proxy_request_buffering off;
 
+    add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
     add_header "Access-Control-Allow-Origin" "*";
 
     location / {
       set $url        UPSTREAM;
       proxy_pass      $url;
-      proxy_redirect  $url http://$host:PORT;
+      proxy_redirect  $url SCHEME://$host:PORT;
 
       # Add AWS ECR authentication headers
       proxy_set_header  X-Real-IP          $remote_addr;
@@ -65,7 +73,7 @@ http {
     location ~ ^/v2/.*/blobs/[a-z0-9]+:[a-f0-9]+$ {
       set $url        UPSTREAM;
       proxy_pass      $url;
-      proxy_redirect  $url http://$host:PORT;
+      proxy_redirect  $url SCHEME://$host:PORT;
 
       # Add AWS ECR authentication headers
       proxy_set_header  X-Real-IP          $remote_addr;
@@ -88,7 +96,7 @@ http {
       set                    $saved_redirect_location '$upstream_http_location';
       proxy_pass             $saved_redirect_location;
       proxy_cache            cache;
-      proxy_cache_key        $scheme$proxy_host$uri$request_method;
+      proxy_cache_key        CACHE_KEY;
       proxy_cache_valid      200  1y;
       proxy_cache_use_stale  error timeout invalid_header updating
                              http_500 http_502 http_503 http_504;

files/ssl.conf

@@ -0,0 +1,8 @@
+ssl_certificate_key REGISTRY_HTTP_TLS_KEY;
+ssl_certificate     REGISTRY_HTTP_TLS_CERTIFICATE;
+
+ssl_protocols TLSv1.2;
+ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
+ssl_prefer_server_ciphers on;
+
+add_header Strict-Transport-Security max-age=31536000;

files/startup.sh

@@ -34,13 +34,30 @@ echo Using resolver $RESOLVER and $UPSTREAM [$(dig +short  ${UPSTREAM_WITHOUT_PO
 CACHE_MAX_SIZE=${CACHE_MAX_SIZE:-75g}
 echo Using cache max size $CACHE_MAX_SIZE
 
+CACHE_KEY=${CACHE_KEY:='$uri'}
+echo Using cache key $CACHE_KEY
+
+SCHEME=http
 CONFIG=/usr/local/openresty/nginx/conf/nginx.conf
+SSL_CONFIG=/usr/local/openresty/nginx/conf/ssl.conf
+
+if [ "$ENABLE_SSL" ]; then
+  sed -i -e s!REGISTRY_HTTP_TLS_CERTIFICATE!"$REGISTRY_HTTP_TLS_CERTIFICATE"!g $SSL_CONFIG
+  sed -i -e s!REGISTRY_HTTP_TLS_KEY!"$REGISTRY_HTTP_TLS_KEY"!g $SSL_CONFIG
+  SSL_LISTEN="ssl"
+  SSL_INCLUDE="include $SSL_CONFIG;"
+  SCHEME="https"
+fi
 
 # Update nginx config
 sed -i -e s!UPSTREAM!"$UPSTREAM"!g $CONFIG
 sed -i -e s!PORT!"$PORT"!g $CONFIG
 sed -i -e s!RESOLVER!"$RESOLVER"!g $CONFIG
 sed -i -e s!CACHE_MAX_SIZE!"$CACHE_MAX_SIZE"!g $CONFIG
+sed -i -e s!CACHE_KEY!"$CACHE_KEY"!g $CONFIG
+sed -i -e s!SCHEME!"$SCHEME"!g $CONFIG
+sed -i -e s!SSL_INCLUDE!"$SSL_INCLUDE"!g $CONFIG
+sed -i -e s!SSL_LISTEN!"$SSL_LISTEN"!g $CONFIG
 
 # setup ~/.aws directory
 AWS_FOLDER='/root/.aws'

hosts

@@ -1,2 +1,2 @@
 [docker-registry-proxy]
-registry-proxy.example.com
+localhost ansible_connection=local

roles/docker-registry-proxy/defaults/main.yml

@@ -8,3 +8,7 @@ docker_proxy_backend_resolver: "8.8.8.8"
 docker_proxy_ecr_access_id:
 docker_proxy_ecr_secret_key:
 docker_proxy_ecr_region:
+
+docker_proxy_ssl_enabled:
+docker_proxy_ssl_host_path: /registry
+docker_proxy_ssl_container_path: /opt/nginx

roles/docker-registry-proxy/files/certificate.pem

@@ -0,0 +1,32 @@
+-----BEGIN CERTIFICATE-----
+MIIFcTCCA1mgAwIBAgIJALWdZASytQRkMA0GCSqGSIb3DQEBCwUAME8xCzAJBgNV
+BAYTAkRFMRUwEwYDVQQHDAxEZWZhdWx0IENpdHkxFTATBgNVBAoMDGVTYWlsb3Jz
+IExURDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTE5MDkwMjA4NTYzNloXDTI5MDgz
+MDA4NTYzNlowTzELMAkGA1UEBhMCREUxFTATBgNVBAcMDERlZmF1bHQgQ2l0eTEV
+MBMGA1UECgwMZVNhaWxvcnMgTFREMRIwEAYDVQQDDAlsb2NhbGhvc3QwggIiMA0G
+CSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCrznLzCWlKJO4fXD/E8hx4cXUqbd9U
+VwpcAzWq3xGjC6gettYSp171elDXj23ddDJ1wwOl2U0bjN/DceRCl4Tnb3O86fzt
+Bwj8xA/stYGvZQqOvEhSfFh85qvDf1niu2uW1Zx4kMemUvNdhpXsqa9RPSu0Mb0G
+ZeHnCQkuz3KTjUMhQqwomg/6BY4G7tDmCZYsZezGSgAgoa+Q4vffW+H8S9nuKi8o
+EXUf9NuJHUTjtdgcVcCihPj54jXAQsqS9JzWSWxDnKxTaOZuRWEkG+vqGoKEL5+q
+PeH8x8aAM7wErsdxTqTV4XCJU0nS9om1Z6sz0Lrva/loyKciulTO8jYWqZIuBL8J
+GVeizcoYl9KcW8I66XkeuYWlNWCsWhGii7zEWVcXDdSuLCv1wLagRE9MJHoT64nd
+KAR2UQml/MSPlz8419K4r87hcVmNU6FFBP2RZO7UGW5eHbbKT+whhDdBTaig2NNL
+Ml7pVFq6ciemNr6IVsTuzS0VPJpuOoZa86+6UPqgw49jg4cBlKxgJxanDy2a2Hbm
+zx2dPAkbz3kMKDeBmgFzk3xt10czPXvIXUSnJdhi8NdQUCBHdu9yjT3s7Cc12NCQ
+3H0NPWUklHlqJrPY9IbGnNrmJblwZ7hyrI3eISV9njQf3etL2QdDXksLJpgeQsv7
+vcXEBje+aluNqQIDAQABo1AwTjAdBgNVHQ4EFgQUyBLOw8nyo6W4BJWI9L24TTta
+RbcwHwYDVR0jBBgwFoAUyBLOw8nyo6W4BJWI9L24TTtaRbcwDAYDVR0TBAUwAwEB
+/zANBgkqhkiG9w0BAQsFAAOCAgEAkYdRTaZuVAcxY1MI7V8PVnvTDxJ1izAWsr2W
+aQMSX0UxAZ/Aed+Q056Ya50/x5ffSigHY5Dap4eP36i/4+dhIeoaMpRjlz/sWCb5
+fE6judBvrMlMwrnKpi/eN0QC1KiCptPbPVPyonRj1ydrvQTPPDxLSeqgzCn3q5kf
+Gb6VlPDhj/CmIoVXkA9gYNlCwSpZ49DJJ2gTmI+MXolXnlZCvXuR+VsgTsjn96vH
+j0AczAZ+g7gD8XTl/K9Z/gcs06DcmMonYrgOGuaFDDiEuBwgABo8gajCFg4xwxi2
+bw7B+opMrOXH9ZGhaoF2eySDGXXgw7TLRqkGXDghZNzWajnGuN7vSaiQfnr0EmW+
+020gJDDUZyc9Ky165SQe3Bfin1cLc2W6mZYmV9lDtUYw80Gth52L9uOiEBIbV6mI
+ZBZFyslxQ7IYWOxseoU9xrxzscTtxa+MGs47w6Hzxh38zYPe/I0Yt62yMRtUmJJ7
+ebQZti7qLeW+QOZAruUzei7fpOZOrq8vy4GBUm0pkg8eOVdDaCAQAWMwaGaHH9/5
+q+AvDLjvR6zvJdV2dxA3XsVmcIA45zOA2mZkRrcTWyf5DZ7bDcQvSnd7R4anv3hG
+YJXeKyzcI7SWfMxo0hU6p9fv66xYn6x9d5oA/ZU/5XRn1bFL7kuKj7BmB+LcS0BE
+XipqaCA=
+-----END CERTIFICATE-----

roles/docker-registry-proxy/files/key.pem

@@ -0,0 +1,52 @@
+-----BEGIN PRIVATE KEY-----
+MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQCrznLzCWlKJO4f
+XD/E8hx4cXUqbd9UVwpcAzWq3xGjC6gettYSp171elDXj23ddDJ1wwOl2U0bjN/D
+ceRCl4Tnb3O86fztBwj8xA/stYGvZQqOvEhSfFh85qvDf1niu2uW1Zx4kMemUvNd
+hpXsqa9RPSu0Mb0GZeHnCQkuz3KTjUMhQqwomg/6BY4G7tDmCZYsZezGSgAgoa+Q
+4vffW+H8S9nuKi8oEXUf9NuJHUTjtdgcVcCihPj54jXAQsqS9JzWSWxDnKxTaOZu
+RWEkG+vqGoKEL5+qPeH8x8aAM7wErsdxTqTV4XCJU0nS9om1Z6sz0Lrva/loyKci
+ulTO8jYWqZIuBL8JGVeizcoYl9KcW8I66XkeuYWlNWCsWhGii7zEWVcXDdSuLCv1
+wLagRE9MJHoT64ndKAR2UQml/MSPlz8419K4r87hcVmNU6FFBP2RZO7UGW5eHbbK
+T+whhDdBTaig2NNLMl7pVFq6ciemNr6IVsTuzS0VPJpuOoZa86+6UPqgw49jg4cB
+lKxgJxanDy2a2Hbmzx2dPAkbz3kMKDeBmgFzk3xt10czPXvIXUSnJdhi8NdQUCBH
+du9yjT3s7Cc12NCQ3H0NPWUklHlqJrPY9IbGnNrmJblwZ7hyrI3eISV9njQf3etL
+2QdDXksLJpgeQsv7vcXEBje+aluNqQIDAQABAoICAQCjIxe3expFUyfhE2FiC1vJ
+akKNFWNY3IVztYCCTeqbXXg4IfjIIbFjes/Ev+bcv3cipxiRpPM4092t4jmSmfmT
+IRtPKQgHsgRwr2NHq1oHR/RscJBj8rq7bvVuX0DksH6K7S70tNU/M8ju59r4rG9S
+vrj/E7OfnaKSFNxpXIY5YYt6y6pZq2C8UgX4w1AM/tFgOzBHYQEZ+y2QcFRZ/Q9q
+2EOJiPjaHSmQPJsxaV9+sa8RyMNwDr+z136en01nmWpLd9CbqutfEF6uGqcQ+Ipc
+8us3xVjg+H3b3363QWipMaUkDD8s8DJB05pn/b3pSRUh0HOQ4IAlZVJ/AVuCXi/I
+amEVq5hClLL16OhyM4u/50BxSn673jNCi3uRPxcJlnR0cFy8u6XEs8rbU1ezxlz3
+SJBpTfXyvvWhNdvEQzNy+AOf8XQLzKgcnoLYYx9nhFUBv3pSU+7hW11RKmjHSu9v
+0NNcUGq+Ig3QTB/4CTM1YJ/usL9kVdJY+tK1wVKHiIm0O58fcyrfN05kUpqJ5NMg
+4ByeXkm8JR8A9jiNmJy/bBuFEIXTsxalsQjonGw4WHcRZCke0eqMSOgzp50CNad2
+NIRqNWD0EInTatXtjTQ+zbCkDoUdgW4NhCnmf4OxVLEpKIYFw/y1271mVZp9zbYU
+aAfaTXi/mZ+hAspQlTOPAQKCAQEA3BhyO90ACUtKbbBPwUVgEHxtsKZ1lyIgyGMQ
+D8PDh+ixdvRYD4m4rewTzY346kaEPcWzjKUp2sU0G1/moBQWLBb9gKyBll8LKSNG
+yA5MAMENyI9rIFpzOoJkkTrnu8iwIPXjVgAShrRqBa+eKAa5XEOR6x+L5UhIEZIQ
+mfCqyPAKKieODiykqqmDURCadRC3LrIbjDSdnX6VEMS7Sun6pNRz7s1u5CYnLQGZ
+QhZeRMkcdmYuAThfwWpX/GdtIxqM08jWX9RFKpMiSisKX8YsXv0W0IbesuDj/bkc
+4BQou03sEUJxC6P8O+jiKtOkygwTdqmjIpDRf1EHiPGibv+f+QKCAQEAx9VfJ/5e
+zmLWwEEqSxpwWpMQdB5ir68VjnnvFNDVms/XVHr/EV7TuI092cXJdntqqynCZGBA
+IP1Wv2eSqGibIyOXdbTXv1qWmSxZdeECIj11vtRQ5etK977/F0llNYav6WAn3pDp
+0IRzofaD6SEFhTJKoGiv86gFcqm2tO4lrTU1B30KnqKfYER6mUdBwto///Rwrpoy
+B5+EWbrjJmKOqKuXP+M/YnlQXeBtyVHlQlaog2sea9OSCJEbTrmkZqJ1ZXmBH8T+
+D91QOF+5rxPVKLG5Ybnfen7Fu3dHAotD8WM77iQR+EDmffvrxS+ddZUXzINS9jml
+kKkaG9zOvEcnMQKCAQEAyIo1u7nYSJ+jh2I4qT9PEnZtc6GYT0a3XB53CgYzaOhq
+mpp0imPQNBiAyrBrdvsdjzNOL/5lroI0wiSVfJIQyceA3/dOc/bRsoAEBFCSi7Vb
+m7yhvW7swwkAHRvw/bcUVFP2+etC9h345Ilpr8rApgKjN/sceqNrlybhnYId+sxM
+VrCHzP58Y0vk7L4WHkhGwHNkilF+s3wc0pSOmumqiPlTUOk5+wOQen+UZxT+e+pK
+1s6vaEk3ZoJA/Sg31t5gJrA+ND6zbuF1QuMIps9oqnwsh3/79jzXP92lI776hf+v
+8uH5IsQeFXBScvc4lSh/q4VRsTMGz9zC4tJYUI718QKB/3qNYM4mMf5gn1NIo6dr
+j3v8tRqBiAQ2XAIExZr+eAF5dZVZ2RPOFAoalNP5eJQxHDncYlssrCePNqQr4MVn
+Yb0rFrgZMDcqVzGZAURJugVFq/BcRUC8DD3j5I1jda5d64Q0dD8KoFpA4KlzhXJz
+ze7h6OJ3UXEcmjq32lUbt/+BogP1q42eLh/b31QhXzMgph9SychKyGPkcEaXVrcz
+ukm28gs8UqMRwzfPa4ULtI36l14BU6bNGcInO5gMQcav209gNNBG/4i7MXdhPX8h
+qphKZmaIl4WIObu+as4kmoZvVVG2zU5yfujEltNXYDm8Ndw2rapTsDYHfvuXbzII
+cQKCAQBW3LAfFAkmu1+NJBXYt86rftOF+VSNWkN1/YkPwIMX1y647aVMGMegr7yF
+xUh1DSQQAuD2ACzII1ufoUWRrhdCMsgr3o9b0ApCXQwTaaFsZjGIr33bsnqNHW3e
+FJEfTrNW5PLTkkEjJQH0N/6W0TRowjpYSpgRz/fpJjdFLmQ1A+RLVoyHCVq/Qhzj
+Ywk6hsYjI432aebdFH8pqWl8Hhcq6DW9jAyKkuVnX/p60OZ6tp6cZ75nIj7bdB7W
+zcrUs4/igRY8HUwZlQJK5X2D+LWuN1Ag8DBbbjOmqziKDBikV/GmOcuCRgltckrT
+UFg2hiaXvnBuMgGHodqIzeQarqKv
+-----END PRIVATE KEY-----