Have you ever tried to set up a new virtual machine with
custom certificates? The
docker-machine create command offers a couple of
options to set custom certificates. However, to me, it was not clear, which
option has what effect. This article discusses which certificates and keys are
involved and which can be overridden with custom files.
Involved certificates and keys
Before I discuss the options of
docker-machine, I’d like to show what
certificates and keys are involved. Generally speaking, an HTTP connection
secured with SSL/TLS can involve up to four different parties. Obviously, we
need a server and a client. A client initiates the connection to the server. The
server is in possession of a private key and a certificate. The certificate is
issued by a third party (unless self-signed), the Certification Authority (CA).
The CA asserts that the identity of the server is correct. The CA itself has a
private key and a certificate (usually self- or cross-signed). The client trusts
the CA to
verify the identity of the server and therefore accepts the certificate sent
by the server. A secure connection can be established. This is how most of the
every-day HTTPS connections with your browser work.
When the docker-client (the docker command-line tool
docker) connects to a
remote docker daemon (
dockerd) via a secure connection (port 2376 by default),
the server (
in this case) should also request a certificate from the client in order to
reject unauthorized clients. The server only accepts clients if they have a
certificate from a particular CA, here called Client-CA. You see, there are up
to four parties involved when we use HTTPS client authentication. This is
illustrated in the following figure.
The picture becomes even more complicated when you consider that the docker
daemon becomes a client itself when it connects to a remote registry. In this
case, we can use the above figure again, replacing
dockerd with the remote registry. This introduces three new parties: two
CAs and the remote registry. It also introduces a new key and a new certificate
as a client. However, this complication is not the main focus of
this article. I will treat
dockerd mostly as the server in this article
The question is now, how can we create a new
docker-machine with custom
The command-line tool
docker-machine provides four general and one specific option to manage TLS
credentials when creating a new machine. Version
0.13.0’s built-in help lists them as
# general options --tls-ca-cert CA to verify remotes against [$MACHINE_TLS_CA_CERT] --tls-ca-key Private key to generate certificates [$MACHINE_TLS_CA_KEY] --tls-client-cert Client cert to use for TLS [$MACHINE_TLS_CLIENT_CERT] --tls-client-key Private key used in client TLS auth [$MACHINE_TLS_CLIENT_KEY] # only for 'docker-machine create' --tls-san option Support extra SANs for TLS certs
By simply reading the built-in help, I didn’t understand which option overwrites which key or certificate in the above picture. After some investigations, I can now shed some light on this. The first key observation is that there is only a single CA issuing the server certificate and used to authenticate clients. The following figure summarizes the relation between the CA, the client and the server.
docker-machine create uses the keys and certificates located at
~/.docker/machine/certs. This directory hosts the key and certificate for the CA
ca.pem), and the key and certificate for the client (
cert.pem). The client certificate was issued by this CA. The first four
command line arguments can be used to shadow these files. If you add
--tls-ca-cert for example, it will use the default files, except for the CA
certificate. If this is a custom one, it most likely doesn’t match the default
CA’s private key and therefore provisioning the machine will fail.
If you want to use different client credentials, use the
--tls-client-key options. Since this still uses the
default CA, you need a certificate issued by the default CA, otherwise, you won’t be
able to connect to the server.
If you want to use
a different CA, use the
--tls-ca-key arguments. The
certificate should be a self-signed certificate matching the specified key.
If you use a different CA, you also need to specify a custom client certificate.
docker-machine uses the default ones which are not issued by the
custom CA and won’t allow you to connect the server.
The private key and certificate of the server are always generated when you
docker-machine create. This is convenient because the machine’s IP
address is not known in advance but must be included in the certificate. If your
machine is accessible via other IP addresses or domain names use the
--tls-san option to add additional Subject Alternative Names to the
certificate. To issue the server certificate,
docker-machine uses the
CA’s private key: either the default key or the one specified via
--tls-ca-key. If you specify a custom CA key,
docker-machine will not store
the key for you.
It is also interesting to note, that if you use the default options,
docker-machine will use the default files for all machines. Machines created
without any options share the same CA certificate, client key and client
In case you were wondering how to deal with custom certificates when the docker daemon itself becomes the HTTP client of a remote registry, it becomes a bit less user-friendly. You need to manage the certificates inside the virtual machine yourself. Check the docker help for more information.
The following table summarizes the relevant command-line arguments. The File
column specifies where the machine-specific files corresponding to the
command-line arguments are stored. All machine-specific files are located in
~/.docker/machine/machines/<MACHINE_NAME>. The Default column specifies the
files that are used when the command-line arguments are omitted. The default files
are stored in
||Private key of the Certification Authority. The server certificate is signed with this key. A custom key is not stored. Must be used in conjunction with
||Self-signed certificates of the Certification Authority used for client and server authentication. Must be used in conjunction with
||Private key of the client. Must be used in conjunction with
||Client certificate. Must be issued by the CA (custom or default) in order to connect to
||none||Automatically created certificate, issued by the CA (custom or default). The option