Wireguard Logo Netmaker Logo

I’ve recently been working on setting up a personal VPN “infrastructure” with wireguard. Previously, I’ve been using a very simple setup to tunnel from my home connection to a server hosted in the US, to get around ISP web filtering.

Recently, when out and about in a coffee shop, I found myself needing access to files on my desktop workstation, and had to traipse home to complete my work. This pushed me over the edge into finally setting up a home VPN, in order to securely access my home LAN from the internet.

While wireguard is easy to configure, it (deliberately) doesn’t include any features for provisioning new clients or managing configurations generally. This is where netmaker comes in. It’s a configuration management layer for wireguard, capable of pushing out wireguard configurations to clients. It’s capable of provisioning complex fully meshed networks, but we can use it to manage a fairly simple wireguard setup.

In this article I’ll describe how I run netmaker with docker-compose, and how I handle some aspects of the configuration. There are also steps to bring the whole thing up. In part 2, I’ll describe the LAN gateway implementation, and part 3 will go over the “personal VPN” aspect.

Network Diagram

The plan is to configure the LAN server as a netmaker server instance, with the VPN running the netclient service which automatically recieves configuration updates from the server. The LAN and mobile devices will be configured as what netmaker calls “external clients,” which involves them running the ordinary wireguard client, configured with config files or QR codes automatically generated from within the netmaker UI.

Configuration

Here is my docker-compose.yml file for the netmaker server instance:

version: "3.4"



services:

  netmaker:

    container_name: netmaker

    image: gravitl/netmaker:v0.8.5

    volumes:

      - sqldata:/root/data

    environment:

      SERVER_HOST: "${EXTERNAL_DOMAIN}"

      SERVER_API_CONN_STRING: "${INTERNAL_IP}:8001"

      SERVER_GRPC_CONN_STRING: "${EXTERNAL_DOMAIN}:4444"

      GRPC_SSL: "on"

      DNS_MODE: "off"

      SERVER_HTTP_HOST: "${INTERNAL_IP}"

      SERVER_GRPC_HOST: "${INTERNAL_IP}"

      API_PORT: "8001"

      GRPC_PORT: "50051"

      CLIENT_MODE: "off"

      MASTER_KEY: "${MASTER_KEY}"

      CORS_ALLOWED_ORIGIN: "*"

      DATABASE: "sqlite"

    ports:

    - 8001:8001

  netmaker-ui:

    container_name: netmaker-ui

    depends_on:

      - netmaker

    image: gravitl/netmaker-ui:v0.8.5

    links:

      - "netmaker:api"

    environment:

      BACKEND_URL: "http://${INTERNAL_IP}:8001"

    ports:

      - 8002:80

  caddy:

    image: caddy:latest

    container_name: caddy

    restart: unless-stopped

    ports:

      - 4443:443

      - 4444:4444

    volumes:

      - ./Caddyfile:/etc/caddy/Caddyfile

      - caddy_data:/data

      - caddy_conf:/config

    extra_hosts:

      - host.docker.internal:host-gateway

  netclient:

    container_name: netclient

    depends_on:

      - netmaker

    image: gravitl/netclient:v0.8.5

    network_mode: host

    privileged: true

    cap_add:

      - NET_ADMIN

      - SYS_MODULE

    volumes:

      - /etc/netclient/config:/etc/netclient/config

      - /usr/bin/wg:/usr/bin/wg

    environment:

      SLEEP: 10



volumes:

  sqldata: {}

  caddy_data: {}

  caddy_conf: {}

And is the Caddyfile for the reverse proxy, which should live next to the compose file:

{

        admin off

        email ${EMAIL}

}



# gRPC

https://${EXTERNAL_DOMAIN}:4444 {

        reverse_proxy h2c://netmaker:50051

}

Some notes on this configuration:

  • We have a few variables which need to be supplied:
    • $INTERNAL_IP is the LAN ip of the server machine.
    • $EXTERNAL_DOMAIN is some publically routable domain name pointing to your home internet connection’s public IP. A dynamic DNS service will be useful here.
    • $MASTER_KEY is a random alphanumeric string serving as authentication secret for netmaker’s backend API. The documentation supplies this handy bash oneliner: tr -dc A-Za-z0-9 </dev/urandom | head -c 30 ; echo ''
    • $EMAIL is your email for obtaining let’s encrypt certificates

  • CLIENT_MODE is off on the server, and netclient is run out of its own container.

    As one typically wishes for the machine running the netmaker server (which, remember, only manages wireguard configurations, and doesn’t actually implement the tunnels) to also be part of the wireguard mesh, we also want to run the netclient binary on the same machine. To this end the netmaker server has the CLIENT_MODE option, which as you might imagine, helpfully spins up netclient inside the same container.

    The problem is that this built-in client automatically joins all wireguard networks created on the server, (and rejoins them if removed.) One of my networks is going to be configured to tunnel any and all internet traffic out via the VPS, which is on the other side of the world and is only there to defeat ISP filtering when necessary, typically on client machines. Having my home server forced to operate in this way all the time is unacceptable for a number of reasons.

    To fix this, we simply run our own separate netclient instance in its own container, and connect it to the wireguard networks as desired.

  • DNS_MODE is turned off. This is mainly due to incompatibilities with the environment I’m running the server on, (a synology NAS.) More generally I do not have a DNS setup for my home LAN, I just remember the IP addresses. There aren’t a lot of them. Local DNS is a project for another day.

  • We’re not putting the server behind an HTTPS endpoint. The netmaker documentation demonstrates setting it up behind the lightweight Caddy reverse proxy, as well as the more familiar ngnix. I’m not doing that because the server and its web UI are only going to be accessible from inside my home network, which I do not consider to be a secure environment.

    The setup detailed in this post enables me to tunnel back into the network securely from outside, from my phone or laptop, allowing me to access the netmaker server and to add new clients to the tunnel from offsite. In the event that I don’t have access to my phone or another device already part of the wireguard mesh, the VPS is permanently joined. I can ssh into the VPS using FIDO2 authentication via my yubikey token, so I will always be able to tunnel back into my home LAN from wherever I am.

    The thing exposed to the outside is the gRPC interface that netmaker uses to communicate with the clients, and this has its own automatically configured SSL encryption, which we have turned on via the GRPC_SSL flag.

  • Only netmaker’s gRPC port, 50051, is being put behind SSL, via the lightweight Caddy reverse proxy. One could of course use nginx, but caddy is a good fit here as it automatically obtains the certificates for us, it’s small, and it’s easy to set up.

    This port is the only one that will be opened on the router and exposed to the internet, and so it’s the only one that really needs SSL. Both the web UI and the http API accessed by that web UI are exposed on the docker host, and will be accessed directly from the browser with no encryption. I’m happy with this because I don’t not consider my home LAN to be a secure environment. If I wanted to use SSL inside the network I’d have to use self signed certificates, with the annoyances that entails.

    The setup detailed in this post enables me to tunnel back into the network securely from outside, from my phone or laptop, allowing me to access the netmaker server and to add new clients to the tunnel from offsite. In the event that I don’t have access to my phone or another device already part of the wireguard mesh, the VPS is permanently joined. I can ssh into the VPS using FIDO2 authentication via my yubikey token, so I will always be able to tunnel back into my home LAN from wherever I am.

Setting it up

Netmaker server

  • First install Wireguard on the server, as detailed here.
  • Place the above docker-compose.yml file in a convenient place, and ensure the variables are defined correctly.
  • Note: You may need to amend the /usr/bin/wg:/usr/bin/wg bind mount in the netclient service to whereever the wg binary is installed on your system.
  • Ensure that EXTERNAL_DOMAIN points to your external IP address.
  • On your router, forward TCP port 50051 to your netmaker server. This is for the netmaker gRPC connections, which it uses to do its job of distributing the configurations.
  • Additionally, forward UDP ports starting at port 51821, with one for each wireguard network you would like to set up. For example, I’m using two networks, so I have UDP ports 51821-51822 forwarded.
  • Create the netclient config directory with sudo mkdir -p /etc/netclient/config
  • Bring up the compose stack with sudo docker-compose up -d and monitor the logs with docker-compose logs -f

You should see output akin to the following, after docker has downloaded the images and started the containers:

netclient      | [netclient] joining network

netclient      | 2021/11/08 11:14:58 error decoding token

netclient      | 2021/11/08 11:14:58 illegal base64 data at input byte 0

netclient      | [netclient] Starting netclient checkin

netclient      | 2021/11/08 11:14:58 [netclient] running checkin for all networks

netmaker-ui    | >>>> backend set to: http://<INTERNAL_IP>:8001 <<<<<

caddy          | {"level":"info","ts":1636370034.685589,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}

caddy          | {"level":"warn","ts":1636370034.688002,"msg":"input is not formatted with 'caddy fmt'","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}

caddy          | {"level":"warn","ts":1636370034.688278,"logger":"admin","msg":"admin endpoint disabled"}

caddy          | {"level":"info","ts":1636370034.6884842,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00035dd50"}

caddy          | {"level":"info","ts":1636370034.6885617,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}

caddy          | {"level":"info","ts":1636370034.689109,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["<EXTERNAL_DOMAIN>"]}

caddy          | {"level":"info","ts":1636370034.6893475,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}

caddy          | {"level":"info","ts":1636370034.6893585,"msg":"serving initial configuration"}

caddy          | {"level":"info","ts":1636370034.6893992,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/data/caddy"}

caddy          | {"level":"info","ts":1636370034.6894226,"logger":"tls","msg":"finished cleaning storage units"}

caddy          | {"level":"info","ts":1636370034.6904533,"logger":"tls.obtain","msg":"acquiring lock","identifier":"<EXTERNAL_DOMAIN>"}

caddy          | {"level":"info","ts":1636370034.7785852,"logger":"tls.obtain","msg":"lock acquired","identifier":"<EXTERNAL_DOMAIN>"}

caddy          | {"level":"info","ts":1636370035.6691058,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["<EXTERNAL_DOMAIN>"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"<EMAIL>"}

caddy          | {"level":"info","ts":1636370035.6691432,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["<EXTERNAL_DOMAIN>"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"<EMAIL>"}

caddy          | {"level":"info","ts":1636370036.1192043,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"<EXTERNAL_DOMAIN>","challenge_type":"tls-alpn-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}

caddy          | {"level":"info","ts":1636370036.5058181,"logger":"tls","msg":"served key authentication certificate","server_name":"<EXTERNAL_DOMAIN>","challenge":"tls-alpn-01","remote":"172.18.0.1:45608","distributed":false}

caddy          | {"level":"info","ts":1636370036.6690965,"logger":"tls","msg":"served key authentication certificate","server_name":"<EXTERNAL_DOMAIN>","challenge":"tls-alpn-01","remote":"172.18.0.1:45610","distributed":false}

caddy          | {"level":"info","ts":1636370036.8740158,"logger":"tls","msg":"served key authentication certificate","server_name":"<EXTERNAL_DOMAIN>","challenge":"tls-alpn-01","remote":"172.18.0.1:45612","distributed":false}

caddy          | {"level":"info","ts":1636370036.9339938,"logger":"tls","msg":"served key authentication certificate","server_name":"<EXTERNAL_DOMAIN>","challenge":"tls-alpn-01","remote":"172.18.0.1:45614","distributed":false}

caddy          | {"level":"info","ts":1636370037.1912491,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/<REDACTED>"}

caddy          | {"level":"info","ts":1636370038.0497818,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/<REDACTED>"}

caddy          | {"level":"info","ts":1636370038.0507696,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"<EXTERNAL_DOMAIN>"}

caddy          | {"level":"info","ts":1636370038.0507824,"logger":"tls.obtain","msg":"releasing lock","identifier":"<EXTERNAL_DOMAIN>"}

netmaker       | 2021/11/08 11:13:53 [netmaker] connecting to sqlite

netmaker       |

netmaker       |     ______     ______     ______     __   __   __     ______   __

netmaker       |    /\  ___\   /\  == \   /\  __ \   /\ \ / /  /\ \   /\__  _\ /\ \

netmaker       |    \ \ \__ \  \ \  __<   \ \  __ \  \ \ \'/   \ \ \  \/_/\ \/ \ \ \____

netmaker       |     \ \_____\  \ \_\ \_\  \ \_\ \_\  \ \__|    \ \_\    \ \_\  \ \_____\

netmaker       |      \/_____/   \/_/ /_/   \/_/\/_/   \/_/      \/_/     \/_/   \/_____/

netmaker       |

netmaker       |  __   __     ______     ______   __    __     ______     __  __     ______     ______

netmaker       | /\ "-.\ \   /\  ___\   /\__  _\ /\ "-./  \   /\  __ \   /\ \/ /    /\  ___\   /\  == \

netmaker       | \ \ \-.  \  \ \  __\   \/_/\ \/ \ \ \-./\ \  \ \  __ \  \ \  _"-.  \ \  __\   \ \  __<

netmaker       |  \ \_\\"\_\  \ \_____\    \ \_\  \ \_\ \ \_\  \ \_\ \_\  \ \_\ \_\  \ \_____\  \ \_\ \_\

netmaker       |   \/_/ \/_/   \/_____/     \/_/   \/_/  \/_/   \/_/\/_/   \/_/\/_/   \/_____/   \/_/ /_/

netmaker       |

netmaker       |

netmaker       | 2021/11/08 11:13:57 [netmaker] database successfully connected

netmaker       | 2021/11/08 11:13:57 [netmaker] no OAuth provider found or not configured, continuing without OAuth

netmaker       | 2021/11/08 11:13:57 [netmaker] Agent Server successfully started on port 50051 (gRPC)

netmaker       | 2021/11/08 11:13:58 [netmaker] REST Server successfully started on port 8001 (REST)

netclient      | 2021/11/08 11:15:09 [netclient] running checkin for all networks

Some things to note:

  • netclient is printing some warnings on lines 2 and 3, these can be ignored.
  • On line 8 caddy complains that the Caddyfile is not formatted to it’s liking - caddy fmt does not seem to fix this. Oh well.

  • On line 17-29 we see caddy automatically obtaining let’s encrypt certificates for the gRPC endpoint.

  • Now, browse to http://<INTERNAL_IP>:8002. You should see the blue netmaker UI.
  • Create a username and password, and log in.

Netmaker is now set up and ready for networks to be created and nodes to be added.

This concludes part 1. Part 2, “home LAN gateway” is here, and part 3, “personal VPN tunnel” is here.