Caging the PowerDNS API

A while ago I decided to start using wildcard certificates for some of my domain names. I never liked the fact that every time I registered a new cert, my subdomain would be added to the public Certificate Transparency logs, creating a public inventory of my services and infrastructure. I kind of just lived with it though, as doing ACME challenges via HTTP just felt operationally simpler than using the DNS challenge.

I host my own DNS using the PowerDNS Authoritative Nameserver, and I've recently started putting some of my services behind Traefik. Traefik can talk to the PowerDNS API to create temporary DNS records to allow ACME DNS challenges to work, which is necessary to get wildcard certificates.

The trouble with the PowerDNS API, is that it gives you full access to do anything. If you give an application an API key to interact with PowerDNS, it can read all zones, create new ones, delete them, create/edit/delete any records it wants, read the server config, fetch DNSSEC private keys, read TSIGs etc.

This made me a bit uncomfortable as Traefik is public Internet facing, so is directly exposed to attacks. I want it to be able to speak to the PowerDNS API for one simple reason, to read, create and delete specific DNS records in specific zones, matching specific qtypes and qnames. I don't want it to be able to do anything else. So to address this problem, I created an API proxy to sit in between Traefik and PowerDNS, called PDNS-API-Cage, also available at Docker Hub.

A simple configuration looks like this:

# PDNS API service is the upstream
upstream:
  url:          http://10.88.1.54:8081
  server_id:    localhost
  api_key_file: /run/secrets/pdns_api_key

# Where we expose the PDNS-API-Cage proxy port
listen: 10.88.2.10:8080

# And the admin interface for container health check
admin_listen: 127.0.0.1:9090

# The cages - Each cage has different API key(s) and different policies defining access it grants
cages:
  traefik:
    # Traefik uses the traefik cage and gets a preset including everything needed to do ACME challenges
    preset: acme
    # Only allow these ACME challenges for the following two zones that I host DNS for
    zones:
      - example.com
      - grepular.com
    # The API key(s) we will give to Traefik to allow access. Usually just one, but can be multiple if needed for key rollover purposes
    tokens:
      - id: "@@TRAEFIK_TOKEN_ID@@"
        secret_sha256: "@@TRAEFIK_TOKEN_SHA256@@"

The @@...@@ values are placeholders. Running docker run --rm grepular/pdns-api-cage token new traefik prints the token to give to Traefik, along with the id and secret_sha256 to put in their place.

Traefik does not know the PowerDNS API key, only the proxy does. Traefik is given a token as configured in the tokens field, which gives it the access as defined in the "cage" in which the token exists. In the above example, we have a preset named "acme" that just gives it the access it needs to do ACME challenges for the example.com and grepular.com zones only. The acme preset was built from, and is tested against, every ACME client with a PowerDNS integration I could find, including lego (which Traefik uses), acme.sh, certbot and Caddy. The preset is just a convenience though, any ACME client should be able to work through the cage given the right configuration. Although I built this application to do ACME challenges, it is not limited to that. You can configure a wide variety of different types of access. For example:

cages:
  dyndns:
    zones: [example.com]
    allow:
      - name:        home.example.com
        types:       [A]
        changetypes: [REPLACE]
        ttl:         {min: 60, max: 3600}
        max_records: 1
        content:     ipv4
        reads:       true
    tokens:
      - id: "@@DYNDNS_TOKEN_ID@@"
        secret_sha256: "@@DYNDNS_TOKEN_SHA256@@"

You might use the above configuration for Dynamic DNS. It allows a client to only set the home.example.com A record, to a single IPv4 address with a TTL between 60 and 3600. It also allows that client to read the existing A record only, and no other records from the zone.

Using/writing secure software is important to me. PDNS-API-Cage is a sub 10MB docker image (under 4MB to download). It's a single binary written in golang (open source) placed on top of a scratch image. It also uses seccomp and landlock to restrict its syscall, file system and network access.

A daily CI pipeline runs to automatically update its version of go, and any dependencies (which are minimal) it might have. Dependency updates have a 7 day cool off period due to supply chain attacks. An update that fixes a known vulnerability (CVE) in the version we're using skips the cool off period, unless our VEX triage has shown that the vulnerable code can't be reached from PDNS-API-Cage. If an update changes what ships, the pipeline cuts a new patch release by itself; minor and major releases need my PGP signed git tag. Images are signed with cosign (keyless, via GitLab's OIDC identity), builds are byte for byte reproducible, and every release is checked by three vulnerability scanners (Trivy, Grype and Docker Scout) as well as govulncheck. It comes with an automatically generated SBOM and an automatically updated OpenVEX document.

An example compose file with further useful hardening features used:

services:
  pdns-api-cage:
    image: grepular/pdns-api-cage:latest
    volumes:
      - /var/lib/operations/pdns-api-cage/config.yaml:/config/cage.yaml:ro,z
    secrets:
      - pdns_api_key
    read_only:  true
    cap_drop:   [ALL]
    ipc:        none
    mem_limit:  128m
    pids_limit: 64
    security_opt:
      - no-new-privileges:true

secrets:
  pdns_api_key:
    file: /var/lib/operations/pdns-api-cage/pdns_api_key

In my own hosted setup, the PDNS API port is only accessible from the PDNS-API-Cage container. The PDNS-API-Cage container exists on two docker networks, one so that it can only talk to the PDNS API port, and one so that only Traefik can talk to it. It has no access to the public Internet at all as it doesn't need it.

If you run PowerDNS and have handed its API key to anything, whether that's Traefik, certbot or a home-grown dynamic DNS script, it's worth asking what that key could do in the wrong hands. With PDNS-API-Cage in between, the answer becomes "only what you wrote down". The README covers the full configuration and tooling, and DESIGN.md the threat model behind it. Feedback and bug reports are welcome. Like this article

Want to hire me? Get in touch