Docker Failed to Query External DNS Server: What It Means and How to Fix It

systemd-resolved stub listener address leaking into containers

This is the single most common cause of this error on Ubuntu 18.04 and later, Debian with systemd-resolved enabled, Arch Linux, and any other distribution that points /etc/resolv.conf at systemd-resolved's stub listener at 127.0.0.53.

When dockerd reads /etc/resolv.conf and finds nameserver 127.0.0.53, it recognizes the address as a systemd-resolved stub and switches to reading /run/systemd/resolve/resolv.conf instead -- Docker's own code does this automatically. However, this behavior has edge cases. If the upstream servers in the stub file are themselves unavailable, or if the stub file was not populated correctly, Docker ends up forwarding queries to addresses that time out. In other configurations, the daemon may forward to 127.0.0.53 anyway, which cannot be reached from within a container's isolated network namespace since loopback is per-namespace.

On distributions that do not use systemd-resolved at all -- Alpine Linux, Gentoo, older Debian without systemd, and any system using openresolv or direct /etc/resolv.conf management -- this exact loopback failure is less common but still possible. openresolv is a framework that manages /etc/resolv.conf on behalf of DHCP clients, VPN clients, and other network managers on non-systemd systems. If openresolv is configured with a local caching resolver like dnsmasq or pdnsd that binds to 127.0.0.1, it will write nameserver 127.0.0.1 into /etc/resolv.conf. Docker inherits that address and fails for the same reason: 127.0.0.1 inside a container namespace refers to the container's own loopback, not the host's. On Alpine and similar systems without systemd, Docker does not attempt any special resolv.conf fallback logic -- it reads the file as-is and uses whatever addresses it finds. The fix is the same as everywhere else: set explicit upstream resolvers in /etc/docker/daemon.json. On Alpine specifically, the file path and systemctl restart command are the same; only the init system differs (service docker restart instead of systemctl restart docker).

IPv6 loopback address in resolv.conf

The same namespace boundary problem that makes 127.0.0.53 unreachable from containers applies equally to IPv6 loopback addresses. On systems where /etc/resolv.conf contains nameserver ::1 -- which appears on hosts running a local DNS resolver like Unbound or dnsmasq bound to the IPv6 loopback -- Docker's embedded resolver inherits that address and tries to forward queries to it. From inside a container's network namespace, ::1 refers to the container's own IPv6 loopback, where nothing is listening on port 53. The result is an i/o timeout identical in structure to the IPv4 variant.

This variant is less common than the 127.0.0.53 case but regularly catches engineers off guard because they have already confirmed that 127.0.0.x addresses are absent from their resolver list without checking the IPv6 column. Inspect the full resolv.conf output, not just the first line:

# Check for both IPv4 and IPv6 loopback addresses in the host resolver config
$ grep nameserver /etc/resolv.conf
nameserver ::1    # <-- IPv6 loopback: will fail inside containers
nameserver 127.0.0.1    # <-- also a loopback: same problem

# Check what the embedded resolver would actually use
$ grep nameserver /run/systemd/resolve/resolv.conf

The fix is identical to the IPv4 loopback case: set explicit, routable upstream resolver addresses in /etc/docker/daemon.json. If you want to keep using a local resolver on the host, configure it to bind to the Docker bridge address (172.17.0.1 by default) rather than loopback, and point Docker at that bridge address as described in the systemd-resolved section above. The same DNSStubListenerExtra pattern works for dnsmasq and Unbound -- both support binding to additional interfaces.

Upstream DNS server unreachable or blocking UDP port 53

The error message i/o timeout specifically means Docker sent a UDP packet to the named DNS server and never received a reply. This happens when the upstream server address is correct and reachable in principle but queries time out because of network firewall rules, the server being overloaded, or the server being on a network segment not routable from the Docker bridge network.

On cloud instances, this manifests when the host's DNS resolver is a metadata-service address that is directly routable from the host but only reachable via the host network stack, not from inside a container's bridge-isolated namespace without NAT rules in place.

• AWS EC2: The VPC resolver lives at 169.254.169.253 (or the base of the VPC CIDR +2, e.g. 10.0.0.2). The link-local 169.254.x.x address is unreachable from inside Docker bridge namespaces by default because the kernel does not route link-local traffic through NAT. The VPC base+2 address is generally reachable as long as Docker's MASQUERADE rules are intact, but some hardened AMIs block it.
• Azure VMs: The Azure DNS resolver at 168.63.129.16 is a special platform address routed at the hypervisor level. Traffic to this address from inside a container bridge namespace typically drops silently at the virtual switch -- it never reaches the hypervisor. Use Azure's public recursive resolvers or your own DNS forwarder as an explicit daemon.json entry instead.
• GCP Compute Engine: The metadata resolver at 169.254.169.254 and the VPC resolver at 8.8.8.8 (available by default in most GCP projects) behave differently. The metadata address has the same link-local problem as AWS. Set 8.8.8.8 explicitly in daemon.json on GCP, or use your project's internal resolver IP if you have a private zone.

The common thread across cloud providers is that their internal resolver mechanisms rely on hypervisor-level routing tricks that do not extend into guest network namespaces. Understanding this distinction explains why a container host that resolves names perfectly at the shell level can still fail to resolve anything from inside a container -- the two are not using the same network path.

iptables rules flushed or corrupted

Docker relies on iptables for two things that are both relevant to DNS: MASQUERADE rules in the nat table that allow containers to reach external IP addresses, and DNAT rules that redirect container DNS traffic to the embedded resolver. If these rules are flushed -- by a system reboot that did not properly restore them, by a third-party firewall manager like ufw or firewalld overwriting Docker's chains, or by manual iptables -F -- DNS queries from containers fail at the network layer before they ever reach an upstream server.

The symptom here is the variant of the error message that reads write: operation not permitted rather than i/o timeout. That indicates the kernel rejected the packet outright rather than allowing it to be sent.

VPN clients changing the host's DNS configuration

VPN clients frequently update /etc/resolv.conf or the systemd-resolved configuration with internal corporate DNS servers that are only reachable through the VPN tunnel interface. Docker's embedded resolver captures those internal server addresses as its upstreams. When the VPN is disconnected, or when the VPN routes do not extend into Docker's bridge network subnets, queries to those internal servers time out. The embedded resolver then logs the error for every forwarded query.

There are three distinct patterns within this root cause, each with a different fix:

• VPN disconnects after Docker starts, resolver list becomes stale. Docker reads its upstream resolvers once at daemon start. If a VPN client rewrites /etc/resolv.conf after the daemon is already running, Docker's embedded resolver continues forwarding to the pre-VPN resolvers. When those addresses become unreachable -- because the VPN tunnel is down -- every query times out. The fix here is either the networkd-dispatcher automation described in the Fixes section, which automatically updates daemon.json and restarts Docker on network changes, or manually restarting Docker after each VPN connect or disconnect event.
• VPN is connected but its DNS server is not routable from Docker's bridge namespace. Some VPN clients push an internal DNS server address that is only reachable via the VPN tunnel's virtual interface. Docker's bridge network does not route through that interface -- it routes through the host's default gateway. Queries reach the tunnel adapter from the host but are sent from a bridge namespace that has no route via the tunnel. The fix is to add the VPN DNS server as an explicit entry in daemon.json but also ensure the routing rules cover Docker's bridge subnet. Alternatively, use a public resolver as the fallback entry so containers still resolve external names when the VPN DNS is unreachable.
• Split-tunnel VPN where internal names must go to the VPN resolver and external names go elsewhere. This is the most complex case. Docker's daemon.json does not support split DNS -- you cannot configure it to send .corp.internal to one resolver and everything else to another. The architecturally correct solution for this case is to make systemd-resolved listen on the Docker bridge address (as described in the systemd-resolved fix section) and configure systemd-resolved's per-domain routing to send internal names to the VPN resolver and external names to a public resolver. Docker then points at the bridge address as its single upstream and inherits the full split-DNS routing intelligence from systemd-resolved.

Docker Swarm mode and overlay network DNS

Docker Swarm introduces a second layer of DNS on top of the per-host embedded resolver. Each Swarm overlay network gets its own virtual IP (VIP) resolver that handles service discovery across nodes. When a container in a Swarm service queries another service name, the request goes to the embedded resolver at 127.0.0.11, which recognizes Swarm service names and returns the VIP. External queries are then forwarded upstream as on any other user-defined network. The failed to query external DNS server error can appear in Swarm with two distinct causes that are worth separating.

The first is the same systemd-resolved stub leak described above -- the embedded resolver on each Swarm node inherits the host's upstream resolvers, and if those resolvers are loopback addresses, external queries fail on every node. The fix is the same: set explicit upstream resolvers in daemon.json on each node in the Swarm. There is no central place to configure DNS for an entire Swarm cluster; each node must be configured individually.

The second is a documented interaction between Swarm's overlay network iptables rules and port publishing. When a service has a published port (via --publish in docker service create or ports: in a Compose stack deployed as a Swarm stack), Swarm adds ingress routing rules to handle load balancing across the routing mesh. On some Docker versions and kernel configurations, these ingress iptables rules conflict with the MASQUERADE rules Docker needs for outbound traffic from those same containers, producing the write: operation not permitted variant of this error. The affected service can resolve internal service names via VIP but cannot reach external DNS servers at all.

To confirm which scenario you are in, test DNS from inside a Swarm task container rather than from a standalone container:

# Find the task container ID for a running Swarm service
$ docker ps --filter "name=myservice" --format "{{.ID}}"

# Exec into it and test both internal and external resolution
$ docker exec -it <task-container-id> nslookup myservice
$ docker exec -it <task-container-id> nslookup sudowheel.com

# If external fails but internal succeeds, it is the Swarm iptables conflict
# Restart Docker on the affected node to rebuild the ingress chain
$ sudo systemctl restart docker

If restarting Docker resolves the external DNS failure for a Swarm service with published ports but the error returns after containers are rescheduled, the root issue is the interaction between the ingress network rules and the MASQUERADE chain. Upgrading Docker Engine to the latest patch release is the most reliable fix, as this interaction has been corrected in several successive Moby releases. If you cannot upgrade immediately, deploying the service without a published port temporarily can confirm whether the port mapping is triggering the conflict.

nftables migration conflicts on newer kernels

On Linux distributions that have migrated from iptables to nftables as the default packet filter framework -- including Debian 11+, Fedora 30+, and RHEL 8+ -- Docker's iptables rules and the system's nftables ruleset can coexist incorrectly. Docker uses iptables (specifically the iptables-legacy backend) to write its DNAT and MASQUERADE rules. If the system's firewall is managed through nftables directly, those iptables rules may not take effect in the right table priority, causing DNS traffic from containers to be silently dropped before it reaches Docker's embedded resolver or before forwarded queries can exit the host.

The diagnostic symptom is subtle: docker run --rm alpine nslookup sudowheel.com fails, yet sudo iptables -t nat -L POSTROUTING -n | grep MASQUERADE shows Docker's rule present. The rule is present -- but in the wrong iptables backend. Verify with sudo iptables-legacy -t nat -L POSTROUTING -n and compare the output. If Docker's MASQUERADE rule only appears in the legacy backend while your firewall is using the nft backend, you have a split-table conflict. The fix is to ensure Docker's iptables backend matches the system's active backend, or to disable Docker's built-in iptables management and write equivalent nftables rules manually.

Internal network containers leaking DNS via host loopback (CVE-2024-29018)

This root cause is distinct from the others because it combines the failed to query external DNS server error with a security implication. When a container is attached only to a Docker internal network (created with the --internal flag or the internal: true attribute in a Compose file), that network is supposed to be fully isolated from external traffic. Prior to Moby versions 26.0.0, 25.0.4, and 23.0.11, Docker's embedded DNS resolver bypassed that isolation entirely for DNS queries.

The mechanism: when the host's /etc/resolv.conf pointed to a loopback address such as 127.0.0.53, the resolver forwarded queries to that address via the host's loopback device -- completely outside the container's network namespace -- rather than routing the query through the container's isolated namespace. This meant a container on an internal network, which should have had no external connectivity, could reach external authoritative DNS servers. The Moby project security advisory noted that an attacker with a foothold in a compromised container could encode exfiltrated data in DNS subdomains and have them answered by attacker-controlled authoritative nameservers, even on supposedly isolated networks.

On patched Docker versions, containers on internal networks no longer forward DNS queries to external servers. If you upgraded Docker and noticed that containers on internal networks suddenly cannot resolve external names -- they should not have been able to before either, correctly speaking. Check your network design: anything that needs external DNS resolution should not be on an internal network.

Embedded DNS resolver returning SERVFAIL for HTTPS and SVCB record types

Docker's embedded resolver at 127.0.0.11 handles A and PTR record types reliably but does not understand newer record types including HTTPS (type 65) and SVCB (type 64), which are used by modern browsers and HTTP/3-capable clients. When a browser like Chrome sends a DNS query for a container-served hostname, it automatically issues both an A record query and an HTTPS record query in parallel. Docker's embedded resolver responds correctly to the A query but returns SERVFAIL for the HTTPS query. Some clients interpret that SERVFAIL as a complete DNS failure, refusing to load the page even though the IP address was returned successfully.

The symptom is specific: a service resolves correctly when queried with nslookup or curl from the command line but fails to load in a browser, and inspecting the browser's network tab shows a DNS error rather than a connection error. If this matches your situation, the fix is not in daemon.json -- it requires placing a full-featured resolver like CoreDNS in front of Docker's embedded resolver, configured to forward A queries to 127.0.0.11 and handle HTTPS and SVCB queries with appropriate responses rather than SERVFAIL.

The standard approach is to run CoreDNS as a container on the same Docker network, give it a static IP, and point containers at it via the dns key in your Compose file or in daemon.json. CoreDNS forwards container-name queries and external A queries to 127.0.0.11 as normal, but responds to HTTPS and SVCB queries with NOERROR and an empty answer rather than SERVFAIL, which satisfies browsers without disrupting resolution. A minimal Corefile to accomplish this looks like:

# Forward all queries to Docker's embedded resolver
# and suppress SERVFAIL for unsupported record types
.:53 {
    errors
    health
    ready
    # Return NOERROR with empty answer for HTTPS and SVCB instead of SERVFAIL
    template ANY HTTPS {
        rcode NOERROR
    }
    template ANY SVCB {
        rcode NOERROR
    }
    # Forward everything else to Docker's embedded resolver
    forward . 127.0.0.11
    cache 30
    loop
    reload
    loadbalance
}

Run CoreDNS as a container alongside your services. Give it a predictable IP on the Docker network and point your other containers at it:

services:
  coredns:
    image: coredns/coredns:latest
    volumes:
      - ./Corefile:/Corefile:ro
    networks:
      appnet:
        ipv4_address: 172.28.0.2

  myapp:
    image: myapp:latest
    dns:
      - 172.28.0.2    # CoreDNS container IP
    networks:
      - appnet

networks:
  appnet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16

The template plugin intercepts HTTPS (type 65) and SVCB (type 64) queries before they reach the forward directive, returning an empty NOERROR response. Browsers treat an empty NOERROR as "this record type does not exist" rather than a DNS failure, which is the correct behavior. The A record lookup proceeds normally through the forward path and the browser connects successfully. The cache 30 directive adds a 30-second TTL cache, which reduces the number of upstream queries to Docker's embedded resolver and can measurably improve container startup performance when many containers resolve the same names at once.

Docker Desktop on macOS and Windows

Docker Desktop runs containers inside a Linux virtual machine managed by Docker itself. The entire iptables and systemd-resolved layer described in this article lives inside that VM, not on your Mac or Windows host. This changes the failure modes significantly.

On macOS, the most common cause of this error is DNS filtering software -- little Snitch, Pi-hole configured as a system resolver, corporate MDM profiles that intercept DNS -- intercepting or blocking the queries the VM sends out through the host's network stack. The VM cannot reach the upstream resolver because the host-side filter is dropping or redirecting the packet. The fix is to explicitly allow DNS egress for the Docker Desktop process in your filter, or to set a public resolver in Docker Desktop's settings under Resources > Network rather than in a daemon.json on disk.

On Windows with WSL 2, Docker Desktop uses a Hyper-V utility VM and DNS is proxied through the Windows host's resolver. If the Windows host uses a VPN that changes DNS settings, containers may lose resolution when the VPN connects or disconnects. The fix mirrors the Linux approach -- set explicit upstream resolvers -- but the daemon.json location on Windows is %USERPROFILE%\.docker\daemon.json and changes take effect after restarting Docker Desktop from the system tray, not via systemctl.

docker build fails to resolve names while running containers work fine

This is a frequently overlooked gap that trips up engineers who have already fixed DNS for their running Compose services. When docker build executes a layer that installs packages -- apt-get update, pip install, npm install -- and DNS resolution fails only during the build, not during docker run, the cause is almost always the network difference between build and runtime.

By default, docker build runs its build container on the default bridge network (docker0), not on a user-defined network. The default bridge does not use Docker's embedded DNS resolver at 127.0.0.11. Instead, it injects a filtered copy of the host's /etc/resolv.conf directly into the build container, stripping loopback addresses. If the host's resolv.conf contains only 127.0.0.53 and nothing else, the filtered file is empty and the build container has no nameserver to query.

Meanwhile, your running Compose services work because Compose always creates a user-defined bridge, which activates the embedded resolver. That resolver forwards to whatever you set in daemon.json. The fix is the same: set a dns array in /etc/docker/daemon.json. That setting applies to both the embedded resolver on user-defined networks and to the resolv.conf that gets injected into the default bridge network. If you set "dns": ["8.8.8.8", "1.1.1.1"], the build container will receive those addresses rather than a filtered and possibly empty resolv.conf.

Alternatively, pass --network=host to a specific build if you are on a trusted development machine and want the build to use the host's full resolver stack directly. Do not use --network=host in production CI environments or with untrusted build inputs, as it removes network isolation from the build process entirely.

This behavior extends to multi-stage builds. Each FROM stage in a multi-stage Dockerfile runs as a separate build container, all on the default bridge, all subject to the same DNS rules. If stage 1 downloads a base image layer and stage 2 runs npm install, both stages need working DNS. The daemon.json fix covers all stages uniformly -- once set, every stage in every build on that host picks up the configured upstream resolvers. There is no per-stage DNS override syntax in a Dockerfile; the fix must be at the daemon level or via a --network flag passed to the entire docker build command, not per stage.

CI/CD environments: ephemeral runners and absent systemd

CI/CD runners -- GitHub Actions hosted runners, GitLab CI shared runners, CircleCI machine executors -- introduce a distinct set of DNS failure conditions that differ from both desktop and server deployments. The failure mode depends on the runner type.

Hosted runners (GitHub Actions ubuntu-latest, GitLab shared runners). These are fresh Linux VMs provisioned per job. They have Docker pre-installed but systemd-resolved is either not running or not the primary resolver. The /etc/resolv.conf file on these VMs typically points to a hypervisor-level stub resolver at a cloud-specific address rather than at 127.0.0.53. On most GitHub Actions runners the resolver is at 168.63.129.16 (Azure) or a similar platform address. Docker inherits this address and containers may fail to reach it for the same reason cloud VMs fail -- the hypervisor route is not accessible from inside a bridge namespace. The recommended fix for CI pipelines is to add an explicit daemon.json configuration step at the start of the job, before any Docker commands run:

steps:
  - name: Configure Docker DNS
    run: |
      # Write daemon.json before any Docker commands
      sudo mkdir -p /etc/docker
      echo '{"dns":["8.8.8.8","1.1.1.1"],"dns-opts":["ndots:1","timeout:2","attempts:2"]}' \
        | sudo tee /etc/docker/daemon.json
      sudo systemctl restart docker

  - name: Build image
    run: docker build -t myimage .

Self-hosted runners on VMs or bare metal. Self-hosted runners persist between jobs and have the same DNS configuration issues as any server Docker deployment. The systemd-resolved stub leak and VPN conflicts described in this article apply in full. Because runners are long-lived, a DNS misconfiguration can affect every job that runs on that machine until the runner is reconfigured. Add the same daemon.json DNS settings and restart Docker as a one-time setup step when provisioning the runner, not as a per-job step.

Runners inside Docker containers (Docker-in-Docker). When the CI job itself runs inside a Docker container and invokes Docker commands via the Docker socket or via DinD, there are two nested DNS layers: the outer container's resolver and the inner daemon's resolver. The outer container gets its DNS from the CI platform's host configuration. The inner Docker daemon reads its own /etc/resolv.conf -- which inside a container is what Docker injected there. If the outer container's resolver is 127.0.0.11 (user-defined network) and that address is what gets inherited by the inner daemon, the inner daemon tries to forward queries to the outer container's embedded resolver. Whether that works depends on the network topology of the DinD setup. The reliable fix is to pass explicit DNS servers to the inner daemon via its own daemon.json, the same as any other deployment.