Proxmox LXC: Why apt's network wait times out three times a day

If you run Debian containers on Proxmox, open one and look at its journal:

journalctl --since -2d | grep -i wait-online

There's a good chance you'll see this pair, two or three times a day:

systemd-networkd-wait-online[...]: Timeout occurred while waiting for network connectivity.
apt-helper[...]: E: Sub-process /lib/systemd/systemd-networkd-wait-online returned an error code (1)

Nothing is actually broken. The container's network is fine the whole time. But it's noise in every log review, and it makes apt's scheduled jobs sit for the full timeout before giving up on the "is the network up?" check.

What's going on

Two network stacks are installed in the container, and only one of them is doing anything.

1. ifupdown actually configures the network. The Debian container template writes eth0 into the classic /etc/network/interfaces, and that's what brings the interface up.

2. systemd-networkd is also running, but it manages nothing. Check it:

   $ networkctl list
   IDX LINK  TYPE     OPERATIONAL SETUP
     1 lo    loopback carrier     unmanaged
     2 eth0  ether    routable    unmanaged
   

   Every link says unmanaged.

3. apt asks the idle one. apt's daily timers run apt-helper wait-online. When systemd-networkd is active, that hands off to systemd-networkd-wait-online, which waits for networkd to report a managed link as online. networkd manages no links, so the answer never comes, and the wait times out every time.

The fix

Only do this if networkctl list shows every link as unmanaged. If networkd configures any interface on your container, this is not your problem, so stop here.

systemctl mask --now systemd-networkd.service systemd-networkd.socket systemd-networkd-wait-online.service

Then check that nothing you care about moved:

ip -4 addr show eth0          # same address as before
ip route show default         # same gateway as before
/usr/lib/apt/apt-helper wait-online; echo "exit $?"    # now returns immediately with exit 0

mask --now stops the units and stops anything from starting them again. Your interface keeps running, because ifupdown owns it. (On systemd 257 you may notice systemd-networkd-persistent-storage.service go inactive too. It's a helper that only exists for networkd, and it stops along with it.)

To undo it:

systemctl unmask systemd-networkd.service systemd-networkd.socket systemd-networkd-wait-online.service
systemctl start systemd-networkd

Doing it across many containers safely

If every container came from the same template, they probably all have this. I fixed them one at a time from the host with a guard: record the address and default route, mask, compare, ping, and roll back and stop at the first difference:

U="systemd-networkd.service systemd-networkd.socket systemd-networkd-wait-online.service"
for id in $(pct list | awk 'NR>1 && $2=="running"{print $1}'); do
  managed=$(pct exec $id -- sh -c 'networkctl list --no-legend | grep -vc unmanaged')
  [ "$managed" = "0" ] || { echo "$id: networkd manages links - skipped"; continue; }
  before=$(pct exec $id -- sh -c 'ip -4 -o addr show eth0; ip route show default')
  pct exec $id -- systemctl mask --now $U >/dev/null
  after=$(pct exec $id -- sh -c 'ip -4 -o addr show eth0; ip route show default')
  if [ "$before" != "$after" ]; then
    pct exec $id -- systemctl unmask $U; pct exec $id -- systemctl start systemd-networkd
    echo "$id: network changed - rolled back, stopping"; exit 1
  fi
  echo "$id: done"
done

Across eleven containers, none tripped the guard, and the timeouts stopped.

For new containers, add the same mask to your post-create checklist.

Scope

I verified this on Debian 13.7 containers (systemd 257) on Proxmox VE 9.2. Other template versions may differ, so run the networkctl check first. It's the whole safety test.