Boot taking a long time

[Paul Groves]

Hi everyone.

I am having some trouble with Ubuntu Server 18.04 networking.

The system has two onboard NICs used in a network bond (bond0) in
balance-rr mode. The network bond is working fine with both or either
NIC connected.

When I boot the system from off or reboot I am presented with:

A start job is running for Wait for Network to be Configured

This is displayed on the screen for approximately 3 minutes at which
point the system boots normally. How can I prevent this timer coming up?

I have this problem on two servers both running ubuntu 18.04.1 and with
the exact same configuration.

Here is my netplan configutration which was automatically generated at
setup.

Am I doing something wrong?

Paul

***@web:~$ cat /etc/netplan/50-cloud-init.yaml
network:
bonds:
bond0:
addresses:
- 192.168.0.9/24
dhcp4: false
dhcp6: false
gateway4: 192.168.0.1
interfaces:
- eno1
- enp2s0
nameservers:
addresses:
- 192.168.0.5
search:
- groves
parameters:
mode: balance-rr
ethernets:
eno1:
addresses: []
dhcp4: false
dhcp6: false
enp2s0:
addresses: []
dhcp4: false
dhcp6: false
version: 2

[Tom H]

Post by Paul Groves
I am having some trouble with Ubuntu Server 18.04 networking.
The system has two onboard NICs used in a network bond (bond0) in
balance-rr mode. The network bond is working fine with both or either
NIC connected.
A start job is running for Wait for Network to be Configured
This is displayed on the screen for approximately 3 minutes at which
point the system boots normally. How can I prevent this timer coming up?
I have this problem on two servers both running ubuntu 18.04.1 and with
the exact same configuration.

What's the unit that's blocking?

Check with "systemd-analyze blame" and "systemd-analyze critical-chain
[<unit>]" where/why the delay is.

You might also want to add "systemd.log_level=debug" to your kernel
cmdline in irder to get more logs.

[Paul Groves]

Post by Tom H
What's the unit that's blocking?
Check with "systemd-analyze blame" and "systemd-analyze critical-chain
[<unit>]" where/why the delay is.
You might also want to add "systemd.log_level=debug" to your kernel
cmdline in irder to get more logs.

The last boot for some reason was quicker at around 1 and a half mins.
It can be anywhere from about this to 3 and a half mins.

See below the output of those two commands. It seems that
systemd-networkd-wait-online.service is the problem.

I will have to arrange downtime to reboot the system and change the
kernel parameters. Although the problem is easily replicable. All that
needs to be done is install Ubuntu 18.04.1 Server from the normal ISO
and configure a network bond. Then this happens.

It doesn't happen when you just have a normally configured interface
though, so I am wondering if it is where the interfaces eno1 and enp2s0
are disabled in netplan due to being slaves to bond0 the system is
waiting for them to come up, but of course they never do.

Is there a way in netplan of saying that an interface is optional?

Paul


***@web:~$ systemd-analyze blame
1min 4.712s systemd-networkd-wait-online.service

***@web:~$ systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@"
character.
The time the unit takes to start is printed after the "+" character.

graphical.target @1min 59.079s
└─multi-user.target @1min 59.079s
└─apache2.service @1min 35.369s +23.709s
└─basic.target @1min 34.917s
└─sockets.target @1min 34.915s
└─lxd.socket @1min 34.890s +23ms
└─sysinit.target @1min 34.735s
└─cloud-init.service @1min 33.499s +1.182s
└─systemd-networkd-wait-online.service @28.784s +1min 4.712s
└─systemd-networkd.service @28.703s +74ms
└─network-pre.target @28.701s
└─cloud-init-local.service @5.109s +23.590s
└─systemd-remount-fs.service @4.509s +594ms
└─systemd-journald.socket @4.374s
└─system.slice @4.362s
└─-.slice @4.349s

[Tom H]

Post by Paul Groves

Post by Tom H
What's the unit that's blocking?
Check with "systemd-analyze blame" and "systemd-analyze critical-chain
[<unit>]" where/why the delay is.
You might also want to add "systemd.log_level=debug" to your kernel
cmdline in irder to get more logs.

The last boot for some reason was quicker at around 1 and a half mins.
It can be anywhere from about this to 3 and a half mins.
See below the output of those two commands. It seems that
systemd-networkd-wait-online.service is the problem.

Indeed.

Post by Paul Groves
I will have to arrange downtime to reboot the system and change the
kernel parameters. Although the problem is easily replicable. All that
needs to be done is install Ubuntu 18.04.1 Server from the normal ISO
and configure a network bond. Then this happens.
It doesn't happen when you just have a normally configured interface
though, so I am wondering if it is where the interfaces eno1 and enp2s0
are disabled in netplan due to being slaves to bond0 the system is
waiting for them to come up, but of course they never do.
Is there a way in netplan of saying that an interface is optional?
1min 4.712s systemd-networkd-wait-online.service
character.
The time the unit takes to start is printed after the "+" character.

I set up a similar netplan config.


/etc/netplan # vi ubu.yaml


/etc/netplan # cat ubu.yaml
network:
bonds:
bond0:
interfaces:
- dummy0
- dummy1
addresses:
- 192.168.0.9/24
gateway4: 192.168.0.1
dhcp4: no
dhcp6: no
parameters:
mode: active-rr
ethernets:
dummy0:
dhcp4: no
dummy1:
dhcp4: no


/etc/netplan # netplan generate


/etc/netplan # cd /run/systemd/network


/run/systemd/network # ls
10-netplan-bond0.netdev 10-netplan-bond0.network
10-netplan-dummy0.network 10-netplan-dummy1.network


/run/systemd/network # cat *
[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=active-rr
[Match]
Name=bond0

[Network]
LinkLocalAddressing=ipv6
Address=192.168.0.9/24
Address=192.168.0.9/24
Gateway=192.168.0.1
ConfigureWithoutCarrier=yes
[Match]
Name=dummy0

[Network]
LinkLocalAddressing=no
Bond=bond0
[Match]
Name=dummy1

[Network]
LinkLocalAddressing=no
Bond=bond0


/run/systemd/network #


bond0.network somehow ends up with two "Address=" address lines. I
haven't seen this before but this might be an 18.10 or 19.04 problem
rather than an 18.04 one. But I don't see why that would slow down the
boot process.

You could try copying the .network and .netdev files to
"/etc/systemd/network/" (and edit them, perhaps by removing the
LinkLocalAddressing and ConfigureWithoutCarrier to simplify
everything; even though they don't look like they'd slow anything
down) and move the yaml file out of "/etc/netplan/" to see what
happens.

Anything in the output of "journalctl"? Or 'systemctl status
systemd-networkd-wait-online.service"?

[Paul Groves]

It turns out my suspicion was correct.

Here is my netplan configuration as generated by the Ubuntu Server Setup
wizard on the install disc:

network:
bonds:
bond0:
addresses:
- 192.168.0.9/24
dhcp4: false
dhcp6: false
gateway4: 192.168.0.1
interfaces:
- eno1
- enp2s0
nameservers:
addresses:
- 192.168.0.5
search:
- groves
parameters:
mode: balance-rr
ethernets:
eno1:
addresses: []
dhcp4: false
dhcp6: false
enp2s0:
addresses: []
dhcp4: false
dhcp6: false


The problem is that the system was waiting for the network interfaces
eno1 and enps2s0 to come up and causing the delay at boot. Of course,
the interfaces configured this way will never come 'up'.

As suggested on another mailing list I modified the configuration to the
following.

ethernets:
eno1:
addresses: []
dhcp4: false
dhcp6: false
optional: true
enp2s0:
addresses: []
dhcp4: false
dhcp6: false
optional: true


This has now resolved the problem. Because the interfaces are now set as
optional they are not required to be up in order to boot the system.

Then once the system is booted and the network bonding module is loaded
bond0 comes up.

Should this not be set automatically?