For some weird reason, I can not get my OpenVPN server to come up at boot time using systemd on an Ubuntu Linux 16.04 LTS server. I have tried a few settings but failed so far.
So I twitted about it:
Dear Systemd,
AT LEAST TELL ME WHAT THE FUCKING ERROR IS!!!#linux #sysadmin #systemd #ubuntu #centos #debian #openvpn
— nixCraft (@nixcraft) August 5, 2016
Here is my /lib/systemd/system/openvpn.service file:
# cat /lib/systemd/system/openvpn.service
# This service is actually a systemd target, # but we are using a service since targets cannot be reloaded. [Unit] Description=OpenVPN service After=network.target [Service] Type=oneshot RemainAfterExit=yes ExecStart=/bin/true ExecReload=/bin/true WorkingDirectory=/etc/openvpn [Install] WantedBy=multi-user.target
I wanted to run services after the network is up so that OpenVPN can bind to specific IP and port. The After=network.target tells to wait until the nework is “up”.
My fix
I finally gave up and simply wrote cron job that will start after each system reboot:
@reboot sleep 100;/etc/init.d/openvpn restart;/etc/init.d/ssh restart;/etc/init.d/squid reload
The @reboot forces to run job once, at startup. In this case, restart the openvpn server, and bind sshd/squid to openvpn port:
$ ss -tulpn
OR
$ netstat -tulpn
Sample outputs:
tcp 0 0 10.8.0.1:80 0.0.0.0:* LISTEN 1549/lighttpd tcp 0 0 0.0.0.0:53 0.0.0.0:* LISTEN 1524/dnsmasq tcp 0 0 10.8.0.1:22 0.0.0.0:* LISTEN 1756/sshd tcp 0 0 10.8.0.1:3128 0.0.0.0:* LISTEN 1675/(squid-1) tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 1732/openvpn
There must be a way to run services (OpenVPN) after the network is up. However, I failed to figure it out. If anyone knows how to fix this problem without using cron, let me know in the comments section below.
🐧 10 comments so far... add one ↓
| Category | List of Unix and Linux commands |
|---|---|
| Disk space analyzers | df • ncdu • pydf |
| File Management | cat • tree |
| Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
| Network Utilities | NetHogs • dig • host • ip • nmap |
| OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
| Package Manager | apk • apt |
| Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
| Searching | grep • whereis • which |
| User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
| WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
Can’t you just look through the ‘start on boot’ process in the source code?
Same with me. Had to rc.local it. In my case it’s an openvz VPS. I’ve seen some people saying the same problem occurs with LXD.
No LXD or openvz here.
Vivek: Problem with openvpn and related services that bind to ip addresses are that it is started before network interface has assigned ip address.
So, fix is very simple.
You must:
1. Enable NetworkManager-wait-online.service, or systemd-networkd-wait-online.service (depend which type you use)
systemctl enable NetworkManager-wait-online.service
or
systemctl enable systemd-networkd-wait-online.service
2. Tell openvpn service to start after network interfaces has ip addresses (after start network-online.target, which is started after NetworkManager-wait-online.service or systemd-networkd-wait-online.service), so add/change these to lines to service file to [Unit] section:
[Unit]
After=network-online.target
Wants=network-online.target
3. Don’t update openvpn service file directly in /lib directory, but make a local copy to /etc:
sudo cp /lib/systemd/system/openvpn.service /etc/systemd/system/openvpn.service
sudo vim /etc/systemd/system/openvpn.service
4. Finally reload systemd daemon:
sudo systemctl daemon-reload
5. Check if openvpn.service symlink in directory /etc/systemd/system/multi-user.target.wants/ is pointed to /etc service file (copy), not to /lib, so:
openvpn.service -> /etc/systemd/system/openvpn.service
If not, simple disable and enable openvpn.service, so it point to right file:
sudo systemctl disable openvpn.service && sudo systemctl enable openvpn.service
5. Restart computer, and openvpn will start after network
this might actually work. I will try out soon. thanks!
I had this issue before, even after the solution above. Netnomada is right about the service trying to bind to an IP not yet available and his solution will fix the issue in most installations.
You’ll run into issues if you’ve IPv6 enabled, in that case, the solution won’t work because your system will run IPv6 DAD (duplicate address detection) and there’s an old bug where the IP won’t be available but the network-online.target would be fired.
I reported that issue here: https://github.com/systemd/systemd/issues/2037 and seems to be patched. However it might take some time to get to your system.
Tld;dr: apply the solution above and if you’ve IPv6, DAD might become an issue, disable it with net.ipv6.conf.eth0.accept_dad = 0 at /etc/sysctl.conf
Instead of copying the whole file, you could just add a “fragment” file:
mkdir /etc/systemd/system/openvpn.service.d
cat </etc/systemd/system/openvpn.service.d/wait-online.conf
[Unit]
After=network-online.target
Wants=network-online.target
<< EOF
Seems <<EOF > got eaten by wordpress.
This works for me. I think I must have done something crazy like reading the documentation:
-bash-4.3$ pwd
/etc/systemd/system/multi-user.target.wants
-bash-4.3$ ls -l openvpn@server.service
lrwxrwxrwx. 1 root root 40 Jan 1 2015 openvpn@server.service -> /usr/lib/systemd/system/openvpn@.service
Concept “Running Services After the Network is up” is detailed described in systemd website:
https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/#runningservicesafterthenetworkisup
Hope this helps