Ansible reboot a Debian/Ubuntu Linux for kernel update and wait for it

last updated in Categories ,

How can I reboot a Debian or Ubuntu Linux server/host remotely using an Ansible playbook for kernel update and wait for it to come back again?

Introduction: It is pretty standard to update a large number of cloud servers or bare metal server using Ansible IT automation or DevOps tool. When new kernel installed, you must reboot the Debian or Ubuntu Linux server. This page shows how to reboot the machine using shell or command module and wait for it to come back.

Ansible modules you need to use

  1. apt – Manages apt packages for Debian/Ubuntu Linux such as install a new package or update package.
  2. command or shell – Execute commands in nodes using shell module. Executes a command on a remote node using command module. Use any one of the module to reboot the box when kernel updated.
  3. wait_for_connection – Waits until remote system is reachable/usable.

Ansible reboot a Debian/Ubuntu Linux for kernel update and wait for it

Let us see how to use these tree Ansible modules to reboot a Debian/Ubuntu Linux kernel update and wait for it to come back online again.

Update your Debian or Ubuntu box in Ansible

The playbook should be as follows:

      - name: Update all packages
        apt:
            update_cache: yes
            upgrade: dist

Make Ansible to wait for a server to reboot and continue playbook to work with shell module

Update playbook as follows:

      - name: Reboot box if kernel/libs updated and requested by the system
        shell: sleep 10 && /sbin/shutdown -r now 'Rebooting box to update system libs/kernel as needed' 
        args:
            removes: /var/run/reboot-required
        async: 300
        poll: 0
        ignore_errors: true

If the file /var/run/reboot-required exists on a Debian or Ubuntu Linux (see How to find out if my Ubuntu/Debian Linux server needs a reboot), you need to reboot the Linux server using the shutdown/reboot command. Please notice that I added sleep command before shutdown -r now, then use async with shell module. It forces the playbook the shell module asynchronously. The removes is some sort of condition. If A filename called /var/run/reboot-required does not exist, the shutdown command will not be run. We only need to run the reboot command or shutdown command if kernel updated.

Reboot and wait for reboot to complete in Ansible

The final puzzle of waiting is as follows:

      - name: Wait for system to become reachable again
        wait_for_connection:
            delay: 60
            timeout: 300

Ansible now waits for a total of 300 seconds. I also added 60 number of seconds to wait before starting to poll.

Reboot a Host Remotely using an Ansible Playbook

Ansible reboot a Debian/Ubuntu Linux for kernel update and wait for it complete example:

# update.yml 
---
- hosts: cluster
  tasks:
      - name: Update all packages on a Debian/Ubuntu
        apt:
            update_cache: yes
            upgrade: dist
      
      - name: Reboot box if kernel/libs updated and requested by the system
        shell: sleep 10 && /sbin/shutdown -r now 'Rebooting box to update system libs/kernel as needed' 
        args:
            removes: /var/run/reboot-required
        async: 300
        poll: 0
        ignore_errors: true
      
      - name: Wait for system to become reachable again
        wait_for_connection:
            delay: 60
            timeout: 300

      - name: Verify new update (optional)
        command: uname -mrs
        register: uname_result
      - name: Display new kernel version
        debug:
            var: uname_result.stdout_lines

Here is my hosts file displayed using the cat command:

[all:vars]
ansible_user=vivek
ansible_port=22
ansible_python_interpreter='/usr/bin/env python3'
ansible_become=yes
ansible_become_method=sudo
 
[cluster]
mysql2 ansible_host=192.168.1.10
mysql1 ansible_host=192.168.1.11
mysql3 ansible_host=192.168.1.12

Now just it as follows:
$ ansible-playbook -i hosts update.yml
Ansible reboot a Debian/Ubuntu Linux for kernel update and wait for it

Conclusion

You just learned how to reboot a Debian or Ubuntu Linux server remotely using an Ansible playbook and wait for it to continue operation. For more info see Ansible docs here.

This entry is 1 of 2 in the Ansible Reboot a Machine/Server Tutorial series. Keep reading the rest of the series:
  1. Ansible reboot a Debian/Ubuntu Linux for kernel update and wait for it
  2. Ansible reboot Linux machine or server with playbooks

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Notable Replies

  1. I got my solution from the ansible issue tracker #14413 (sorry can only add two links…) and everybody there uses async: 1. As you set poll: 0 you do not really check the return status and so async: 300 becomes irrelevant. Just do not make the mistake I made and set async: 0 which is the same as leaving it out and results in a fatal error every time. :wink:

    You should also reference the issue “Proper reboot module as action plugin” and the related pull-request “[WIP] Add reboot action plugin”. I think a proper module for rebooting is important because there are so many caveats. Unfortunately the momentum for the PR seems to be lost.

    One more thing I did is, split up the check for reboot requirement and actual reboot. That way I can trigger the reboot from multiple sources and it is always done at the very end.

    tasks:

    - name: 'Reboot if necessary'
      stat:
        path: /var/run/reboot-required
      register: result
      changed_when: result.stat.exists
      notify: 'restart server'
    

    handlers:

    - name: 'Reboot server'
      shell: 'sleep 5 && shutdown -r now "Rebooting to complete system upgrade"'
      become: yes
      async: 1
      poll: 0
      listen: 'restart server'
    
    - name: 'Wait for host to become available again'
      wait_for_connection:
        delay: 30
        connect_timeout: 10
        sleep: 10
        timeout: 300
      listen: 'restart server'
  2. Thanks for posting your solution.

Continue the discussion www.nixcraft.com

Participants