Linux Hotplug a CPU and Disable CPU Cores At Run Time

Posted on in Categories , , , , , , , , last updated May 26, 2015

I would like to dynamically enable or disable a CPU on a running system. How do I hotplug a CPU on a running Linux system? How do I disable cpu cores on a Linux operating system at run time?

Linux kernel does supports cpu-hotplug mechanism. You can enable or disable CPU or CPU core without a system reboot. CPU hotplug is not just useful to replace defective components it can also be applied in other contexts to increase the productivity of a system. For example on a single system running multiple Linux partitions, as the workloads change it would be extremely useful to be able to move CPUs from one partition to the next as required without rebooting or interrupting the workloads. This is known as dynamic partitioning. Other applications include Instant Capacity on Demand where extra CPUs are present in a system but aren’t activated. This is useful for customers that predict growth and therefore the need for more computing power but do not have at the time of purchase the means to afford. Please note that not all server supports cpu hotplug but almost all server can support disabling or enabling cpu core on a Linux operating systems. There are couple OEMS that support NUMA hardware which are hot pluggable as well, where physical node insertion and removal require support for CPU hotplug. This tutorial will explain how to hotplug a cpu and disable/enable core on a Linux.

List all current cpus and cores in the system

Type the following command:
# cd /sys/devices/system/cpu
# ls -l

Sample output:

total 0
drwxr-xr-x 4 root root    0 Apr  2 12:03 cpu0
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu1
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu2
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu3
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu4
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu5
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu6
drwxr-xr-x 4 root root    0 Feb 15 07:06 cpu7
-rw-r--r-- 1 root root 4096 Apr  2 12:03 sched_mc_power_savings

I’ve total 8 core cpu logically started from cpu0 to cpu7. To get more human readable format, try:
# lscpu
Sample outputs:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 45
Stepping:              7
CPU MHz:               2000.209
BogoMIPS:              4001.65
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              20480K
NUMA node0 CPU(s):     0-7,16-23
NUMA node1 CPU(s):     8-15,24-31

Under each directory you would find an “online” file which is the control file to logically online/offline a processor.

How do I logically turn off (offline) cpu#6 ?

Warning: It is not possible to disable CPU0 on Linux systems i.e do not try to take cpu0 offline. Some architectures may have some special dependency on a certain CPU. For e.g in IA64 platforms we have ability to sent platform interrupts to the OS. a.k.a Corrected Platform Error Interrupts (CPEI). In current ACPI specifications, we didn’t have a way to change the target CPU. Hence if the current ACPI version doesn’t support such re-direction, we disable that CPU by making it not-removable. In such cases you will also notice that the online file is missing under cpu0.

Type the following command:
# echo 0 > /sys/devices/system/cpu/cpu6/online
# grep "processor" /proc/cpuinfo

How do I logically turn on (online) cpu#6 ?

Type the following command:
# echo 1 > /sys/devices/system/cpu/cpu6/online
# grep "processor" /proc/cpuinfo

Sample session:

Fig.01: Howto enable and disable CPU core in  on a multi-core server
Fig.01: Howto enable and disable CPU core in on a multi-core server

Once done, you can can actually remove CPU if your BIOS and server vendor supports such operation.

How do I verify cpu is online and offline?

Type the following cat command to see a list of cpus which are online:
# cat /sys/devices/system/cpu/online
To see a list of all offline cpus, run:
# cat /sys/devices/system/cpu/offline

Further readings:

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

Share this on (or read 24 comments/add one below):

24 comment

  1. I’m really curious about this. Can anyone comment if they’ve actually tried it? I’m not sure if any of the x86_(64) systems I’ve ever worked with support this capability – anyone know of systems that do?

  2. You need high end servers such as IBM Power6 series to actually add or remove CPU / memory without rebooting the server. This is called Capacity Upgrade on Demand (CUoD). Other vendor such as Sun and HP also provides the same stuff. Your average server only support hard disk hot-plug. Another example is HP ProLiant ML570 G2 business server powered by Intel Xeon CPUs. Ask, your vendor for CUoD aware server and you will get the list. Be prepared to pay extra for all these fancy features ;)

    HTH

  3. I tried to offline a core in my i386 (kernel 2.6), but I was notified a permission denied.

    [email protected]:~$ echo 1 > /sys/devices/system/cpu/cpu1/online
    bash: /sys/devices/system/cpu/cpu1/online: Permission denied
    [email protected]:~$ sudo echo 1 > /sys/devices/system/cpu/cpu1/online
    bash: /sys/devices/system/cpu/cpu1/online: Permission denied

    Can anybody tell me y? Is it by default enabled in the latest versions?
    Thx

    S.

  4. I have put cpu 1 to 3 offline.
    When I online a cpu I get the following message:
    (root @ rhel5)
    # echo 1 > /sys/devices/system/cpu/cpu1/online
    -bash: echo: write error: Invalid argument

    Anyone else got the same message and a solution to this problem?

  5. For the people confused by “echo: write error: Invalid argument”: Is the CPU already enabled (if trying to enable it) or disabled (if trying to disable it)? AFAICT, the kernel produces an error when you try to put the CPU in the state it is already in, instead of just ignoring the request.

    For the people getting permission errors:
    Use “echo 0 | sudo tee /sys/devices/system/cpu/cpu6/online” instead of “sudo echo 0 > /sys/devices/system/cpu/cpu6/online”.

    As Vivek points out, you need to be root to hotplug a CPU. You might think “But I used sudo”. However, when you use “>”, bash (or your shell, if you use something else) is responsible for opening the file that you’re trying to write to, and then setting that opened file as the destination for stdout (file descriptor 1). bash will then execute “sudo echo 0”, which will cause sudo to do its magic and run “echo 0” as root. Unfortunately, bash isn’t running as root, so it will fail to open /sys/devices/system/cpu/cpu6/online.

    By instead piping the output of the echo to sudo tee, tee becomes responsible for opening the file and writing to it. Since tee is run under sudo, it can actually succeed.

  6. I only have a file

    /sys/devices/system/cpu/online that contains the id of cores like that 0-15. How can I disable one of cores? How can I know online cores number by shell script commands?

    Minxia

  7. See this doc for more on this feature. https://www.kernel.org/doc/Documentation/cpu-hotplug.txt

    Q: Why can’t i remove CPU0 on some systems?
    A: Some architectures may have some special dependency on a certain CPU.

    For e.g in IA64 platforms we have ability to sent platform interrupts to the
    OS. a.k.a Corrected Platform Error Interrupts (CPEI). In current ACPI
    specifications, we didn’t have a way to change the target CPU. Hence if the
    current ACPI version doesn’t support such re-direction, we disable that CPU
    by making it not-removable.

    In such cases you will also notice that the online file is missing under cpu0.

  8. Christian: changing a state on a CPU core does not remove it from Linux. Thus, the device would still be polled and would require power. One way to disable a core would be outside of the OS. If supported, one would change the number of CPU cores within the bios. This of course would require an outage.

  9. If my server has 2 physical processor and I want to disable and remove only one processor which has 2 cores if I make offline the CPU cores as per above given commands to remove one physical processor but I dont want to get stop/kill/terminate any job which is handeling by that CPU which i want to remove.

    Is there any possibility to migrate all running process from one CPU/Cores to another CPU/Cores or stop handelling any new request to any specific CPU/cores.

  10. It is safe to make a cpu offline and bring it back online if the CPU is stuck with softlock error with CPU showing 100% utilization ? Will it affect the applications which were being served before the CPU went into a softlock ?

Leave a Comment