≡ Menu

CentOS: rpmdb: PANIC: fatal region error detected; run recovery error and solution

I am using a CentOS Linux v6.5 on my home server and I am getting the following error when I execute yum command:

rpmdb: PANIC: fatal region error detected; run recovery
error: db3 error(-30974) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db3 - (-30974)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed

How do I fix this problem on a CentOS / RHEL based system?

Tutorial details
DifficultyEasy (rss)
Root privilegesYes
RequirementsNone
Estimated completion time5m
This is quite the messy situation. You may fix this by cleaning out rpm database. To minimize risk, make a backup of files in /var/lib/rpm/ using cp command:

mkdir /root/backups.rpm.mm_dd_yyyy/
cp -avr /var/lib/rpm/ /root/backups.rpm.mm_dd_yyyy/

To find list of __db* file, enter:
# ls -l /var/lib/rpm/_*
Sample outputs:

-rw-r--r-- 1 root root   24576 Jan 28 04:00 /var/lib/rpm/__db.001
-rw-r--r-- 1 root root  229376 Jan 28 04:00 /var/lib/rpm/__db.002
-rw-r--r-- 1 root root 1318912 Jan 28 04:00 /var/lib/rpm/__db.003
-rw-r--r-- 1 root root  753664 Jan 28 04:00 /var/lib/rpm/__db.004

To fix this problem, try:
# rm -f /var/lib/rpm/__db*
# db_verify /var/lib/rpm/Packages
# rpm --rebuilddb
# yum clean all

Verify that error has gone with the following yum command
# yum update
Sample outputs:

Loaded plugins: downloadonly, fastestmirror, security
Determining fastest mirrors
 * base: centos.mirrors.tds.net
 * extras: centos.corenetworks.net
 * updates: mirror.fdcservers.net
base                                                     | 3.7 kB     00:00
base/primary_db                                          | 4.4 MB     00:00
extras                                                   | 3.4 kB     00:00
extras/primary_db                                        |  19 kB     00:00
updates                                                  | 3.4 kB     00:00
updates/primary_db                                       | 2.0 MB     00:00
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-431.5.1.el6 will be installed
---> Package kernel-firmware.noarch 0:2.6.32-431.3.1.el6 will be updated
---> Package kernel-firmware.noarch 0:2.6.32-431.5.1.el6 will be an update
---> Package mysql-libs.x86_64 0:5.1.71-1.el6 will be updated
---> Package mysql-libs.x86_64 0:5.1.73-3.el6_5 will be an update
---> Package nss.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package nss-sysinit.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss-sysinit.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package nss-tools.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss-tools.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package openldap.x86_64 0:2.4.23-32.el6_4.1 will be updated
---> Package openldap.x86_64 0:2.4.23-34.el6_5.1 will be an update
---> Package p11-kit.x86_64 0:0.18.5-2.el6 will be updated
---> Package p11-kit.x86_64 0:0.18.5-2.el6_5.2 will be an update
---> Package p11-kit-trust.x86_64 0:0.18.5-2.el6 will be updated
---> Package p11-kit-trust.x86_64 0:0.18.5-2.el6_5.2 will be an update
---> Package tzdata.noarch 0:2013i-1.el6 will be updated
---> Package tzdata.noarch 0:2013i-2.el6 will be an update
---> Package wget.x86_64 0:1.12-1.8.el6 will be updated
---> Package wget.x86_64 0:1.12-1.11.el6_5 will be an update
---> Package yum-plugin-downloadonly.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-downloadonly.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-plugin-fastestmirror.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-fastestmirror.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-plugin-security.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-security.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-utils.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-utils.noarch 0:1.1.30-17.el6_5 will be an update
--> Finished Dependency Resolution
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-358.18.1.el6 will be erased
--> Finished Dependency Resolution
 
Dependencies Resolved
 
================================================================================
 Package                     Arch      Version                Repository   Size
================================================================================
Installing:
 kernel                      x86_64    2.6.32-431.5.1.el6     updates      28 M
Updating:
 kernel-firmware             noarch    2.6.32-431.5.1.el6     updates      13 M
 mysql-libs                  x86_64    5.1.73-3.el6_5         updates     1.2 M
 nss                         x86_64    3.15.3-6.el6_5         updates     822 k
 nss-sysinit                 x86_64    3.15.3-6.el6_5         updates      40 k
 nss-tools                   x86_64    3.15.3-6.el6_5         updates     358 k
 openldap                    x86_64    2.4.23-34.el6_5.1      updates     265 k
 p11-kit                     x86_64    0.18.5-2.el6_5.2       updates      94 k
 p11-kit-trust               x86_64    0.18.5-2.el6_5.2       updates      71 k
 tzdata                      noarch    2013i-2.el6            updates     448 k
 wget                        x86_64    1.12-1.11.el6_5        updates     483 k
 yum-plugin-downloadonly     noarch    1.1.30-17.el6_5        updates      21 k
 yum-plugin-fastestmirror    noarch    1.1.30-17.el6_5        updates      28 k
 yum-plugin-security         noarch    1.1.30-17.el6_5        updates      38 k
 yum-utils                   noarch    1.1.30-17.el6_5        updates     102 k
Removing:
 kernel                      x86_64    2.6.32-358.18.1.el6    @updates    116 M
 
Transaction Summary
================================================================================
Install       1 Package(s)
Upgrade      14 Package(s)
Remove        1 Package(s)
 
Total download size: 45 M
Is this ok [y/N]: y
Downloading Packages:
(1/15): kernel-2.6.32-431.5.1.el6.x86_64.rpm             |  28 MB     00:01
(2/15): kernel-firmware-2.6.32-431.5.1.el6.noarch.rpm    |  13 MB     00:00
(3/15): mysql-libs-5.1.73-3.el6_5.x86_64.rpm             | 1.2 MB     00:00
(4/15): nss-3.15.3-6.el6_5.x86_64.rpm                    | 822 kB     00:00
(5/15): nss-sysinit-3.15.3-6.el6_5.x86_64.rpm            |  40 kB     00:00
(6/15): nss-tools-3.15.3-6.el6_5.x86_64.rpm              | 358 kB     00:00
(7/15): openldap-2.4.23-34.el6_5.1.x86_64.rpm            | 265 kB     00:00
(8/15): p11-kit-0.18.5-2.el6_5.2.x86_64.rpm              |  94 kB     00:00
(9/15): p11-kit-trust-0.18.5-2.el6_5.2.x86_64.rpm        |  71 kB     00:00
(10/15): tzdata-2013i-2.el6.noarch.rpm                   | 448 kB     00:00
(11/15): wget-1.12-1.11.el6_5.x86_64.rpm                 | 483 kB     00:00
(12/15): yum-plugin-downloadonly-1.1.30-17.el6_5.noarch. |  21 kB     00:00
(13/15): yum-plugin-fastestmirror-1.1.30-17.el6_5.noarch |  28 kB     00:00
(14/15): yum-plugin-security-1.1.30-17.el6_5.noarch.rpm  |  38 kB     00:00
(15/15): yum-utils-1.1.30-17.el6_5.noarch.rpm            | 102 kB     00:00
--------------------------------------------------------------------------------
Total                                            18 MB/s |  45 MB     00:02
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
2:irqbalance-1.0.4-8.el6_5.x86_64 is a duplicate with 2:irqbalance-1.0.4-6.el6.x86_64
  Updating   : nss-sysinit-3.15.3-6.el6_5.x86_64                           1/30
  Updating   : nss-3.15.3-6.el6_5.x86_64                                   2/30
  Updating   : nss-tools-3.15.3-6.el6_5.x86_64                             3/30
  Updating   : p11-kit-0.18.5-2.el6_5.2.x86_64                             4/30
  Updating   : kernel-firmware-2.6.32-431.5.1.el6.noarch                   5/30
  Installing : kernel-2.6.32-431.5.1.el6.x86_64                            6/30
  Updating   : p11-kit-trust-0.18.5-2.el6_5.2.x86_64                       7/30
  Updating   : openldap-2.4.23-34.el6_5.1.x86_64                           8/30
  Updating   : wget-1.12-1.11.el6_5.x86_64                                 9/30
  Updating   : yum-plugin-fastestmirror-1.1.30-17.el6_5.noarch            10/30
  Updating   : mysql-libs-5.1.73-3.el6_5.x86_64                           11/30
  Updating   : yum-plugin-downloadonly-1.1.30-17.el6_5.noarch             12/30
  Updating   : yum-utils-1.1.30-17.el6_5.noarch                           13/30
  Updating   : yum-plugin-security-1.1.30-17.el6_5.noarch                 14/30
  Updating   : tzdata-2013i-2.el6.noarch                                  15/30
  Cleanup    : kernel-2.6.32-358.18.1.el6.x86_64                          16/30
  Cleanup    : kernel-firmware-2.6.32-431.3.1.el6.noarch                  17/30
  Cleanup    : yum-plugin-fastestmirror-1.1.30-14.el6.noarch              18/30
  Cleanup    : yum-plugin-downloadonly-1.1.30-14.el6.noarch               19/30
  Cleanup    : yum-utils-1.1.30-14.el6.noarch                             20/30
  Cleanup    : yum-plugin-security-1.1.30-14.el6.noarch                   21/30
  Cleanup    : tzdata-2013i-1.el6.noarch                                  22/30
  Cleanup    : openldap-2.4.23-32.el6_4.1.x86_64                          23/30
  Cleanup    : nss-tools-3.15.3-3.el6_5.x86_64                            24/30
  Cleanup    : nss-sysinit-3.15.3-3.el6_5.x86_64                          25/30
  Cleanup    : nss-3.15.3-3.el6_5.x86_64                                  26/30
  Cleanup    : p11-kit-trust-0.18.5-2.el6.x86_64                          27/30
  Cleanup    : p11-kit-0.18.5-2.el6.x86_64                                28/30
  Cleanup    : wget-1.12-1.8.el6.x86_64                                   29/30
  Cleanup    : mysql-libs-5.1.71-1.el6.x86_64                             30/30
 
 
Removed:
  kernel.x86_64 0:2.6.32-358.18.1.el6
 
Installed:
  kernel.x86_64 0:2.6.32-431.5.1.el6
 
Updated:
  kernel-firmware.noarch 0:2.6.32-431.5.1.el6
  mysql-libs.x86_64 0:5.1.73-3.el6_5
  nss.x86_64 0:3.15.3-6.el6_5
  nss-sysinit.x86_64 0:3.15.3-6.el6_5
  nss-tools.x86_64 0:3.15.3-6.el6_5
  openldap.x86_64 0:2.4.23-34.el6_5.1
  p11-kit.x86_64 0:0.18.5-2.el6_5.2
  p11-kit-trust.x86_64 0:0.18.5-2.el6_5.2
  tzdata.noarch 0:2013i-2.el6
  wget.x86_64 0:1.12-1.11.el6_5
  yum-plugin-downloadonly.noarch 0:1.1.30-17.el6_5
  yum-plugin-fastestmirror.noarch 0:1.1.30-17.el6_5
  yum-plugin-security.noarch 0:1.1.30-17.el6_5
  yum-utils.noarch 0:1.1.30-17.el6_5
 
Complete!
 
Tweet itFacebook itGoogle+ itPDF itFound an error/typo on this page?

{ 10 comments… add one }

  • danfalos May 28, 2014, 2:05 pm

    Thanks alot for this information.Thanks again.

  • Cihan June 29, 2014, 2:28 am

    That was very helpful and made me solve the problem in minutes. Thanks!

  • Dave July 28, 2014, 10:44 am

    Well written article and very helpful. Thank you – Resolved my problem.

  • Jackson August 24, 2014, 10:24 pm

    Thanks for this. I had a power outage during an update that caused some corruption. Solved my issue.

  • Raz October 10, 2014, 12:04 am

    Very helpful.
    Thanks a lot

  • Vijay January 22, 2015, 10:09 am

    Awesome solution….i was stuck in this issue for past 3 days….the above steps resolved my issue….thanks man!!!

  • Robert Morley May 19, 2015, 6:21 am

    Thank you! As someone who only has linux installed for one specific purpose and has nearly no understanding of it beyond that, this gave me exactly what I needed.

  • Aleksey Tsalolikhin July 1, 2015, 3:08 pm

    Thank you for having consistently excellent and useful content!!

  • virtual July 13, 2015, 3:32 pm

    Never written comments. However, in earnest I have learned a lot Linux and UNIX from cyberciti.biz, specifically your NixCraft site. With guys like Vivek Gite you can’t go wrong.

    Great and trusted work buddies – is all I can say.

    Wow.

  • Turrican August 21, 2015, 6:18 am

    This solved my rpm database problem after a filesystem crash on a SLES machine, thank you very much!

    Under SLES, if you have zypper installed instead of yum, you need to run “zypper clean” instead of “yum clean all”.

    Thanks again!

Leave a Comment