CentOS: rpmdb: PANIC: fatal region error detected; run recovery error and solution

by on March 9, 2014 · 5 comments· LAST UPDATED March 9, 2014

in , , ,

I am using a CentOS Linux v6.5 on my home server and I am getting the following error when I execute yum command:

rpmdb: PANIC: fatal region error detected; run recovery
error: db3 error(-30974) from dbenv->open: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db3 - (-30974)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed

How do I fix this problem on a CentOS / RHEL based system?

Tutorial details
DifficultyEasy (rss)
Root privilegesYes
RequirementsNone
Estimated completion time5m
This is quite the messy situation. You may fix this by cleaning out rpm database. To minimize risk, make a backup of files in /var/lib/rpm/ using cp command:

mkdir /root/backups.rpm.mm_dd_yyyy/
cp -avr /var/lib/rpm/ /root/backups.rpm.mm_dd_yyyy/

To find list of __db* file, enter:
# ls -l /var/lib/rpm/_*
Sample outputs:

-rw-r--r-- 1 root root   24576 Jan 28 04:00 /var/lib/rpm/__db.001
-rw-r--r-- 1 root root  229376 Jan 28 04:00 /var/lib/rpm/__db.002
-rw-r--r-- 1 root root 1318912 Jan 28 04:00 /var/lib/rpm/__db.003
-rw-r--r-- 1 root root  753664 Jan 28 04:00 /var/lib/rpm/__db.004

To fix this problem, try:
# rm -f /var/lib/rpm/__db*
# db_verify /var/lib/rpm/Packages
# rpm --rebuilddb
# yum clean all

Verify that error has gone with the following yum command
# yum update
Sample outputs:

Loaded plugins: downloadonly, fastestmirror, security
Determining fastest mirrors
 * base: centos.mirrors.tds.net
 * extras: centos.corenetworks.net
 * updates: mirror.fdcservers.net
base                                                     | 3.7 kB     00:00
base/primary_db                                          | 4.4 MB     00:00
extras                                                   | 3.4 kB     00:00
extras/primary_db                                        |  19 kB     00:00
updates                                                  | 3.4 kB     00:00
updates/primary_db                                       | 2.0 MB     00:00
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-431.5.1.el6 will be installed
---> Package kernel-firmware.noarch 0:2.6.32-431.3.1.el6 will be updated
---> Package kernel-firmware.noarch 0:2.6.32-431.5.1.el6 will be an update
---> Package mysql-libs.x86_64 0:5.1.71-1.el6 will be updated
---> Package mysql-libs.x86_64 0:5.1.73-3.el6_5 will be an update
---> Package nss.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package nss-sysinit.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss-sysinit.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package nss-tools.x86_64 0:3.15.3-3.el6_5 will be updated
---> Package nss-tools.x86_64 0:3.15.3-6.el6_5 will be an update
---> Package openldap.x86_64 0:2.4.23-32.el6_4.1 will be updated
---> Package openldap.x86_64 0:2.4.23-34.el6_5.1 will be an update
---> Package p11-kit.x86_64 0:0.18.5-2.el6 will be updated
---> Package p11-kit.x86_64 0:0.18.5-2.el6_5.2 will be an update
---> Package p11-kit-trust.x86_64 0:0.18.5-2.el6 will be updated
---> Package p11-kit-trust.x86_64 0:0.18.5-2.el6_5.2 will be an update
---> Package tzdata.noarch 0:2013i-1.el6 will be updated
---> Package tzdata.noarch 0:2013i-2.el6 will be an update
---> Package wget.x86_64 0:1.12-1.8.el6 will be updated
---> Package wget.x86_64 0:1.12-1.11.el6_5 will be an update
---> Package yum-plugin-downloadonly.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-downloadonly.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-plugin-fastestmirror.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-fastestmirror.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-plugin-security.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-plugin-security.noarch 0:1.1.30-17.el6_5 will be an update
---> Package yum-utils.noarch 0:1.1.30-14.el6 will be updated
---> Package yum-utils.noarch 0:1.1.30-17.el6_5 will be an update
--> Finished Dependency Resolution
--> Running transaction check
---> Package kernel.x86_64 0:2.6.32-358.18.1.el6 will be erased
--> Finished Dependency Resolution
 
Dependencies Resolved
 
================================================================================
 Package                     Arch      Version                Repository   Size
================================================================================
Installing:
 kernel                      x86_64    2.6.32-431.5.1.el6     updates      28 M
Updating:
 kernel-firmware             noarch    2.6.32-431.5.1.el6     updates      13 M
 mysql-libs                  x86_64    5.1.73-3.el6_5         updates     1.2 M
 nss                         x86_64    3.15.3-6.el6_5         updates     822 k
 nss-sysinit                 x86_64    3.15.3-6.el6_5         updates      40 k
 nss-tools                   x86_64    3.15.3-6.el6_5         updates     358 k
 openldap                    x86_64    2.4.23-34.el6_5.1      updates     265 k
 p11-kit                     x86_64    0.18.5-2.el6_5.2       updates      94 k
 p11-kit-trust               x86_64    0.18.5-2.el6_5.2       updates      71 k
 tzdata                      noarch    2013i-2.el6            updates     448 k
 wget                        x86_64    1.12-1.11.el6_5        updates     483 k
 yum-plugin-downloadonly     noarch    1.1.30-17.el6_5        updates      21 k
 yum-plugin-fastestmirror    noarch    1.1.30-17.el6_5        updates      28 k
 yum-plugin-security         noarch    1.1.30-17.el6_5        updates      38 k
 yum-utils                   noarch    1.1.30-17.el6_5        updates     102 k
Removing:
 kernel                      x86_64    2.6.32-358.18.1.el6    @updates    116 M
 
Transaction Summary
================================================================================
Install       1 Package(s)
Upgrade      14 Package(s)
Remove        1 Package(s)
 
Total download size: 45 M
Is this ok [y/N]: y
Downloading Packages:
(1/15): kernel-2.6.32-431.5.1.el6.x86_64.rpm             |  28 MB     00:01
(2/15): kernel-firmware-2.6.32-431.5.1.el6.noarch.rpm    |  13 MB     00:00
(3/15): mysql-libs-5.1.73-3.el6_5.x86_64.rpm             | 1.2 MB     00:00
(4/15): nss-3.15.3-6.el6_5.x86_64.rpm                    | 822 kB     00:00
(5/15): nss-sysinit-3.15.3-6.el6_5.x86_64.rpm            |  40 kB     00:00
(6/15): nss-tools-3.15.3-6.el6_5.x86_64.rpm              | 358 kB     00:00
(7/15): openldap-2.4.23-34.el6_5.1.x86_64.rpm            | 265 kB     00:00
(8/15): p11-kit-0.18.5-2.el6_5.2.x86_64.rpm              |  94 kB     00:00
(9/15): p11-kit-trust-0.18.5-2.el6_5.2.x86_64.rpm        |  71 kB     00:00
(10/15): tzdata-2013i-2.el6.noarch.rpm                   | 448 kB     00:00
(11/15): wget-1.12-1.11.el6_5.x86_64.rpm                 | 483 kB     00:00
(12/15): yum-plugin-downloadonly-1.1.30-17.el6_5.noarch. |  21 kB     00:00
(13/15): yum-plugin-fastestmirror-1.1.30-17.el6_5.noarch |  28 kB     00:00
(14/15): yum-plugin-security-1.1.30-17.el6_5.noarch.rpm  |  38 kB     00:00
(15/15): yum-utils-1.1.30-17.el6_5.noarch.rpm            | 102 kB     00:00
--------------------------------------------------------------------------------
Total                                            18 MB/s |  45 MB     00:02
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
** Found 1 pre-existing rpmdb problem(s), 'yum check' output follows:
2:irqbalance-1.0.4-8.el6_5.x86_64 is a duplicate with 2:irqbalance-1.0.4-6.el6.x86_64
  Updating   : nss-sysinit-3.15.3-6.el6_5.x86_64                           1/30
  Updating   : nss-3.15.3-6.el6_5.x86_64                                   2/30
  Updating   : nss-tools-3.15.3-6.el6_5.x86_64                             3/30
  Updating   : p11-kit-0.18.5-2.el6_5.2.x86_64                             4/30
  Updating   : kernel-firmware-2.6.32-431.5.1.el6.noarch                   5/30
  Installing : kernel-2.6.32-431.5.1.el6.x86_64                            6/30
  Updating   : p11-kit-trust-0.18.5-2.el6_5.2.x86_64                       7/30
  Updating   : openldap-2.4.23-34.el6_5.1.x86_64                           8/30
  Updating   : wget-1.12-1.11.el6_5.x86_64                                 9/30
  Updating   : yum-plugin-fastestmirror-1.1.30-17.el6_5.noarch            10/30
  Updating   : mysql-libs-5.1.73-3.el6_5.x86_64                           11/30
  Updating   : yum-plugin-downloadonly-1.1.30-17.el6_5.noarch             12/30
  Updating   : yum-utils-1.1.30-17.el6_5.noarch                           13/30
  Updating   : yum-plugin-security-1.1.30-17.el6_5.noarch                 14/30
  Updating   : tzdata-2013i-2.el6.noarch                                  15/30
  Cleanup    : kernel-2.6.32-358.18.1.el6.x86_64                          16/30
  Cleanup    : kernel-firmware-2.6.32-431.3.1.el6.noarch                  17/30
  Cleanup    : yum-plugin-fastestmirror-1.1.30-14.el6.noarch              18/30
  Cleanup    : yum-plugin-downloadonly-1.1.30-14.el6.noarch               19/30
  Cleanup    : yum-utils-1.1.30-14.el6.noarch                             20/30
  Cleanup    : yum-plugin-security-1.1.30-14.el6.noarch                   21/30
  Cleanup    : tzdata-2013i-1.el6.noarch                                  22/30
  Cleanup    : openldap-2.4.23-32.el6_4.1.x86_64                          23/30
  Cleanup    : nss-tools-3.15.3-3.el6_5.x86_64                            24/30
  Cleanup    : nss-sysinit-3.15.3-3.el6_5.x86_64                          25/30
  Cleanup    : nss-3.15.3-3.el6_5.x86_64                                  26/30
  Cleanup    : p11-kit-trust-0.18.5-2.el6.x86_64                          27/30
  Cleanup    : p11-kit-0.18.5-2.el6.x86_64                                28/30
  Cleanup    : wget-1.12-1.8.el6.x86_64                                   29/30
  Cleanup    : mysql-libs-5.1.71-1.el6.x86_64                             30/30
 
 
Removed:
  kernel.x86_64 0:2.6.32-358.18.1.el6
 
Installed:
  kernel.x86_64 0:2.6.32-431.5.1.el6
 
Updated:
  kernel-firmware.noarch 0:2.6.32-431.5.1.el6
  mysql-libs.x86_64 0:5.1.73-3.el6_5
  nss.x86_64 0:3.15.3-6.el6_5
  nss-sysinit.x86_64 0:3.15.3-6.el6_5
  nss-tools.x86_64 0:3.15.3-6.el6_5
  openldap.x86_64 0:2.4.23-34.el6_5.1
  p11-kit.x86_64 0:0.18.5-2.el6_5.2
  p11-kit-trust.x86_64 0:0.18.5-2.el6_5.2
  tzdata.noarch 0:2013i-2.el6
  wget.x86_64 0:1.12-1.11.el6_5
  yum-plugin-downloadonly.noarch 0:1.1.30-17.el6_5
  yum-plugin-fastestmirror.noarch 0:1.1.30-17.el6_5
  yum-plugin-security.noarch 0:1.1.30-17.el6_5
  yum-utils.noarch 0:1.1.30-17.el6_5
 
Complete!
 
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 5 comments… read them below or add one }

1 danfalos May 28, 2014 at 2:05 pm

Thanks alot for this information.Thanks again.

Reply

2 Cihan June 29, 2014 at 2:28 am

That was very helpful and made me solve the problem in minutes. Thanks!

Reply

3 Dave July 28, 2014 at 10:44 am

Well written article and very helpful. Thank you – Resolved my problem.

Reply

4 Jackson August 24, 2014 at 10:24 pm

Thanks for this. I had a power outage during an update that caused some corruption. Solved my issue.

Reply

5 Raz October 10, 2014 at 12:04 am

Very helpful.
Thanks a lot

Reply

Leave a Comment

Tagged as:

Previous Faq:

Next Faq: