Hi
My firends website had some kinds of problems such as high load in average and so much after restarting apache
also apache got down after restarting it too.
The Yum Would show up an error like this
error: db3 error(28) from dbenv->open: No space left on device
I dig further into this and found out that the partition size is okay by issuing df -h, I researched and figured out that the partition / is out of INodes by issuing the following command

df -i

which would output the following

root@linux [/var/asl/data/audit]# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/vg_linux-lv_root
                     3276800  3276800  0    4% /

This was now brief for me that the INodes in this partition was 0 so that any new data such as logs and any thing else would fail in addition to the yum that would return an absolute error
How To Solve yum update error error: db3 error(28) from dbenv->open: No space left on device

Finging where is using too much INodes

So now is the time to finding out where is using too much Inodes
To find it out I used the following shell comination in the directory /
before that you need to go to / diretory using cd

cd /
for i in *; do echo -e "$(find $i | wc -l)\t$i"; done | sort -n

it will take some while according to your hard speed, mine was an SSD which took arround 15 seconds
What was printed out in my server is in here

1       mnt
1       scripts
1       srv
17      tmp
35      boot
98      bin
243     sbin
355     root
411     lib64
766     dev
851     opt
1580    etc
1581    selinux
2097    hdd2
5926    lib
10662   sys
43905   home
99665   usr
189514  proc
3172738 var

I went into the /var by using cd /var and ran that script once more
in /var directory what was printed out about INodes was

root@linux [/var]# for i in *; do echo -e "$(find $i | wc -l)\t$i"; done | sort -n
1       cvs
1       games
1       local
1       mail
1       nis
1       opt
1       preserve
1       yp
2       db
2       empty
16      profiles
17      tmp
19      lock
22      named
32      run
38      cache
218     log
412     spool
1492    cpanel
6245    lib
3164214 asl

It was now brief to me that the asl log files from mod security was using too much of INodes that means to me the websites on the server was under too much attack that made the server being out of INodes
I deleted the extra logs loged out and now ran the df -i and the following is what I got printed out from the partitions INodes

 df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/vg_linux-lv_root
                     3276800  112872 3163928    4% /

Yum worked pretty nice now

yum update
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.ash.fastserv.com
 * extras: yum.singlehop.com
 * rpmforge: mirror.lug.udel.edu
 * updates: centos.mirror.netriplex.com
base                                                                                                                                             | 3.7 kB     00:00
extras                                                                                                                                           | 3.5 kB     00:00
rpmforge                                                                                                                                         | 1.9 kB     00:00
updates                                                                                                                                          | 3.5 kB     00:00
updates/primary_db                                                                                                                               | 1.4 MB     00:00
Setting up Update Process
No Packages marked for Update

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.