A new record? 2006 January 24 23:56
Posted by diamond in : Random , 1 comment so farToday i wanted to backup all data on a 250G drive i have in preperation for rebuilding the storage in the server (switching to 4x 250G w/ raid5). I set rsync running, went out for a while, got back about 50 minutes later and noticed it was still trying to index the drive. Now there’s a lot of data on it (~221G), but it still shouldn’t be taking that long. Upon inspection, it was the apache mod_cache dir that was killing it. I’d told mod_cache to allocate 1G of a disk cache.
I had a quick look at the directory in question. It had 4097 subdirs. I looked into a few of them, they had varying amounts of between 80 and 100 subdirs. Hmm. How many subdirs could there possibly be? So i ran a test.
find . -type d > ~/dir.list
Guess how long it took. Go on. The real answer:
real 352m38.230s
user 0m57.195s
sys 3m46.477s
Just under _6 hours_ to list all the directories. That’s pretty damn scary. And how many subdirs were there?
root@fluff:/space/.data/apache-cache(0)# wc -l ~/dir.list
745917 /root/dir.list
So, is this a record? 745k directories. I’ve no idea what size the cache actually is, finding out is likely to take, well, hours ,-) I can tell you one thing though, there’s no way i’ll be using apache as a proxy until it becomes somewhat less psychotic. I’m also wondering if this ext3 partition was created with b-tree dir indexing, would be interesting to know. Anyone know a way of finding out?
*Update*
root@fluff:/space/.data/apache-cache(0)# time du -hs .
9.1G .
real 472m45.334s
user 0m44.290s
sys 4m41.508s
