post.thing.net

headlines | about |

SERVER CRASH

It all started Monday early afternoon when I could not reach this very post.thing.net web site. I noticed that the other thing.net sites were down too and checked with Walter in Berlin, who proceeded to restart the Apache web server, only to see an extremely high load on it. Not knowing what's going on we checked with Jan who immediately diagnosed a disk failure. Bad news. We had upgraded to a new server just about a year ago, a new machine with two big drives and a RAID setup (redundant array of independent disks, originally redundant array of inexpensive disks). As it turned out, apparently both drives had suddenly problems and the RAID system, which should make recovery from such failures tolerable, failed. So now everything had to be copied manually to drive partitions that were still ok. Turned out that a lot of junk had accumulated over the months, some email folders had about 5G of data in them. Who in hell needs 5G of email crap? Well, nobody gives a hoot, it's just the new normal. I just wondered about all the new huge data centers that are being build, gobbling up resources like water and electricity, for what? For gigatons of silly videos, pictures and annoying, stupid, useless spam. So anyway, we had to move mountains of trash, copying for days, since the sick drive was really going slow. As I write this we are still moving data... but at least the web sites are up and https://webmail.thing.net is working for now.

More info and perhaps some reflections later. Thanks to Eva and Franco for the picture. It really cheered me up after a distressing 48 hours...


Thanks for the update, Wolf, and Happy Thanksgiving

Wolf

Thanks for the update. Like most Thingamajigs, I had two days with no email, one with just webmail, but now the POP protocols seem to be in place, and my usual service is fully restored, just in time for Thanksgiving.

I want to thank you for your diligence and for the obvious pains you, Walter and Jan have taken to get things back together.

I make it a weekly practice to empty my Inbox (and also purge the Trash) so the data does not accumulate in the folder and become a burden. I would urge all others to follow this practice. Just a simple weekly flushing can clear the system and make it much healthier.

Servers like Gmail, where my storage is currently 66% full (with 6.7 GB of 10.1 GB used), not only spoil us, they advance a model that is not particularly ecological. As you noted, this is now the "new normal". It makes us cyber-storage pigs, dotting the landscape with corporate data storage facilities just to accommodate the glut. A cautionary lesson for the orgy of consumption that characterizes Thanksgiving and the entire holiday season.

In any case, Happy Turkey Day!!