Hi All,

Okay so a lot of you are wondering why downloads are offline. It's not some nefarious scheme or any planned maintenance. The simple fact of the matter is that Chii, the server that serves all the zips and rar files, had a hardware problem (the first drive with the OS on it died completely) and she wouldn't boot up.

I ordered a reload and she was back up this morning. However I still have to run integrity checks on the drive where all the downloads are stored, and since she was reloaded with a "clean" OS, I have to re-install Apache, PHP, Lighttpd, and so on and then test everything.

I'm not going to change the estimated fix time since all this will take some time to do - I estimate at least a day anyway. However, as far as I can tell, the downloads are intact (all 127gb of them for MTS2), it looks like I'll be able to get everything working fine.

Just be patient and she'll be back soon.

Regards
Delphy

Note this is an informational thread ONLY. You may not reply here - and do not post in Site Issues about it!

But I wanna download NOWWWWWW!!111 *whinewhinebitchmoan* I will die without being able to download and I can't possibly do anything else for a day like play with what I already have, play another game, or go outside in the sunshine. *sobs* YOU'RE MEAN!

But of course you can't go out in the sunshine! It's Chii! Ain't no suuunshine when she's goooooone!

HP, as a moderator of this forum, may I remind you that abusing the site owner will not be tolerated, unless you intend to feed him sausages afterwards!!

Must... not... reply... with... innuendo!

Update: Chii as a server is back online and all the download data folders copied across with, as far as I can tell, no data loss. However, some of the key configuration files for Apache, PHP and Lighttpd did get corrupted, and I don't have backups of those... so I'll need to reconstruct those ones manually, basing them off other servers. Chii, however, had some unique scripts that no other server used (specifically the one that controlled access to the files via cookies) and so that will take a little while to redo.

So the estimate still stands - hopefully at some point tommorow I will be bringing Chii back for a limited usage run to test things out under load.

In the meantime, please don't try and download anything or use existing bookmarks to download things - only download when I say you can, ie when the message at the top changes. Thanks.

Increasing the estimated fix date since I find I have to do a large data integrity check and cleanups of the old drive.

Okay, so, turns out that when I said I didn't think we'd lost any data, I was incorrect.

The primary drive from chii had a ton of read write errors on it - so much so that currently all downloads past mid-october last year (when the server was first setup - remember, we moved servers back then) are basically toast.

The backup drive has so many read write errors on it that it wont even boot the machine with it plugged in and it just goes to kernel panic mode.

As it stands I have managed to recover most downloads prior to Mid-October last year (some 127gb worth), but anything after attachmentid 826742 is gone. I am not entirely convinced that *all* downloads prior to then are intact and am investigating further to see how many prior to that date are ok.

Suffice to say I am extremely annoyed at the situation and beleive that there may be some other hardware issue on Chii becuase to have 2 drives fail so utterly is extremely rare.

I'm going to see if there is a way of getting the primary drive looked at professionally to see if we can recover any more data, but right now it's not looking hopeful.

Latest status: They are attempting to rebuild the RAID array from the secondary drive to see if that works.

If it doesn't I've ordered a temporary server for another go at the data transfer, since I don't trust the drive controller on Chii.

More later when I have more info.

Okay RAID rebuild failed - the backup drive is completely toast.

I'm going to get the primary drive shipped to me so I can run my own recovery on it. However this will obviously take some days.

So, basically what I'm doing is re-installing the basic stuff on Chii (gcc, libjpeg etc) so I can get PHP compiled. Once that happens I'll run a script that tells me which files are actually missing from the filesystem, and then we can identify the threads.

Once we know which threads are affected, I'll make a custom banner appear at the top of each of them, asking for members or the creator to resubmit the files. This way we can hopefully rebuild anything missing.

I apologise for this and for the delay - it's obvious there was a massive failure of the backup systems here, which only ever comes to light when something actually goes wrong. Going forward I'm going to investigate more stringent and effective backup solutions but for now we'll have to see how bad the damage really is.

Okay, I've restored what I can from the broken old primary drive. The secondary drive backup is, as I have said, unusable.

I've compiled up a list of all possible missing files, which breaks down into the following:

Total files: 7418
Threads affected: 3760

The list can be found here: http://db.modthesims2.com/missingfiles.html

My next task will be to do what I planned to do... set Chii up with Apache/lighttpd etc, so that people can upload and download again.

While I am doing that I would like to request that people look at the list and see if they have any of the downloads in question, in the original zip or rar version. This way once uploads are back hopefully we can start filling in any holes.

Last edited by Delphy : 13th Mar 2009 at 3:53 AM.

Just a note, if you hit CTRL-F (CMD-F for you filthy Mac-lovers) on that page Delphy linked to, you can search the page easily. That way you can type in your own name, filename, or name of a creator whose stuff you know you have (or whatever you want to search for) and see if it's listed.

Last edited by HystericalParoxysm : 13th Mar 2009 at 3:41 AM.

Okay I've added a longer version of the missing files list. This one includes the dump from rar/zip (which includes the CRC values of each package file in the rar version) and also includes the "full" filename (ie MTS2_123456_Username_filename.zip).

Since it's rather large I've zipped it. You can download it here: http://db.modthesims2.com/missingfileslong.zip

Chii is configured and back up, along with Skuld too. All threads with missing have had a banner added and a thread written on how to re-attach files.