Film-Tech Forum ARCHIVE: Dolby Show Store down failed array

This is topic Dolby Show Store down failed array in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=001775

Posted by Eric Robinson (Member # 2966) on 01-25-2014, 04:32 PM:

We have a dolby show store which reported a failed drive in the array. After booting into the RAID BIOS, the unit is reporting that we have a 3 disc array in RAID 0 configuration. The fourth drive is not recognized by the BIOS so we have removed it. I am suspecting that one of the 3 drives is also bad thus causing the BIOS to think this is a RAID 0 instead of a RAID 5 configuration. What I am wondering is if a Dolby Show Store which has 4 drives in a RAID 5 configuration has 2 failed drives...is there any recovery possible? Also, I know that the show store has a bootable Linux flash on board, but are there Dolby application files on the RAID drives also? That is can a Dolby Show Store be resurrected from 4 blank drives?

Posted by Mark Gulbrandsen (Member # 72) on 01-25-2014, 04:56 PM:

I recently had this happen on a DSS-200 that I had recently installed new Hitachi 1 TB drives in. The unit lost two drives and failed to work. Now I will admit this is extremely rare but it can and does happen. The DSP-100 and DSS-200 Raid card are pretty similar and there is likely no easy way to recover the data once the parity striping is lost. At the location this happened to me at I had the 4 original Seasick 500GB drives ready to go sitting in the 4 spare slots and all the manager had to do was power down, swap the four drives over and re-boot and load content or live play the content. Then I replaced the drives after the New Year
You'll have to get new drives and reinstall the OS then the content.

Mark

Posted by Eric Robinson (Member # 2966) on 01-25-2014, 05:18 PM:

Thanks Mark, I suspected the RAID degraded beyond a point of recovery. I was just informed that the Show Store was showing 2 bad drives. I spoke with Dolby already and they are emailing me a link to the OS download. So, it looks like I can bring this back to life after a little bit of work.

Posted by Marco Giustini (Member # 4544) on 01-25-2014, 05:30 PM:

Does the unit still boot up and work properly?
I've seen some units where one drive fails and a second is working just enough to operate the server but not enough to allow a rebuild.
I've never seen the RAID card reporting a RAID 0.
You may want to enter the RAID BIOS and see what it says. That may save you a reinstall - which is easy and gives you a 'new' system but you'll lose all your settings, playlists, ftp's, cues (you can save the latter).

Posted by Mark Gulbrandsen (Member # 72) on 01-25-2014, 06:32 PM:

Yes, if you go into RAID card bios you can select RAID 0. I think it automatically defaulted to a JOBD mode. JOBD = Just A Bunch Of Disks. Many raid cards also allow that.

Anyway, all you now have left is JBOD!

Posted by Marcel Birgelen (Member # 6801) on 01-25-2014, 08:37 PM:

If your RAID controller reports a RAID 0 container, where once a RAID 5 container used to live, I would rather suspect the controller than just the disks. The situation could still have been triggered by a double disk failure, but it should not happen. If the second disk in a RAID 5 array fails, the data becomes inaccessible, but the controller should not go haywire.

RAID controllers do not conform to some kind of standard, but most of them save the essential parts of the RAID configuration both in NVRAM and as metadata on the disks itself (usually on all the disks, so a singe disk would be sufficient to get at least the configuration back). The NVRAM configuration is useful in cases where no disks show up, because of e.g. a backplane failure. The data on the disks can be used if the controller needs to be swapped, the new controller can then read the metadata and "import" the existing array.

In some cases, a failed RAID array can be (partially) restored by forcing a rebuild. If it works, you should back up all your essential stuff and still get new disks and rebuild the array from scratch.

Alternatively, there are also data recovery services, but I doubt that would be even remotely useful for data on a Show Store.

Posted by Steve Guttag (Member # 268) on 01-26-2014, 07:42 AM:

I'm curious as to which software you are currently running. I ask because one rarely if ever goes into the RAID software anymore. Since system 4.3.x, the Dolby software self-configures the RAID.

Running an Install disc isn't such a bad thing. It will clean out any issues that have been building on each other. The down aides are reloading any default content as well as any serial cues.

I have only had the double-drive failure happen to me once or twice. As such, I'm more inclined to change drives if they hit of issues (high reallocated sector counts or errors in the SMART logs).