Film-Tech Forum ARCHIVE: Digital Projector

my profile | my password | search | faq & rules | forum home

»	Film-Tech Forum ARCHIVE » Operations » Digital Cinema Forum » Digital Projector (Page 3)

This topic comprises 3 pages: 1 2 3

Author

Topic: Digital Projector

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 02-10-2008 01:37 PM

Another interesting point to ponder is the info in this article.

July 18th, 2007
Why RAID 5 stops working in 2009
Posted by Robin Harris @ 6:18 am

Categories: RAID, Disk drives

The storage version of Y2k? No, it’s a function of capacity growth and RAID 5’s limitations. If you are thinking about SATA RAID for home or business use, or using RAID today, you need to know why.

RAID 5 protects against a single disk failure. You can recover all your data if a single disk breaks. The problem: once a disk breaks, there is another increasingly common failure lurking. And in 2009 it is highly certain it will find you.

Disks fail
While disks are incredibly reliable devices, they do fail. Our best data - from CMU and Google - finds that over 3% of drives fail each year in the first three years of drive life, and then failure rates start rising fast.

With 7 brand new disks, you have ~20% chance of seeing a disk failure each year. Factor in the rising failure rate with age and over 4 years you are almost certain to see a disk failure during the life of those disks.

But you’re protected by RAID 5, right? Not in 2009.

Reads fail
SATA drives are commonly specified with an unrecoverable read error rate (URE) of 10^14. Which means that once every 100,000,000,000,000 bits, the disk will very politely tell you that, so sorry, but I really, truly can’t read that sector back to you.

One hundred trillion bits is about 12 terabytes. Sound like a lot? Not in 2009.

Disk capacities double
Disk drive capacities double every 18-24 months. We have 1 TB drives now, and in 2009 we’ll have 2 TB drives.

With a 7 drive RAID 5 disk failure, you’ll have 6 remaining 2 TB drives. As the RAID controller is busily reading through those 6 disks to reconstruct the data from the failed drive, it is almost certain it will see an URE.

So the read fails. And when that happens, you are one unhappy camper. The message “we can’t read this RAID volume” travels up the chain of command until an error message is presented on the screen. 12 TB of your carefully protected - you thought! - data is gone. Oh, you didn’t back it up to tape? Bummer!

So now what?
The obvious answer, and the one that storage marketers have begun trumpeting, is RAID 6, which protects your data against 2 failures. Which is all well and good, until you consider this: as drives increase in size, any drive failure will always be accompanied by a read error. So RAID 6 will give you no more protection than RAID 5 does now, but you’ll pay more anyway for extra disk capacity and slower write performance.

Gee, paying more for less! I can hardly wait!

The Storage Bits take
Users of enterprise storage arrays have less to worry about: your tiny costly disks have less capacity and thus a smaller chance of encountering an URE. And your spec’d URE rate of 10^15 also helps.

There are some other fixes out there as well, some fairly obvious and some, I’m certain, waiting for someone much brighter than me to invent. But even today a 7 drive RAID 5 with 1 TB disks has a 50% chance of a rebuild failure. RAID 5 is reaching the end of its useful life.

Update: I’ve clearly tapped into a rich vein of RAID folklore. Just to be clear I’m talking about a failed drive (i.e. all sectors are gone) plus an URE on another sector during a rebuild. With 12 TB of capacity in the remaining RAID 5 stripe and an URE rate of 10^14, you are highly likely to encounter a URE. Almost certain, if the drive vendors are right.

As well-informed commenter Liam Newcombe notes:

The key point that seems to be missed in many of the comments is that when a disk fails in a RAID 5 array and it has to rebuild there is a significant chance of a non-recoverable read error during the rebuild (BER / UER). As there is no longer any redundancy the RAID array cannot rebuild, this is not dependent on whether you are running Windows or Linux, hardware or software RAID 5, it is simple mathematics. An honest RAID controller will log this and generally abort, allowing you to restore undamaged data from backup onto a fresh array.

Thus my comment about hoping you have a backup.

Mr. Newcombe, just as I was beginning to like him, then took me to task for stating that “RAID 6 will give you no more protection than RAID 5 does now”. What I had hoped to communicate is this: in a few years - if not 2009 then not long after - all SATA RAID failures will consist of a disk failure + URE.

RAID 6 will protect you against this quite nicely, just as RAID 5 protects against a single disk failure today. In the future, though, you will require RAID 6 to protect against single disk failures + the inevitable URE and so, effectively, RAID 6 in a few years will give you no more protection than RAID 5 does today. This isn’t RAID 6’s fault. Instead it is due to the increasing capacity of disks and their steady URE rate. RAID 5 won’t work at all, and, instead, RAID 6 will replace RAID 5.

Originally the developers of RAID suggested RAID 6 as a means of protecting against 2 disk failures. As we now know, a single disk failure means a second disk failure is much more likely - see the CMU pdf Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You? for details - or check out my synopsis in Everything You Know About Disks Is Wrong. RAID 5 protection is a little dodgy today due to this effect and RAID 6 - in a few years - won’t be able to help.

Finally, I recalculated the AFR for 7 drives using the 3.1% AFR from the CMU paper, using the formula suggested by a couple of readers - 1-96.9 ^# of disks - and got 19.8%. So I changed the ~23% number to ~20%.

| IP: Logged

Tristan Lane
Master Film Handler

Posts: 444
From: Nampa, Idaho
Registered: Feb 2002

posted 02-11-2008 05:28 PM

Interesting Article.

I wonder how long before RAID and Platter drives are replaced entirely by solid-state memory.

SSD flash memory "disks" are getting larger by the week. They do have their drawbacks, but it is only a matter of time before spinning platters and read/write heads are a thing of the past.

Flash memory seems to be getting old as well. It seems that there are a couple of new technologies on the horizon offering much better performance in terms of memory.

*

quote: Steve Guttag
Doremi has an D/A card that I have been less than impressed with.

AGREED - This card was a common failure item when I was servicing for Christie. In addition to failure, the analog sound card seemed to have grounding issues (it may have been somewhere else in the Doremi Chassis, but 50% of the Doremi's installed in the field needed to have additional grounding provided to eliminate hum.

| IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999

posted 02-12-2008 09:15 PM

quote: Mark Gulbrandsen
Instead of talking to Dolby why not talk to the guys at Christie

As I am working on a quote for one (CP2000ZX)...I did indeed talk with the Christie guys today...Nope...the CP2000-ZX is not Dolby 3D capable at this time. Christie is working on it but it will likely require a smaller version of the filter disc. As of today, there is no ETA on a Dolby 3D capable version of this projector.

As to RAID drives...a problem arises for the Cinema industry at the moment...the drives may be getting too big. By nature, cinemas are going to change out the content on those drives relatively often...once the keys go bad on the contenet...there is no need to keep the files. At some point you can always delete to make room for the current stuff.

If you don't and keep getting bigger and bigger drives...it now takes progressively longer to rebuild one or even move all of the content.

Now as DCinema moves beyond 2K...the need for larger drives will be there...but for 2K content...I think they currently can hold something like 12 or 15 typical features plus trailers and such...

Steve

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 02-12-2008 10:08 PM

Steve,
Thanks for the Update on the ZX... I too have two to do for Dolby 3-d but its going to be later in the year. The thing thats funny about that was the guy that taught our training class and told us it had been done is also the guy in Canada that qualifies all the parts that go into the projectors. It'll happen eventually though.

Tristian,
Don't think that solid state memory won't also fail, or need re-formatting at some point in its life...

If we can keep the raid disks to under 500gb each I think we will all only experience the once in a blue moon array failure... kinda like loosing an ignitor or doide... its going to happen to everyone sooner or later. Thing is that rebuilding the array isn't ALWAYS going to work either... drives will fail completely or intermittantlly and this is where attempts at rebuilding will get one into bigger trouble. Keeping one extra spare drive in the booth is hence very important! Dual drive failures are pretty rare as long as the array is rebuilt immediately and the drives are physically ok, but intermittant problems on drives can lead to corrupting the entire data structure of the RAID Array. Complete drive replacement every so many years or hours will more than likely be in the forcast to maintian the reliability of the RAID. Several IT friends of mine say that it seems like when drives get to be more than 4 to 5 years old they can no longer be trusted so they are for the most part automatically replaced at 3 to 4 year intervals. We will probably find ourselves doing the same thing

Mark

| IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999

posted 02-13-2008 07:09 AM

Remember too...unlike most users of Raid arrays...cinemas have the originals (movies) on site so long as they are playing the picture...even if one had to completely start from scratch, they could get back up using the content in the cinema at the time and only need to put on (immediately) what is needed for that week/day.

With a system like Dolby's...where you have individual show stores as well as the show library...you also have several redundant systems as the library can load to any theatre and any theatre can send to any theatre. In short...if you have a complete dump and were starting from a complete rebuild, in a cinema, you are likely to have what you need in the cinema as either an original or on one of the other (many) drives throughout the cinema. I agree however...it could become like belts or oil changes...you simply change out drives after a period of time just to better your odds at not having a failure that would take the theatre down for a period of time (potentially hours if the cinema HAS a spare drive).

Because of the whole KDM thing...you will also see cinemas keeping complete spare SYSTEMS on site to ensure if a piece of equipment fails they are not waiting on the keys to get them back up.

Steve

| IP: Logged

Scott Norwood
Film God

Posts: 8146
From: Boston, MA. USA (1774.21 miles northeast of Dallas)
Registered: Jun 99

posted 02-13-2008 08:03 AM

How would the key situation work for a spare server?

Suppose you have a single screen theatre and want complete redundancy. You buy two projectors and two servers. Then, what? Do you get two sets of keys, one per server/projector, for each feature? Or do you only get one key, with the promise that a second one can be generated quickly in the event of a server/projector failure? How long does it actually take to get new keys in these situations?

Where, exactly, is the hostid (on which each key is based) stored? Is it in the server or the projector? Is the hostid module removable and/or easily swappable in the event of a system swap/replacement?

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 02-13-2008 08:15 AM

quote: Steve Guttag
Because of the whole KDM thing...you will also see cinemas keeping complete spare SYSTEMS on site to ensure if a piece of equipment fails they are not waiting on the keys to get them back up.

Actually, if a server was swapped out not counting a hard drive you would need to have new KDM's issued for that screen. The servers have a HASH line in them to prevent the wrong KDM from being loaded in the wrong server. The Server serial number is also loaded into a line of the KDM in the form of HEX code. Now I suppose you could have KDM's issued for the spare server as well but if this whole dial up KDM placement fiasco does take hold that would make the new KDM mandatory...

quote: Steve Guttag
With a system like Dolby's...where you have individual show stores as well as the show library...you also have several redundant systems as the library can load to any theatre and any theatre can send to any theatre.

Another problem with using the Show Library is if you loose it then you loose Show Manager Client for the entire complex and hence you loose all your remote stations. It won't shut down a booth with a smart operator though as he can go around when the shows that are playing have stopped and take them off the network one by one and make them free standing players. Also note that a smart tech will store all the config files for servers and projectors on a memory dongle and keep it on himself. This allows instant re-activation of a piece of gear if a board or unit has to be replaced. The files are relatively small so its possible to store many screens on a single memory stick. Another set of backup config files stored in the managers office key box is also a good idea!!

quote: Scott Norwood
Suppose you have a single screen theatre and want complete redundancy. You buy two projectors and two servers. Then, what? Do you get two sets of keys, one per server/projector, for each feature? Or do you only get one key, with the promise that a second one can be generated quickly in the event of a server/projector failure? How long does it actually take to get new keys in these situations?

Its possible some studios would oblige on that... it only takes a matter of minutes to get a key via internet anyway and the KDM people are generally available 24/7/365. BTW: Important screenings and Premieres do use redundant systems.

Mark

| IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999

posted 02-13-2008 09:13 AM

At this point, there is no prohibition on having more units than you have screens and getting KDMs for the spare unit. Mind you, the picture is watermarked on all of the DCinema systems...you can't use the spare to go and do nasty things...it will STILL be traced back to the theatre.

The problem arises that IF you have a player go down and you don't have spare on site with the KDM for it...you can get a replacement unit sent in but you will then be having to get the various distribution networks on board with the new box so that proper KDMs will work with it. However, if you have spare player on site and are already receiving KDMs for it...all one must do is put it in and configure it for that particular screen/projector.

One of the nifty things about Dolby's system is that only the Show Player is tied to the KDM...not the server or store...thus, that is the only box that one would want a complete spare, server wise.

Steve

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 02-13-2008 10:44 AM

Thats only partly true... a problem in the Show Store could also prevent the kdm from loading in since thats the only ingest point for it... via either usb or cd rom. Changing out either of them takes but minutes...akin to swapping out an X-L movement in the film world. As long as you have records of network addresses and such its a cinch. By the time you're ready your new kdm would already be there.

Mark

| IP: Logged

David Zylstra
Master Film Handler

Posts: 432
From: Novi, MI, USA
Registered: Mar 2007

posted 02-13-2008 01:29 PM

I can gaurantee you that the studios won't be authorizing KDMs for any spare units (most have stated this publicly).

So far the distributors have been extremely quick in sending KDMs for replacement servers - just a quick call to the right number, tell them the new server serial number and within 5-10 minutes you have a KDM (the AccessIT weekly bulletins include a list of what distributors to call for each film).

| IP: Logged

All times are Central (GMT -6:00)

This topic comprises 3 pages: 1 2 3

Printer-friendly view of this topic

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.