Film-Tech Forum ARCHIVE: High Capacity and Compatibility server in 2016

my profile | my password | search | faq & rules | forum home

»	Film-Tech Forum ARCHIVE » Operations » Digital Cinema Forum » High Capacity and Compatibility server in 2016 (Page 3)

This topic comprises 3 pages: 1 2 3

Author

Topic: High Capacity and Compatibility server in 2016

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999

posted 03-21-2016 12:55 PM

I will say...I've never had a problem getting GDC to remote in in VERY short order. If I know they are going to need to remote in to set a serial number, I tend to give them a heads up on the time frame I anticipate needing them...they have ALWAYS been very attentive to that sort of thing.

Your other comments still standing. Though I guess it would be wise to plan on the big clone fest in the near future for our older servers.

| IP: Logged

Scott Norwood
Film God

Posts: 8146
From: Boston, MA. USA (1774.21 miles northeast of Dallas)
Registered: Jun 99

posted 03-21-2016 02:02 PM

The non-mirrored boot disk on the GDC is pretty much unforgiveable. The disks may be reliable and GDC support may be great, but a) disks are cheap, b) computers are really good at doing repetitive tasks like keeping a disk mirror in sync, and c) not every D-cinema system has Internet connectivity.

The "right" way to do this would be to have the OS on a mirrored pair of disks (which could be small) and the content on a RAID (honestly, a RAID 1 mirror would work fine with today's 4TB and larger disks, though a RAID 5 or RAID 6 would be better for the sort of capacity that Steve wants). Having the OS on the RAID is not ideal, but not really a problem as long as the read speed of the RAID is fast enough to serve the needs of both the OS and the content and the content can be easily re-loaded if the RAID ever needs to be re-built.

| IP: Logged

Adam Martin
I'm not even gonna point out the irony.

Posts: 3686
From: Dallas, TX
Registered: Nov 2000

posted 03-21-2016 02:19 PM

Don't forget that it's a software RAID, so whether or not the machine can handle RAID-1 + RAID-5 + playback all at the same time is questionable. Based on the performance I've experienced, it can barely handle RAID-5 + playback, and forget it if it's rebuilding or choking on disk errors.

| IP: Logged

Scott Norwood
Film God

Posts: 8146
From: Boston, MA. USA (1774.21 miles northeast of Dallas)
Registered: Jun 99

posted 03-21-2016 04:49 PM

Mirroring is easy and safe to do in software, and should have very little performance penalty. RAID 5 or 6 is CPU-intensive on writes and isn't really safe to do in software, anyway (or, for that matter, in hardware without a battery-backed cache on the RAID card). Software RAID 5 or 6 should not be CPU-intensive on reads, though.

But, really, a 4TB mirror should be plenty sufficent for the average cinema, with respect to capacity and throughput. And, yes, I realize that 4TB drives did not exist when many of these servers were initially sold.

| IP: Logged

Marcel Birgelen
Film God

Posts: 3357
From: Maastricht, Limburg, Netherlands
Registered: Feb 2012

posted 03-21-2016 05:08 PM

For any non-mirrored setup (think RAID1 or RAID10), you definitely want double parity (e.g. RAID6) for those huge but rather slow multi-terabyte SATA disks.

Whether or not the GDC single OS-disk is an excusable SPOF depends on the actual failure rate, since there are many other SPOFs in those servers, it's not entirely fair to bring this forward as a design failure, without real numbers to back it up. Flash storage tends to be quite durable if it's primarily being used for read operations, actually often more durable than rotating rust.

What bugs me though, is the fact that you need them to remote in to the machine to fix their stuff, that shouldn't be a requirement. Just hook-up an USB stick and reload the OS and you should be good to go. If it's really just because of the serial number, then that's just a shoddy design. You could easily save a copy the serial number onto some ROM on the mainboard.

quote: Scott Norwood
Mirroring is easy and safe to do in software, and should have very little performance penalty. RAID 5 or 6 is CPU-intensive on writes and isn't really safe to do in software, anyway (or, for that matter, in hardware without a battery-backed cache on the RAID card). Software RAID 5 or 6 should not be CPU-intensive on reads, though.

There are software implementations of RAID schemes which are considered safer than most hardware implementations. ZFS is an interesting example, it takes data consistency to another level. Btrfs is another example, but it's not entirely there yet.

Also, with modern CPUs with something like 32 cores at your disposal, it's really a no-brainer to do both a rebuild of a large software RAID environment, ingest content and play it back at the same time, CPU-wise. The problem here is to squeeze this amount of performance out of a bunch of sluggish SATA spindles though, especially for small arrays. But this problem would be present in hardware RAID setups too.

A smart software RAID implementation could also dynamically throttle rebuild speeds if other higher priority operations, like playback are running. Not that this would be entirely impossible in a hardware RAID setup, but it's not trivial to make it happen.

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 03-21-2016 06:45 PM

Failure rate is really irrelevant in a GDC because you have both the option of ordering it with an extra OS drive already installed in the bay or taking a half hour and clear Ning your customers existing drive and either saving that image or burning a new drive that you can leave in the drive cage. All the servers I install are on sliders and it takes all of five minutes to slide it out pull the top cover and move two cables over to the backup drive. It just doesn't get any easier, faster or simpler.

Mark

| IP: Logged

Brad Miller
Administrator

Posts: 17775
From: Plano, TX (36.2 miles NW of Rockwall)
Registered: May 99

posted 03-22-2016 12:44 AM

Doremi's design is actually simpler and easier. Just swap the OS chip.

| IP: Logged

Scott Norwood
Film God

Posts: 8146
From: Boston, MA. USA (1774.21 miles northeast of Dallas)
Registered: Jun 99

posted 03-22-2016 05:21 AM

quote: Marcel Birgelen
There are software implementations of RAID schemes which are considered safer than most hardware implementations. ZFS is an interesting example, it takes data consistency to another level.

Agreed that ZFS is great, but it is not appropriate for D-cinema or other real-time applications, where a guaranteed rate of throughput is needed. Dolby and Doremi (and possibly GDC...I don't know) use XFS, which specifically provides that functionality.

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 03-22-2016 10:11 AM

I agree with you guys on the mirrored raid thing. It's a no brainier, but it's a big cost factor for the manufacturer and consumer. Doremi and GDC both use boards with very similar imbedded Intel based software raid controllers. I have yet to see that controller reliably do both a mirrored pair and a raid 5 triple set. Also, the amount of work to combine those software raid functions into the manufacturers OS system is a lot of time and money. At that point in time a hardware raid card makes way more sense and can do that job very reliably.

Marcel, cloning a drive should really be done at installation and I use a separate computer to do that, no USB! Have I cloned everyone's drive... Certainly not, but I have done it and made tests and I will be cloning the older ones in the very near future. That old drive then will become a backup drive left in place.

And almost all of my maintainance work is done during shut down hours. I don't know many guys that work on plex's while they are running, unless a screen goes down or you are changing out equipment. The nice thing about working on just one brand of server is I also carry a spare GDC OS drive in my parts kit and spare Media Drive Sets that are already raided and ready to go should they ever be needed. So what's the big deal? Oh, and GDC does indeed throttle their raid depending on what's going down on. Playback takes precedence over just about anything else going on in their raid. I can remember when their raid was not throttled and playing back 4K content and ingesting content at the same time could make it skip.

Brad, same thing on the SX-3000.

Mark

| IP: Logged

Steve Guttag
We forgot the crackers Gromit!!!

Posts: 12814
From: Annapolis, MD
Registered: Dec 1999

posted 03-22-2016 10:55 AM

I believe Marcel was referring to if you did have a theatre go down in a plex...the only way to make an OS drive is borrow one from another screen to make your clone.

Yes...they do throttle back...to the point of "why bother." With the DSS line...it is full steam ahead while showing the movie...ingest or transfers can happen at speed.

| IP: Logged

Marcel Birgelen
Film God

Posts: 3357
From: Maastricht, Limburg, Netherlands
Registered: Feb 2012

posted 03-22-2016 10:56 AM

quote: Scott Norwood
Agreed that ZFS is great, but it is not appropriate for D-cinema or other real-time applications, where a guaranteed rate of throughput is needed. Dolby and Doremi (and possibly GDC...I don't know) use XFS, which specifically provides that functionality.

ZFS might not be the best suited candidate for D-Cinema right now, but the guaranteed rate of troughput other filesystems claim really doesn't matter all that much here. ZFS will perform just fine if the hardware is fine and will give you the bandwidth you need. Keep in mind that none of the servers out there are running real-time Operating Systems, they're running Linux or Windows, which are as non-real-time as it could get. They work, because nothing else that could interfere is going on on the system and providing sufficient headroom as to not run into any problems during normal operations.

quote: Mark Gulbrandsen
I agree with you guys on the mirrored raid thing. It's a no brainier, but it's a big cost factor for the manufacturer and consumer. Doremi and GDC both use boards with very similar imbedded Intel based software raid controllers. I have yet to see that controller reliably do both a mirrored pair and a raid 5 triple set. Also, the amount of work to combine those software raid functions into the manufacturers OS system is a lot of time and money. At that point in time a hardware raid card makes way more sense and can do that job very reliably.

The future will most likely be software defined storage and not so much hardware. The embedded RAID you're referring to is a half-baked attempt at RAID and is what has given "software RAID" a bad name. Everything is better than those Frankenraid implementations, pushing the hard work into the device driver to save a few bucks on dedicated hardware. Doremi doesn't use this kind of RAID though, it uses Linux software RAID. Based on my experiences with that, I prefer proper hardware RAID instead of Linux software RAID.

quote: Mark Gulbrandsen
Marcel, cloning a drive should really be done at installation and I use a separate computer to do that, no USB! Have I cloned everyone's drive... Certainly not, but I have done it and made tests and I will be cloning the older ones in the very near future. That old drive then will become a backup drive left in place.

The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

quote: Mark Gulbrandsen
And almost all of my maintainance work is done during shut down hours. I don't know many guys that work on plex's while they are running, unless a screen goes down or you are changing out equipment.

I usually prefer to carry out disruptive maintenance on Friday night... Hey, who doesn't like a little excitement now and then?

| IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007

posted 03-22-2016 02:32 PM

quote: Marcel Birgelen
The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

Which is what Dolby do [Smile]

| IP: Logged

Mark Gulbrandsen
Resident Trollmaster

Posts: 16657
From: Music City
Registered: Jun 99

posted 03-22-2016 02:32 PM

quote: Marcel Birgelen
The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

You can't because of all the partitions involved. That's why I carry a spare drive in the truck. Changing the drive is much quicker anyway if your server is on slide out rails. Many of my customers could even do that drive change in a pinch. I really don't want to drive 600 mils just to change an OS drive but certainly would if my customer felt he or she couldn't.

I have several really old GDC's that will be getting overhauls before the summer films hit. Complete clean up, new fans, new filters, new drives, etc.

Will be doing the same to some of the older S-2 NEC's. I have several out there with what seems like a half billion hours on them. Some five and six year old projectors will be partly dismantled and cleaned. Then several fans replaced, new back up ICP battery and fan also installed.

No point in waiting for the parts to die and letting them have down time...

Mark

| IP: Logged

Marco Giustini
Film God

Posts: 2713
From: Reading, UK
Registered: Nov 2007

posted 03-22-2016 07:03 PM

ok. While you drive umpteen miles and clone/replace those drives (in 5 minutes) I will download an 'INSTALL' version of my Dolby software. Then I'll watch some TV. Let me know if you need a hand! [Smile]

(yes, I do clean the servers too and needless to say that RAID arrays seldom fail - but single drives do. If the customer does not ignore the warnings and quickly replace the failed drives, chances are that I will never need that INSTALL media.)

| IP: Logged

All times are Central (GMT -6:00)

This topic comprises 3 pages: 1 2 3

Printer-friendly view of this topic

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.