Film-Tech Forum ARCHIVE: High Capacity and Compatibility server in 2016

This is topic High Capacity and Compatibility server in 2016 in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=002620

Posted by Steve Guttag (Member # 268) on 03-17-2016, 09:24 AM:

It appears that the day I've dreaded is fast approaching. I need to find a suitable alternative to the Dolby DSS line of servers.

The DSS line of servers had key features that made them particularly suited to our core customer base.

1) They could store as much as you want. That is, you can go up to 16TB if you desired though most of our customers are finding 6TB to be a comfortable size.

2) Ingest of any movie or content transfer while shows are running without glitches.

3) Run most any frame rate imaginable. The DSS line is good up to 60fps on CAT862s, 120fps on CAT745s and up to 30fps on 4K.

3) Easy integration between the servers so that SPLs and content could easily move about without external TMS.

4) Great logging that is accessible.

5) 4K and 4K upgradable.

So as I look out at the current offerings. Some are immediately discounted because they are projector specific (Alchemy and IMB S2) though we use a lot of Barco projectors, it is tough to standardize on the Alchemy except for certain customers that would never need it to be in a mixed environment.

In the high capacity front...GDC has their Enterprise grade storage BUT that brings along the SX-3000 that HAS to be shut down before the projector, but more importantly, GDC will not publish their frame rates (despite requests) beyond the basics. I can't specify them if the manufacturer will not back them up to run things like 24-60fps on 2D and 24-60fps on 3D. I've received 25fps 3D. Will it run it? Who knows but GDC won't publish that it will. GDC also chokes on ingest while the show is running as well as content transfer while the show is running. Their logs are locked too. I will not be beholden to paying their tech support fees to analyze logs. Their tech support is GREAT but holding the logs hostage is just plain wrong. I also HATE that it deletes shows as they run rather than leaving a history on the schedule. Integration is okay if you use their TMS and at least the annual fee (bi-annual normally) isn't stiff but still, would rather have the option of only pay if you WANT the upgrade (perhaps it does that...I haven't been in the situation of it nagging).

Then there is the Doremi/Dolby stuff. The ShowVault/IMB can do up to 6TB if all four drives are stuffed with 2TB but this device is getting a bit old and still uses the PCIe cable thing, which makes it tethered. I'm sure that the HDMI port is also stuck in its time. That isn't its fault...all equipment is a "snapshot" of the era in which it was made. But if I'm moving towards a new platform, I'd hate to be doing it again in 6-months.

Then there is the Dolby IMS-2000...coming from the IMS-1000 it gives me the shivers. I also think the UI for the IMS series is about as bad as I've ever seen. Really poor color/contrast choices and things are just scattered...one almost has to use the hot buttons to make sense of it. The on board storage is only 2TB (WAY TOO SMALL) but has the options of a NAS storage. Has anyone done NAS storage with it? Can one confidently play content from the NAS to it? Thus far, my only experience (limited) with NAS storage has been with Christie and that hasn't been a pleasant experience.

So when festivals roll around and one has to load up on the servers (A LOT)...what is everyone doing to allow for the big storage?

The Dolby/Doremi TMS seems to want to only work with their products (big surprise there) and has that nagging reoccurring fee so there are two strikes against it. These sites wouldn't lend themselves to a TCC or AA type TMS, necessarily though perhaps a GDC TMS but now we are talking separate computers to run the TMS. The Dolby TMS had EVERY server be a TMS terminal. It just worked so well for the people we deal with.

So what say you? Just remember, whatever you are suggesting, it has to have large storage, integrate well, and play most anything that walks in the door (not necessarily from Hollywood).

Posted by Leo Enticknap (Member # 534) on 03-17-2016, 11:08 AM:

quote: Steve Guttag
So when festivals roll around and one has to load up on the servers (A LOT)...what is everyone doing to allow for the big storage?

DSS200s with 8-port RAID cards substituted, and 8 x 2TB drives. A contact is very keen on the Alchemy and has been trying to persuade me to consider one if we upgrade our older NEC to a Barco (which may be on the cards in the next few months). While the front end looks reasonable from the demo and the specs cover what we'd want to do with it, a ceiling of 4TB is simply not enough for a venue that needs to ingest a five-day fest's worth of DCPs before the opening night.

Dolby are continuing to support the DSS200 until April 2018 (at least, this is my understanding), and so I have no plans to revisit this issue until around fall 2017, when I intend to look at what's on the market at that time. If, by then, 4TB laptop drives are available and the Alchemy can use a RAID of three of them, then that would be just about enough space to work for us. But for the moment, I have no plans to retire our DSS200s.

Posted by Gordon McLeod (Member # 33) on 03-17-2016, 01:10 PM:

Overall i find GDC reliable but i dislike the locked logs and as of late we seem to be finding more bad RAM
I have always heard good things about the QUBE server but have yet to play with one

Posted by Steve Guttag (Member # 268) on 03-17-2016, 04:59 PM:

The locked logs are a non-starter. But even more so...their specs are for don't cover the range of frame rates I HAVE TO SUPPORT...I was mistaken above in that they do support 2K 3D at 60fps already. But they don't list 4K/30...which is essential if you want to run the DCP of Oklahoma!. I'm sure we'll see something at CinemaCon.

Doremi's servers have normally be the darling's of the festival circuit because they run most everything and that is a big plus. But they do have size limitations of 6TB and you have to use the 4 drive systems (ShowVault or 2K4)...and then there is the TMS connectivity.

Posted by Scott Norwood (Member # 30) on 03-17-2016, 05:19 PM:

One plus for the Doremi with its IMB is that it is possible to mix frame rates in a single playlist without glitches at the frame rate changes. This is true with the Barco projectors, at least.

Also, it is easy to get content off of the Doremi and onto a blank CRU or USB drive. This has occasionally been useful. The Dolby does this, too, but I do not believe that it is possible in any officially supported way on the GDC (if it is, let me know).

I have no idea on the TMS issues, though. I just run this stuff from time to time...I don't install or maintain it.

Posted by Stephan Shelley (Member # 8587) on 03-17-2016, 05:56 PM:

There was a similar thread a while back. The USL IMS was mentioned as a possible candidate.

Posted by Marcel Birgelen (Member # 6801) on 03-17-2016, 06:08 PM:

I'm afraid that, whether we like it or not, the future will be vendor-provided IMSes and external NASes if you want more storage than can be fitted inside the local storage of the IMS.

The reason why this will probably win the race is not because it's technically the best solution, but simply price. The projector vendors will simply undercut anything other on bundle price. Once the IMS becomes a mandatory item on any new projector model, they're essentially killing the competition.

The situation could be somewhat improved if Barco and Christie would improve and standardize their API. At least it would open up the possibility of someone else building the ultimate TMS, which can handle mixed vendor setups without being a pain in the butt...

Posted by Steve Guttag (Member # 268) on 03-17-2016, 09:56 PM:

I'm not convinced of that at all. Christie backed off by discontinuing the Solaria 1/1+ and bring out the CP2208 which allowed anybody's IMB to be used. Barco's Alchemy has taken the first step of replacing the ICP to break the 500Mbs bottleneck...hence it can handle higher bandwidth. However, if a new ICP were to emerge it would negate the Alchemy inherent advantage.

I fear you may be right on the NAS type storage thing though I personally prefer local storage.

As for USL...storage limitations are a problem for it, from what I can tell but I'm looking at it real hard.

Posted by Marco Giustini (Member # 4544) on 03-18-2016, 02:52 AM:

I blame Dolby for not integrating the Doremi servers into the DSS line. They removed the DSS line without having a suitable replacement ready. Lots of customers with small Dolby TMS servers cannot replace a faulty server - or add a new one. Dolby and Doremi don't even talk to each other (unless Dolby finally stopped removing the small files when ingesting content).
Doremi 2K series have not received a software upgrade for ages. 2.6.4 works but there are lots of minor bugs which would be nice to see resolved. On the other hand the DSS series keeps being developed at the usual pace of "we fix 1 bug and we introduce 2 brand new ones" - sounds like ASDA [Smile]

.

If Barco keep doing a good job with the Alchemy, they will wipe away all the competition from the market.

Posted by Steve Guttag (Member # 268) on 03-18-2016, 06:25 AM:

4.9 "fixes" the removal of the small files, as you call them,

Dolby's mistake was discontinuing the DSS line. They've essentially told an entire market to "go away" and "please use this product we just bought that you didn't chose to use before." There are plenty of proponents of Doremi products...they ARE the most installed server. But that doesn't mean that it is best/right server for every application. Between the DSS and Doremi products, they had many bases pretty well covered. They threw a sizable portion of their market away in the hopes it would return.

And you are correct, they didn't really have an upgrade path for those in the DSS line. They think that their TMS will harmonize the two products and I don't doubt it will for some but definitely not for all or, I predict, most. Mind you, if you are using a full-featured TMS system, the role of the show player (SMS) is greatly diminished because most user interaction is at the TMS level, not SMS. But for singles, Art, special venue...it is just the opposite...there is a lot of interaction at the SMS level. The more you want hands on, the more you want some interaction between servers and how the SMS interface is. You definitely don't want a WEB interface to the SMS as your only means of interaction in this sort of application.

So back to the original question...what server adequately replicates or substantially has the positive features of the Dolby DSS line? I'd say that GDC is a near miss but with some deal-breaking gaping holes (Logs, quirky shut down procedures, lack of guaranteed frame rates, and vanishing schedules). In typical multiplexes...these are not such big issues but in some of our venues...definite deal breakers.

Posted by Tom Bert (Member # 5719) on 03-18-2016, 07:09 AM:

On the list of supported formats/framerates for Alchemy, these are publically available:
2K2D: 24, 25, 30, 48, 50, 60, 96, 100, 120
2K3D: 24, 25, 30, 48, 50, 60
4K2D: 24, 25, 30, 48, 50, 60
4K3D: 24, 25, 30, 48

... and also fractional framerates

Posted by Carsten Kurz (Member # 5396) on 03-18-2016, 07:45 AM:

From my limited experience with the ICMP (we are still in testing), this server has a very high potential. Too bad it is limited to Barcos, but of course, Barco has a good reason to do so.

Steve - there is an older statement from Doremi regarding disc support >2TB. That expresses that it shouldn't be much work to do it, and that they didn't investigate further so far because there was little demand for it. Probably related to TMS/Library servers taking over high capacity needs to a large extent. But there is certainly a chance that DP2k4 or ShowVault will support larger discs in the near future.

Also - as you said earlier in other threads - Doremi is the only real remaining solution for a classic mediablock/HD-SDI. So, having both options under the same manufacturer/software level should be a big plus for you as well. I would press on Dolby for the larger disc support. The fact that there hasn't been a major Doremi software update AFTER the Dolby takeover should mean something. I am nearly sure there is something behind the corner. Maybe even an upgrade to their now slightly outdated IMB.

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 03-18-2016, 07:49 AM:

quote: Steve Guttag
I'm not convinced of that at all. Christie backed off by discontinuing the Solaria 1/1+ and bring out the CP2208 which allowed anybody's IMB to be used. Barco's Alchemy has taken the first step of replacing the ICP to break the 500Mbs bottleneck...hence it can handle higher bandwidth. However, if a new ICP were to emerge it would negate the Alchemy inherent advantage.

I do appreciate Christie's move to remove their own rather failed attempt of an IMS as a mandatory item for their entry level projectors. Maybe it's a sign they're silently ditching their own IMS implementation? But still, I'm not really all that convinced this strategy would stick. Instead of rolling their own, they might as well buy it from someone else, like Doremi/Dolby and sell this as a bundle.

As for Barco, it looks like they're quite committed to their Alchemy strategy and they've essentially eliminated the ICP. As for now, it doesn't look like Texas Instruments is putting a whole lot of efforts in producing a new ICP design, so what we see right now might also be the beginning of the end of the ICP itself, because nobody wants to get stuck at this 500Mb barrier. The upside of this is that it could remove some redundant components, remove restrictions and get costs down, but the downside will also be less compatibility between vendors, which will make it harder for IMB/IMS manufacturers to support multiple vendors.

quote: Steve Guttag
I fear you may be right on the NAS type storage thing though I personally prefer local storage.

I also do prefer local storage. Maybe one of the IMS vendors creates a proper mSAS implementation, where you can just connect a storage shelf to it. It would be the closest thing to local storage with a decent amount of capacity.

The development of higher capacity traditional 2.5" hard disks seems to be stuck, but multi-terabyte SSDs are gaining traction and might become an affordable alternative within the near future. Samsung unveiled a 16 TByte SSD in a 2.5" package last year for example.

Posted by Steve Guttag (Member # 268) on 03-18-2016, 08:33 AM:

Tom. Yes, the Alchemy has many things going for it...including the widest range of frame rates (both DCP and Alternate Content)...but it has a storage limitation of 4TB. It also is exclusive to Barco...which for me isn't a deal breaker in and of itself for MOST of our sites but the 4TB limit would be in the special venue/art circuit. They are all having us roll up to 6TB on the Dolby stuff now. Though the difference between their existing 3TB and a 4TB system would be over 5 features, typically. But if you are a "festival" type venue...you can't have enough storage. An immense amount of time is spent getting content loaded and ready for show...you might have 5 different titles on a single day. Being able to load/transfer while a show is going is a biggie too.

Carsten,

My guess on the Doremi gear is that all engineering time is being spent on the IMS2000 and their TMS (the future, if you will) and the existing stuff is "stable" in their minds.

GDC does still manufacture the SX-2001 so they too have a traditional HDSDI/Mediablock solution as well as a traditional server/IMB (SX2000AR) as well as are more IMSesq like solution of the SX-3000 but at least they kept the storage external so it can actually have a LOT of storage (16TB)...so GDC has actually kept their options very open, in that respect. They also have an IMB announcement at CinemaCon so we'll see what is up there sleeve there.

I do hope to see some advancement by Dolby on the traditional front. Then again, I hope they come to their senses on the DSS line and ramp up again (no reason to believe it will happen...just hope it does and that other parts of the world have been more vocal about it than I).

Posted by Tom Bert (Member # 5719) on 03-18-2016, 09:09 AM:

On the limited onboard storage of ICMP (and IMS in general): we do know of 3TB, 6TB and even 13TB SSD disks that could be compatible. The question is: how much are you willing to pay to have all content available onboard? Current price points of those disks would put the price of the storage >50% higher than the server price.

On the "Barco only" topic: feel free to convince Christie and NEC to become "Alchemy-compatible" :-)

Posted by Steve Guttag (Member # 268) on 03-18-2016, 11:10 AM:

I don't want to pay any particular premium on the drives...shouldn't have to. Barco needs to allow for larger on-board storage. I'm guessing there is no NAS option, at this time?

As for Christie and Barco allowing for the ICMP...I suspect that the problem would be both ways. In theory, since everyone uses the same ICP...the ICMP should be able to work on any DLP projector.

One of the things that DCI was supposed to bring about was this avoidance of proprietary formats that lock one into a particular brand. The SDDS, Dolby Digtal and DTS debacle being the prime example of what they wanted to avoid. That is why all of the IMB slots use the same connectors/real estate to ensure ANYONE's IMB should work. Christie, at first tried to box out Dolby on their IMB slot but gave in (though they always had a "generic" option).

In my opinion, any projector manufacturer that is in the server business should be compelled to have their servers be compatible with any other DLP projector after a brief period (say 1-year after release). So of like when Atmos came out...it only worked on Dolby servers but in relatively short order all of the other servers could handle it, including the Barco Alchemy.

I don't have a problem with a manufacturer having "hooks" in their stuff such that when it resides in their own equipment that things unique to that equipment can take advantage of those features but the general part of being a server with normal serverly duties should be universal regardless of projector it is living in.

Posted by Marco Giustini (Member # 4544) on 03-18-2016, 12:38 PM:

quote:
4.9 "fixes" the removal of the small files, as you call them,

I see, I thought it was supposed to be changed with 4.5! [Wink]

(not joking)

Posted by Mark Gulbrandsen (Member # 72) on 03-18-2016, 12:57 PM:

I predicted all this would happen two or three years ago and I would resist installing any piece of gear if it comes under the following headings....

Any of the IMS servers. They are absolute crap!

Any server where you are stuck with only it's on board storage.

Anything where the OS also resides with the on board media storage.

Anything that doesn't allow external networking in of media via gigabit or esata connection for live playback of media.

Mark

Posted by Steve Guttag (Member # 268) on 03-18-2016, 01:57 PM:

You are dead wrong on the OS not residing on the RAID. While you could argue that OS should be on a mirror with 2 drives. Having the OS on a separate internal, never to be seen drive is like a silent cancer...you don't know you have it until you are struck.

Having the OS on the RAID has yet to be an issue.

Posted by Mark Gulbrandsen (Member # 72) on 03-18-2016, 05:52 PM:

Actually a wise technician will know it exists and any of them can be changed out in under 10 minutes and you'd be back up and running before your server with the raided OS has even reloaded. Oh did you lose your media too when your raid crashed? I've seen that happen.

Of course I am still waiting on that OS drive failure to happen. It simply hasn't, and out of 336 servers that use them. Some are six years old now. The newer solid state drives will likely never fail as they are not written to constantly like media drives are.

Mark

Posted by Carsten Kurz (Member # 5396) on 03-18-2016, 07:29 PM:

As far as I know, Doremis do already support GUID partitioned drives. I think their kernel is just one step away from generic support for larger drives anyway. I don't think it is much work to implement it. And I don't think all their resources are bound to the IMS2000. I am sure the fact that there have been no updates on their base software does not mean they are busy with something else. I think a new release is around the corner with some substantial improvements. Sure also the Dolby takeover ate some resources while defining the future Dolby/Doremi ecosystem.

From my perspective, it is strange they gave up the CAT745, because techwise, it seems it was the better IMB than the Doremi IMB (okay, except for the secure clock). And I guess, with Gigabit-Ethernet-Connectivity, it should have been easier to marry a ShowVault with a CAT745 than a 220 with a Doremi IMB.

Okay, let's say I'm getting a bit mad here - Doremi software 3.0 will offer a choice of the classic Doremi GUI, their WEB-GUI, or, a classic Dolby GUI simulation...
Plus, of course, support for drives >2TB ;-)

- Carsten

Posted by Dave Macaulay (Member # 813) on 03-18-2016, 08:29 PM:

I expect Doremi will support larger drives on the IMS eventually. Possibly on chassis servers as well, but I think development is... relaxed... on those.
Samsung just announced a 15TB 2.5" SSD... I think using those would be enough for most festivals?

Posted by Marco Giustini (Member # 4544) on 03-19-2016, 03:38 AM:

quote:
Of course I am still waiting on that OS drive failure to happen. It simply hasn't,

Mark are we really discussing this again? Your 300 + drives will ALL fail at some point. It's not a possibility it's a fact.

Posted by Mike Blakesley (Member # 26) on 03-19-2016, 09:46 AM:

Might not matter though. We had to replace our server due to the media block failing. It was 5+ years old at the time and still running the original OS drive.

Posted by Steve Guttag (Member # 268) on 03-19-2016, 06:03 PM:

To be clear, Mike...didn't HAVE to replace your server. You chose to replace it due to the high cost of mediablock replacement. It would still have been cheaper to change the mediablock. I think it is absurd to can a server in just 5 years.

While we have reduced some DSS100/DSP100s to backup duty...I've never retired a DSS server due to inability to make it run reliably. I've retired a GDC SX2001 server due to Mediablock failure. I was about to do another one but GDC cut a deal on putting the server back under extended warranty knowing that it would need a mediablock.

So, at this point, retiring a server seems like a silly thing to do as it is still cheaper to fix what you have.

The problem with the OS from GDC is you HAVE to source it from GDC or you have to clone (bit-for-bit) your drive before it craters and it will be a snapshot of the day you clone it.

Posted by Mark Gulbrandsen (Member # 72) on 03-19-2016, 07:11 PM:

The server has not been canned and in fact has been converted into a rather nice TMS. This will allow Milke to switch over to Satellite down load which will save him money over what it costs to have hard drives shipped to his location. I had two media blocks die last summer, both were in booths that run rather warm in the summer. You me location lost its air conditioner no upstairs and that one died while they were awaiting the new units to be installed.

quote: Steve Guttag
The problem with the OS from GDC is you HAVE to source it from GDC or you have to clone (bit-for-bit) your drive before it craters and it will be a snapshot of the day you clone it.

So do you source your Dolby OS from Qube or something. That's all pretty irrelevant. OS's for this gear have to come from the respective manufacturer. And cloning it is actually an advantage because that carries the server serial number over to the new drive. Something you'd have to have GDC remote in to do with a fresh OS drive from them. They are very good about that stuff though.

In the next three or so years you are also likely to see full size servers vanish from manufacturer price lists. It's way too much needless expense for customers and manufacturers to endure and IMB type servers are the way it's going to go. They also save a tremendous amount of power, you drop 300 watts per screen!

Mark

Posted by Marco Giustini (Member # 4544) on 03-20-2016, 04:06 AM:

Dolby's OS is a generic install. It does not have to be tied to the specific machine. You can leave an install USB with the customer for emergencies and they'll be able to restore the server by sourcing new hdd's (in an emergency any hdd would work) and reinstalling the OS in 45 minutes. (With some minor network configuration involved of course).

Most of the time you won't need to worry about that as the raid rarely fails - unless you ignore the warnings.

Can a gdc customer do that at 10pm on a Saturday?

Posted by Steve Guttag (Member # 268) on 03-20-2016, 08:40 AM:

quote: Mark Gulbrandsen
So do you source your Dolby OS from Qube or something. That's all pretty irrelevant. OS's for this gear have to come from the respective manufacturer.

It is amazing how you jump on the wrong part of an argument. The problem isn't sourcing the OS from the manufacturer (duh) but that the way GDC does it, if your OS drive craters, you have to buy an OS drive from them rather than just loading the OS. The OS for Dolby comes on every version of software and any version you are entitled to has it...use your own hard drives...they even publish a lists of tested/known certified drives rather than try to keep it a big secret. Then again, I know people that claim most any drive will work in a Dolby DSS because of their hardware RAID...including mix/match of different brands/models/duties. GDC and Doremi have to use specific drives and even you admit GDC can't run when the drives are near capacity.

Lets face it...the people that bought your systems you are telling to replace the stuff already and the shows with the new stuff...don't look one bit better for their money.

As for server companies making full servers...I would agree that is going to diminish...not because it should be because the IMB/IMS type solution will become more and more attractive. But back on topic...they need sufficient storage space options. I think that will happen over time.

All of that said, I STILL prefer the separate server approach. In fact, the only thing I don't like about the HDSDI systems is the Enigma module itself that can do wonky things or fail for no good reason and OEMs (particularly Barco) refuse to have them repaired.

Posted by Mark Gulbrandsen (Member # 72) on 03-21-2016, 11:43 AM:

quote: Steve Guttag
if your OS drive craters, you have to buy an OS drive from them rather than just loading the OS.

And duh back to you sir. You only have to get it from GDC if you can't clone another drive... say if you are a single screen. The OS drive is divided into a number of partitions. You store other data on that same drive like back ups of older OS systems so you can immediately revert to one if need be. That is actually a nice feature I used twice or three times during the 8.xx update sequence. Anyway, I could still clone another good drive from another like GDC server in a multiplex, install in in the downed server, have GDC remote in to set the S/N then load in the setting that were saved on the booth computer and be back on the screen before you had your OS back on to a new set of drives. Cloning a full drive is quite easy and they are only 500 GB drives so go quickly. Even if we got both OS's back on at the same time you'd still be missing your media and spending another hour loading all that back on. I'd spend 30 seconds making up a new play list and bam hit the screen with the show. Of course I have never HAD to do this with a GDC, although I have tried cloning the drive and that is easy.

The age that many of these OS drives are now I will be cloning them as I go around and replacing them during minor tear downs of the older servers as well as replacing some of the fans.

Mark

Posted by Marco Giustini (Member # 4544) on 03-21-2016, 12:29 PM:

quote: Mark Gulbrandsen
Anyway, I could still clone another good drive from another like GDC server in a multiplex, install in in the downed server, have GDC remote in to set the S/N then load in the setting that were saved on the booth computer and be back on the screen before you had your OS back on to a new set of drives. Cloning a full drive is quite easy and they are only 500 GB drives so go quickly

Easy peasy.

So you have 2xUSB adaptors with you all the time to plug the HDD's into your computer and run, say, Clonezilla (*). Clearly the owner will be happy to lose another screen while you clone that HDD - oh wait, you never had such occurrence while other auditoria are playing, they are all idle for you to borrow the drive.
(*) super fast procedure as you will be cloning from USB to USB on the same computer. Even considering the max theoretical speed of a USB port (you see where I am going) of 480Mbit/s (60MB/s) that will take 142 minutes and during that time TWO auditoria will be unavailable.

Then GDC remotes in. Assuming GDC are available 24/7 (not an American company as far as I know) then that means that all your cinemas have remote access. No exceptions.

Obviously you have a stock of HDD's in your car and you are also available 24/7 to drive instantly (transporter?) to wherever is needed for that 5 minutes job.

If that happens to a single screen customer and your internet is down then I guess a GDC representative will be waiting for you at the cinema with an HDD?

Posted by Steve Guttag (Member # 268) on 03-21-2016, 12:55 PM:

I will say...I've never had a problem getting GDC to remote in in VERY short order. If I know they are going to need to remote in to set a serial number, I tend to give them a heads up on the time frame I anticipate needing them...they have ALWAYS been very attentive to that sort of thing.

Your other comments still standing. Though I guess it would be wise to plan on the big clone fest in the near future for our older servers.

Posted by Scott Norwood (Member # 30) on 03-21-2016, 02:02 PM:

The non-mirrored boot disk on the GDC is pretty much unforgiveable. The disks may be reliable and GDC support may be great, but a) disks are cheap, b) computers are really good at doing repetitive tasks like keeping a disk mirror in sync, and c) not every D-cinema system has Internet connectivity.

The "right" way to do this would be to have the OS on a mirrored pair of disks (which could be small) and the content on a RAID (honestly, a RAID 1 mirror would work fine with today's 4TB and larger disks, though a RAID 5 or RAID 6 would be better for the sort of capacity that Steve wants). Having the OS on the RAID is not ideal, but not really a problem as long as the read speed of the RAID is fast enough to serve the needs of both the OS and the content and the content can be easily re-loaded if the RAID ever needs to be re-built.

Posted by Adam Martin (Member # 641) on 03-21-2016, 02:19 PM:

Don't forget that it's a software RAID, so whether or not the machine can handle RAID-1 + RAID-5 + playback all at the same time is questionable. Based on the performance I've experienced, it can barely handle RAID-5 + playback, and forget it if it's rebuilding or choking on disk errors.

Posted by Scott Norwood (Member # 30) on 03-21-2016, 04:49 PM:

Mirroring is easy and safe to do in software, and should have very little performance penalty. RAID 5 or 6 is CPU-intensive on writes and isn't really safe to do in software, anyway (or, for that matter, in hardware without a battery-backed cache on the RAID card). Software RAID 5 or 6 should not be CPU-intensive on reads, though.

But, really, a 4TB mirror should be plenty sufficent for the average cinema, with respect to capacity and throughput. And, yes, I realize that 4TB drives did not exist when many of these servers were initially sold.

Posted by Marcel Birgelen (Member # 6801) on 03-21-2016, 05:08 PM:

For any non-mirrored setup (think RAID1 or RAID10), you definitely want double parity (e.g. RAID6) for those huge but rather slow multi-terabyte SATA disks.

Whether or not the GDC single OS-disk is an excusable SPOF depends on the actual failure rate, since there are many other SPOFs in those servers, it's not entirely fair to bring this forward as a design failure, without real numbers to back it up. Flash storage tends to be quite durable if it's primarily being used for read operations, actually often more durable than rotating rust.

What bugs me though, is the fact that you need them to remote in to the machine to fix their stuff, that shouldn't be a requirement. Just hook-up an USB stick and reload the OS and you should be good to go. If it's really just because of the serial number, then that's just a shoddy design. You could easily save a copy the serial number onto some ROM on the mainboard.

quote: Scott Norwood
Mirroring is easy and safe to do in software, and should have very little performance penalty. RAID 5 or 6 is CPU-intensive on writes and isn't really safe to do in software, anyway (or, for that matter, in hardware without a battery-backed cache on the RAID card). Software RAID 5 or 6 should not be CPU-intensive on reads, though.

There are software implementations of RAID schemes which are considered safer than most hardware implementations. ZFS is an interesting example, it takes data consistency to another level. Btrfs is another example, but it's not entirely there yet.

Also, with modern CPUs with something like 32 cores at your disposal, it's really a no-brainer to do both a rebuild of a large software RAID environment, ingest content and play it back at the same time, CPU-wise. The problem here is to squeeze this amount of performance out of a bunch of sluggish SATA spindles though, especially for small arrays. But this problem would be present in hardware RAID setups too.

A smart software RAID implementation could also dynamically throttle rebuild speeds if other higher priority operations, like playback are running. Not that this would be entirely impossible in a hardware RAID setup, but it's not trivial to make it happen.

Posted by Mark Gulbrandsen (Member # 72) on 03-21-2016, 06:45 PM:

Failure rate is really irrelevant in a GDC because you have both the option of ordering it with an extra OS drive already installed in the bay or taking a half hour and clear Ning your customers existing drive and either saving that image or burning a new drive that you can leave in the drive cage. All the servers I install are on sliders and it takes all of five minutes to slide it out pull the top cover and move two cables over to the backup drive. It just doesn't get any easier, faster or simpler.

Mark

Posted by Brad Miller (Member # 2) on 03-22-2016, 12:44 AM:

Doremi's design is actually simpler and easier. Just swap the OS chip.

Posted by Scott Norwood (Member # 30) on 03-22-2016, 05:21 AM:

quote: Marcel Birgelen
There are software implementations of RAID schemes which are considered safer than most hardware implementations. ZFS is an interesting example, it takes data consistency to another level.

Agreed that ZFS is great, but it is not appropriate for D-cinema or other real-time applications, where a guaranteed rate of throughput is needed. Dolby and Doremi (and possibly GDC...I don't know) use XFS, which specifically provides that functionality.

Posted by Mark Gulbrandsen (Member # 72) on 03-22-2016, 10:11 AM:

I agree with you guys on the mirrored raid thing. It's a no brainier, but it's a big cost factor for the manufacturer and consumer. Doremi and GDC both use boards with very similar imbedded Intel based software raid controllers. I have yet to see that controller reliably do both a mirrored pair and a raid 5 triple set. Also, the amount of work to combine those software raid functions into the manufacturers OS system is a lot of time and money. At that point in time a hardware raid card makes way more sense and can do that job very reliably.

Marcel, cloning a drive should really be done at installation and I use a separate computer to do that, no USB! Have I cloned everyone's drive... Certainly not, but I have done it and made tests and I will be cloning the older ones in the very near future. That old drive then will become a backup drive left in place.

And almost all of my maintainance work is done during shut down hours. I don't know many guys that work on plex's while they are running, unless a screen goes down or you are changing out equipment. The nice thing about working on just one brand of server is I also carry a spare GDC OS drive in my parts kit and spare Media Drive Sets that are already raided and ready to go should they ever be needed. So what's the big deal? Oh, and GDC does indeed throttle their raid depending on what's going down on. Playback takes precedence over just about anything else going on in their raid. I can remember when their raid was not throttled and playing back 4K content and ingesting content at the same time could make it skip.

Brad, same thing on the SX-3000.

Mark

Posted by Steve Guttag (Member # 268) on 03-22-2016, 10:55 AM:

I believe Marcel was referring to if you did have a theatre go down in a plex...the only way to make an OS drive is borrow one from another screen to make your clone.

Yes...they do throttle back...to the point of "why bother." With the DSS line...it is full steam ahead while showing the movie...ingest or transfers can happen at speed.

Posted by Marcel Birgelen (Member # 6801) on 03-22-2016, 10:56 AM:

quote: Scott Norwood
Agreed that ZFS is great, but it is not appropriate for D-cinema or other real-time applications, where a guaranteed rate of throughput is needed. Dolby and Doremi (and possibly GDC...I don't know) use XFS, which specifically provides that functionality.

ZFS might not be the best suited candidate for D-Cinema right now, but the guaranteed rate of troughput other filesystems claim really doesn't matter all that much here. ZFS will perform just fine if the hardware is fine and will give you the bandwidth you need. Keep in mind that none of the servers out there are running real-time Operating Systems, they're running Linux or Windows, which are as non-real-time as it could get. They work, because nothing else that could interfere is going on on the system and providing sufficient headroom as to not run into any problems during normal operations.

quote: Mark Gulbrandsen
I agree with you guys on the mirrored raid thing. It's a no brainier, but it's a big cost factor for the manufacturer and consumer. Doremi and GDC both use boards with very similar imbedded Intel based software raid controllers. I have yet to see that controller reliably do both a mirrored pair and a raid 5 triple set. Also, the amount of work to combine those software raid functions into the manufacturers OS system is a lot of time and money. At that point in time a hardware raid card makes way more sense and can do that job very reliably.

The future will most likely be software defined storage and not so much hardware. The embedded RAID you're referring to is a half-baked attempt at RAID and is what has given "software RAID" a bad name. Everything is better than those Frankenraid implementations, pushing the hard work into the device driver to save a few bucks on dedicated hardware. Doremi doesn't use this kind of RAID though, it uses Linux software RAID. Based on my experiences with that, I prefer proper hardware RAID instead of Linux software RAID.

quote: Mark Gulbrandsen
Marcel, cloning a drive should really be done at installation and I use a separate computer to do that, no USB! Have I cloned everyone's drive... Certainly not, but I have done it and made tests and I will be cloning the older ones in the very near future. That old drive then will become a backup drive left in place.

The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

quote: Mark Gulbrandsen
And almost all of my maintainance work is done during shut down hours. I don't know many guys that work on plex's while they are running, unless a screen goes down or you are changing out equipment.

I usually prefer to carry out disruptive maintenance on Friday night... Hey, who doesn't like a little excitement now and then?

Posted by Marco Giustini (Member # 4544) on 03-22-2016, 02:32 PM:

quote: Marcel Birgelen
The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

Which is what Dolby do [Smile]

Posted by Mark Gulbrandsen (Member # 72) on 03-22-2016, 02:32 PM:

quote: Marcel Birgelen
The point is: You shouldn't have to clone anything, you should just have to hookup the install media and reload the system from there, restore a previously back-upped configuration and be golden.

You can't because of all the partitions involved. That's why I carry a spare drive in the truck. Changing the drive is much quicker anyway if your server is on slide out rails. Many of my customers could even do that drive change in a pinch. I really don't want to drive 600 mils just to change an OS drive but certainly would if my customer felt he or she couldn't.

I have several really old GDC's that will be getting overhauls before the summer films hit. Complete clean up, new fans, new filters, new drives, etc.

Will be doing the same to some of the older S-2 NEC's. I have several out there with what seems like a half billion hours on them. Some five and six year old projectors will be partly dismantled and cleaned. Then several fans replaced, new back up ICP battery and fan also installed.

No point in waiting for the parts to die and letting them have down time...

Mark

Posted by Marco Giustini (Member # 4544) on 03-22-2016, 07:03 PM:

ok. While you drive umpteen miles and clone/replace those drives (in 5 minutes) I will download an 'INSTALL' version of my Dolby software. Then I'll watch some TV. Let me know if you need a hand! [Smile]

(yes, I do clean the servers too and needless to say that RAID arrays seldom fail - but single drives do. If the customer does not ignore the warnings and quickly replace the failed drives, chances are that I will never need that INSTALL media.)