This is topic Leave the machine on or off at the end of the night? in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.


To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=000797

---

Posted by Frank Cox (Member # 6258) on 04-29-2011, 11:30 PM:
 
Do you leave your digital projector and server turned on overnight, or do you switch the whole works off at the end of the day?

It's always been my habit to switch the power to everything in my projection room off (all of the power hookups in that room go through a bank of switches on the wall) so with the digital setup I'm rebooting the whole works every day. Alternatively I could just leave the server and/or projector on standby. What do you recommend?
 

---

Posted by Kurt Zupin (Member # 2751) on 04-30-2011, 01:15 AM:
 
We shut down the projectors every night but we leave or servers on. They take a while to boot up and never had any issue with leaving them on.
 

---

Posted by Tony Bandiera Jr (Member # 2365) on 04-30-2011, 11:53 AM:
 
IF and ONLY IF your servers are on a quality UPS (Tripp-Lite makes a really good one for under $200) it is best to leave the servers on 24/7.

If your servers are not on a UPS, then you are stupid. There I said it and don't care.

As for the projector my instinct says to leave it on "Standby" rather than a hard power-down....but I would like to hear from the Grand Poo-Bah (Brad), Steve G and Sam C since they have more digital install experience than I do.
 

---

Posted by Steve Guttag (Member # 268) on 04-30-2011, 02:15 PM:
 
My opinion on the matter is...

Anything that moves air is also a "dirt-pump" too. So how clean the room is will greatly affect how filthy the equipment gets from the act of just leaving it on. This is particularly true of the projectors. I recently cleaned a projector that was in operation only 6-months. And despite all of the filtering of the projector, I was able to improve the light by over 6.4% by cleaning JUST the Dolby filter wheel! If you add up the cleaning of the other glass/mirror items...the light improvement was into the double-digits. The booth was not particularly filthy at all, either. The back of the lens proved to be a BIG dust/dirt collector, just like with film. These machines were turned off each night. One could expect twice as much dirt had they been left on the extra 12-hours just to make noise a night.

On the server side of things, you don't have an optical element BUT I have found servers absolutely clogged full of dirt too...which compromises the ability to remove the heat, which will lead to failure. On stupid thing most companies do is have their air flow to be from the Front to the Back of the equipment that is racked...which is dumb. The air INSIDE of the rack is controllable. QSC got this right, they pump from rear to front. Thus, one can put filters on INTAKE blowers to pressurize the rack (or pedestal) and let the equipment suck on nice filtered (and cool) air, then spit it out as warmer air for the booth's HVAC to deal with (heat load). The other drawback to the preset designs is that each piece of equipment blows its high air into the rack to help heat up the other equipment...just plain dumb.

That is the dirt side to the equation. Then there is the operational side. I've found that it is best to have at least a regular schedule of rebooting the equipment. It really seems to clear out problems before they start showing themselves. A reboot will often cause the equipment to purge memory, check the time/date and start things up fresh. So, even if you don't want to turn things off every night, I suggest you reboot everything at least once a week.

At present, all of my customers turn the DCinema equipment off nightly that are NOT part of a NOC. Most NOCs will require it to remain on 24-hours so they can monitor it and possibly handle updates during the down-hours. As such, I recommend having a regular schedule where you merely reboot the equipment periodically...once a week is sufficient and in fact, once a month would probably be okay too but harder to keep track.

-Steve
 

---

Posted by Mike Blakesley (Member # 26) on 05-01-2011, 12:59 PM:
 
I've wondered about this too. On the one hand, I've heard that the hardest thing on a computer is the boot-up. And, one of the first things to go is the switches. But I can see the dirt-accumulation issues too. We only have a max of two shows a day usually, so it seems to make more sense to turn it off, but still... there probably is no hard and fast good answer to this one, I guess. You're going to compromise something either way.
 

---

Posted by Steve Guttag (Member # 268) on 05-01-2011, 04:16 PM:
 
With computers, I generally run 24-hours...I have had HDD get "sticktion" from being off. Even with computers, I find I have to reboot them, periodically to clear out stuff and get them running back at peak. There is often a way to do a software reboot too that will accomplish most of the same thing as a hardware power-cycle.
-Steve
 

---

Posted by Scott Norwood (Member # 30) on 05-01-2011, 04:58 PM:
 
I'm not saying that Steve is wrong, but I would argue that any piece of hardware or software that requires regular reboots to avoid issues is broken as designed and that the memory leaks (or other issues) need to be fixed by the manufacturer, not through a silly workaround by the end-user.

I have had various servers and pieces of network hardware (not in the cinema industry) that had uptimes of 1000+ days without issues. Eventually, hardware or OS upgrades and/or power failures forced reboots, though.

(So, yes, computers in general--especially those with spinning disks (as opposed to SSDs)--are best kept running 24/7. I am not qualified to say if this applies to D-cinema hardware or not, however.)
 

---

Posted by Terrence Meiczinger (Member # 5059) on 05-01-2011, 10:17 PM:
 
Power consumption is something to consider as well. If you leave everything on in a rack and it is using 1500w (just a guess) @ $.15/kwh (New England) thats gonna be $1000 per rack a year. That adds up for larger theaters. There would have to be real statistical evidence that power cycling reduces the equipment's life expectancy by a significant amount and increases the failure rate.

I've done power cycling and thermal tests for enterprise network routers. We'd power cycle them 100,000+ times and vary environmental temperatures. The failure rates were not significantly different from units in the field under normal operating conditions.
 

---

Posted by Mike Blakesley (Member # 26) on 05-01-2011, 10:59 PM:
 

quote: Scott Norwood

---

I'm not saying that Steve is wrong, but I would argue that any piece of hardware or software that requires regular reboots to avoid issues is broken as designed

---

quote: Tony Bandiera Jr

---

If your servers are not on a UPS, then you are stupid. There I said it and don't care.

---

Right on Tony ! Ditto for the projector's electronics section and the local network switch. You should however have at least a 1KVA UPS or larger. I typically use a 1.6kva Tripp Lite that runs about $800.00 on line. It'll run the server, switch and projector electronics for about 15 min. before it shuts down. Give plenty of time to go around a large multiplex and do proper power downs on the gear. ALso filters out all the nasty stuff that might come in on the AC line.

Leaving a projector in standby and your server powered on is actually what you must do if you are monitored for VPF's or hooked to a contracted NOC. I have spent hours cleaning grease and dirt from digital projector's lamphouses that have auto align for the lamp or no filtering on the lamphouse air intake. So having a projector whose entire air intake is filtered is a big advantage. The projector will remain more or less spotless inside but you will for sure need to change filters AT the perscribed intervals. Ditto for servers... if your server has air intake filters as the GDC does then leave it on if you're running 10 to 12 hours a day and put your projector in standby mode. Change the filters at the prescribed interval. Installing your server on rack slides makes changing filters an easy thing to do.

If you run just a few hours a day then I reccomend shutting it all down for the night as you are just wasting power. All this gear with the exception of Qube(DATASAT) runs on Linux and a daily boot up or boot down isn't going to hurt anything software related. As for the drives... failures are rare but can happen weather you leave them on or off. The MTBF point in the HD's life span will be reached much quicker if it is left on all the time. So IMHO you basically have the same chance for a drive failure leaving it running or powering it down. SATA drives are cheap and you can have spares all ready to go in the server or if your server allows you can play back directly off the DCP anyway. A SSD can always be installed if the OS drive fails since it is a smaller size drive and they are way more reliable. SSD's are just not cost effective for a large raid 5 at this point in time.

As for having to reboot to clear issues... preojectors and servers are getting better and the software that runs them is more refined. Reboots are not a thing of the past but are generally required far less frequent then in the early days of digital.

Mark
 

---

Posted by Scott Norwood (Member # 30) on 05-02-2011, 12:14 PM:
 
As this article indicates, MTBF is a load of crap as applied to hard disks. In real life, they fail on a bathtub-shaped curve: most will either fail early or will last for a number of years. The key is to keep data on redundant disk arrays, keep spare disks on the shelf, and keep good backups of any valuable data.

And I agree completely that a good UPS is important for any sort of critical computer or network hardware.
 

---

Posted by Mark Gulbrandsen (Member # 72) on 05-02-2011, 02:10 PM:
 
Scott,
Thats hilarious that Seagate claims 1.5 million hours for a Cheeta SATA Drive, SCSIdroves would typically last longer though because the build quality is better. More like 150,000 hours is realistic for any SATA drive. I think the manufacturers MTBF is always averaged with disks always left running. Of course they are going to want to show the best spec they can. The bathtub shape comes into play when those testing the drives also take into account that some are turned off and on on a daily or more often basis. From my experience with SATA drives leaving them on or off is meaningless as to their actual life span. It's going to vary all over the place no matter what. With SCSI drives leaving them always running guarantees you a much longer life span and they are just flat out better built. I have some 146 gb scsi drives here that are in excess of 8 years old and they still run quietly and perfectly normal. Simple math says they have about 75,000 hours on them after running for 8 continous years. The main thing is not to subject any running drives to any vibration at all, vibration is what will kill them really fast. No arguement on running RAID levels for redundancy except a number of the common servers out there do not make that an option for the OS drive. So in that case a SSD would be a wiser choice as they are uber reliable and for read only which they would be doing in the case of an OS drive it would also reduce boot up time dramatically.

Mark
 

---

Posted by Bill Enos (Member # 440) on 05-02-2011, 07:32 PM:
 
And what is the absolutely first thing the service tech does---turns it off and back on, even if smoke is billowing
from it.
 

---

Posted by Olivier Lemaire (Member # 5579) on 05-02-2011, 11:33 PM:
 
Out there (EMEA), most of the servers I can see (roughtly 920 deployed at 20110503) run 24x7, of course secured (UPS and auto-extinction procedure in case of UPS low battery).

This have advantages: we can upgrade freely, and so keep a running fleet pretty up-to-date.
This have drawback: you sometimes have to call the operator for him to reboot it's Doremi unit to have a firmware patch applied as it's unit is running 155 days uptime...
 

---

Posted by Chris Slycord (Member # 4239) on 05-02-2011, 11:58 PM:
 
With the memory leak issues and what-not, since all these major server manufacturers have been using linux I wonder why they haven't built in support for kexec, which would give them the ability to "restart" the OS without physically rebooting the hardware. It's a part of the linux kernel that lets you automatically load any kernel of your choice with any settings you want that would normally be thrown in your bootloader.

So all the HW keeps running, which makes them not all that prone to failure from being shut down all the time while having all parts of the OS reloaded, which avoids all the leak issues. And this speeds up the reboot cycle as you avoid having to wait for it to load the BIOS and such.
 

---

Posted by Frank Cox (Member # 6258) on 05-03-2011, 12:22 AM:
 
Generally speaking, it's not the kernel that's at fault with regard to such things as memory leaks. I don't know enough about how cinema servers work to be able to provide citations, but the standard Linux kernel (defined as Linus Torvald's official kernel found at kernel.org) is pretty solid unless someone who doesn't know what he's doing has added some wonky proprietary drivers to it. That's not beyond the realm of possibility, though most programs run in userspace so they don't and can't affect the kernel. The fundamental design of Linux as a multi-user operating system means that as a user the worst you can do is crash your own session but the rest of the system should keep on ticking.

I was somewhat surprised to see that my cinema server runs Fedora Core 5; that distribution is quite out-of-date as it was originally released in 2006 and official support (bugfixes, etc) are generally available for only about 13 months for each Fedora release. The current version of Fedora available today is 14 and 15 will be released later this month, so as you can see Fedora is a fast-moving distribution that's intended to include the cutting-edge of Linux technology -- it's not generally recommended for situations where you require long-term operation such as servers. It's more-or-less a test-bed for Red Hat Enterprise Linux.

I personally use Centos (a free version of Red Hat Enterprise Linux). RHEL and its clones (Centos and Scientific Linux being the most well-known) are intended for use in data centers and the like and would, in my opinion, be a better fit for use in applications like cinema servers than Fedora. The folks who have to support these things could then take advantage of bug fixes and enhancements provided by Red Hat instead of having to roll-their-own. But what do I know...
 

---

Posted by Mark Gulbrandsen (Member # 72) on 05-03-2011, 08:06 AM:
 

quote: Chris Slycord

---

I wonder why they haven't built in support for kexec, which would give them the ability to "restart" the OS without physically rebooting the hardware.

---

quote: Frank Cox

---

I was somewhat surprised to see that my cinema server runs Fedora Core 5; that distribution is quite out-of-date as it was originally released in 2006 and official support (bugfixes, etc) are generally available for only about 13 months for each Fedora release.

---

quote: Mark Gulbrandsen

---

For now all my sites that are non vpf shut down completely every night.

---