Film-Tech Forum ARCHIVE: What causes the screen to go green and the server to crash?

This is topic What causes the screen to go green and the server to crash? in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=001902

Posted by Amanda Mundin (Member # 3386) on 05-15-2014, 04:51 PM:

We have an NEC NC800C and Doremi DCP-2000, they're both around seven and a half years old now, tonight while running Noah the screen went green, not a green cast over the image but a complete bright solid green and the server crashed, the server was completely unresponsive so we had to turn off & back on using the power switch, when it rebooted after a minute or two it started itself automatically where the film left off. The film has been running fine all week without problems, the projector and server have been running fine too, is this likely to be something on its way out or just a random crash without reason?

Posted by Marcel Birgelen (Member # 6801) on 05-15-2014, 05:07 PM:

There's nothing obvious in the diagnostics?

Maybe you just ran into a once-in-so-many bugs. But the first thing I would check is, after seven and a half years, if all the fans are still working and nothing is overheating. Other easily fixable contenders are defective memory banks.

Posted by Marco Giustini (Member # 4544) on 05-15-2014, 05:40 PM:

Have you had your server serviced regularly? If so, then you should check the logs, looks like the decoder crashed. It could be content or hardware.
If the server has not been updated and cleaned regularly, call an engineer [Smile]

Which software are you running?

Posted by Joe Elliott (Member # 4006) on 05-15-2014, 09:52 PM:

Have you tried turning it on and back off again?( from IT Crowd)

Seriously though, how long had it been since the server had been rebooted? I see many odd and rare errors on servers that I find have not been rebooted in 6 months to a year.

Posted by Monte L Fullmer (Member # 2797) on 05-15-2014, 11:23 PM:

Usually, it's a common practice to do full server/projector shutdowns once every week to keep stale memory from happening.

Posted by Mark Gulbrandsen (Member # 72) on 05-16-2014, 01:03 PM:

Yea, and when that stale memory begins getting moldy... well... them you'll have to put the projector in the garbage! Phew!

Posted by Amanda Mundin (Member # 3386) on 05-17-2014, 04:07 PM:

Thanks for all the advice. Nothing obvious that we can see in Diagnostic Tool. We haven't had the server serviced in a good while, in the past problems have occurred shortly after the Server has been out and moved around for Servicing etc. so we tend to think while everything's working fine it's best left alone. Apparently it has crashed once or twice in recent months when not running films and needed rebooting. We're currently running 2.2.3-0 software.

Posted by Carsten Kurz (Member # 5396) on 05-17-2014, 05:41 PM:

That's a pretty old software version. And a .0, which mostly is not a good idea to keep longer than necessary ;-)

As the NC800 is a series I projector, there are potential issues when updating the Doremi to DCI compliant current firmware, you should get an experienced tech in to do the upgrade, he could give the server a cleanup etc. as well.

- Carsten

Posted by Marco Giustini (Member # 4544) on 05-18-2014, 02:55 AM:

What Carsten said.

And what firmware are you running?

Posted by Dave Macaulay (Member # 813) on 05-18-2014, 07:49 AM:

In diags/hardware check the cpu temperature and that the cpu fan is turning OK (~2500 rpm). Also worthwhile to open the server and clean the fan and heatsink (air blower can or compressor, small toothbrush). What you describe is what I've seen when the cpu overheats and turns itself off. They will usually run ok with just a stalled cpu fan because the other fans move a lot of air around in there, but a dirt clogged heatsink and slow fan can cause shutdown when it's working hard playing a movie - they don't generally shut down when just idling.
After 7 years you may find a severely clogged heatsink and very dirty fans. Just be very careful with the black block Dolfin card. And the Dolfin battery most likely needs be replaced by now.
While it's open, reseat the memory sticks and the SATA connectors.

Posted by Leo Enticknap (Member # 534) on 05-18-2014, 11:38 AM:

Might also be worth replacing the thermal grease between the CPU and the heatsink, too. It does go stale over time and lose its conductive ability. I find with PCs that it's a good idea to have a good clean inside the case (while compressed air is good for nooks and crannies, my usual plan of attack is a soft artists' paintbrush followed by a hoover) every six months or so, and then to replace the fans and grease and clean the contacts of the memory boards, terminals and PCI cards (with a lint-free cloth and rubbbing alcohol) every couple of years.

PC case fans at least get gradually noisier over time as the bearings wear, but it's boiling the frog syndrome - you don't really notice until you replace them. Maybe the ones in DCP servers are of a higher grade (e.g. with better bearings), but for such a mission critical application as that I'd want to replace them reguarly as an insurance policy against failure, anyway.

Posted by Marco Giustini (Member # 4544) on 05-18-2014, 05:35 PM:

Also remember that brushes and hoovers are good friends with static. Professional cleaning companies have special static-free brushes and hoovers.

Most of the time nothing nasty will happen, I know. But every now and then you just kill the electronics.

Posted by Kevin Markwick (Member # 2698) on 05-18-2014, 05:51 PM:

We had the same issue some years ago with the same DSN combination, AA never really got to the bottom of it and after the DSN period was up I insisted on a swap out before I bought the kit. We've had no further problems thankfully.

My only real advice would be to have the equipment given a good service and the software updated on the server and the PJ, if that's possible.

If you've already done that then I'm sorry I can't help further!

The lamps on these things are becoming prohibitively expensive now. We are ditching ours very soon.

Posted by Leo Enticknap (Member # 534) on 05-18-2014, 10:49 PM:

Marco - point taken; many thanks. My dislike of air duster spray is that you're simply moving the crud to somewhere nearby, not getting rid of it. Antistatic brushes and non-conductive vacuum nozzles are a sensible precaution, though.

Posted by Marco Giustini (Member # 4544) on 05-19-2014, 01:04 AM:

When I use air duster i always make sure I put the end of the vacuum cleaner nearby so most of the dust is sucked away. Not sure that removes static risk though. Not saying I have an antistatic kit, mine was just a word of caution to recommend professionals to open a server: if they break it, they pay for it!

Posted by Amanda Mundin (Member # 3386) on 05-21-2014, 01:01 PM:

We're running 21.2k Firmware. Thanks for all the advice we'll look at getting it serviced.

Posted by Antti Nayha (Member # 4980) on 05-22-2014, 05:23 AM:

There's at least 2.13f which was out in 2012, I don't know if there's been further updates since.

Posted by Vegard Olsen (Member # 8102) on 05-25-2014, 04:04 AM:

When you say unresponsive, is it completely frozen, or just slow/not opening diagnostics etc? If the latter the problem could be with the security manager and/or dolphin. Have a look in /doremi/logs/kern.log and see if there's lots of time outs and/or decoder crashes.

Would regardless advise to upgrade all components of the Doremi. Software, security manager and firmware. Use the -full version of the security manager (sbc405) as the -0 version doesn't upgrade everything.

Posted by Dave Macaulay (Member # 813) on 05-25-2014, 08:40 AM:

With a series 2 projector, current Doremi server versions (non-IMB) are S/W 2.6.4, F/W 21.03s, SM 2.0.17, and uboot 2.0.7; you should step through s/w v2.4.4 on the way to 2.6.4 if your installed version is older.
All except s/w require DLMs to load. You need to do the uboot as well as the main -full SM, the SM dlm has a uboot license but the ingest doesn't update it. Run a set of diags through the doremitechno.org log analyzer after updating (open to anyone, look under "tools"). Ignore out of date "errors", it will show "no ntp server found" and such from months ago.
Make sure the projector is on current versions as well. I'm not sure what that is now for the 800; call NEC support and find out. If the installed version is really old you have to step through several versions. Ballantyne/Strong support was not up to date on this procedure when I had to do a NC2000 from v2.1, I recommend asking NEC directly about it.

Posted by Marco Giustini (Member # 4544) on 05-25-2014, 04:39 PM:

If you ever decided to move to DCI firmware, you MUST check on the logs if your NEC can support DCASM. Failure to do so will result in a non working system (you cannot downgrade DCI software).

Posted by Antti Nayha (Member # 4980) on 05-30-2014, 11:53 AM:

quote: Antti Nayha
There's at least 2.13f which was out in 2012, I don't know if there's been further updates since.

Just for the record: Reading through this thread again, I realized that I was talking about NEC NC800C software when everyone else was obviously discussing Doremi firmware versions. [Smile]

Please disregard.