Film-Tech Forum ARCHIVE: DSS200 sudden shut-off

This is topic DSS200 sudden shut-off in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=003449

Posted by Carsten Kurz (Member # 5396) on 10-02-2018, 06:30 AM:

A fellow projectionist wrote to me about one of his DSS200 shutting off with no appearent reason. Unfortunately, often enough to be an issue, but not often enough to find a pattern. In all instances, no one was close to the machine to actually see it happening. As far as I see, the log analyzer doesn't locate these shutdowns (so far we are not even sure wether they are real shutdowns or hard shut offs). Any idea where to look in the logs to find something usable?
Did anyone experience this on a DSS200 before?
For now, I suggested to put that machine on a UPS.

- Carsten

Posted by Frank Cox (Member # 6258) on 10-02-2018, 12:08 PM:

I have no experience with this particular machine, but with general purpose computers a sudden shutdown without any warning is a sign of a bad power supply.

Posted by Greg Routenburg (Member # 1742) on 10-02-2018, 12:10 PM:

I've seem that behaviour on a DSS100 with a motherboard that was getting flaky. I've also seen the power supply enclosure cause these sorts of problems. It's always been a tricky issue to troubleshoot because you need to swap a part and then wait to see if it recurs. If it only happens once or twice a week, that's a long troubleshooting cycle.

Posted by Dave Macaulay (Member # 813) on 10-02-2018, 01:10 PM:

The UPS is a good idea, eliminating flaky power as the cause. It only takes a power loss for well under a second to shut it down.
Since the DSS is normally shut down by pulling the power it isn't going to log that as a problem event.
I would pull and swap the PSUs, really just to clean the contacts. With redundant supplies weird things can happen with bad connections or when one is flaky: make sure the server runs OK with both when used alone.
Unexplained shutdowns are usually the PSU but PCs with the bad capacitors from a few years ago also do that. I don't think the DSS boards have this problem but it is worth opening it up, lifting the media block out of the way, and looking for bulging or leaking capacitors.

Posted by Jim Cassedy (Member # 4115) on 10-02-2018, 02:31 PM:

I've personally never experienced a power supply problem with any of the
many DSS200's I've worked with, although that doesn't say one can't happen.

The DSS200 has redundant power supplies. If one goes down, it will operate
OK on one, although you will get a constant annoying alert beep from the
somewhere on the motherboard to make you aware of the problem.

As Dave said- - pull & swap the supplies, and clean the contacts a bit.

The DSS200 will shut down due to power-glitches or minute power outages,
so the next thing I would suspect is your actual AC power source.
As others have suggested - - a UPS will help eliminate that possibility.
(And even if it does turn out to be a problem with the DSS power supplies,
a UPS is a good investment and it really should be on one anyway.

Good Luck!

Posted by Mark Gulbrandsen (Member # 72) on 10-02-2018, 03:40 PM:

I'd start by checking the logs. Power supply issue probably...

Mark

Posted by Leo Enticknap (Member # 534) on 10-02-2018, 06:06 PM:

As Dave points out, if the motherboard suddenly loses power and dies, no log entry will be created to record that fact. All that you'll see are the bootup entries that were created when the server was restarted.

quote: Dave Macaulay
With redundant supplies weird things can happen with bad connections or when one is flaky: make sure the server runs OK with both when used alone.

I found this out through trial and error with a Doremi Showvault recently. The fault reported was that it was reluctant to start, and that the button had to be pressed three or four times before it eventually did (this was a multi-use venue that only showed movies 2-3 times a week, and shut down the server when it wasn't in use).

By the time I got there, it wouldn't start at all with both PSUs in place. The LED on one was totally out, and the other was flickering amber. With the former completely out of the chassis, and the latter moved into the other slot, the server booted. But it wouldn't with any other configuration of the PSUs.

Needless to say, we replaced both of them, but that temporary arrangement was enough to get them through the wait for the new ones to arrive.

Posted by Ioannis Syrogiannis (Member # 3235) on 10-03-2018, 04:13 AM:

Should we assume that by shutting off, we mean rebooting, given that it is a DSS200 and when there is power supply, it is on? (Except for the cases that the BIOS settings has gone bad.)

UPS is always a good idea, PSU is always a good guess.
If none of the above work for you, try checking the memory modules.
And if your fellow projectionist have time, propose to him to burn a live CD with MemTest86, run the short test and afterwards let it run the extended one.
We have to acknowledge here that if the issue (shutting off) appears in the midst of the test, it could be some wasted time.

In any case, I've seen that in a DSS220 and chances are the problem may occur in a DSS200 also.

Posted by Leo Enticknap (Member # 534) on 10-03-2018, 05:11 AM:

Very good suggestion - testing the RAM. I've experienced several PCs (either mine, or those of co-workers, friends and relatives) throw completely random BSODs over the years, and the fault turned out to be with the RAM boards. So much so, in fact, that if someone comes to me complaining about this and I'm satisfied that the occurrences are truly random (e.g. not when a specific program was run, driver starts to be used, etc.), then the RAM is the first thing I'd suspect and investigate.

Usually, either heat cycling had compromised the connection (in one case I saw, one of the plastic clamps at the end of the slot had actually popped open), or a RAM board had actually gone bad, which MemTest86 revealed.

Posted by Marco Giustini (Member # 4544) on 10-03-2018, 11:44 AM:

kind of weird as the DSS200 is designed to power on when power is applied - hence if the PSU caused the unit to shut off, it should then power up again shortly afterwards.
If you cannot see signs of a software shutdown on the server on the logs, then the server is probably shutting off abruptly and that should lead to the PSU as it's been said.

Posted by Leo Enticknap (Member # 534) on 10-03-2018, 02:26 PM:

I guess the thinking behind that is that if there is a power outage, the server will come back up when the power does, without human intervention being needed.

Posted by Carsten Kurz (Member # 5396) on 10-08-2018, 11:47 AM:

Seems this is getting more annoying. Before, they noticed their servers shut down during nights, unnoticed. Now, they are having show interruptions. Servers shutt off, and do not restart automatically. After they push the front-button, the server starts normally, and the show is able to proceed.

They just had a clean, new hard disc set install and current software install, but it seems the problem remains. I told them they should try swapping the power supplies, or, supplying only one of them with a cable alternately, to find out wether there is a pattern. Is there a way to power a DSS200 from a plain vanilla PC power supply (non-redundant, of course)?

- Carsten

Posted by Ioannis Syrogiannis (Member # 3235) on 10-08-2018, 05:05 PM:

I can confirm that it can be done to power a DSS100.
One will have to remove the connections from the PSUs backplane to the motherboard and peripherals and replace them with the ones from the "vanilla PC PSU". That's it.
Of course, in order for that to happen, the top lid of the server will have to remain open. The common PC PSU does not fit into it.
I had that done once, but only for troubleshooting and testing purposes and not in a cinema environment.
At that particular period, with the PSUs in question being unavailable from Dolby, it was the only way to power up the server, and all worked fine. If I remember well, there weren't even any indications to point the difference on logs.

I can't see why one couldn't do the same on a DSS200. The difference here, of course, is that you have the 2x2ATX (yellow/black/12VDC) power supply to the cat.862 that I didn't have in the DSS100 due to DSP100 media block. Also, the 8pin (yellow/black/12VDC) connection just by the main motherboard one. So, I presume that the PSU should offer a number of connection modules to fill the powering needs.

Posted by Carsten Kurz (Member # 5396) on 10-09-2018, 10:39 AM:

There must be a power distribution board between the two power supplies and server mainboard/CAT862 and RAID backplane. I was never able to get a closer look at that one.

- Carsten

Posted by Ioannis Syrogiannis (Member # 3235) on 10-09-2018, 11:03 AM:

What I call the "PSUs backplane". That's actually the board the PSUs are clipping when full in the bays, makes the power supply redundant and has all the wires, distributing power where needed.
It isn't prone to issues, but when it has one, it must be the more time-consuming to change.

P.S. I wouldn't know for how long the server would work in the proper temperature, if the lid is open and the airflow created from the fans gets minimized. Its a factor worth considering here.

Posted by Stephan Shelley (Member # 8587) on 10-10-2018, 05:24 PM:

Is it possible the motherboard battery need replacing? Make sure after you do to set it to always power on after power failure. I think it defaults to last state.