Film-Tech Forum ARCHIVE: DOLBY DSS/DSP 100 lost time/date

This is topic DOLBY DSS/DSP 100 lost time/date in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=003606

Posted by Allan Barnes (Member # 5178) on 02-28-2019, 12:05 AM:

I think I can just replace the DSS battery... But how to enter the correct date & time? NOW if needed can the DSP100 four (4) hardwired batteries can be replaced... my most excellent service guy (Dave) says Dolby isn't doing this anymore. CAN SOMEONE RECOMMEND A SOLUTION or A TECH FIRM that has done this repair before. ADVICE & ABUSE WELCOME.

Posted by Marco Giustini (Member # 4544) on 02-28-2019, 04:15 AM:

I don't understand what the problem is. Have you lost the date? Does it reset to 00:00 every time you power down your system?
You probably just need to connect the system to an NTP server - you can also get into the BIOS and change the date yourself.

Be aware that the secure clock - in the mediablock - cannot be adjusted more than a few minutes. If memory serves, Dolby servers will base the show start on the Secure clock while the displayed clock is the system clock which is a software clock based on the hardware clock from the MB! [Smile]

Also, if an NTP sever is unavailable, the system will sync its software clock to the mediablock at boot.
The tricky part is to have all those clocks synchronised! Step one is definitely an NTP but a *single* NTP server in the list will seldom work (the server will likely blacklist you very soon if the system keeps polling time). It's a long subject but the NTP server is indeed a start.

I believe you can safely replace the coin battery on the DSS motherboard. When it comes to the mediablock, be VERY cautious as if power is lost you will lose the certificates making the mediablock a big doorstop - but I don't think the DSS100 mediablock features a battery on the mediablock?

Posted by Carsten Kurz (Member # 5396) on 02-28-2019, 04:45 AM:

Replacing the coin battery on the mainboard in the DSS100 should be the first step indeed, followed by a clock/date set in the BIOS. That's all standard PC procedures, means, if you are not keen to do it, any 'PC guy' will be able to perform it.

Then boot and see what happens.

I think I have never read anything about the secure clock/battery in a DSP100?!

- Carsten

Posted by Steve Guttag (Member # 268) on 02-28-2019, 07:43 AM:

The CR2032 battery on the motherboard is definitely changeable.

When the server is booting up, tap on the "DEL" key until you get to the BIOS screen. Set the date and time to UTC time, Hit F10 to save/exit and then let it boot up.

The problem with changing batteries on the DSP100 is losing the certificate. Dolby won't do it anymore and I believe the 3rd party firm that was repairing them a few years ago has also stopped. The newest DSP100 out there is now about 10 years old with most 10-15 years old now.

Changing the time on the DSP100 isn't hard...it is done from the UI by setting the "secure clock". Depending on your software version, you can move it a little or a lot. It had a good clock so one would think you could get it set reasonably accurately still.

Posted by Sean McKinnon (Member # 612) on 02-28-2019, 12:20 PM:

I don't know of this applies to the DSS100 but I always power on any server or IMB for 30 minutes before changing the battery. I have seen certificates lost after a server has been off all night and the battery was removed without powering on first.

Posted by Marco Giustini (Member # 4544) on 02-28-2019, 12:41 PM:

Sean, it depends on the server. Doremi Dolphin card is like that: server must be powered up first, then powered down and then you have very little time to swap the mattery (2 minutes?). Powering them up charges a capacitor which will keep the voltage up while the battery is missing.
Other servers will need a backup battery installed before the main battery is removed.
Other servers won't require batteries - such as cat.862 if not mistaken.

Posted by Carsten Kurz (Member # 5396) on 02-28-2019, 12:51 PM:

Are there any specs on the DSP100 battery? Photos? While the DS100 is severly limited as far as supported frame rates go, I would still pity someone who loses a server (or working backup) just because a 1US$ item. That's an interesting question which of the current parts needing batteries may have a chance to have them replaced in operation or by supplying a temporary backup (as GDC suggests to do).

I usually replace Dolphin batteries WHILE the server is running. It's a bit fiddly, but, with plastic or isolated pliers, and with a plastic sheet beneath, it's more or less a safe procedure. I have heard of Dolphin cards losing their cert even when the server had been powered up long enough before the change.

- Carsten

Posted by Steve Guttag (Member # 268) on 02-28-2019, 12:59 PM:

Wait until the ICP batteries start to go! Then wait to see how people respond to Barco's position of buy a new ICP module (unless one has an extended warranty...which may not be of any use since I think they top out after 10-years). There ought to be a depot for getting things like Enigmas and ICPs repaired for common/known failures (Enigmas have died just due to a failed cover switch).

Posted by Carsten Kurz (Member # 5396) on 02-28-2019, 02:15 PM:

I have seen pictures of ICPs with a socketed BR2330 cell + a soldered backup cell. Although I understand there are some ICP revisions with different backup cell types - some seem to have a rechargable Lithium cell with a strictly limited life, while others seem to have supercaps - I would assume the supercaps to live longer. Both should be easy to replace, even if some soldering is necessary.

Don't you think it should be possible to replace/bridge the backup battery when/while the button cell is changed?

What about CAT862 batteries? Given the number of sites running these systems, there are interesting times ahead...

Will there be business for independent techs to perform repairs, or will manufacturers revise their repair/replacement policies once it becomes imminent by numbers?

- Carsten

Posted by Steve Guttag (Member # 268) on 02-28-2019, 05:17 PM:

The CAT862 is a supercap.

The ICP batteries are two different batteries. The changeable one is the RTC battery, the other one is the certificate battery.

Posted by Carsten Kurz (Member # 5396) on 02-28-2019, 05:51 PM:

Didn't we have a discussion and/or doubts a while ago about wether one is a backup for the other?

One would think that, if the soldered-in cell is a backup for the replacable, that would be the only reason how the RTC could survive a cell replacement? I know it's not important as it resyncs on next boot anyway, but... Unless there is another 'bridge' cell nearby, e.g. a supercap.

Would be nice to hear from the ICP manufacturers wether these two cells work independently, or backup each other. Anyway, I guess it is not impossible to desolder or clip-off the soldered-in cell while bridging it externally with a wired cell, then put a new one in. The most interesting question is - when do you feel it being urgent/necessary to perform such a risky operation on a still working ICP...
Probably when we hear about the first ICPs dying following a pattern.
The same probably applies to the CAT 862. Even if a DSS200 is powered all time, after 10 years or so in a warm environment, the cat862 backup battery could be dead without ever being used, but could possibly be replaced as long as the supercap is still okay.

Anyway, at least for Barco projectors, Barco will be happily replacing debrained ICPs with ICMPs instead of rebraining them...

- Carsten

Posted by Allan Barnes (Member # 5178) on 03-01-2019, 01:37 AM:

Many thanks. My DSS100 now has and is holding the current DATE & TIME. Working on the DSP100 now... hoping it isn't a "door stop" with no certificates.

Posted by Carsten Kurz (Member # 5396) on 03-01-2019, 05:43 AM:

There may be no need to change the batteries on the DSP100 for now. Did Dolby ever issue a Bulletin on them? Did you try to reboot both and play encrypted content after you reset the DSS100 clock?

I think I have never seen the inside of a DSP100.

- Carsten

Posted by Steve Guttag (Member # 268) on 03-01-2019, 06:19 AM:

Every time I've changed an ICP battery, it has lost the date/time. Barco does not automatically reset the clock, you have to do it with your computer. Christie resets itself as does the NEC.

It is the NEC service manual that calls out the two ICP batteries as Certificate and RTC.

Posted by Bruce Cloutier (Member # 9615) on 03-01-2019, 10:09 AM:

Since we are discussing batteries and the RTC I thought I would detail that for the JNIOR just for information's sake. Many of you running decade old Series 3 JNIORs likely have dead batteries. You probably don't notice either because the JNIOR remains powered 24/7 or your application does not depend on time and date. Some use the Task Manager and the RTC setting is more important for that. It does help us in debugging when timestamps in logs are accurate.

The Series 3 batteries are soldered in. My fault in looking back. We went with the 2032 cells in a holder for the Series 4. I tried a smaller cell in the 412DMX but will likely move back to the 2032 when the board is revised (if at all). If you feel compelled to replace a Series 3 battery, I would recommend getting a leaded battery cell holder rather than to try to find a suitable replacement to be soldered in place. Any 3V cell will work.

On the JNIOR the battery retains the RTC through power off. It also holds the non-Flash file content. Basically that is anything in the file system not part of the /flash subfolder. We typically place log files in the non-volatile SRAM as opposed to the Flash. Kevin does some logging in some applications to the /flash folder. This behavior is a legacy thing in trying to remain compatible with the Series 3. In the Series 3 the Flash is limited, slow and subject to wear. So we used it sparingly. In the Series 4 we have been providing more and more Flash space and are using better technology. It might be possible to move the entire file system into the flash. I just can't get rid of the SRAM without impacting things that rely on immutable blocks (like the MODBUS client and server functions). The problem is that we are single-sourced on that SRAM and our hand may be forced some day.

Anyway, if the JNIOR can get to the Internet it will poll an NTP server at boot and every 4 hours thereafter to update the RTC.

Posted by Marcel Birgelen (Member # 6801) on 03-01-2019, 05:21 PM:

quote: Carsten Kurz
Would be nice to hear from the ICP manufacturers wether these two cells work independently, or backup each other.

AFAIK there is just one manufacturer of DLP ICPs: Texas Instruments. Barco, NEC and Christie just put their specific front bezel on it.

Also, I was in the understanding that the battery powering the certificate storage has a capacitor backup that should survive the battery swap, the RTC one doesn't have one.

Then again, there are numerous different revisions and versions (e.g. 2K v.s. 4K), like you indicated.

Posted by Carsten Kurz (Member # 5396) on 03-01-2019, 09:11 PM:

I can't see a SuperCap close to the socketed and soldered in cells on ICPs. Also, if the cert battery is soldered in, having a backup supercap doesn't make much sense.

My NEC manual is not actually clear on that:

'The coin battery (Panasonic BR2330) mounted on the ICP board provides electrical power for maintaining tamper detection,time, and date information while the power to the projector is off. If the battery voltage drops, the tamper detection circuit is activated and the security key is erased. Once the security key has been erased, the projector requires repair at the factory'
...
'• Although data is maintained while the battery is being replaced for approximately 3 hours by the sub-battery built into the ICP board (when the sub-battery is fully charged), please replace the battery quickly. To ensure that the sub-battery is fully charged, turn the projector power supply on for 30 minutes or more before replacing the battery'

Now, the trouble is, who's interested in messing around with a working ICP to find out what's happening if...

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 03-04-2019, 02:28 AM:

I've recently pulled the ICP in our DP4K-19B as part of a cleaning job. It has two of those BR2330 batteries close together, one is supposed to be the "RTC battery", without a backup, and the other the "Certificate battery", with a backup.

I'm not sure what IC on the board is holding the certificates, but it doesn't necessarily have to be close to the battery cells.

Also, if the capacitor designed to provide the backup to just the piece of memory that keeps the certificates alive for about 3 hours, you probably don't need a "supercap", but an ordinary capacitor might do the trick already.

Posted by Carsten Kurz (Member # 5396) on 03-04-2019, 07:26 AM:

But where's the sense in having a backup battery for a soldered in cell?

- Carsten

Posted by Dave Macaulay (Member # 813) on 03-04-2019, 11:18 AM:

The ICP battery that can be replaced is for the ICP clock, which pretty much just gives time stamps to log entries. After changing it a Barco has an error and you need to reset the ICP clock to UTC. Christies don't seem to care. Not sure about NEC.
The other soldered in battery... apparently for certificates. None have died for me yet. I assume the ICP is dead once that battery dies. Possibly they can be repaired after that?

Posted by Carsten Kurz (Member # 5396) on 03-04-2019, 11:45 AM:

Many things CAN be repaired. The question is, who would do it?! Barco hasn't the best credit there, and, we have all seen the issues with bricked enigma boards. Many cinemas paid for replacements when suddenly TI fixed it in software...

Is the lost ICP cert a TI thing, or can each OEM rebrain their ICPs?

Also, this is most certainly not field-doable, so, screens will be down, even if some company feels ready to perform the operation?

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 03-04-2019, 02:12 PM:

I've never looked at it with so much detail, but I decided to pull the board out again. The left battery is indeed soldered on and not placed in a socket...

I guess those BR2330 cells don't come in rechargeable versions?

Posted by Carsten Kurz (Member # 5396) on 03-04-2019, 04:18 PM:

Not with that specific number. It's a bit strange that both are the same type, one soldered in, one replaceable. Then again, they may experience different amounts of current drain, and the one keeping the time is not so critical. Also Harold once mentioned that very low current drains can be problematic with cell holders, so a soldered in battery might be safer for a critical very low current source.

I think Doremis IMS1000 features a soldered-in rechargabe button cell sustaining certificate memory. You do not have to replace it periodically, but as every rechargable cell, it has a limited life span (around 10 years they say - moot point, as I hardly doubt any IMS1000 will make 10 years...).

So, what is better?

Also, so far the industry has been rather lucky with their suppliers - but what if a manufacturer goes out of business and there is no 'generic' replacement possible, because the certificate restore hardware is not there?

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 03-04-2019, 05:09 PM:

A non-rechargeable battery usually outlasts a rechargeable battery in longevity on a "single charge" compared to a rechargeable battery of the same volume.

I guess they simply expect the "certificate battery" to outlast the useful life of the board. But it's for sure this board does have a ticking self-destruct bomb.

The RTC circuit will most likely be more power hungry. You need to drive an electronic oscillator and the circuit that calculates and updates the time and date. That will probably consume more power than simply keeping some memory alive, while there is no external power.

I don't really understand why there needs to be an off-line powered RTC on there. Can't the ICP pull the time from the projector? Does it need to know what the time is when powered down? There is no circuitry alive that can write to log files anyway?

The whole security chain is built on just a select few entities having access to stuff like certain certificates and private keys. Once they disappear, so will the ability to repair those products.

In this particular case, the ICP is actually a TI board, but I doubt Texas Instruments would ever honor a repair request. The certificates in there could also be manufacturer-specific. Has anybody ever tried the ICP across manufacturers? I guess the thing is sufficiently expensive not to try such an experiment, in order to brick it.

So, if the manufacturer would cease to exist or simply decides to entirely abandon it, with it would go the ability to replace it.

This is hardly avoidable and also not, in any way, illegal. But, what is an interesting case though, is that we've almost clearly established that this board does have an expiration date, although it's unclear when exactly that will be. Also, there is no real solution to fix it, as that's being blocked on purpose.

You could ask yourself if it's actually legal to sell such products, without explicitly informing the customer about it.

Now, there might be an infinite amount of nuances, depending on your jurisdiction, but in general, products with a non-advertised built-in auto-expire self-destruct are considered defective...

Posted by Carsten Kurz (Member # 5396) on 03-04-2019, 05:57 PM:

A while ago, Qube Cinema claimed they were no longer able to renew the certificates for their early DCI server line, the XP-D. They never became fully open as to why. Someone mentioned WIN XP 'expiration' being the reason for it, as these systems were running XP.

Some of the systems abandoned that way were only a few years old.

In that case, the manufacturer still existed, but refused to service these systems. Additionally, these systems life span was exceptionally short (most other manufacturers have their SPB certs expire in the late 2020's or even 2040's.

These systems were certainly out of warranty, but I doubt the general 'out-of-warranty' excuse would hold through court here...

- Carsten

Posted by Marco Giustini (Member # 4544) on 03-05-2019, 04:52 AM:

it's a shame that some devices are 'designed to fail' like that. We all assume that the board is not going to be useful in 10 years time but we all know that original Mediablocks and servers can still be used nowadays and they get replaced just because they are not supported anymore. And this applies to basically everything that is manufactured today, in all departments.

Posted by Bruce Cloutier (Member # 9615) on 03-05-2019, 08:52 AM:

quote: Marcel Birgelen
The RTC circuit will most likely be more power hungry. You need to drive an electronic oscillator and the circuit that calculates and updates the time and date. That will probably consume more power than simply keeping some memory alive, while there is no external power.

Off topic... Pet Peeve Alert!

This drives me nuts. With the level of technology we implement today they still insist on using the same silicon RTC implementation designed for the first digital watches in the late 60s. That oscillator drives counters which break time into seconds, minutes, hours, day of month, month, and year. All of that takes power. When all we need is a 64-bit millisecond counter. Every OS first reads that RTC and calculates the millisecond count since some epoch. That math is cumbersome in and by itself and could be avoided. Dumb!

Okay, as you were.

Posted by Marcel Birgelen (Member # 6801) on 03-05-2019, 09:20 AM:

I remember the Qube Cinema dilemma. Didn't they eventually issue a "last update" which extended the certificate for another 10 or so years?

quote:
This drives me nuts. With the level of technology we implement today they still insist on using the same silicon RTC implementation designed for the first digital watches in the late 60s. That oscillator drives counters which break time into seconds, minutes, hours, day of month, month, and year. All of that takes power. When all we need is a 64-bit millisecond counter. Every OS first reads that RTC and calculates the millisecond count since some epoch. That math is cumbersome in and by itself and could be avoided. Dumb!

Well, couldn't agree more on that. Actually, I'm pretty particular when it comes down to time and time-keeping... most of this stuff has been implemented in awkward fashions and the circuitry of any RTC should just be a simple 64 bit integer that's increased with "1" every millisecond, which would also keep power usage at a minimum.

I've been a fan of TAI64 timestamps for a while now. It's based on International Atomic Time, which, in contrary to UTC, isn't affected by leap-seconds.

As you might know, leap seconds are a pretty big problem, especially if we are "inserting time", skipping over a second or a half isn't such a problem, but encountering the same timestamp twice, is bound to break stuff, especially in things like databases, that create unique transaction IDs based on timer values.

TAI64 uses a bunch of simple system libraries to convert "atomic time" into "meat space time".

Obviously, those libraries need to be maintained and need to be made aware of new leap seconds, or the clock will skew over time. Then again, every now and then some local dictator moves themselves to another timezone or some government takes the wise decision to mess around with daylight saving time, so those libraries need constant maintenance anyway.

Posted by Harold Hallikainen (Member # 5405) on 03-05-2019, 09:46 AM:

In recent products, I've been using a 1 Hz interrupt in the microcontroller to generate a 32 bit signed time stamp. NTP is run at power up and on an occasional basis to initialize the counter and to update it. When doing an update from NTP, I adjust the period register in the right direction to correct the clock speed. Eventually corrections become very rare. This is used in non-secure products like the LSS-200. The clock is used only in logging. There, the time stamp is stored in a 64 bit field. When the log is displayed, a user defined time zone offset and DST offset is applied before passing the integer into time.c for conversion to ASCII for display. I currently do DST determination in javascript on the web UI which then passes a DST flag back to the web server in the device. I am looking for a solution to the 2038 issue where the 32 bit signed counter rolls over (goes negative).

In media blocks, as I've mentioned before, and as Carsten wrote recently, we have had problems with battery holders with very low currents. Therefore, we are now soldering batteries in. Further, super capacitors do not hold enough energy to run the security chip for a long time. In our latest design, we do use a super capacitor to power the security chip. That keeps the chip from drawing any power from the battery for about 2 weeks after main power shutdown. In a typical theater, the battery will never see any drain from the security chip.

The security chip holds the private key, includes an RTC that is used to authorize KDMs. The security chip also has inputs for the tamper switches. A fair amount of current goes through these tamper switches resulting in more battery (or super capacitor) drain. Switches have the same issue as battery holders with very low currents. We run higher currents through the switches when main power is present to prevent the very low current issues. When a tamper occurs, the security chip records the time of the tamper and the type of tamper. There's more stuff going on in the security chip, but that's a fair amount of it.

Harold

Posted by Marcel Birgelen (Member # 6801) on 03-05-2019, 02:19 PM:

1 Hz seems more than reasonable for something that's essentially sleeping.

quote: Harold Hallikainen
I am looking for a solution to the 2038 issue where the 32 bit signed counter rolls over (goes negative).

I guess you don't need to log anything from before 1970?

When you've got the source of all the stuff running on it, why not change it from int to uint? That gives you another 68 years or so, long enough to make it somebody else's problem. [Wink]

Posted by Harold Hallikainen (Member # 5405) on 03-05-2019, 08:21 PM:

True, but I do not have the source for time.c . Calculating out the date for dates beyond 2038 would be interesting. I really don't know how they do it for the signed 32 bit integer. I just call the function and back comes a date and time string!

Harold

Posted by Marcel Birgelen (Member # 6801) on 03-06-2019, 02:46 AM:

I think this is the moment where Bruce should chime in and tell you about the virtues of having your own OS. [Wink]

We're getting a bit off-topic, but well...

Usually, systems that use signed 32-bit integers for their timestamp, with 1970-1-1 as the epoch, simply can't handle any date beyond 2038 and they wrap around. Also, negative timestamps are seen as dates BEFORE the UNIX epoch. This is how it's handled according to POSIX standards, but many libraries violate that standard and some produce arbitrary results.

Now, if there are no updates for your OS and/or development stack and if you don't have access to the sources of your OS, but the system is still functional, even after it wraps around from 2^31-1 to -2^31 and users interface with your system only via your own software... Well, then you could do some kind of hack. You could replace all time functions with your own time functions, which essentially are a wrapper around the existing time functions. If you encounter a negative timestamp, you add 2^31-1 to it. Instead of time.h, you'll just need to include harold_time.h.

Maybe this is a gross oversimplification, but I don't really know the nitty-gritty details obviously. [Wink]