GDC TMS - gotcha with upgrading and accumulation of junk temporary files

[Leo Enticknap]

I thought this would be worth writing up, in case it saves anybody else some pain.

During planned maintenance at a site yesterday morning, I noticed that the TMS was very sluggish, and would occasionally say "Not responding" on the window title bar. I saw that it had not been updated in a long time (the splash screen had a 2022 copyright date), and so decided to see if rebooting the computer and updating the TMS app would fix it. That was when the fun started. The installer stalled here:

image.png

After 20 minutes I called GDC, and was told that there is a known issue with older versions of the TMS app, whereby it creates multiple temporary files in C:\Users\Administrator\AppData\Local\Temp , but does not delete them when it's done with them. More recent versions fix this bug, but the installer deletes the accumulated temporary files before actually installing the app. I was advised to be patient and wait.

Two hours later I was running out of time on site, and needed to take action. So I force quitted the installer, and took a look in that folder...

image.png

It wasn't done counting when I took this screenshot - the grand total was 20.8 million junk files, totaling 210.3 gigabytes!

At this point I turned to ChatGPT for advice (sorry!). After it asked me for details of the computer, operating system, and storage, it estimated that Windows File Explorer (and, most likely, the GDC installer) would take two to three weeks to delete them all. It suggested a workaround, involving the robocopy command line utility that is built into Windows. It told me to open a command line shell with administrator privileges and enter the following:

mkdir C:\Empty
robocopy C:\Empty C:\Users\Administrator\AppData\Local\Temp /MIR /R:0 /W:0 /NFL /NDL /NJH /NJS /NP

This has the effect of "mirroring" an empty folder onto the folder containing the junk files, obliterating the junk files in the process.

On this TMS, the robocopy process took 14 hours and 17 minutes to complete.

Here is the gotcha: check for the presence of millions of junk files before launching the GDC update installer, because the first thing the installer does is to delete the pre-existing version of the TMS. If it then tries to delete millions of junk files after that, you're buggered, because the TMS is unusable until you've gotten rid of them. Although I didn't do it this way (because at the time, I didn't know to), I would guess that if you start the robocopy deletion process as the first step, you can continue to use the old version of the TMS software until the junk file deletion is complete.

After robocopy got done, I was able to install the new version of the TMS app without any problem, and it's now running happily.

[Frank Cox]

Yet another example (among many others) of extremely sloppy don't-give-a-damn programming from GDC.

[Jim Cassedy]

I vaguely recall there was something similar I had to do at two different locations
which had Dolby DSS200/Cat862 systems which hadn't been purged in a very long
time. There was a log file or something else that kept filling up, and it would eventually
start affecting playback performance. You had to get into the linux system and issue
some machine-level command and that would get rid of all the 'junk files', which would
take a little while- - I remember having to let it run overnight once to do this.

[Mark Gulbrandsen]

Yet another example (among many others) of extremely sloppy don't-give-a-damn programming from GDC.

All computer OS's have glitches in them Frank.... Even Linux does.

[Leo Enticknap]

I had to do at two different locations which had Dolby DSS200/Cat862 systems which hadn't been purged in a very long time. There was a log file or something else that kept filling up, and it would eventually start affecting playback performance.

I think this is the security log in the Enigma link decrypter in a Series 2 projector. If the Enigma was on version 1.6 or earlier, that in combination with a cat862 gave you a Catch 22: if the log storage on the Enigma filled up, the Enigma wouldn't work until it had been purged; but with a DSS200/cat862, you couldn't connect to it to purge it in that condition. From a Doremi or GDC server, you could. The bug was fixed in Enigma firmware version 1.8.

Agreed with Mark - all software has bugs, and while I'm not a fan of everything that GDC does, my experience is that they are responsive on technical support issues and do try to fix bugs as they are discovered. The annoying aspect of this one is that I could have mitigated its impact if I'd known about it at the outset of my attempt to update this TMS, hence writing it up here.

[Frank Cox]

All computer OS's have glitches in them Frank.... Even Linux does.

Sure, but there's a point where having an occasional bug (that gets fixed) spills over into carelessness and neglect, doing the bare minimum to keep those support dollars rolling in.

If I'm writing a program and create a temporary file (as in this example here), the very next thing I write after opening the file is the function that closes and deletes that temporary file, then I'll write the the function that actually uses the data. In this way there's no chance that it gets left behind and this type of bug/error is therefore not going to happen. If there's a danger of the program crashing before it deletes the file, then I'll add a conditional call to the delete function as part of the startup the next time the program runs.

I don't even claim to be a professional programmer, but if I can figure this out then there's no reason why the pros can't do the same thing.

[Harold Hallikainen]

It would be nice if companies provided documentation on such issues and workarounds until the problem is resolved. Another thing that would have been nice in the new version would be to let the new version install, then SLOWLY over time delete the old temp files. I don't think they needed to be deleted all at once since the system was working (slowly). It would just get better over time as the old files were deleted. I once helped someone whose computer was really slow. It was due to never expiring cookies (or expiring some time next century).

I think all software has bugs, it's just a matter of how serious they are and how long they will take to appear. One fun one in the IRC-28C had to do with the lease timer. The server sets a lease in the IRC and renews it with further requests. The IRC saves that set lease value and decrements the lease timer each second. But, let's say the lease timer decremented 500 ms ago when the set new lease or lease renewal arrives. If I put the lease time into the counter, the lease would expire 500 ms early since 500 ms have already gone by. So, I actually set the lease timer to 1 second more than requested to prevent a lease timeout. This worked great! Until Sony did an update where they sent a lease request for 0xffff ffff ffff ffff ffff ffff ffff ffff seconds (maximum possible 32 bit unsigned integer). I then added 1 second to that (giving zero!), and the lease immediately timed out. So... there's always something! Logs out of the IRC revealed what was happening, and it was a simple firmware update.

[Jim Cassedy]

I think this is the security log in the Enigma link decrypter in a Series 2 projector.

That sounds about right. Both the locations which had that problem had Series 2 NEC projectors
which were horribly behind in various updates. Although they were two totally separate venues,
they had almost identical equipment, which had been installed, (but obviously not well maintained)
by the same vendor, at around the same time period of each other.