Film-Tech Forum ARCHIVE: Disk Cleanup

This is topic Disk Cleanup in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=003074

Posted by Gronw Roberts (Member # 9141) on 07-25-2017, 09:42 AM:

I'm finding that my Doremi server is saying it's fuller than it should (i've had more films in it in the past without it filling the drive up so much) so I think it's time for a disk cleanup. I'm not very familiar with the process so I wanted to check with you before I do anything silly!

I got to the point last night where I anylized the disks but I need an explination of the options given - Orphan assets, Incoming assets and Incompete composition playlist.

I don't want to click everything and do the cleanup only find that a file I need is missing!

Thanks in advance.

Posted by Carsten Kurz (Member # 5396) on 07-25-2017, 10:33 AM:

In your situation with a single, standalone server, you can just choose all three options. All playable content will remain on the RAID.
Make sure no playout, ingest, etc. are running when you do this.

Are there any size/numbers given behind each option after you analyzed?

- Carsten

Posted by Leo Enticknap (Member # 534) on 07-25-2017, 11:30 AM:

The way I used to do this was to keep all the DCPs that were to stay on the server permanently (frequently played ads and trailers, snipes, technical test DCPs, transitions, etc.) on an external drive, export all the automation cues and configuration settings (if possible and applicable) and then periodically, just nuke the whole server by doing a factory reset/clean install of the software, then reconfigure/restore/reingest. That gets rid of everything unneeded, defrags the drives and gets rid of any RAID errors (e.g. reallocated sectors) that aren't physical ones.

Posted by Gronw Roberts (Member # 9141) on 07-25-2017, 11:47 AM:

Hi Carsten.

Orphan - 406086277KB (free's up 387.3GB)
Incoming - 18467489KB (free's up 178.8GB)
Incomplete - It was a similar number to the Incomplete asset, I actually ran that last night because I knew that wasn't going to affect playable files because I had to use it before to solve an ingest problem.

Can you explain to me what they are? I'm trying to learn as I go!

Thanks Leo, I'll keep that in mind.

Posted by Marco Giustini (Member # 4544) on 07-25-2017, 12:49 PM:

imagine you ingest some content and the transfer fails at 90%. The server won't delete the content which is already on the server - and fully validated - so if you try ingesting it again with a good drive, it will resume from where it left.

Over time those "orphan" files (not linked to any CPL's) will fill up your hard drive!

Posted by Frank Cox (Member # 6258) on 07-25-2017, 01:34 PM:

I don't get it.

If I have files 1,2,3 and 4 to ingest and the transfer fails on file 3, then I have files 1 and 2 and part of 3 on my server.

Then I do it again and it starts with 3(?) and 4, so now I have the whole thing ingested.

So where does the orphan file come from? I don't think it would be very common to have an ingest fail and just say "Oh well, I'll just ingest something else instead", and not take steps to complete the failed ingest? You were probably ingesting it for reason, after all.

Posted by Carsten Kurz (Member # 5396) on 07-25-2017, 01:57 PM:

The server may leave the partial part/reel on the server, and reingest it again.

It may not sound smart, but only the Doremi programmers would be able to explain this sort of behavior.

Skipping an ingest after it failed is not too unusual. Not everything I ingest is targeted at immediate screening. Sometimes mid-ingest you notice you picked the wrong VF or OV and cancel.

What's important for Gronw is that the Doremi doesn't clean up files light-heartedly, these files are in temporary directories, or are not part of the Doremis DCP/asset database. I have never heard of issues after doing a full cleanup. Of course you should never do it WHILE an ingest is running (e.g. to free up some space in the last possible moment). That will probably delete files that have been/are currently being ingested, thus the current ingest will not be complete and will leave more orphan files.

Grown - the amount of data you quote is considerable. Go ahead. But still, it is a good measure to make sure you have the distribution discs for immediate upcoming shows still on site before you do any content clean ups.

With the Doremi file manager, one could probably have a look into these directories and find out what it is or at least when it has been copied.

Then again, sometimes a server crashes or is shut down during an operation. In a larger TMS environment, more scenarios are possible, e.g. files first being copied to the server through network transfers, for being processed later.

- Carsten

Posted by Gronw Roberts (Member # 9141) on 08-05-2017, 06:22 AM:

Thanks all, I meant to post earlier but got side tracked! I went ahead and cleaned up the drives. Great stuff, there's plenty of room now, it's making life much easier.