Ghost DCPs showing up on NTFS ingest?

[Ryan Gallagher]

October is here, time for "spooky" tech support...

Does anyone know a windows based solution to fix this. Our DCP2000 server will detect ingestible DCPs on our jump drive even though the original files have been deleted and recycle bin emptied.

I have 5 right now I can't seem to clear from being visible in the scan.

I assume it's some kind of file system management quirk between windows and linux handling of NTFS. Prefer not to reformat the drive to resolve if there is a more direct path.

[Ryan Gallagher]

Things tried so far:

Remounting on windows to confirm the recycle bin is empty.
chdsk D: /f
chdsk D: /sdcleanup

Have not tried anything from root on Doremi or another linux system yet.

[Carsten Kurz]

Have you tried rebooting the server?

[Marco Giustini]

did you format the jump drive?

Alternative the nuclear option is to use DISKPART.
Open it in a CMD prompt
LIST DISK (will show you the available disks)
SELECT DISK n (where n is the ID of the disk you want to wipe)
LIST DISK (MAKE SURE the disk you want to wipe has an * by it)

CLEAN

Mind you, CLEAN will wipe the disk. No confirmations. If you make a mistake, it's gone.

Now open "change disk partitions" (can't remember how it's called exactly but it'll show up if you search "partitions")
You'll see a drive with a black missing partition

Create new partition
Format it NTFS

done.

=============

Before you go nuclear, have you got "show hidden and system files" enabled on your Windows? It might be as simple as that. Enable that and check the usb stick again.

[Carsten Kurz]

What does 'jump' drive mean? And why do you prefer to not re-format? A standard windows quick-format should wipe the MFT completely within a few seconds. Are there files on these drives you need to keep?

[Marco Giustini]

ah - I missed the mention of "not formatting", so removing the partition is even worse

I'd make sure you don't have hidden files then.

[Tony Bandiera Jr]

October is here, time for "spooky" tech support...

Does anyone know a windows based solution to fix this. Our DCP2000 server will detect ingestible DCPs on our jump drive even though the original files have been deleted and recycle bin emptied.

I have 5 right now I can't seem to clear from being visible in the scan.

I assume it's some kind of file system management quirk between windows and linux handling of NTFS. Prefer not to reformat the drive to resolve if there is a more direct path.

My first question is, do the "ghost" DCP's cause ingest problems when trying to ingest what you want to ingest? If not, the easiest solution is just ignore them.

So what about the files you wish to keep? Can you manually save them to another drive (or server) first? if so, your best (only?) option is to save those files on said other drive, format the problem drive, then transfer them back onto the problem drive after reformatting. Welcome to the wonderful world of D-Crapma.

"Prefer not to reformat the drive to resolve if there is a more direct path." Well I'd prefer we never went down this digital rabbit hole, but here we are. And like it or not we now have to bow down to our digital overlords and do as they say. Just reformat the drive and transfer the wanted files back. That is going to be your only option with the least risk and best chance of success without creating a whole 'nother problem.

And just wait till the studios decide to use AI to control and distribute content, that's when the real fun begins. Mark my words (and this post), it's gonna happen.

[Ryan Gallagher]

Thanks. all. Obviously format or diskpart is a nuclear option. I was just curious if anyone had seen this with NTFS drives before and had a workaround.

We don't have a lot of large drives handy and some other keeps tend to live on that drive. So that was my reason for preferring a direct solution.

The problem exists between boots of the DCP2000. It does not cause any ingest problems other than to clutter up the scan. (Though I've never tried ingesting one of the ghost entries).

I will check the windows settings and make sure hidden files are not hidden, that is worth a shot.

[Jason Sharp]

What platform did you originally try to delete the content from? Windows?
I wonder if there is another directory on the drive that contains the asset map or CPL.
I don't recall if you can delete content from a removable drive on the DCP2000 but you could try that.

[Dave Macaulay]

An NTFS drive can only be read by a DCP2000 so nothing on it can be modified. Deleted files are still there until overwritten, maybe the server is finding the data somehow even though they are renamed as deleted files? An undelete program can recover files but not the original filenames.

[Ryan Gallagher]

What platform did you originally try to delete the content from? Windows?
I wonder if there is another directory on the drive that contains the asset map or CPL.
I don't recall if you can delete content from a removable drive on the DCP2000 but you could try that.

Windows 11 is the machines the drive is managed with. But there it has been mounted read only to Macs a few times too. I got distracted by a pre-show audio issue yesterday and did not investigate hidden files. (Intermittent L Chan volume issue, due to a seating/strain issue of the analog DB25 on our CP750 apparently)

[Jim Cassedy]

Our DCP2000 server will detect ingestible DCPs on our jump
drive even though the original files have been deleted and recycle bin emptied.

I know you specifically mentioned that your issue is happening in a WINDOWS environment,
but I have run into a similar situation when creating DCP's on MAC computers. I have several
MAC's, all of them loaded with 3rd party software (ExtFS for MAC) which allow me to format
media and read & write files in EXT2 and NTFS. While that worked fine, I discovered that it
worked best with newly formatted media, or else I would run into the "phantom file" problem,
usually on DoRemi servers. I also had a small (cheap!) Mp3 player that would gag on music
files I had created to use for walk-in music. I eventually learned that MACs used some sort
special files that had something to do with their OS indexing system. So I found a program
called "USB CLEAN". It wasn't free. I think I paid about $5us for it in the app store, and I
would run this when ejecting any DCP or MP3 files I'd made on my MAC and it would
automatically delete all the extra MAC-specific files on my media that weren't needed.

[Carsten Kurz]

Maybe what the Doremi sees there is Mac resource fork files, usually when files have been transfered from a native Mac filesystem to e.g. FAT/FAT32 or NTFS. They also may appear when you packed/unpacked a folder with a ZIP tool. These file names start with '._' , and the Doremi may mistake an ._ASSETMAP.XML file for an actual ASSETMAP.XML file when scanning/displaying drive content - but the ingest manager will fail when actually trying to ingest one of them. It is safe to delete these files.

[Ed Gordon]

This might be what you are seeing:

Hidden ._ files (e.g., ._reed movie) appearing alongside videos on a Mac are AppleDouble resource fork files generated by macOS to store metadata when saving to non-Apple formatted drives (such as FAT32 or exFAT).
These files are harmless and do not impact the actual video data, but they often clutter file management.


To resolve or prevent this issue:

• Reformat the external drive to APFS or Mac OS Extended (Journaled) if it will be used exclusively with Macs.
• Use third-party tools like BlueHarvest or Onyx to automatically hide or remove these files if the drive must remain compatible with Windows.
• Delete manually if preferred, though macOS may recreate them upon future saves to the drive.
• Properly eject external drives to avoid file corruption or persistent metadata issues.

[John Thomas]

Do the DCPs show up in the ingest manager only when the drive is connected, or perhaps also when the drive isn't connected as well?

And you're SURE the recycle bin is empty?