Recommended NAS for Deluxe eCinema transfers?

[James Gardiner]

Its not possible to use the disk in the Deluxe box as a storage area, as it does not have control of it. (DCPs can disappear without notice)
Catcher is designed to have its own storage, so Cinemas can have 10, 20, or more TB in it. It then ingests content from any Deluxe/Qube, whatever boxes are in your cinema and gives you a long term LMS (Library management system) of DCP. It will simply auto ingest any DCP it sees on a provider's box and store them until it's the oldest file and automatically deletes it (Tho you can mark a file as no-delete.). Good if you occasionally bring an old film back and don’t have to re-pay an MG to get a copy.

I did write a tool that scanned ftp directories and made a database of where all CPLs were ingested. Scanned once per day, walked the tree on SMSs and any other LMS type device, you could then search for the existence of a specific CPL on all devices it scanned. Technically, you could then use this as a wider database of stored CPLs. But I didn't think this tool was that useful to cinema owners. Its more an enterprise feature. For example, you need to manage content and where it is, make sure its deleted for security reasons etc. (Big post house or studio type need)

[Ryan Gallagher]

I’ve not used Ecinema distribution, but I suppose one could configure the NAS with two volumes, one for dedicated deluxe delivery, and one as the storage pool that cannot be remotely nuked by deluxe (which cinema catcher could utilize).

But I kinda had the impression deluxe required hardware, and was not just a software solution?

If cinema catcher was not in use I’d be considering tools like cron, rsync, and ftp to facilitate the desired pool storage and auto ingests. One has to be careful though about how partially transferred files are handled. I know our doremi would break SimpleDCP’s sync process if it tried to ftp ingest a DCP before all the downloading from offsite was complete. We had to poke them to resync that title if this occurred.

[Sebastian Binz]

Thanks James, got it.
In my case the Deluxe box is a Synology NAS with tons of storage, so it does not make sense for me to have another large storage on the Ubuntu box on which the catcher app is curently running.
If I would mount a folder on the NAS from the catcher box, I would probably end up having catcher ingest the file via FTP from the NAS, just to put it back onto the shared directory on the NAS - which sound like not really making sense.

Picking up on Ryan's idea:
Would it be possible to run catcher within a docker container on the synology NAS?

The deluxe tool (its actually called Sharc by CinePostProducution here in Europe, working in a similar fashion) would be downloading DCPs into /download on the NAS and then catcher would ingest from that location into a folder on the same NAS (e.g. /catcher/storage). From there one could then initiate the ingest into the ICMP.

Would this work? I would probably need to fiddle around with the FTP config in order to avoid the two FTP servers created by synology and catcher getting into a conflict.


Coming to your last idea: I would think that there is indeed a certain need for such a tool scanning remote locations and handling the ingest to a cinema server. At least here in Germany there are at least 3 different companies running eDelivery solutions. Some provide their own hardware (GoFilex), some run as a software on a Windows PC or a Synology NAS (SHARC) so you will evenutally end up in a situation where there is more than one box with a large storage attached accessible via FTP. In that scenario it makes sense to have someone like catcher to act like a man in the middle and tell the ICMP to ingest the DCP directly from any of these FTP sources, rather than ingesting it into a storage controlled by catcher and having a second ingest job into the ICMP.

[James Gardiner]

From my experience.. Having GoFlix (Deluxe), Qube, Sharc, Startrek (4 vendors).
GoFlix have a small Dell mini desktop. They delete things very quickly. This is a problem as its the main supplier here so you need to move it of if you plan to keep it around.
Qube, small Dell mini PC, delete only when needed.
Sharc (nas based) tend to leave things around for a long time. Probably until the disk gets full.

They all offer software but force typical cinemas to install a dedicated Internet connection and a hardware box. Only if you're a non-commercial site do they allow PC based tools.

I have asked numerous times why they force sites to install multiple internet links, run dedicated (eats more power) hardware. Adding extra links to a site is a pain as locations are not always wired up to do it, can cost thousands to rectify.

My preference would be docker containers. Then I could run, with my TMS, on a server running ProxMox, as many systems I need. One for TMS, POS server, one for each delivery service. Far more reliable. Utilise our own corporate link that we could upgrade to 1Gbe. Much faster, much cheaper to run, much more reliable. But for reasons they will not say, they will not allow it.
But I do believe it will eventually happen/go-virtual as its likely for reasons the cinema owners will not linked as.. Why would they keep it secret. Plus cinema owners will get sick extra costs.
My worry is they could do secret audits of the players, like in the VPF days. Nothing stopping them. The devices have the access to do that as they can talk to the Players directly.

catcher does run as docker compose system, but I am not sure it would run well under a gutless NAS cpu. catcher does a lot more then this simple LMS feature. Its AutoKDM toolset is very useful as it has good diagnostic tools to allow faster follow up on why KDMs may not be working as expected.

I was also wondering. My main service portal I am every always improving, uses much of the same code, It converts all incoming TLR/ADV into mp4 so you can view them via the browser. I could add that to catcher at some point. Plus add the Leq(m) measurement system so you can QC the audio levels via the browser.

Currently, I am spending all my time converting the main portal from older angularjs to svelte modern javascript front end framework. Makes it a lot faster to use, especially when dealing with large tables of data as it only downloads the visible data and not the full tables. Which get amazingly larger than I ever expected. for example, SO many KDMs. loading the KDM table in AutoKDM is far to slow when its at typical use levels.

[Ryan Gallagher]

Having looked more at what all catcher does, I would agree relying on the synology cpu is not ideal. As James suggests a dedicated “server” or vm is better.

But if you are willing to accept some file handling performance hit, and have the network to support it, pointing catcher at a networked share or network block device for its storage folder seems doable? So if the existing NAS is large enough it could perhaps be used in that role too, just not the application host?

If so, segregating Deluxe and Catcher's storage volumes or quotas might be advisable... so that one does not over-utilize the resource and squeeze the other?

[James Gardiner]

The catcher always keeps enough Space available to store a DCP, so a few hundred gig (Configurable). And yes, I run a test catcher with the storage directory mounting from my large TrueNAS storage array. Works well enough, especially if you have 10Gbe (Obviously I do, its cheap now.) If you change the size of the share. Would likely just delete files until it was in the limit or grow until it hits the limit.

If I was going to do it cheap, I would get something like...
Dell OptiPlex 7080 SFF i5 10500 3.1GHz 8GB 256GB for refurbished, about $300 usd. drop in a 10Gbe hard disk, or connect to a NAS. Possibly drop in a half height 10Gbe card.
16gig would be better though. Doing a lot of Disk IO works better with more memory.

The biggest issue is,, you need to be comfortable with Linux administration in setting it all up. (Mounting a storage drive, etc.. Editing setup files for catcher) It's not windows point and click. But really, you don't want it to be, as you tend to get these systems remotely administered. And command line rules. For example, why ACE was converted to Linux based. AAM, Linux based. All the better TMS or player are Linux based.

[Justin d'Entremont]

I find all this interesting, up here in Canada the "Deluxe Box" is a 1RU server provided by a 3rd party called "Cinesend". It's just a branded ASRock server, it runs some form of Linux and has an SSD for boot drive, and a 4x ~4Tb RAID array for the content. The content lives on there for usuallly about 4-6 weeks before it gets overwritten.

We also have a 1RU "MovieTransit" server that I've never received a single peice of content from, and a 1RU "GlobalDCP" server from GlobalMagnetique for French content coming out of Quebec.

None of them force us to have a separate internet connection for them, mind you we have 1Gbps x 1Gbps fiber connection so I doubt they have room to complain in that department. They can't seem to push more then ~850Mbps download onto a single server, though the connection is good up to the rated 1Gb.

I've been meaning to make changes to our network in terms of security, and place all these 3rd party content servers in their own VLAN so they can't talk to anything else aside from the internet, but that the screen servers can have FTP access into them.

[Frank Cox]

up here in Canada the "Deluxe Box" is a 1RU server provided by a 3rd party called "Cinesend". I

Actually, Cinesend was its own separate company that tried to compete with Deluxe as a movie distributor for a short time. Then Deluxe bought them out/took them over, and what was Cinesend was absorbed and disappeared, except for the name on the front of their black box.

[Justin d'Entremont]

Actually, Cinesend was its own separate company that tried to compete with Deluxe as a movie distributor for a short time. Then Deluxe bought them out/took them over, and what was Cinesend was absorbed and disappeared, except for the name on the front of their black box.

It's my understanding (primarily via support tickets when our last unit crapped out) that Cinesend is still a separate company, but entered into some sort of service contract with Deluxe that allowed Deluxe to install their Gofilex software on their servers. It was cheaper and faster for them to do that, then to purchase and roll out hundreds of new servers themselves. When we've had hardware issues, Deluxe refers us to Cinesend's support department. That being said, seems that Cinesend's website no longer has any reference to "DCP delivery to cinemas", and when I login to their portal, there's no longer anything to do with DCP delivery in there, so maybe they sold off the division to Deluxe?

Someone feel free to correct me if they have better info then this.

[Frank Cox]

I dunno. All I know is that I got this box from Cinesend and they sent me movies through that for a while, and then suddenly I'm was told to start paying Deluxe to put movies on it for me.

[Pete Naples]

Why is the end user/customer providing the hardware?

Here in the UK, Lansat/Gofilex is king of the hill for content download. Unique mainly deal with adverts.

Both companies provide and support the hardware and broadband connection serving it.

[James Gardiner]

All this delivery entity competition is nuts.
Initially, distributors thought they would be emailing a DCP. Stupid but understandable at the time.
Ironically, we are now at the stage that the internet has improved so much that emailing (or ftp) a DCP is actually possible for likely 50% plus sites in the world.
All these custom internet connections and privately administered extra networks cost a considerable amount. It is becoming a Fax machine compared to Email situation. At some point, these entities are going to collapse, compared to just setting up a FTP server on a Cloud server with 100GBe network speed. The industry is not in a position to support cost inefficient implementations.

Its strange that in some regions cinemas and distributors pay the delivery entity, while on other its free, (Or billed to Cinema via the distributors). While many regions REQUIRE the entity also insytall a dedicated internet link...

DCPs are encrypted. The design was implemented to cater for DCPs going via the post. meaning, it was expected the data would leak.
I mention this as, every content delivery entity also have PC-computer applications that allow the DCPs to be downloaded to ANY location. And in some situations, its expected that smaller part time cinemas download the DCP to a home office to be taken to the cinema for ingest.
The same process can be used to back up any cinema Internet link. (And typically, all ISPs share infrastructure anyway so a link goes down.. likely all links go down.)
Commercially, I see no reason why we need to spend all this extra money on dedicated links and managed services.

Its, like DCDC, are now moving many sites over to internet delivery (Away from satellite). WHY, because satellite makes no sense if everyone has high speed internet. Satellite cannot compete. And eventually managed services, simply to create a situation to clip the ticket, will also find it hard to exist, like Fax->Email.
As a business owners looking to the future of what will be needed to implement the business of cinema. Laser projectors, solar panels., ubiquity of hi speed internet. How we implement and save running costs for the business of cinema will evolve.

[Scott Norwood]

The main advantage of satellite is multicast. I don't know what it costs to send a DCP by satellite, but having every cinema download that same DCP separately could potentially be very expensive (at AWS's standard egress charges, a 100GB DCP would cost someone $9 for every download (well, slightly less with volume discounts), plus the storage cost for as long as it needs to remain available. A release to 1000 cinemas would thus cost $9k. Is satellite more expensive than that? (Maybe it is. I don't know.) There is also something to be said for not being dependent upon the public Internet or public cloud services.

[James Gardiner]

If you know what you are doing, you can make the egress costs much smaller.
Data transfer costs has always been a big issue as, everyone knows, the cloud providers RAPE customers in that area specifically. So don’t use them, or use some services that specifically focus on those needs.
AWS is a rip off for this type of implementation, but backblaze, or filemail (Who focus on data storage and a transfer in the cloud) are a small fraction of the price.

The other issue with satellite is in the DCP world, we end up with a lot more _OV DCPs then you expect, fragmenting and making satellite un commercial. For the US. it works as you have a lot of cinemas needing the same _OV, But Asia and Europe, much more difficult. Your footprint (Total number of target locations) are minimal. I don't follow it that close, but I expect a lot of those locations have fallen back to basic internet delivery as it is more commercial, and become cheaper everyday as internet becomes cheaper every day. (Higher speeds, same price unlimited download)

The industry really needs to adopt a torrent type technology for content delivery implementation. Delivery costs would be marginalised to near $0. But don;t mention that evil word. The studios lose their mind. Would not support it in any form due to the nightmares connected to it. But really. it would be perfect for this. (Who knows, it may allready be implemented in some delivery technologies.)

[Scott Norwood]

Torrents are great for things like Linux ISOs, but I wouldn't want to depend upon that distribution mechanism for commercial content that absolutely, positively has to get somewhere in a reasonable amount of time. Agreed that AWS S3 is expensive (and a single-vendor service), but it is difficult to beat their global reach and reasonable record for service reliability.