Film-Tech Forum ARCHIVE: Any film companies using large capacity flash drives?

This is topic Any film companies using large capacity flash drives? in forum Digital Cinema Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=16;t=003845

Posted by Frank Cox (Member # 6258) on 11-03-2019, 11:23 PM:

Flash drives seem to be getting to a point where they are both large enough and cheap enough that they could replace CRU drives for movie distribution.

Not that CRU drives are particularly fragile in my experience so far -- I've only had one bad one that I couldn't use at all.

Flash drives wouldn't have problems with missing screws, bent hard drives, broken-off lever arms and so on. They would be lighter and probably cheap enough in big quantities that they wouldn't need to be sent back, saving both freight costs for the theatre and tracking costs for Deluxe.

What do you think?

Posted by Carsten Kurz (Member # 5396) on 11-04-2019, 06:05 AM:

I think they still cost at least twice as much as conventional spinning discs of the same capacity. I haven't seen a flash based distribution drive so far. Also I haven't seen a speed improvement during ingests from my own flash SSDs (as ingests are not just a plain copy process). They will probably perform a lot faster during duplicating, though. Keep in mind, with more and more distribution taking place over satellite and broadband, they are sitting on a huge inventory of spinning discs, and the 'disc shipping business' certainly is on the decline. I wouldn't switch to flash drives in that situation. As you said, I rarely ever experienced a dead or malfunctioning spinning distribution drive in 6 years. Maybe one.

Or do you mean USB flash memory sticks? I think those being able to compete pricewise with spinning discs are still very slow and unreliable.

- Carsten

Posted by Leo Enticknap (Member # 534) on 11-04-2019, 07:44 AM:

I use USB flash sticks to distribute the DCPs I make all the time, though admittedly I am not a large scale commercial operation such as a post house. I just do it as a sideline for a few people on an occasional basis.

They are a little slower to write (my rendering machine will write to a typical USB3 flash stick at about 90-100 MB/s, and to a CRU via SATA at 120-130. On an old school server that only has USB2, they are significantly slower to ingest. On a newer server or IMS with USB3, ingest speeds are about the same for USB3 flash sticks and CRU drives, IMHO.

While GB for GB, USB3 flash sticks are still more expensive at the higher capacities, you don't need the higher capacities for a typical DCP project. In this neck of the woods, a 128GB stick is now around $20-30, and a 256GB one is $40-50. The cheapest spinning rust drive you can buy to put in a CRU cartridge is around $50 for a 500GB one (plus the cost of the CRU cartridge itself, if you don't have one).

But you're never going to need that extra 300-400GB to distribute a typical DCP. As against which, you can pop a USB stick in a padded envelope and ship it by regular mail ($5 to anywhere in the country, or $20 if it has to be certified), whereas for a CRU drive, you need to put it in a Pelican case (or similar), and shipping will cost you a bare minimum of $30-40, and likely a lot more for long-distance and/or rush shipping.

I agree with Carsten that the big distributors (e.g. Deluxe Technicolor) will likely never move to this method of distribution, because:

- They already have millions of spinning rust drives in CRU cartridges, plus the shipping infrastructure for them.

- A significant number of theaters still have old school servers, for which any ingest method other than CRU and through the media LAN will be very slow.

- Online delivery will likely supersede physical media for DCP delivery before existing inventories of CRU drives wear out, and old school servers are replaced.

But for small scale operations here and now, requiring delivery to theaters that cannot download feature DCPs, the advantages of using USB flash sticks vastly outweigh the one drawback for the end user (slow ingest on older servers), IMHO.

Posted by Scott Norwood (Member # 30) on 11-04-2019, 08:27 AM:

My complaints with USB flash drives for DCP distribution are a) the size and b) the labelling (or lack thereof). At least in the context of a "film" festival, there is such a thing as "too small." A standard-size USB flash drive is easily lost or misplaced. The easiest way to work around this is to put them in envelopes, but those are also easily mis-placed and the flash drives sometimes fall out. Worse, flash drives usually show up with any sort of label or information about the contents. I have used tie-on "price tags" (Staples sells them) to label these, since most flash drives have a hook or notch that is big enough for a string to attach the tag to the drive.

Another benefit of shipping CRU disks is that they are immediately identifiable as "digital cinema content" (at least by people who work for festivals or theatres). Flash drives could contain anything, not necessarily DCPs.

Agreed that this will all (I hope) become mostly moot in the forseeable future, and movies will just be downloaded at the theatre level, with physical shipping to continue for the venues that do not have high-speed Internet connections.

Posted by Carsten Kurz (Member # 5396) on 11-04-2019, 08:31 AM:

Leo - can you list a specific device that worked out nicely for you? Occasionally I do buy a brand-quality 128GB or 256GB usb stick, but those at the affordable prices you quote are still dead slow (writing and reading, compared to USB3.0 discs). I can buy a portable 500GB USB 3.0 spinning disc (Toshiba brand) at around 45US$ here. I can see some companies using the same type of disc for commercial mass distribution (nowadays, only ARRI is sold on CRU here it seems).

- Carsten

Posted by Dave Macaulay (Member # 813) on 11-04-2019, 09:47 AM:

Not on topic but I've seen a several CRU cases with notebook size drives lately. I pick up the CRU and think that it's empty... although why anyone would steal a 500gB or 1tB drive, I dunno. I opened one to see what was in it, found a spinning drive not an SSD.
And - entirely off topic - a SATA drive has maybe 10 connections total for power and signal... why does the CRU system have so many tiny and easily broken connector pins? I've had to replace several bays in servers or ingest modules in the last year or two due to pins bent/broken in that connector.

Posted by Scott Norwood (Member # 30) on 11-04-2019, 09:56 AM:

SATA uses 22 pins for data and power. SAS uses 29 pins (there is a secondary data path). The CRU DX115 supports both types of drives. Agreed, though, that there seem to be more than 29 pins there, though I have never actually counted them.

Posted by Carsten Kurz (Member # 5396) on 11-04-2019, 10:11 AM:

I see many CRU cases equipped with 2.5 drives nowadays as well. They had to drill some extra holes into the CRUs to mount them. It may be that notebook drives are actually more robust during shipping than (cheap) desktop drives (built for mobile use, and smaller mass).

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 11-04-2019, 10:27 AM:

2.5" drives are also often used in servers or in IMSes as we now see. Those drives indeed tend to be a bit more resilient during transport. On the other side, a 2.5" SATA SSD is pretty much indestructible. Since they only need to be written on once every single distribution cycle, they could last "almost forever". They may still be a bit more expensive than the cheapest rotating rust counterpart, but since they're lighter and more durable, you may save in shipping, since you also need less padding.

One downside is potential compatibility issues.

Posted by Brad Miller (Member # 2) on 11-04-2019, 03:46 PM:

For those needing a hefty sized USB3 flash drive at a very good price, you can do no better than the ones from MicroCenter. Every one I've used has been just as fast as the best-reviewed name brands.

MicroCenter 256GB USB 3.1 Flash Drive $23

MicroCenter 128GB USB 3.1 Flash Drive $12

At those prices, they are officially disposable, as it's cheaper to toss them than to pay for return shipping.

Posted by Carsten Kurz (Member # 5396) on 11-04-2019, 04:26 PM:

Many cheap USB3 flash sticks use a fast cache memory. They are blazing fast when transferring ordinary office type files, but the write speed drops considerably when writing larger files up to the full capacity as you would with a DCP. Well, you could probably pop 6 of them into a hub and copy a DCP off an SSD to them while you go out for a coffee, so...

- Carsten

Posted by Marcel Birgelen (Member # 6801) on 11-04-2019, 06:09 PM:

I even have a not-so-cheap 64 GByte SanDisk drive here that does the same. The first 100-or-so megabyte go at USB 3 speeds and then it drops dramatically.

But I've got a 256 GByte really expensive one, also a SanDisk one, that sustains about 200 MByte/s all the way through. This one apparently uses an SSD controller and flash inside and is almost as expensive as a comparable SSD.

Posted by Dave Macaulay (Member # 813) on 11-04-2019, 08:14 PM:

"SATA uses 22 pins for data and power"

A SATA cable has 7 conductors (2 serial signal pairs and 3 grounds). The drive power connector has +5V and +12V plus ground, total 3 conductors.

SAS is very similar, the drive data gets two serial links (two wires each) plus grounds. Power is pretty much the same.

10 conductors +/- a few. Certainly not 22 or 29.

Posted by Marcel Birgelen (Member # 6801) on 11-04-2019, 09:40 PM:

Physically, I don't think there is all that much difference between SAS and SATA. SAS controllers are, for example, compatible with SATA drives (but not vice versa).

The main difference is the signalling protocol. SATA essentially uses IDE commands, while SAS uses SCSI commands on the bus. SAS allows for more complex commands and far greater addressing flexibility. That's why you can put multiple shelves of disks (both SATA and SAS disks) on a single SAS port, for example.

Posted by Leo Enticknap (Member # 534) on 11-04-2019, 11:36 PM:

quote: Carsten Kurz
eo - can you list a specific device that worked out nicely for you?

I've always had good results with the Lexar S75 range: they write at 90-100 MB/s, will ingest at 120-130 in an IMS2000 or an Alchemy via USB3, and I've never bought one that was DOA, or had a report of refusal to ingest from an end user.

The only drives I've had consistently bad results with are these PNY ones. I've never been able to write at more than 20-30 MB/s to them, I've had a few reports of them being unreadable in the USB3 jacks of IMSes and only working in USB2 ports at very slow ingest speeds, and I've had to return two that were DOA.

I've just ordered one of the 256GB Micro Center ones that Brad posted the link to, and will be interested to experiment with it - thanks Brad.

Posted by Gordon McLeod (Member # 33) on 11-05-2019, 10:40 AM:

i have always been happy with Patriot usb sticks they even have a 1tb

Posted by Scott Norwood (Member # 30) on 11-05-2019, 11:33 AM:

The 29-connector SAS pinout is here.

SAS pinout

Not sure why there are so many grounds or multiple same-voltage power pins, but that's how the connector was designed. There are 29 pins in total.

Posted by Marcel Birgelen (Member # 6801) on 11-05-2019, 12:27 PM:

From what I heard, the extra pins are primarily used for in-line power supply. A SAS bus can carry a higher load of devices without external power than SATA.

Also, the voltage ranges on a SAS bus are higher than on SATA, allowing for longer cables, but that has nothing to do with the amount of pins.

I guess some of the pins can also be used for data in a newer protocol revisions. A bit like how Ethernet evolved over standard 8-wire CatX cabling.