Film-Tech Forum ARCHIVE: Are digital intermediates uncompressed?

This is topic Are digital intermediates uncompressed? in forum Film Handlers' Forum at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=1;t=008024

Posted by Lyle Romer (Member # 1266) on 05-23-2006, 08:56 AM:

The subject says it all. If so, 4K Digital Intermediates must be HUGE files.

Posted by John Hawkinson (Member # 1135) on 05-23-2006, 10:19 AM:

Are you distinguishing between lossy compression and lossless compression? For sure they are stored with some amount of lossless compression, but I can't imagine we would care...

--jhawk

Posted by Dave Macaulay (Member # 813) on 05-24-2006, 11:49 PM:

Apparently they are not compressed as compression would just slow down processing, disk space is cheaper than processor time. Scanning and film recording take considerable time, so only a few minutes of film should need to be online at any time - there are a lot of very large disk arrays involved though. After film recording the data is saved on tape until the project is complete, after that it can be archived (if someone wants to pay for the tapes) or discarded. Once a project is complete, there isn't much use for the data; film is a much more reliable storage medium than any magnetic one yet devised. You already have the camera negative and the final output film, the digital data is just representing them.

Posted by Aaron Haney (Member # 747) on 05-25-2006, 01:19 AM:

EFilm says a 4K digital intermediate is 50 megabytes per frame. (EFilm was recently responsible for the 4K DI for "The Da Vinci Code", see here.)

They are probably stored in some uncompressed high dynamic range format like Cineon or OpenEXR, although I wouldn't be surprised if a company like EFilm had their own format. Like Dave said, it is doubtful there is even any lossless compression involved, except perhaps for offline archives, as the time to compress / decompress every time you touch the data on disk would likely just slow down the workflow. Certainly there would be no lossy compression going on, as that would severely impact quality.

Posted by Tao Yue (Member # 836) on 05-27-2006, 01:51 PM:

OpenEXR supports various compression methods (all lossless), see page 10 of: http://www.openexr.com/TechnicalIntroduction.pdf

Hard to say whether it's faster to store it uncompressed on disk. Memory bandwidth is greater than bus bandwidth is greater than hard drive transfer rate. If a fast compression method (RLE) is used with a fast multi-core processor, it may come out a wash.

You'd have to try it to see -- there are other limitations that don't necesssarily show up when you compare raw bandwidth of the various components.

Posted by Frank Angel (Member # 248) on 05-27-2006, 02:57 PM:

quote: Aaron Haney
EFilm says a 4K digital intermediate is 50 megabytes per frame.

So what again does 4K refer to? 4K is a value....of what?

Posted by Mike Schindler (Member # 1309) on 05-27-2006, 03:06 PM:

quote: Frank Angel
So what again does 4K refer to? 4K is a value....of what?

Lines of vertical resolution

Posted by Frank Angel (Member # 248) on 05-27-2006, 03:34 PM:

Then it should be so indicated....imagine if we did that with everything:
24
1K
30M
16

instead of
24fps
1KHz
30M ohms
16FtL

So how about for digital projection:
4Klvr

Posted by Bobby Henderson (Member # 840) on 05-27-2006, 03:45 PM:

A "4K" image can actually be in any number of different frame sizes. The only consistent value making it "4K" is the horizontal axis of the frame, measuring 4096 pixels.

A 1.33:1 frame would measure 4096 X 3072 pixels.
8-bit color (per color channel): 36MB,
16-bit color (per color channel): 72MB.

A 1.85:1 frame would measure 4096 X 2214 pixels.
8-bit color: 25.9MB, 16-bit color: 51.9MB

A 2.39:1 frame would measure 4096 X 1714 pixels.
8-bit color: 20.1MB, 16-bit color: 40.2MB

Note: these files sizes are for RGB color mode.

Most lossless image file formats (everything from Photoshop and TIFF images to specialized formats like Scitex and Cineon) can use compression systems like LZW to cut file sizes roughly in half without permanently throwing away any data.

You can get better levels of compression if the image is graphical in nature and has large areas of flat, identical color. The newest implementations of LZW use some of the same tricks utilized by the GIF format.

Posted by John Hawkinson (Member # 1135) on 05-29-2006, 08:34 AM:

For clarity, Mike and Bobby said two different things.
I believe Mike is incorrect when he answers Frank, "4K is a value....of what?"; "Lines of vertical resolution "

It is 4K pixels per line.

Frank, it's a sad reality that people use the term "4K" a lot, so it's come into our speech without more clear units. But please don't make up a new abbreviation -- that will just engender lots of confusion (especially if it's wrong [Embarrassed]

).

Bobby went on to say, "The newest implementations of LZW use some of the same tricks utilized by the GIF format." Err, LZW is the compression format used by GIF. This doesn't make a lot of sense, seeing as GIF's compression hasn't changed since 1989 (or 1987, depending on how you look at it).

--jhawk

Posted by Mike Schindler (Member # 1309) on 05-29-2006, 09:25 PM:

jhawk,

If it's 4K pixels per line, then would it not also be 4K lines of vertical resolution? If anything, it seems like lines of vertical resolution is more informative, because it describes in what direction there are 4K pixels per line.

Posted by John Hawkinson (Member # 1135) on 05-29-2006, 09:39 PM:

4K pixels in each horizontal line.
This is why the number stays constant across aspect ratios -- because the height of the image changes, but the width (4k) stays the same.

--jhawk

Posted by Mike Schindler (Member # 1309) on 05-30-2006, 09:42 AM:

Exactly. 4K Pixels Per Horizontal Line= 4K Lines of Vertical Resolution

Posted by John Hawkinson (Member # 1135) on 05-30-2006, 09:54 AM:

It is clear to me that "lines of vertical resolution" is not unambiguous. Evidently when you say it, you mean "there are 4k lines, each of them vertical." But my interpretation of the phrase is "there are 4k lines, and those lines define the vertical resolution, i.e. as you travel in the vertical direction you cross 4k of them."

In my view, "vertical resolution" is a measure of how much precision there is in the vertical space, i.e. how many pixels high it is.

Perhaps you see it differently, but I recommend adopting phrasing that is unambiguous.

--jhawk

Posted by Bobby Henderson (Member # 840) on 05-30-2006, 03:23 PM:

I wish those folks would do like professional graphics people do when describing a raster-based image: give the full image dimensions in pixel numbers on both X and Y axis.

4096 X 2214 X RGB 8/24 or RGB16/48 means a lot more to me than some stupidly vague term like "4K". Honestly, "4K" is merely a term for laypeople to use. It communicates very little in professional terms.

Posted by Greg Anderson (Member # 235) on 05-30-2006, 04:12 PM:

I saw a presentation at NAB last month in Las Vegas.
At the time, I thought it was "4K" because the resolution was 7680 by 4320 with 22.2 channel sound (4x the horizontal and 4x the vertical resolution of HDTV... or 16x the total resolution). But now I see that it was actually NHK's "Super Hi-Vision" and actually a step above this 4K that you folks are discussing. Hmmm. Anyway, it was very nice stuff. This is what I always thought HDTV and/or Digital Cinema should look like. But I have no idea when anyone plans to produce or present any feature-length stuff in 7680 by 4320.

Posted by Mike Schindler (Member # 1309) on 05-30-2006, 06:33 PM:

jhawk,

For video, it's common to express picture quality in terms of lines of horizontal resolution. I assumed that lines of vertical resolution would be the easiest way to express what 4K is a value of.

Posted by Ron Curran (Member # 3631) on 05-30-2006, 07:52 PM:

Totally agree, Bobby.

While we are at it, why do we have 4x3 and 16x9 when proper aspect ratios are 1.37, 1.85 and 2.35?

and why do we have 8, 16, 35, 65 and 70 milimeter film when people insist on expressing image areas in fractions of an inch?

Is the intention to confuse?

Posted by John Hawkinson (Member # 1135) on 05-30-2006, 08:35 PM:

Just remember, Ron, the world is not as it seems.

Dimension A is 1.377 +/- 0.001 inches, or 34.975 +/- 0.025 mm, per SMTPE 139 (SMPTE STANDARD for Motion-Picture Film (35-mm) ---- Perforated KS).

So 35mm film is not 35mm wide. [Big Grin]

--jhawk

Posted by Bobby Henderson (Member # 840) on 05-30-2006, 09:02 PM:

All of the shorthand terms, such as "4K" or levels of "megapixels" touted for digital cameras is all -to put it bluntly- laziness. Or just call it taking the path of least resistance.

Something like "1920 by 1080" doesn't roll off the tongue as easily as a far less exact term like "1080P," which is now being twisted into meaning two different things.

Posted by Ron Curran (Member # 3631) on 05-31-2006, 02:07 AM:

Thanks for the info, John. I have changed all the projector rollers over to SMPTE spec +/- .025mm and the image has improved by that much

Posted by Leo Enticknap (Member # 534) on 05-31-2006, 02:38 AM:

quote: Dave Macaulay
Scanning and film recording take considerable time, so only a few minutes of film should need to be online at any time

Unless you're carrying out digital restoration or other post-production work, in which case you might want random access to more footage. But as you say, HDD storage is now so cheap (and getting cheaper all the time), that if you can afford HD scanners and film recorders, you can easily afford several terabytes of HDD space.

Agreed on the timescale, though; I recently had a tour of this lab, and their 6k Cineon scanner was chomping through a reel of 35mm camera negative at the rate of 25 seconds per frame or thereabouts. The owner told me that it took around 3 days or so to scan a typical 1,000 foot reel at maximum resolution.