Film-Tech Forum ARCHIVE: Digital Archiving Artical in NY Times

This is topic Digital Archiving Artical in NY Times in forum Film-Yak at Film-Tech Forum ARCHIVE.

To visit this topic, use this URL:
https://ft-forum.com/ft/cgi-bin/ubb/ultimatebb.cgi?ubb=get_topic;f=8;t=005317

Posted by John Walsh (Member # 168) on 12-23-2007, 01:51 PM:

A not-to-technical, but interesting artical about digital archiving.

The New york Times
LOS ANGELES December 23, 2007
MICHAEL CIEPLY

The Afterlife Is Expensive for Digital Movies‏

TIME was, a movie studio could pack up a picture and all of its assorted bloopers, alternate takes and other odds and ends as soon as the production staff was done with them, and ship them off to the salt mine. Literally.
Having figured out that really big money comes from reselling old films — on broadcast television, then cable, videocassettes, DVDs, and so on — companies like Warner Brothers and Paramount Pictures for decades have been tucking their 35-millimeter film masters and associated source material into archives, some of which are housed in a Kansas salt mine, or in limestone mines in Kansas and Pennsylvania.
A picture could sit for many, many years, cool and comfortable, until some enterprising executive decided that the time was ripe for, say, a Wallace Beery special collection timed to a 25th-anniversary 3-D rerelease of “Barton Fink,” with a hitherto unseen, behind-the-scenes peek at the Coen brothers trying to explain a Hollywood in-joke to John Turturro.
It was a file-and-forget system that didn’t cost much, and made up for the self-destructive sins of an industry that discarded its earliest works or allowed films on old flammable stock to degrade. (Indeed, only half of the feature films shot before 1950 survive.)
But then came digital. And suddenly the film industry is wrestling again with the possibility that its most precious assets, the pictures, aren’t as durable as they used to be.
The problem became public, but just barely, last month, when the science and technology council of the Academy of Motion Picture Arts and Sciences released the results of a yearlong study of digital archiving in the movie business. Titled “The Digital Dilemma,” the council’s report surfaced just as Hollywood’s writers began their walkout. Busy walking, or dodging, the picket lines, industry types largely missed the report’s startling bottom line: To store a digital master record of a movie costs about $12,514 a year, versus the $1,059 it costs to keep a conventional film master.
Much worse, to keep the enormous swarm of data produced when a picture is “born digital” — that is, produced using all-electronic processes, rather than relying wholly or partially on film — pushes the cost of preservation to $208,569 a year, vastly higher than the $486 it costs to toss the equivalent camera negatives, audio recordings, on-set photographs and annotated scripts of an all-film production into the cold-storage vault.
All of this may seem counterintuitive. After all, digital magic is supposed to make information of all kinds more available, not less. But ubiquity, it turns out, is not the same as permanence.
In a telephone interview earlier this month, Milton Shefter, a longtime film preservationist who helped prepare the academy’s report, said the problems associated with digital movie storage, if not addressed, could point the industry “back to the early days, when they showed a picture for a week or two, and it was thrown away.”
Mr. Shefter and his associates do not contend that films are actually on the verge of becoming quite that ephemeral. But they do see difficulties and trends that could point many movies or the source material associated with them toward “digital extinction” over a relatively short span of years, unless something changes.
At present, a copy of virtually all studio movies — even those like “Click” or “Miami Vice” that are shot using digital processes — is being stored in film format, protecting the finished product for 100 years or more. For film aficionados, the current practice is already less than perfect. Regardless of how they are shot, most pictures are edited digitally, and then a digital master is transferred to film, which can result in an image of lower quality than a pure film process — and this is what becomes stored for the ages.
But over the next couple of decades, archivists reason, the conversion of theaters to digital projection will sharply reduce the overall demand for film, eventually making it a sunset market for the main manufacturers, Kodak, Fujifilm and Agfa. At that point, pure digital storage will become the norm, bringing with it a whole set of problems that never troubled film.
To begin with, the hardware and storage media — magnetic tapes, disks, whatever — on which a film is encoded are much less enduring than good old film. If not operated occasionally, a hard drive will freeze up in as little as two years. Similarly, DVDs tend to degrade: according to the report, only half of a collection of disks can be expected to last for 15 years, not a reassuring prospect to those who think about centuries. Digital audiotape, it was discovered, tends to hit a “brick wall” when it degrades. While conventional tape becomes scratchy, the digital variety becomes unreadable.
DIFFICULTIES of that sort are compounded by constant change in technology. As one generation of digital magic replaces the next, archived materials must be repeatedly “migrated” to the new format, or risk becoming unreadable. Thus, NASA scientists found in 1999 that they were unable to read digital data saved from a Viking space probe in 1975; the format had long been obsolete.
All of that makes digital archiving a dynamic rather than static process, and one that costs far more than studios have been accustomed to paying in the past — no small matter, given that movie companies rely on their libraries for about one-third of their $36 billion in annual revenue, according to a recent assessment by the research service Global Media Intelligence.
“It’s been in the air since we started talking about doing things digitally,” Chris Cookson, president of Warner’s technical operations and chief technology officer, said of the archiving quandary.
One of the most perplexing realities of a digital production like “Superman Returns” is that it sometimes generates more storable material than conventional film, creating new questions about what to save. Such pile-ups can occur, for instance, when a director or cinematographer who no longer has to husband film stock simply allows cameras to remain running for long stretches while working out scenes.
Much of the resulting data may be no more worth saving that the misspellings and awkward phrases deleted from a newspaper reporter’s word-processing screen. Then again, a telling exchange between star and filmmaker might yield gold as a “special feature” on some future home-viewing format — so who wants to be responsible for tossing it into the digital dustbin?
For now, studios are saving as much of this digital ephemera as possible, storing it on tapes or drives in vaults not unlike those that house traditional film. But how much of that material will be migrated when technology shifts in 7 or 10 years is anyone’s guess. (And archiving practices in the independent film world run the gamut, from studied preservation to complete inattention, noted Andrew Maltz, director of the academy’s science and technology council.)
According to Mr. Shefter, a universal standard for storage technology would go far toward reducing a problem that would otherwise grow every time the geniuses who create digital hardware come up with something a little better than their last bit of wizardry.
As the report put it, “If we allow technological obsolescence to repeat itself, we are tied either to continuously increasing costs — or worse — the failure to save important assets.”
In other words, we could be watching Wallace Beery long after more contemporary images are gone.

Posted by Frank Angel (Member # 248) on 12-23-2007, 03:51 PM:

Hey, we've been saying that for years here. All the AMPAS had to do was read this forum for the last few years and they could have saved the cost of a big investigation.

quote:
only half of a collection of (DVD) disks can be expected to last for 15 years, not a reassuring prospect

Yah, surely not a reassuring prospect for the consumer who will be have to repurchase their NTSC DVD collection with degrading Blu-Ray or HD-DVDs -- which, btw, they bought to replace their Laserdisc collection, which they bought to replace their rotting Laserdisc collection, which they bought to replace their VHS collection!

Then again, figuring in the "imperative to save factor" of how few movies made today anyone would really truly miss 15 years out, the studios might actually come out ahead of the game. [Razz]

And just to throw a little fire on the copy protection issue, this is a damn good reason why the studios should figure out a benign way to let consumers make protection copies of the discs they buy. Along with the ubiquitous, pain-in-the-ass FBI warning, that warning should also include a statement that informs the OWNER of the disc that the law ALLOWS them to make a single protection copy (wouldn't THAT bring tears to your eyes). In fact, one would expect if the studios were playing fair, they should give a warning explaining that the life of the disc is expected to be about 10-15 years and it would be wise for the OWNER to make a LEGAL back-up copy, the way the software companies used to do years ago when you bought a program on a floppy disk. In many cases they gave instructions right in the setup instruction as to how to make a backup copy. Same deal here, but even more so as the industry itself now states publicly that the DVDs have a finite life, unlike their original claim in the courts when they stopped the authors of DVDCopy from selling their copy program. The MPAA claimed there was no need to make backup copies because the DIGITAL medium would LAST FOREVER. Lying scum. [thumbsdown]

Posted by Mike Blakesley (Member # 26) on 12-23-2007, 05:12 PM:

Why would DVDs only last 15 years? I've got 25 year old CDs that are perfect. I know DVDs have a lot more data, but still.

What about DVD-Rs and such?

Posted by Bobby Henderson (Member # 840) on 12-23-2007, 05:24 PM:

"Rot" can be a problem. The glue that bonds together data layers and such can degrade over time, sometimes even a short time. However, 15 years of shelf life is a pretty dubious claim.

I, too, have music CDs that are over 20 years old and still play just fine. Hopefully those DVDs will last just as well.

Posted by John Walsh (Member # 168) on 12-23-2007, 07:39 PM:

Opps.. I meant to post this in a different forum ...

Is there any way to tell how much 'digital rot' a CD has? I was wondering if the error correction in the player might fool a person into thinking their DCs are OK. When is the point where a CD will no longer play?

Posted by Bobby Henderson (Member # 840) on 12-23-2007, 09:38 PM:

Usually when laser rot occurs you can see it on the disc surface. Sometimes glue separation will look like a slightly milky spill under the clear plastic.

You can get the other extreme too. I've had a couple of DVDs that didn't have a single visible blemish to the surface but just would flat out not play anyway.

Posted by Paul Linfesty (Member # 214) on 12-24-2007, 04:20 PM:

Its very possible that the DVDs that are being talked about in the context of the article are the use of DVD-ROMS for long-term data storage (and the author may not have made the proper distinction). This has turned out to be a major problem. While, as Bobby points out, laser-rot can appear in stamped out CD's and DVD's (although increasingly rare, due to higher level of purity of materials used), the burned-type of DVD's (ROMS) have turned out to have a poor track record for use in archival purposes. Several studios were using the DVD-Rs (or + or RAM) for storage and have discovered an almost 40 percent destruction rate already! (This rather bothered me, as I have archived a lot of my video productions, as well as recorded lots of shows off my cable box, on DVD-Rs). I was hoping to be able to save watching them in my old age! (Of course, I just turned 51, so that puts me in that category now!!!)

Posted by Wayne Keyser (Member # 2420) on 12-24-2007, 10:48 PM:

Maybe I'm stupid, but ... umpty thousand PER YEAR to archive a picture in digital form? The cost must be coming from someplace I don't see.

Of course, it may ... but hey, considering just the feature, ancillary materials aside, for the sake of blue-sky figgerin':

Let's say four copies on four hard drives in four locations ... throw in a dvd-rom or two with the same material in the same box with each hard drive, just in case. Don't trust to the wisdom of the major companies, make every movie deal include a pittance, say $40k, to be invested and the proceeds (maybe 5% annually) to fund an annual or biannual swapout to four new drives and four new sets of discs. The cost to store four shoebox-sized units, per year ... well, what does a bank safety-deposit box rent for? Administrative costs and annual swapout costs, modest, certainly well within the foreseeable profits of the "preservation cost" endowment, especially if that kind of money drove an enterprise solution like a digital archive service that just kept records and performed the yearly "refresh."

What have I missed?

Posted by Allison Parsons (Member # 2728) on 12-25-2007, 02:28 AM:

quote: Bobby Henderson
"Rot" can be a problem

Ask any Laserdisc owner!

Posted by Cameron Glendinning (Member # 3516) on 12-25-2007, 03:44 AM:

quote: Wayne Keyser
What have I missed?

Good question and I am only guessing an answer, but labour would be a big factor, plus managment, plus higher rents due to the fact that people are now required. I guess that throwing it down an abandoned salt mine in the middle of know where is a really really cheap, on a long term basis for film but I doubt that they can do anything like that with digital.

Anyway as far as the high figures, it must involve hollywood style acountants doing the bookwork! Only they could possibly explain why some movies cost 200 million to make.

Posted by Frank Angel (Member # 248) on 12-25-2007, 01:34 PM:

The problem is that, unlike Laserdiscs which are analog and which, when disc rot began you could easily see the picture slowly deteriorating, CDs and DVDs are digital and the system is replete with error correction. This ec process will allow a disk which as already begun to degrade, to play quite flawlessly.

The owner gets no warning signs that he's about to have a catastrophic lose of data. When the error correction fails, the disk becomes unreadable or so badly distorted as to be useless. But the time those ON and OFF bits are no longer distinguishable, it's too late to worry about making that back up...the horse is already out of the barn. That's why I say that along with all those damn antipiracy warnings, the studios should also instruct the purchaser of their product, how to make a LEGAL protection copy so that in the even of a failure, either by reason of a degradation of the medium or by accidental damage, a backup copy will exist.

There is a precident for this by copyright owners of computer software, which is no less copyright protected than motion pictures or music, and which instructions for a single backup copy was always given with the software installation. The studios should get off their high horse (which they have already beaten half to death) and do the same. Consumer protection advocates should hound them till they do.

Posted by Mark Gulbrandsen (Member # 72) on 12-25-2007, 06:31 PM:

Thais rediclous! They can just store B&W seps like they've always done.

Mark

Posted by Steve Guttag (Member # 268) on 12-25-2007, 08:11 PM:

In fact, duping digital to film for archiving is indeed a booming industry.

However, one thing this article points out is something that most home-movie people have had to contend with. Back in the 8mm (or 16mm, depending on your era), you had a relatively short load with film and as such you always conserved your shots. Thus most of what you shot is something that you would at least consider to put in your final edited reel to show to the family. When Video taping came about...people just let that puppy roll and roll and roll and roll....heck there was at least 2-hours available!

With features, a similar thing has occurred. Studios are NOW accustomed to saving everything just in case one needs/wants to re-edit the directors final last ultimate cut (e.g. Blade Runner) or to use for other purposes. With film, you just store the whole lot. With video, there is apparantly MUCH more stuff and nobody that will decide what will stay and what will go. Do you dupe EVERYTHING to film on the possibility it MIGHT be used at a latter date. The cost of that grows geometrically. You have the initial cost of the dupe...then you have the upkeep cost of keeping the drives in good shape that are in storage and verifying that all is in good. Then on any technology change (we don't use 8" floppies anymore, or 5-1/4 or really 3-1/2...SCSCI drives are out....etc and then there are all of the file systems that could be in use or operating systems that can access the data...so there needs to be a planned migration of the digital media...that as more titles come about begets that many more drives that have to be kept up with. That is much worse than just throwing the lot in a cold-dry storage facility and deal with it when you want it. The archivists really seem to think they can keep film in good shape for many hundreds if not a 1000 years with current storage practices.

Steve

Posted by Wayne Keyser (Member # 2420) on 12-25-2007, 11:00 PM:

Were the industry (or a large part of it) to adopt a system such as I proposed above, the amount of money I proposed, with its annual earnings, would IMO be more than enough to support one or more businesses whose sole task is to do the following:

Retrieve the stored hard drives every year or two years, transfer the data to their computers (checking the transfer for errors and cross-checking with the other archive sites), and storing it again on new hard drives (of whatever advanced style comes along in the future) and new discs. SCSI drives are old-hat now, but I'm still running one on my second computer - just because new drives have come along doesn't mean the old style won't be usable on systems designed for it for a couple of years, then the data gets copied to new styles. Such businesses could be run "light on staff" with economies of volume.

But I feel the ring of truth behind the suggestion that "Hollywood style accountants" are behind the high figure they estimate. I once took a class on production and was surprised by the emphasis on the need to watch the accountants closely - the line between "above the line" and "below the line" expenses has made the differences between profit and loss for many films and many companies.

Posted by John Hawkinson (Member # 1135) on 12-25-2007, 11:30 PM:

Do people really consider 4 hard drives sitting in 4 closets to be reasonable? I don't think that I would. If I were doing this, I'd expect the data to be on multiple live (on-line) RAID systems in seperate geographical locations. And then the costs sound reasonable.

It'll be interesting to see the AMPAS report when it becomes public. It was released privately in November and there were discussions on the AMIA list where it was indicated it would be released publicly after some period of time, and there were also articles in Variety and other trade press.

--jhawk

Posted by Leo Enticknap (Member # 534) on 12-26-2007, 10:05 AM:

The basic, underlying problem is that there's no known way to archive digital data 'offline' with anything approaching the same longevity and reliability as an analogue image on film.

We now know enough about the chemistry of film bases and emulsions, and their behaviour over time, to be reasonably sure that in the right atmospheric conditions, it'll last for centuries (and ironically, nitrate is proving to be the most stable of all the bases that were ever in mainstream use). There is simply no equivalent for data. The options are:

1. Magnetic tape. It also has exacting temperature and humidity requirements, can be subject to chemical decomposition (e.g. sticky shed on some types of polyester mag, or binder layer breakdown on evaporated metal oxide), and presents a major hardware and software obsolescence problem. So major, in fact, that going down this route more or less commits you to a programme of continuous format migration. The bigger your collection, the bigger the cost and complexity of this.

2. Optical discs. Lots of research going on, but the bottom line is that we simply don't know how chemically stable the dyes and substrates are in the long term. There is still a potential obsolescence issue, although as a common form factor appears to be establishing itself, this may be less of a problem than with tape.

3. Hard discs archived offline. Some in the archival world are seriously trying to push this as a solution, while others have serious misgivings. It's certainly true that the cost per gigabyte is falling like a stone and that HDDs don't have any particularly expensive atmospheric requirements; but even in as little as 20 years time, how easy will it be to connect them to the current generations of computers? And what are the mechanical implications of spinning one up after many years of storage?

4. 'Always on' HDD storage, i.e. RAID or MAID arrays. Probably quite reliable, but very expensive and, due to the power input needed, not particularly environmentally friendly. Certainly not a viable option for smaller, public archives on a shoestring budget.

5. Digital data on film emulsion. Some think that this is the great white hope: the proven stability of b/w polyester film, very simple optics and mechanics to read it back but a potentially large storage capacity if you use the whole film width. There was a panel on it at the last but one AMIA conference, but from the presentations there I quickly concluded that the sums didn't add up. They were talking less than a terabyte (four hours of digibeta) on a 2,000 foot roll of 35mm: the cost of preserving film-resolution material this way doesn't bear thinking about, and given that most of the cost to the customer is in stock and processing, this is going to get more expensive, not less, as overall film use continues to reduce. Besides, there is not even a viable system on the market as yet.

So, sadly, I fear that a lot of 'born digital' material is not going to survive in the long term, because the cost and complexity of preserving it is going to be beyond the reach of all but the best equipped archives.

Posted by Mark Gulbrandsen (Member # 72) on 12-26-2007, 11:11 AM:

I would find it hard to believe that any really important stuff is kept on something like a mechanical hard drive. Something like static ROM would be far more stable and more immune to any outside influence. Not all ROM's are permanent but many are and selecting the right one would be pretty easy. This sort of storage would certainly drive the costs up dramatically in this day and age but memory will become much larger and cheaper still as time goes on... and it makes far more sense to say in 40 years pull out the ROM and read it back... while Hard Drives and the equipment to run them will eventually be obsolete... probably sooner than later. The Raid systems although robust and reliable are not going to be with us alot longer either... nor will the electronics necessary to retreive the data off of them.

One entity that learned the hard way about Digital Data Storage is NASA who has lost a good majority of the Mercury, Gemini, and Apollo data tapes to deterioation and age.

If I were going to store any digital production that is actually worth the cost to store it would only be via B&W Seps until the Digital Data Storage realm becomes a bit more stable in itself.

Mark

Posted by Wayne Keyser (Member # 2420) on 12-26-2007, 08:36 PM:

That's why I suggested that you stop looking for permanent storage, and (since digital data can be copied without loss) go for regularly-updated storage in multiple ways, in multiple locations.

One thing we haven't discussed since it was brought up earlier in the thread: today's chosen digital format, like that NASA data mentioned earlier, will sooner or later be totally incompatible with future digital systems. How do you ensure that some 23rd-century company, rediscovering the artistic genius of BIG MOMMA'S HOUSE, will be able to access the carefully-preserved data and turn it into viewable form?

Posted by Leo Enticknap (Member # 534) on 12-27-2007, 03:24 AM:

Exactly. For that reason, some people are advocating using no compression whatsoever for archival storage. The problem there is that an hour of NTSC comes to about 250gb, so you can imagine the volume of data storage involved with even a substantial TV collection, let alone 2k or 4k resolution film. Others believe that lossless compression is the way to go - for example this company uses MJPEG2000 in its automated, robotic videotape archiving system. Most are agreed that lossy compression (e.g. MPEG-2) is an absolute no-no, both for the reason you suggest (will the codecs still be available in the 23rd century?), and because you've introduced permanent generational loss relative to the original. Furthermore, the only way to copy the data thereafter involves decompressing it (some codecs do this differently from others, hence introducing even more digital noise, potentially) and then recompressing it into whatever output format you want.

As for solid state storage, the general consensus at the moment seems to be that the volume of data storage is the major problem. Flash memory degrades over time (i.e. the stuff on your USB stick will still be there in 2-3 months, but it may well not be in 2-3 years and it certainly won't be in 20-30 years), and the more permanent alternatives are so expensive per GB archived that for the amount of data required by a major collection, not even the most lavishly funded archives in the world could contemplate it.

Posted by Stephen Furley (Member # 1277) on 12-27-2007, 04:39 AM:

quote: Leo Enticknap
4. 'Always on' HDD storage, i.e. RAID or MAID arrays. Probably quite reliable, but very expensive and, due to the power input needed, not particularly environmentally friendly. Certainly not a viable option for smaller, public archives on a shoestring budget.

But you can't just forget about it; in our server room we have fifty odd servers, with something like 250 hard discs between them. Most of these are replaced on a five year cycle, but even so e typically have 4-5 discs fail per year; we actually had one go a few hours before we closed for Christmas. 95% of these are SCSI, or on the later servers SAS, drives, with the system usually on a pair of mirrored drives, and the data on a raid 5 array. sometimes with a running hot spare. All of this requires monitoring and maintenance, takes something like 13kVA from the mains, and requires a similar amount of power for the air conditioning to remove the heat produced. I can't remember the total amount of available storage, but it's somewhere around 10 TB.

quote: Leo Enticknap
We now know enough about the chemistry of film bases and emulsions, and their behaviour over time, to be reasonably sure that in the right atmospheric conditions, it'll last for centuries (and ironically, nitrate is proving to be the most stable of all the bases that were ever in mainstream use).

How is polyester in this respect; or is nobody prepared to commit themselves to predicting a figure yet. I have a few reels of Ferrania and Prinz Super-8 film from the late '60s somewhere; the last time I looked, a few years ago, the colour fas faded, but the film looked physically ok. The Kodachrome on Triacetate also seemed to be ok; no vinegar.

I think that, in the longer term, safety colour material will prove to be a bigger preservation problem than B/W nitrate. Look at the restoration that was necessary on 'Vertigo', and I've heard that there are other major films that are equally badly preserved.

I suspect that the issue of obsolete hardware and software will not be too much of a problem for the major formats; if somebody finds a DVD of something important in the future that needs to be transferred then there will probably be a few DVD players somewhere in the world, simply because of the vast number that were produced. I would be more concerned about some of the less successful formats; how about the material which Thames Television produced on MII for example? How many servicable machines for that format still exist. Then you've got to know whether something you've found is worth preserving; If you find a reel of film somewhere you hold it up to the light, and you can see what's on it. If it turns out to be something important you can show people what you've got, and try to get funding to copy it. Even if it's some totally unknown format, say 51mm horizontally running, with seven perfs on on side and five on the other, it wouldn't be too difficult to machine up an optical printer gate for it, even if you had to pull down and register each frame manually. With some 'lost' digital format you can't even tell what's on the media without spending large amounts on building equipment the retrieve the data, and then working out how to decode them.

Posted by Wayne Keyser (Member # 2420) on 12-27-2007, 06:25 AM:

quote:
Then you've got to know whether something you've found is worth preserving;

And as Archive.org and its abundance of new projects is showing (as if we didn't know from our own laments about vanished films) the answer is: "everything is worth preserving."

If you're not convinced, Google "archimedes palimpsest".

Posted by Leo Enticknap (Member # 534) on 12-27-2007, 12:14 PM:

quote: Stephen Furley
All of this requires monitoring and maintenance, takes something like 13kVA from the mains, and requires a similar amount of power for the air conditioning to remove the heat produced. I can't remember the total amount of available storage, but it's somewhere around 10 TB.

Or in other words, about two feature films at 4k resolution.

quote: Stephen Furley
How is polyester in this respect; or is nobody prepared to commit themselves to predicting a figure yet.

As it's an inorganic polymer, the base itself is said to be good indefinitely with no special atmospheric requirements. The problem, with both photographic emulsion and magnetic oxide on polyester, is the binder layer which sticks one to the other. There is some anecdotal evidence of the emulsion peeling off the base of b/w polyester film which is only 10-15 years old. Unlike nitrate and acetate decomposition and colour dye fading, all of which have 20-30 years of research by three separate institutes to tell us how it happens, we really know very little about what polyester film does over time.

quote: Stephen Furley
The Kodachrome on Triacetate also seemed to be ok; no vinegar.

Because it's non-substantive (i.e. the dyes are introduced to the couplers during processing rather than linked to the emulsion in the film before it's exposed), the colour dyes on Kodachrome are very stable over time - to all intents and purposes, as good as an IB print.

quote: Stephen Furley
I think that, in the longer term, safety colour material will prove to be a bigger preservation problem than B/W nitrate. Look at the restoration that was necessary on 'Vertigo', and I've heard that there are other major films that are equally badly preserved.

Digital intermediate technology is now able to help a lot with dye fading. Vertigo was one of the last major all analogue feature film restorations. The basic technology used to do it (printing through filtered light in order to extract three separate colour records from the best surviving elements) works, but is very difficult and expensive. The hardware and software needed to scan at up to 4k resolution, rebalance the colour and then burn out separations is coming down in price and increasing in capability all the time. Agreed entirely that stored in the right atmosphere, we now know that b/w nitrate is incredibly stable stuff. But substantive coupler dye fading can also be slowed down almost to the point of arrest by by cold storage, and digital restoration technologies are coming along in leaps and bounds. So I'm a bit more optimistic about the ability of archives to deal with this stuff in the future. A bigger problem, IMHO, will be films that are inappropriately 'restored' by amateurs who have the kit but not the expertise to use it.

quote: Wayne Keyser
And as Archive.org and its abundance of new projects is showing (as if we didn't know from our own laments about vanished films) the answer is: "everything is worth preserving."

Agreed in principle, but often the issue is not as simple as that. All properly run archives will have an acquisition policy - a statement which says what they do and don't collect, and why. There's no point in collecting material which is already preserved by another archive, and if you have limited preservation resources you may well have to decide that you're not in a position to take certain collections on which are offered to you.

Posted by Stephen Furley (Member # 1277) on 12-27-2007, 12:19 PM:

quote: Wayne Keyser
And as Archive.org and its abundance of new projects is showing (as if we didn't know from our own laments about vanished films) the answer is: "everything is worth preserving."

If you look a a reel of film, and you see that it's a poor quality print, possibly incomplete, with significant damage, of something that you already have much better elements for, then it's probably not worth preserving. There may be exceptions, for example, if it's a Technicolor dye print of something that you have the original negatives for then you wouldn't want to duplicate from it, but it might be useful as a colour reference when making new colour prints. The trouble with digital media, is that you can't tell just by looking at them what they might contain. It might contain an episode of Big Brother, and be labeled as such, but also contain, in the last few minutes, a recording of some historically important event, not known to exist elsewhere, but you don't know that, and if you don't have the correct hardware and software available to play it, you can't easily find out without spending lots of money just on the offchance that there might be something interesting on it.