You may remember that I was the first person on this newsgroup to wax enthusiastically about the possibilities of digital projection of movies, based on my experience at a unique synthesized 3-D theatre in the Orlando area. Well, I'm just grasping the implications of the next steps and where they will lead us. My jaw has dropped open at the possibilities, and digital cinema will probably end up being a dead medium as well.

One of the seminal scenes in science fiction is a scene in Asimov's "Foundation Trilogy" where Hari Seldon encounters a holographic wall, with the accompanying description that "suddenly the surface of a normal looking wall melted away and the room extended to show a full motion color holographic image of a pre-recorded scene, so real as to be indistinguishable from reality." To me, that effect has always been the gold standard of what film should be.

I'm stating for the record that for a single viewer, that technology is possible today. The method of achieving it may ultimately doom both film and theatres as we now know them, except as historical oddities.

Pick up the April Scientific American and flip to the article on tele-immersion on page 50. The concept of creating a complete 3-D model of objects from photographs taken from differing viewpoints has been know for years. The process is used to create topographical maps, recreate accident scenes, build virtual models of structures, etc. Until now, it has been a slow labor intensive process. Doing this real-time for moving video images was unheard of. Now it is possible to create those moving models.

Once the model has been constructed, in CGI the surface is painted with textures. There is also the possibility of painting the surface with information derived from the original photos. You can play around with this on a limited scale for free by downloading a program from: www.photomodeler.com
If you complete one of the program tutorials, you'll see exactly what I mean.

The process of finding points can be automated, as shown in the Scientific American article. One key is to have a number of differing photos or video feeds. You can find more background info at: http://www.advanced.org/teleimmersion.html

The next factor that comes into play is image averaging to remove noise and artifacts from the image being painted onto the model. Anyone who has ever used a "Snappy" video grabber knows how using multiple images of poor quality can create a single image of excellent quality.

Now, about those image feeds- for the type of immersive image generated, the quality of any single camera image showing the exact point of view being displayed is less important than you might think. The image being displayed is NOT the original camera image, but a paint of the averaged images over a model. The paint can have much greater definition than any individual image.

What is required on the front end? To get the raw data needed, an array of cheap video cameras could easily do the job. For example a grid rack of a dozen cameras (three high, four wide on one foot centers) would be more than adequate. In fact, it would probably be a wise move to record key historical events like that starting now.

What happens to create the magic? The massive computational power of the image processing array creates an output of 3-D model and high-def map of the surfaces.

How does it get displayed? By use of head tracking technology already existing, and a new auto stereo screen. Face recognition software can already compute the exact location of eyes and facial features for the head tracking.

NYU is working on what may be a real solution to the problem of auto stereo displays.: http://www.mrl.nyu.edu/projects/autostereo/

NYU uses an idea for an active shutter screen that is brilliant in concept. The best way to describe it is to point you to their javascript aplet. http://www.mrl.nyu.edu/~perlin/demos/autoshutter-talk.html

Click on all the text links and move the head with your mouse, and you'll understand the concept.

The use of all this technology for tele-immersion won't happen for quite a while, because the income stream isn't big enough to support it. However, an investment in an image processing setup, and smaller display units that use the processed image files in a less intense fashion, can create a new market for stories where the viewer is up close and personal with the actors. The cost of shooting could actually be less than shooting 35mm, while yielding image qualities surpassing that of 70mm 3-D.

Will this happen overnight? Of course not, but it will happen. How do I know? Lets just say the great psycho-historian Hari Seldon told me it would.

In short, science fiction is about to become science fact yet again. Hold on to your hats.

 

------------------
Greg Mueller
Amateur Astronomer, Machinist, Filmnut
 

To do this kind of "immersion" would probably take all the supercomputers at NASA, the CIA and the NSA combined... and then some!
 

Part of the reason-d'etre for the system was to test the capabilities of Internet II, which has a huge bandwidth. The primary problem they seem to be having is one of latency, which wouldn't affect any use of the medium for prerecorded movie plots.

Remember that once the original model and image map are constructed, much of the work is done. The data stream, while large, is not so large as to be totally unwieldy. The computer re-creating the image doesn't need to be as powerful as the one doing the original processing.

If I hadn't seen the article, I think I would be as just as skeptical of this being possible within our lifetimes.

 

------------------
Greg Mueller
Amateur Astronomer, Machinist, Filmnut
 

Seeing the difference between Creative Labs first expensive attempts at a netmeeting style of teleconferencing just a few years ago and todays version of free MS netmeeting which is almost stable (high praise for any MS product), I have no doubt that the process will be quickly improved, especially if big money is pumped into it by the studios or someone wanting to corner a new market with unlimited potential.

BTW, The Hubble didn't work too well at first, either!

I wonder how many pixels there are in human sight?
So you're going to tell me that somebody is going to make a projector that can (for financial gain) project a video image that exceeds the pixel count that the human eye can distinguish.
Not in our lifetime

------------------
Greg Mueller
Amateur Astronomer, Machinist, Filmnut

However, I'll give you a couple of examples where relatively low-res images regularly fool the casual viewer. (Not including black and white movie scenes involving high cliffs that can sometimes terrify my wife into thinking she's on the precipice with the actors and about to fall.)

The first is a common trick in Ripley type exhibits, where dressed mannequins behind glass appear to be sitting or standing in a furnished room. A face is projected on the head of the mannequin, usually through a beam splitter glass, which -if done properly- makes the mannequin appear to be a living person. Unfortunately, many of these exhibits are poorly maintained or have substituted tv and video tape for direct film projection, and the illusion is lost.

Second is an illusion I saw at the 3-D theatre I mentioned earlier. Above the concession was a video? display of a floating pepsi can that looked exactly like a floating pepsi can. The only tip off was that it was slightly too bright. I'm not sure how the illusion was accomplished, perhaps lenticular, since you could move and see different aspects of the can while it rotated. The setting made a lot of difference in the reality of it.

Third are some 35mm slide 3-D shots that I have projected on a small rear screen, that lacked only movement to make them appear real. 3-D slides, especially larger format where the visible grain structure is lost, can be stunningly real. Visible grain was a major problem with the 24x24mm slides of the Stereo Realist style cameras, and is problematic in 3-D motion picture film. If there were an iMax 3-D viewer for a single person that used a physical shutter for the eyes rather than the leaky LCD shutters, and allowed direct viewing of the film through lenses, without the contrast loss caused by projection, I'd almost guarantee it would appear real enough for you have the feeling of having been party to the scene.

Ghosting, contrast loss, and lack of being able to move the head and see different views all severely limit current 3-D. A fixed single person viewer limits views in such a manner that head movement can't be used to dismiss the illusion, but at the expense of any feeling of interaction with the 3-D image.

The proposed system removes that barrier and allows other, previously unused visual cues to come into play, the importance of which I cover later.

"Indistinguishable from reality" presentations are possible within a controlled environment. What do I mean by controlled environment? For example, the viewer obviously couldn't have a working flashlight to point at objects, (although a virtual one could be used) and the lighting would have to be consistant with the surroundings, or the surroundings be blacked out.

Initial designs of this immersion technology will probably have a peculiar hyper-real look, because the eye selectively focuses on objects, defocussing others. Until tracking is accurate enough to triangulate what the viewer is looking at, and set the range of focus, everything in the immersion will appear in focus. That may be enough of a tip-off for more astute viewers.

That selective focus idea leads to the point that the eye/brain rarely perceives all detail in a scene, especially if it is moving, but reacts to cues. The vase/face illusion is an example of this and of brain fatigue, as are most optical illusions. I contend that the majority of people, when confronted with seemingly solid 3-D objects behind a glass will not be able to distinguish them from reality, especially if, as they move around to look through different parts of the "window" the are presented with geometricly consistant views. Even those that understand the artificiality will quickly adapt, as the brain works to equate the illusion with reality. Brain fatigue at maintaining the knowledge that it is an illusion will draw most people in, just as many people are drawn into movies once the brain has to focus on plot and storyline and ignore the imperfections of the picture on the screen. As people become more accustomed to the process, they undoubtedly will learn tricks to help in the determination of "real or memorex?", but until then the illusion will be as complete as described in "Foundation."

As for the moneymaking potential? Imagine yourself getting to watch Julia Roberts or some other actor standing three feet in front of you while acting out a role, or experiencing the things that 3-D films promised but weren't able to deliver. Imagine "Jurrasic Park" raptors in your face.

Yeah, I'm a 3-D nut. Yeah, I agree this stuff will take a few years. This is the cusp of a new way of viewing images. I don't think any of us are prepared for what tomorrow will bring, any more than I ever was prepared to use my very own personal computer (much more powerful than those used to send man to the moon) to see real images of gas jets streaming out of a pulsar billions of miles away.

------------------
Greg Mueller
Amateur Astronomer, Machinist, Filmnut
 

This is exactly the same technique used in the "Morgana" effect and the "skeletons in your seat" effect when you visit the Haunted Masion in Disneyland. The "Floating Pepsi Can" effect you mention is probably a variation of the Pepper's Ghost illusion.

Point is... this is old, old stuff being passed off as "breakthrough" technology. Strictly kid's stuff compared to the things Lucas did (is doing) in Star Wars.

Same goes for the 3-D glasses you wear in IMAX theatres and the like. It's just the same thing that they used back in the 50's to show 3D monster movies, only they use electronic glasses. (With LCDs or vibrating shutters instead of polaroid lenses or colored gelatin.)

To make a totally immersive, "virtual reality" movie is going to take a LOT more technoligy than that.
 

It has been fun exposing the technology, which will grow whether or not we think it possible. I intend to keep up with it, but I'll let sleeping dogs lie and keep my discoveries closer to the chest in the future.

One of the primary reasons I brought the subject up here was that of all the theatres where I ever directly managed or projected, there is only one still being used as a movie theatre. I feel the march of progress more than some others here.

In retrospect, bringing information on any successor to digital into this forum might not have been the most brilliant idea I've ever had, and that error having been my responsibility, I take your comments in good humor. So, in the same spirit of fun and chain pulling, I'll close my remarks on the subject by saying that there is still an occasional need for good buggy whip makers, and when things _eventually_ do change (and I fully agree it won't be overnight), there may still be a good market for projectors designed in the 1940s and folks to fix them.

Long live the King.
No hard feelings, the next (insert your favorite beverage) is on me.

Now, mercifully invoking usenet law on the thread... "Hitler"

 

There have been a lot of articles in science magazines about the comming technologies of tomorrow. The more respectable mags like Sci-Am usually take the point of view of, "Here's a possibility for you..." I remember one magazine back in the 80's was saying that we'd all have cold fusion reactors powering our cars by the year 2000. That was laughable at best.

Can you re-cite the article so I can read it? Was it the one from this past April? I haven't gotten that mag for the last few months.

I REALLY am interested!

As Jerry Sinfeld said, "They told us we'd have flying cars by now... I want my flying car!!"

A company I worked for about 15 years ago was involved in a virtual reality training cockpit for a fighter aircraft. A one of many problems was having a video projector that was bright enough to fill a pilot's vision to simulate a moving landscape. They solved that problem (although at the cost of new problems) by using one very bright (and very expensive) projector, and several lower light projectors. (They would never get the bid if they used expensive projectors for everything.) There was a device bolted to the helmet to give the computer an X-Y positioning of the pilot's head. It was electronic, not mechanical, so as not to mis-read when the cockpit moved. As he moved his head, the bright projector moved to fill his main vision, only illumining the places where the pilot was actually looking. The others filled in his peripheral vision. This thing had several co-processors (mini-mainframes!), it's own super-optimumized graphics language. They stripped apart and rewrote the operating system to get it to run faster.

I guess my point is: It looked damn good, but was very expensive. And I think that's what we have here: Cool stuff that can be done, no doubt, but not avaiable for many years.
 

The retina contains two different types of photosensors -- rods and cones. Rods are very low resolution -- they mostly detect motion, and cannot detect color. Cones detect color. Interestingly, the rods and cones are not evenly distributed on the retina. The vast majority of the cones are concentrated in a little tiny spot near the middle of the retina, and they are responsible for the phenomenon of high resolution color vision. When you look straight at something, that's the part of your retina that you are using -- the high resolution patch right in the middle.

Outside of that tiny little center patch, your retina has much lower resolution, but better peripheral vision and motion detection.

Try this -- focus your eyes on the first letter of a paragraph of text, and, keeping your eye focused straight on that first letter , try and read the paragraph. You'll find that, even though the entire paragraph is within your field of vision, your brain cannot resolve detail outside of an extremely narrow range -- about 15 degrees.

Here is an excellent page describing the details of this phenomenon:
http://hyperphysics.phy-astr.gsu.edu/hbase/vision/retina.html#c2

How could this phenemonon be exploited in creating a display technology that is both beyond photo-realistic (reality-realistic?), and also computationally feasable?

One possible approach to designing an "immersive" simulation that is indistinguishable from reality would be to design a system that tracks the eyeball, so it "knows where you are looking", and continually presents an extremely high resolution image only where needed -- in the fovea centralis, while presenting a lower-resolution image in the peripheral vision. So instead of having to render some 200 degrees of ultra-high resolution graphics, the system would only need to render a 15 degree view in ultra-high resolution, and could render the rest on a lower resolution.

Of course, this is all speculative, but I think that it's worth keeping in mind that visual perception is non-linear, and the non-linearity of the human eye is an underexploited factor in display technology.

The intrinsic structure of film provides a natural "brute force" solution to the problem of image representation by providing very high resolution across the entire image -- an effective resolution of, as Greg said, some 13 megapixels,

But the sharp "sweet spot" in the eye only contains some 30,000 cones, which corresponds to a resolution of only 173 x 173. This is VERY low resolution.

So here's a proposed immersive VR system --

A low to medium resolution image, filling the complete 200 degree visual field, with an extremely high resolution "spot" superimposed on the low resolution image, where the location of the high resolution spot is determined by tracking the motion of the eyeball.

You might be able to get away with numbers like, 256x256 for the high resolution spot, and 1024x1024 for the low resolution field.

More food for thought.

 

(BTW, I recommend that anyone who has the chance to "fly" in a simulator take advantage of the opportunity. Through an odd set of circumstances, I managed to spend about fifteen minutes in an airline DC-9 simulator, and (surprisingly) managed to land the thing, albeit with a commercial pilot telling me what to do. The thing looks like a box on stilts, but the inside looks just like a DC-9 cockpit. There are CRT monitors to show the view out the cockpit windows. I've obviously never flown a real DC-9 [or anything else, for that matter], but the experience was amazingly realistic...and this was with late-'80s-vintage technology. I'm sure that the newer simulators are even better and more realistic.)
 

------------------
Greg Mueller
Amateur Astronomer, Machinist, Filmnut
 

Before, I was thinking that teleimersion was intended for use in presenting movies,etc. Actually its main use is for teleconferencing, although the article didn't rule out the possibility that it could be used for other things like medical research and ultimately, movies, I suppose.

Still, the article mainly looks at what the future of the technology might hold. Right now, they are using 6 or 7 video cameras, large video displays and either a pair of polarized glasses or a head mounted display. He also mentioned the use of a "computer glove".

The article talks about handling a three-way conference. It says, "Roughly speaking, a cluster of eight two gigahertz Pentium processors with shared memory should be able to process a trio [using 7 cameras] in approximately real time." It is estimated that, "Conveying a single person at a desk, without the surrounding room, at a slow rate of about two frames per second has proved to require about 20 megabits per second but with up to 80-megabit-per-second peaks."

Yes, there's a long way to go but the article gives a good look at the state of the art and the hopes of things to come.

May I add this -- there's a tolerably good possibility that many of us will live to be several hundred years old. Certain of us already are, the first century is the hardest; but that's another story. Thus we will see what you have described come to pass.

This will be just one of the benefits of genetic research and the mapping of the human genome. The biological clock will first be stopped, and then gradually reversed. Cloning technologies will eventually produce such mundane things as a new replacement heart and other knick-knacks.

However death can still occur as a result of an accident if we aren't careful: such as going off the road into a ditch on the way to an Elvis sighting somewhere in the middle of the next century.

Therefore: Drive Carefully!

 

Remember that we do not actually "see" anything. We "see" the light that is reflected off of the objects around us, and this light is slightly different between eyes, do to the spacing of our eyes. Since the principle of 3D photography involved controlling the light that hits the eyes to simulate how we see, no real 3D process will work unless you can control the light that arrives at each eye individually. At present time, this can not be done from the screen, it needs to be at the eye.

The 3D projection in Disney's Haunted House is a great gimick, but it is a trickery that is not transportable to general purpose presentations.

/Mitchell
 

With Field Programmable Gate Arrays, or FPGA's (computers that can reporgram themeselves and literally evolve) are already in use at NASA and being improved upon constantly. Read more, its educational: http://www.newsobserver.com/monday/business/Story/419010p-414835c.html
These kind of computers can solve the tasks that we would normally have to throw server farms at.
Like I said to begin with, kinda, this is just another example of Star Trek becoming reality. Did anyone ever doubt it?

------------------
Jonathan Haglund
Edwards Theatre Circuit, Inc
Park Place 10

Computers are used to design and optimize computers. Some circuits in a processor may be entirely computer designed and generated. Technology is fascinating.

I wonder if photon torpedos are just used xenon bulbs?
 

For example, using a CAD type schematic drawing program, you could draw a circuit of logic devices like AND, OR gates, flip-flops, shift registers, etc. to perform some function (no analog stuff like caps or resistors.) Then, copy that circuit into the gate array.

The gate array itself is an IC with a bunch of pins on it (you buy the size you need to fit your curcuit in.) It's not uncommon for a gate array to have over 500 pins on it.

You load a gate array somewhat like "flashing" an EPROM. It can be "erased" and reprogrammed if you want to change the logic. It operates much faster than an EPROM, so it can used where processing a signal through a microprocessor isn't fast enough or you want to save space by combining everything in one big chip.

That article make it seem like FPGA are new and leading edge technology. They are neat stuff, but have been around for about 10 years. Before that, there were simplified chips called PAL's (programmable array logic) and other similar chips (FPLS's; field programmable logic sequencers) that did much the same thing, but were smaller and could only be burned once.

One of the three novels dealt with a planet which had only about 400 people, each with 10's thousands of robots who did all the work. The "holodeck" process here was a wall of great complexity requiring a platoon of robots on each end twirling dials and adjusting to keep it working. Never did any of the people see each other in person, the distances were too great, but they visited constantly this way.

At the rate technology is moving today, I think none of us can say that "anything" is "impossible." 10-Gig computer processors are just around the corner.

Will projectionists be replaced by robots? Well, maybe it will be tried, but carmike cinemas et al will buy cheap robots which will break down quickly and I Can See It Now!!!! Carmike's El Cheapo robot broke down on the floor, short circuited, shooting sparks and all tangled up in several thousand feet of loose film.