Revealing hidden details

by Steve Towns
Editor, HP Government Solutions

FBI agents investigating a man suspected of murdering young children recently turned to an innovative three-dimensional imaging technique developed by HP Labs to help scour for evidence.

The suspect kept a notebook listing the names of his alleged victims, but he'd torn out the incriminating pages before his arrest. Luckily, a faint imprint of his handwriting remained on the underlying blank pages, and authorities were able to reveal the hidden information using HP Labs' polynomial texture mapping (PTM).

"We photographed the pages, and using PTM, we made the surface of the paper appear highly reflective, which let authorities see those indentations quite a bit better than they could with the naked eye," said PTM inventor Thomas Malzbender, a distinguished technologist in HP's Mobile and Media Systems Laboratory.

After three days of working with the notebook, investigators were able to recover the name of an additional victim.

The evidence never was used during the trial, but the case offers a glimpse of why PTM is drawing the attention of not just other law enforcement agencies, but specialists in such fields as paleontology, archaeology, geology and even art.

PTM at work

These experts have used PTM to get a clearer picture of everything from plant fossils to painters' brush strokes to native American carvings.

Paleontologists at the Swedish Museum of Natural History used PTM to see sharp images of trace fossils of primitive life on a 500 million-year-old stone. Curators at the National Gallery of Art in London, working with PTM on a painting by Pointillist Georges Seurat, were able to see a sailboat the artist had painted over in the course of his work. Other recent PTM users include the San Francisco Police Department, the Tate Britian in London and the West Semitic Research Project at the University of Southern California.

"One of the things about PTM is that it's extremely visual. You just show it to poeple, and they get enthusiastic," says Dan Gelb, an HP Labs researcher who has worked with Malzbender on the technology.

How it works

PTM captures images of three-dimensional objects, then allows users to shine a light on that image from various angles to reveal hidden surface details. The technique also allows users to change the color of the lighting or modify the characteristics of an object's surface, transforming it from dull to metallic, for example.

The result is what is known as a parametric photograph -- something that allows users to dial a couple of knobs to change the position of light in a photograph. The technology is fast, too. It runs in real-time on consumer-level PCs.

Because it is a purely photographic technique, PTM is also much less expensive than other methods of producing images of three-dimensional objects, which rely on costly, elaborate laser scanning systems. PTM is simpler to use, too, because it involves only the complexity of taking the photographs.

To produce a PTM model, users capture between 30 to 50 digital images of an object -- each with the light source in a different location. That's done using a special automated dome with flashbulbs lining its interior and a digital camera mounted in a hole at the top. The device is linked to an HP laptop that automatically collects 50 images at the touch of a key.

(Free software to view and create PTMs is available here.)

Now it's portable

Although the device can be shipped whereever it's needed, the dome can be somewhat cumbersome to use, particularly in a field environment. So Malzbender's colleage Gelb developed a more portable method, gathering images for PTMs using simply a digital camera, a tripod and a shiny object.

"You put a shiny, reflective object into the scene you're photographing, then take a number of pictures while moving around the scene with a handheld flash," Malzbender explains. "It turns out that with a reflective object in the scene -- a cue ball works very well -- you can recover the position of the light source."

The simplified technique will allow forensic specialists to produce PTM images of footprints in the ground, for example, without special equipment.

Shared intelligence

Not only are PTM models relatively simple to create, they're easy to share, because the technology produces an extremely compact representation of an object. PTM models are just 30 percent to 40 percent larger than a single image of an object.

Interactive PTM images easily can be posted to Web sites for wide distribution, which is particularly useful for research disciplines such as archaeology and paleontology, where researchers are reluctant to part with priceless physical artifacts.

Last year, HP researchers teamed with a number of well-known paleontologists to publish a paper on Cambrian trace fossils that were roughly 500 million years old. They published the paper online, which gave readers the ability to download PTMs of the illustrations and try different enhancement methods themselves.

Now the researchers are teaming with several universities to digitally share archaeological artifacts among scholars and make them available on the Web.

Unexpected consequences

PTM's usefulness extends well beyond Malzbender's original intent when he cobbled together the dome's prototype in his basement. The researcher was attempting to solve problems with existing three-dimensional rendering technologies, looking for a way to improve photorealism and image rendering efficiency.

Although the technique is finding its way into sophisticated graphics accelerators inside HP products, PTM also is poised to reshape fields ranging from police forensics to diagnostic medicine.

"It turns out there are a lot of fields that deal with objects that have interesting surfaces on them, and they can benefit from looking at those surfaces under variable lighting conditions," Malzbender says.