Jump to content United States-English
HP.com Home Products and Services Support and Drivers Solutions How to Buy
» Contact HP

HP.com home


Movie makeover

HP and Warner Bros. give old films new life


HP Labs

» Research
» News and events
» Technical reports
» About HP Labs
» Careers @ HP Labs
» Worldwide sites
» Downloads

See a frame from "King Kong" before and after processing.
»See a frame from "King Kong" before and after processing.

Content starts here
Classic films are fragile treasures, but a new system that automates film restoration could preserve them for future generations to enjoy.

By Simon Firth, May 2006

It’s that big scratch across Scarlett O’Hara’s face, the red blur obscuring Robin Hood’s heroics, or the white blotches that pepper King Kong as he climbs the Empire State Building.

Classic movies are fragile treasures. They pick up dust, are easily scratched, and often shrink or fade and get otherwise disfigured as they age.

The most famous will occasionally be painstakingly restored by hand, says HP Labs’ Qian Lin. But of the thousands of other movies sitting in archives around the world, Lin says, “very few go through this manual processing, because the cost is very, very high.”

That’s a big deal to Hollywood studios like Warner Bros., owners of Gone With the Wind, Robin Hood, King Kong and thousands of other classic features.

HP researchers, in collaboration with Warner Bros. Motion Picture Imaging, have come up with a process for automating much of the restoration process and are now looking to bundle their set of media-processing techniques into a single engine, or video-processing pipeline. It’s already yielding some spectacular results.

In test clips Scarlett now sweeps down the stairs to meet Rhett unaccompanied by background glitches; Robin Hood fights his battles in glorious, sharp Technicolor; and King Kong climbs through a clear Manhattan sky.

“We have many of the most beloved and important films ever made in our library, and we feel a strong responsibility to preserve these films so that future generations will be able to enjoy them,” says Chuck Dages, executive vice president of Emerging Technology at Warner Bros. “And from a pure business standpoint, these films are an invaluable asset and need to be properly cared for."

Automating clean-up

Studios need to re-copy their movies every time a new distribution technology comes along, says Dages.

Following VHS video and then DVD, the industry is now moving toward the next generation of optical disc formats: High Definition DVD (HD-DVD) and Blu Ray formats. The new formats offer studios a great economic opportunity, Dages says, but a challenge too.

“Older movies that have been transferred to video,” he explains, “just don’t cut it when they go to the high-definition television formats”

In response, studios are making super-high resolution digital transfers of each frame of many of their library movies. But without special processing, whatever dirt, scratches or other imperfections that mar the original frames are transferred to the digital copies; all in high definition.

Until now the only way to get rid of imperfections, explains Dages, was to have “someone sit with a mouse and laboriously click to take the dirt out.” No wonder, he says, Warner Bros. was hoping “that there was HP technology that could be applied to help automate the process.”

Unique identifiers  

Although HP Labs had never looked at film restoration, researchers had plenty of experience working with still photography and personal video.

“We found that a lot of that expertise can be applied to the film restoration process,” says Lin, who leads the research team.

Researchers were able to apply much of what they'd learned from an earlier investigation of color-sensor technology to address blurring of images in older color library titles. Work on single- lens cameras that de-blurred an area of color using edge information from other patches of color in the frame also proved useful, as did research into video super-resolution, a technique for combining information from nearby frames to increase a particular frame’s resolution.

How it works

The automated restoration process works like this: Each film clip is first run through a set of algorithms that selects out only the frames likely to contain an image artifact. For example, says researcher Amnon Silverstein, “we look for a pixel that changes color dramatically from frame to frame. Or within a frame we look for a small area that is very different from everything else in its neighborhood. And in color, we look for things that are saturated and unique.”

A second pass fixes many of the glitches. When dust creates a bright red, green or blue spot in the frame, for instance, the researchers' software fills it in with information interpolated from the area around it.

Although some particularly complicated frames must still be repaired by hand, automated fixes dramatically reduce the amount of manual work required.

In the cases where the team had access to scans of the original set of color negatives, the images took on a new life.

“Thanks to this technique, you can now look at the chain link in Robin Hood’s armor or the lace in Scarlett O’Hara’s dress and they pop out with incredible detail and a purity of image that you never saw before,” says Richard Place, global manager for all of HP’s Time-Warner accounts.

Different kinds of film offer different challenges

Digitized frames from black and white movies, for example, contain less information, making scratches, dust and other errors harder to spot. Early black and white film also tends to look grainy and suffer from flicker, a result of the poor shutter control on early cameras, giving each frame a different exposure.

Perhaps the team’s biggest challenge was Cinerama. Only 13 features were made using this widescreen technique – a format that requires a cinema equipped with a curved screen and three projectors that run simultaneously – but nearly all are considered classics.

The HP team worked on clips from the 1962 epic, How the West Was Won. Beyond the usual dirt and scratches, Cinerama films doubly distort when transferred to home-viewing formats. Because they were projected onto a curved screen, they look odd when shown on a flat screen. And they suffer from clear overlaps and lines where the three images join.

“To resolve the vertical-line issue, the team borrowed technology for putting ink on paper, which is very similar to putting ink on film,” reports Richard Place. “They then used another algorithm to eliminate the flattening distortion.”

The result, says Warner Bros. executive Dages, “is remarkable. The technology really has turned what has been an issue for many, many years into a solution that allows us to repurpose the film.”

Speeding up the process

HP and Warner Bros. Motion Picture Imaging envision a process that would either run automatically or be ‘dialed’ up or down in sensitivity depending on what is happening in a particular sequence (the more action, the more likely that one of flying arrows, for example, might be mistaken for a scratch, requiring the software’s sensitivity to be turned up).

Cleaning entire films requires enormous computing resources.

“These films take up terabytes of storage,” says researcher Silverstein. “So we’ve been looking at how you can take a film, send it out across a set of servers, process it efficiently, and handle cases when machines are added and removed.”

Each two-hour movie has 172,800 frames – if you're processing that on a single machine at about five minutes per frame, it would take 600 days to complete an entire movie. By using a cluster of servers, engineers can speed up the process considerably.

Such a process would employ state-of-the-art data security from HP, a crucial feature for content owners worried about video piracy.

Film remains archiving choice

It’s not just Hollywood that could benefit from the innovations behind the video-processing pipeline.

There are major theatrical movie archives in India and Hong Kong, for example, as well as large research holdings in public collections like the U.S. Library of Congress.

Although digital is gaining ground, film remains the medium of choice for making movies and high-profile television dramas.

And despite its flaws, says Warner Bros.' Dages, film continues to be the studio’s preferred archival material. “We know,” he says, “we can put a piece of film in the vault and under proper conditions, it will last up to 600 years. It is a great archival media."

Simon Firth is a writer and television producer living in Silicon Valley.

Related links

» Imaging systems research
» Warner Brothers

News and events

» Recent news stories
» Archived news stories

See a frame from "King Kong" before and after HP Labs' film processing.

» See a frame from "King Kong" before and after HP Labs' film processing.

See a frame from "How the West was Won" before and after processing.

»See a frame from "How the West was Won" before and after processing.

See a frame from "How the West was Won" before and after processing.

» See a frame from "How the West was Won" before and after processing.

© All images used in connection with this story appear courtesy of Warner Bros. and are protected by copyright.    
Printable version
Privacy statement Using this site means you accept its terms Feedback to HP Labs
© 2009 Hewlett-Packard Development Company, L.P.