Frustration-free photo books

HP Labs launches Magic Photobook, a breakthrough solution to the most difficult and frustrating steps of photo book creation

by Margie Wylie, October 2009

Magic Photobook
Today’s low-cost, high-capacity digital cameras can be both blessing and curse. While family photographers no longer run out of film (or miss a shot while feverishly reloading), they can wind up with hundreds of pictures from a single event.

Photo books offer a potentially efficient way to turn heaps of snapshots into cherished keepsakes. Instead of printing photos one-by-one and then putting them in albums, people can arrange photos on pages, then print out entire albums.

At least, that’s the idea: "A lot of people who start photo books don’t finish them," says Qian Lin, director of the Multimedia Interaction and Understanding Lab (MIUL) at HP Labs. "It’s just too hard. It takes a lot of time and there are a lot of steps that have to be done manually."

HP Labs has been researching improvements to this challenging process since the idea was first conceived a decade ago, says Lin. Her group recently began sharing the results in a Web prototype. Magic Photobook (try it yourself) automates the most grueling steps of photo book creation, giving users a completed project that can then be customized to taste using intuitive tools.

Lin’s team calls the result “designer-inspired, user-generated photo books.”

Reducing the frustration factor

About 17 percent of US households have tried making photo books, according to a 2009 Photo Marketing Association International (PMA) survey. About two-thirds set photo books aside to "finish at a later time.” Half of the 17 percent have an average 2.5 unfinished books. Forty percent with unfinished books say the process is too tedious or difficult.

Magic Photobook

It's easy to see why: In a typical workflow, photos are streamed into the “holes” of user-selected, predefined templates. Every image is placed on a page, no matter how blurry or poorly composed. Some programs may sort photos by date, but the holes are filled without regard for the subject matter or whether an image should be displayed vertically or horizontally.

Any resulting random groupings or indiscriminate crops must be corrected manually. Users can either swap out poorly cropped photos or change page templates to accommodate the orientation of the photos they have. If the book needs more or fewer pages, it’s time to change templates and start again.

"It's one of the most challenging workflows to automate," says Lin.

Magic Photobook uses image analysis and good design principles to weed out undesirable photos and to group thematically similar photos together. It automatically chooses an optimal number of pages and populates them with the chosen images.

From there, users can customize the album to fit their tastes. They can add, delete or shuffle pictures and captions. They can change the number of pages in the book. With each change, the layout automatically adjusts without cropping, rotating or distorting images. It leaves no blank "holes" to be filled.

The 'magic' behind Magic Photobook

While humans may find sorting photos and assembling an album tedious, it's a fiendishly complex process to replicate in computer software, explains Yuli Gao, the HP Labs researcher responsible for Magic Photobook's algorithms for image selection, grouping and pagination.

"We automate the process in three basic steps: Quality-based image filtering, near duplicate removal and theme-based pagination," says Gao.

Magic Photobook

First, photos are passed through two filters that rate the quality of the image based on focus and color variation. The "blurry filter" detects the strength and frequency of image edges. The "boring filter," as Gao calls it, weighs color variation across the image. So, the sharper and the more colorful an image appears, the higher its quality score.

Next the system searches for duplicate and near-duplicate images by examining time stamps and matching up similar color blobs and "key points," such as the corners of a door on a building. Among similar photos, higher-quality images and those that contain detected faces are rated more "interesting," and thus more likely to be used in the layout.

Once the low-quality images and duplicates are removed, the image collection is clustered into "themes," based on time, color and face similarities among the chosen images. "The goal is to group those photos into meaningful arrangements," Gao says. For example, a group of kids riding on a pony might constitute one theme.

After calculating the strength of the relationship between photos, the algorithm partitions the photos into groups. Photos with the strongest connectedness are grouped on the same pages. Themes are preserved across facing pages, or spreads.

The resulting pages are then fed into Magic Photobook's layout engine with relative weights that reflect their “importance" within the page group and thus their relative size on the page.

Finally, the layout engine generates a photo book based on the images it receives. "It takes virtually any collection of photos and arranges them on the page in a way that makes good use of the space while preserving existing aspect ratios," says Brian Atkins, the HP Labs researcher who wrote the layout engine's algorithm, called Block Recursive Image Composition, or BRIC.

User-perfected photo books

No matter how well the technology works, however, people can't be automated out of the process, says Lin. So, a simple, intuitive interface was essential.

Magic Photobook lets users edit the layout directly and see the results instantly. A drag-and-drop interface makes moving photos around simple. Delete, auto-crop and auto-enhance controls are accessible from within each photograph. Captions can be typed directly into the page. The BRIC layout engine automatically adjusts the layout in response to changes.

Magic Photobook

Even poor quality photos excluded from the layout can easily be added to a book. A blurry picture of baby taking her first step, for example, is a shot you'll never get again, Gao points out. "We are using engineering semantics to approximate human reasoning, not replace it," he explains.

Snapfish, the HP-owned photo sharing site, has begun integrating some of Magic Photobook's technologies into its photo book production process, starting with the BRIC layout engine. Dubbed Magic Layouts, "it lets users create photo book layouts that don't even exist in our current library of templates," says Jaya Savkar, a Snapfish product manager. "This will free up our users to be more creative than ever."

And that's exactly what Magic Photobook is about, Atkins sums up: "It's a daunting task to sit down with 150 photos and try to pull out a theme and tell a story. We're trying to make that not just possible, but easy."