Chromaglyphs

A file of correspondence points is generated for each image in the input image set. Each line contains the 3D world coordinates of a glyph followed by the image coordinates for the same glyph.

Finally, camera pose is calculated and a transformation matrix is created. We perform this step with Reg Willson's implementation of Roger Tsai's camera calibration algorithm.

Glyph color space

Glyph composed of N discs » Larger view

Each glyph is composed of N discs, each with a unique color, chosen from a set of M prototype colors. In the case shown above N=3, M=8. The permutation of disc color uniquely identifies the glyph and is used to lookup the 3d coordinates of the glyph. We must reliably identify these colors under these conditions:

• Variations in view direction
• Shading variations due to scene geometry
• Printing nonlinearities and noise
• Imaging nonlinearities and noise

The image below demonstrates the printing and imaging nonlinearities (photoCD based) involved in mapping the corners of the color space into imaged RGB space for a single view. The nonlinearities are extreme and cause us to limit the number of prototype colors to 6. We reject magenta due to its proximity to red and reserve white as a background color.

Demonstration of printing and imaging nonlinearities (photoCD based) involved in mapping the corners of the color space into imaged RGB space for a single view » Larger view

Limitations of nearest neighbor classifiers

Determination of glyph identity requires classifying each pixel as belonging to one of the M=6 prototype colors. A simplistic approach to this is to classify each pixel in the image as being associated with the closest prototype color in color space. This nearest neighbor classification fails at boundaries between regions where pixels see contributions from more than one color. Note the false colors appearing in the classification below. These difficulties are best understood by looking in color space, a 2D projection of which is shown below. Aliased pixels are shown as small spheres between two of the larger spheres, which represent the prototype colors. Note small amounts of noise along the diagonal between prototypes can perturb the pixels and cause them to be misclassified. We choose to classify each pixel as either pure or aliased, where aliased pixels are some threshold distance away from their closest prototype color. We use only pure colors for determining glyph ID’s, as these are never misclassified.

		Glyph pixels in projected color space » Larger view
		Raw image of a Glyph » Larger view
		Classifiaction of Glyph pixels based on nearest neighbors » Larger view

Occluded Glyph rejection

We must also not misidentify occluding geometry as a glyph. We employ a number of heuristics to ensure that the glyph centers are accurately reported. The glyph extraction method extracts connected regions of foreground colors (non-white), classifies each pixel in the foreground region as belonging to one of 6 prototype colors and then checks the distribution of foreground colors.

• Only N=3 prototype colors are seen in the region. Disregard any region with more than 5% of its pixels not in the top 3 ranked, classified colors.
• The number of pixels in the top 3 ranked colors corresponds to a distribution expected from a glyph geometry of concentric overlapping discs.
• Disc regions are concentric. The spatial centroid of the top 3 ranked pixel colors should be within a few pixels of each other.
• Pixels in the outer band should have a larger standard deviation of position than those in the inner bands. The rank of the the top N=3 color histogram must match the rank of the top 3 spatial standard deviations.
• A glyph must exceed some minimum size to be usable.
• The glyph ID implied by the top 3 ranked colors must have actually occurred on the particular chromaglyph sheet imaged.

Connected region color histogram » Larger view

In addition, we can classify each pixel as being either an pure or aliased pixel. Pure pixels are within some threshold distance in color space to one of the M=6 prototype colors. Aliased pixels are past this threshold distance in color space and presumably are pixels on region boundaries that have contributions from more than one color region. There are further tests we can apply based on this classification.

• Exactly N=3 types of pure pixels are seen in the region.
• Each of the N=3 top ranked buckets must have some minimum number of pixels.

Pure pixels classification » Larger view

Download a sample (.gif file, 209KB) of Chromaglyph wallpaper.