Computer Vision

Recovering 3D Structure from Autostereograms

An automated pipeline for recovering the three-dimensional surface that is encoded within a single-image autostereogram, commonly known as a “Magic Eye” image.

• Computer Vision
• Autostereograms
• Depth Maps
• Python
• JavaScript
• three.js

Background

An autostereogram is a single two-dimensional image that encodes a three-dimensional surface. The encoding is produced by repeating a narrow pattern horizontally while varying the repetition period slightly from one column to the next. When an observer diverges the optical axes so that two adjacent copies of the pattern are fused by the visual system, the local variations in period are interpreted as binocular disparity, and a depth percept is produced.

It follows that the depth information is physically present in the pixel data, represented as spatial variation in the local horizontal period. Because the human visual system is able to decode this representation, the same information is recoverable by computational means. The present tool implements that recovery.

Method

Given a single autostereogram, the analysis proceeds in four stages:

• Period estimation. The dominant horizontal repetition period is estimated by autocorrelation. This period defines the baseline against which all subsequent disparity is measured.
• Base texture recovery. The single pattern tile that was replicated across the image is reconstructed.
• Depth estimation. Each region is matched against its shifted neighbour by block matching. The local disparity, defined as the deviation of the measured period from the baseline, is converted into a depth value at each pixel.
• Surface reconstruction. The resulting depth map is lifted into a mesh or point cloud, rendered in three dimensions, and exported either as a raster image or as a Wavefront .obj point cloud.

Implementation

The reference implementation is a desktop application written in Python using NumPy, Pillow, and Matplotlib, with a PyQt5 interface. The complete pipeline was subsequently ported to JavaScript and deployed as a browser application that executes entirely on the client, so that images are not transmitted to any server. The computationally intensive analysis is performed in a Web Worker in order to keep the interface responsive, and the recovered surface is rendered using three.js. A random-dot stereogram generator is included so that the system can be evaluated without external input.

# serve the web application locally
cd docs
python -m http.server 8000
# then open http://localhost:8000

In the live application the procedure is as follows: a sample is generated, or a user-supplied stereogram is provided; parameters are adjusted; analysis is performed; and the base texture, depth map, and interactive three-dimensional surface are inspected and, if required, exported.