Samenvatting

Axon Digital Design develops modular systems and equipment for audio and video signal processing and monitoring, mainly aimed at the broadcasting industry. The recent successes of 3D film productions have pushed the consumer TV market into a new era: 3D television in the living room. One fundamental problem however is the lack of 3D-enabled content. Producing native stereoscopic 3D video is time consuming and costly. It requires broadcasters to invest in new expensive equipment like stereo cameras and stereo rigs and to hire specially trained stereographers. Realtime 2D-to-3D conversion is a cheaper option since it requires additional hardware only. Unfortunately this is an extremely difficult task for which no optimal generally applicable solution exists.

At Axon Digital Design, an efficient method for the conversion of shallow depth-of-field material has been developed. This method is based on the focal blur cue and it reconstructs the scene depth based on different blur values in the image. Unfortunately, this approach fails for material with a deep depth-of-field in which (nearly) all objects are in focus. Therefore the conversion system needed to be expanded, allowing it to leverage other depth cues as well. An additional depth estimation algorithm based on linear perspective was developed. It leverages dominant lines in the image to find a vanishing point. The detected location is used to reconstruct the scene depth, by generating a depth map with increasing depth towards the vanishing point. Viewer perception tests have shown that this method yields a natural and more realistic depth experience compared to the gravity model. In the gravity model, depth is determined by the vertical position in the image only.

The fusion of these algorithms is based on an estimation
of their confidence. For the focal blur method, the standard deviation of blur values is estimated. The confidence of the linear perspective method is determined by the spatial drift of vanishing point locations across time. Due to the nature of the current depth estimation algorithms, the depth cues they depend on are hardly ever concurrently available. Therefore the fusion module will simply select the best fitting algorithm rather than combining multiple methods simultaneously.

If the focal blur method is considered a confident option, it will always prevail over other methods. Else, if the detected vanishing points are temporally stable, the linear perspective method will be considered the best fitting algorithm. In the special case where neither method returns satisfactory results, the gravity model is applied as a fallback.