Multi-View Geometry and Computational Photography using Non-Classical Cameras

Organizers: Srikumar Ramalingam (MERL, USA), Amit Agrawal (MERL, USA)
Duration: half day
Abstract: This tutorial is meant as an introduction to the design, modeling and implementation of non- classical (multi-perspective) cameras for several computer vision and computational photography applications. The tutorial will provide an overall view of developing a complete system (capture, modeling, and synthesis/reconstruction) as well as provide sufficient details for calibration and modeling such non-central cameras. We hope to provide enough fundamentals to satisfy the technical specialist as well as tools/software’s to aid graphics and vision researchers, including graduate students.

Outline:

  • Section 1: Introduction to Multi-perspective Imaging (capturing more than just pixels)
    • Geometry of rays in non-central cameras
      • Pinhole Cameras
      • Axial Cameras
      • Locally Central and Light Field cameras
      • General Non-Central Cameras
    • Applications (Examples):
      • Digital Refocusing using Light Field Cameras (Lytro etc)
      • Wide-Angle 3D reconstruction
      • Wide-Angle Digital Refocusing using Array of Spherical Mirrors
      • Localization in Urban Canyons
  • Section 2: Modeling Multi-Perspective Cameras
    • Traditional Approaches
      • Pinhole Approximation
      • General Linear Cameras (GLC)
    • Analytical Projection Models
      • Catadioptric Cameras with Camera on Mirror Axis
      • Catadioptric Cameras with General Camera Placement
      • Dioptric Camera with Spherical Ball
    • 4D Light Fields
      • Fourier Slice Theorem
    • Calibration of Non-Central Cameras
      • Plane based calibration
      • Generic Calibration
  • Section 3: Synthesis and Rendering
    • 3D Reconstruction
      • Plane Sweep and Bundle—adjustment for non-central cameras
      • Depth from focus for Light Field Cameras
    • Novel View interpolation
      • Generating un-distorted and perspective views
    • Digital Refocusing
    • Other Applications such as geo-localization
  • Section 4: Future Directions
    • Beyond Conventional Rays: Thinking in Light-paths
    • Theoretical Directions
    • A list of unsolved problems
    • Sharing of code, datasets

Material:

Matlab Code for modeling several catadioptric systems is already available on co-authors website at http://www.umiacs.umd.edu/~aagrawal/software.html. Course slides will be posted at http://www.umiacs.umd.edu/~aagrawal