Hide menu

Computer Vision on Rolling Shutter Cameras

Introduction

Most digital cameras sold today have rolling shutters. These cause geometric distorsions in the acquired images whenever either the camera, or the target is moving. This tutorial describes how classical projective geometry is modified to take a rolling shutter into account. We also cover recent research on how to adapt computer vision algorithms such as structure from motion and video stabilisation to rolling shutter cameras.

  • Time : Sunday, June 17th, 1:30pm - 4:30pm
  • Location : Room 551-B

Intended Audience

Due to the ubiquity of rolling shutter cameras, this tutorial is of general interest to the attendees of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR12). Especially researchers working in geometry based computer vision will benefit from following the tutorial.

Course Schedule

1:30 pm: Introduction, 30min [ slides ]

  1. What is a rolling shutter, and why should we care?
  2. When does computer vision fail on rolling shutter cameras?
  3. CMOS and CCD cameras, how do they work?
  4. Why are most new cameras of CMOS type, with rolling shutter?
  5. Electronic rolling shutters and mechanical rolling shutters
  6. Sensor readout times

Rolling Shutter Geometry, 15min [ slides ]

  1. Review of classical projective geometry
  2. Rolling shutter projective geometry
  3. A new degeneracy in two view geometry

Rectification and Stabilisation, 45min [ slides ]

  1. The full rectification problem, and sources of distorsion
  2. Distorsion models
  3. Three special cases for single frame rectification
  4. Interpolation and smoothing on SO(3)
  5. Optical flow estimation
  6. Camera motion estimation from optical flow
  7. Motion estimation from gyroscopes and accelerometers
  8. Work on multiple motions
  9. Work on parallax
  10. 3D rotation based video stabilization

3:00 pm: Break, 30min

3:30 pm: Rolling Shutter and the Kinect, 15min [ slides ]

  1. Kinect hardware overview
  2. Point cloud rectification
  3. Removal of the structured light pattern
  4. Motion estimation from optical flow and depth

Structure from Motion, 45min [ slides ]

  1. A state of the art pipeline for video structure from motion (SfM)
  2. SfM degradation, when and how
  3. Pre-rectification of point trajectories, pros and cons
  4. Rolling shutter bundle adjustment

Last updated: 2014-12-16