Robot Vision Systems

Course goals

This course will address the topic of implementing robot vision systems using ROS and OpenCV with a special focus on visual computing for robotics. That means, we will use ROS and OpenCV as platforms for implementing own computational schemes and estimation methods for own vision models and visual representations for robotic applications. Combining those schemes and methods with high-level tools provided by ROS and OpenCV is intended to be done in the project part of the course, whereas the lectures concentrate on the fundamental data structures, primarily for dense and sparse matrices, how to manipulate and extend those for your own purposes, how to efficiently use these in ROS nodes and nodelets providing topics and services.

Prerequisites

Course participants are expected to have a solid background in mathematics (linear algebra, numerical methods), signal/image processing, computer vision and C++ programming. Bring your own laptop with internet access and admin rights to install software. Camera supported by Ubuntu 14.04. Access to robotic hardware (if you want to go beyond simulated robots).

Course organization

The course will consist of

lectures, where core features of OpenCV 3 rc1 and ROS will be presented
seminars, where the course participants are supposed to prepare presentations of selected topics
exercises, where the participants should bring their own laptop and will go through essential steps
a project, where the participants are supposed to implement an example application

This course will be based on the OpenCV course given 2012-2013. Compared to that course, we will extend the contents and the number of credits (6+3 instead of 3+3). The course will give 9hp credits (=6 weeks full time work) if the student conduct successfully a project work, is present at least at 80% of the lectures, exercises and seminars, and gives at least one own presentation at the seminar. Without the project work, but otherwise the same rules as above, the course will give 6hp. If the course is only attended 'listen-only', no credit points will be given. Student who have participated in the OpenCV course 2012-2013, may attend the second (ROS) half only, giving 6hp (3hp without project).

Instead of making a manual install of Eclipse, OpenCV, and boost, we will look into the option of using Ceemple. We are also looking into various options of Python packages, e.g. Python(x,y) and WinPython (as well as corresponding Linux variants). We will use OpenCV 3.0 rc1. We are further aiming at using ROS Indigo in a VirtualBox running Ubuntu 14.04 LTS (unless you are running Ubuntu natively) or possibly ROS Jade.

Preliminary schedule

The course will be given during the end of the spring term 2015 and the beginning of the autumn term 2015 . All lectures, exercises and seminars will take place in 'Algoritmen' ( map ). The schedule below might be subject to updates.

lecture 1: course information and OpenCV history . April, 30, Thursday, 13.15 - 15.00
exercise 1: installation of Eclipse, Python, and OpenCV (see links in lecture 1). To be done individually.
lecture 2: dense matrices . May 4, Monday, 15.15 - 17.00
lecture 3: methods on dense matrices . May 7, Thursday, 13.15 - 15.00
seminar 1 (topic 1-4). May 11, Monday, 15.15 - 17.00
seminar 2 (topic 5-8). May 18, Monday, 13.15 - 15.00
lecture 4: sparse matrices and methods . May 20, Wednesday, 15.15 - 17.00
exercise 2: adding functionality to sparse matrices . May 21, Thursday, 13.15 - 15.00
lecture 5: building your own representation . May 25, Monday, 15.15 - 17.00
exercise 3: build and test your own representation . May 27, Wednesday, 15.15 - 17.00
lecture 6: rapid prototyping using Python and Ceemple . May 28, Thursday, 13.15 - 15.00
lecture 7: good design principles . June 1, Monday, 15.15 - 17.00
lecture 8: debugging in OpenCV , Python Wrappers in OpenCV . June 2, Tuesday, 15.15 - 17.00
exercise 4: Python: basics and prototyping using OpenCV . June 4, Thursday, 13.15 - 15.00
lecture 9: ROS - history and installation . October 19, Monday, 15.15 - 17.00
lecture 10: ROS - basics and shell extensions . October 21, Wednesday, 15.15 - 17.00
lecture 14: ROS nodes with Python . November 19, Thursday, 8.15 - 10.00
lecture 15: ROS nodes with C++ . November 30, Monday, 15.15 - 17.00

Seminar topics

Participants might think about which topic they would like to present in the seminar. The topics are typically required C++ topics that are essential to programming in OpenCV, but are not part of OpenCV. If there are more course participants than topics in this list, we will extend the list appropriately. We will assign seminar topics during the first lecture. Each seminar should start with a presentation of about 20 minutes, including examples and time for discussion. This allows us to fit 4 seminar talks into a 2-hours block.