Stereo Camera Bumblebee - NHL Centre of Expertise Computer

Computer Vision: 2D Camera Calibration
26-aug-14
Computer vision
3D Stereo camera Bumblebee
25 August 2014
Copyright © 2001 – 2014 by
NHL Hogeschool and Van de Loosdrecht Machine Vision BV
All rights reserved
Thomas Osinga
[email protected], [email protected]
3D Stereo Camera Bumblebee
Overview:
•
•
•
•
•
Introduction to stereo vision
Camera stereo parameters
Accuracy with different baselengths
Examples
Advantages / Disadvantages
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
2
1
Computer Vision: 2D Camera Calibration
26-aug-14
Introduction to stereo vision
26-aug-14
3D Stereo camera Bumblebee
3
Introduction to stereo vision
• Stereo vision camera’s are used to perform 3D measurements
• Stereo vision is based on the human eyes:
The camera takes two snapshots from different positions.
When a certain object can be identified as a pixel location in
one image and in the other image, then the distance can be
calculated based on the translation of the object pixel
• Some problems in stereo vision:
• Identifying of pixels in multiple images for matching the
same world coordinates
• Correct calibration of both camera’s, so the pixels can be
correlated
• Less accuracy on larger distances
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
4
2
Computer Vision: 2D Camera Calibration
26-aug-14
Introduction to stereo vision
• Identifying of pixels in multiple images for matching the same
world coordinates:
This can be solved with stereo vision algorithms. There are
many algorithms available. A stereo vision SDK is delivered
with the BumbleBee camera’s.
26-aug-14
3D Stereo camera Bumblebee
5
Introduction to stereo vision
• Correct calibration of both camera’s, so the pixels can be
correlated
A stereo rig is used to calibrate the camera’s. The images have
to be mapped to a pin-hole camera model. This image is called
rectified.
Raw image with lens distortion
26-aug-14
Rectified pin-hole image
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
6
3
Computer Vision: 2D Camera Calibration
26-aug-14
Introduction to stereo vision
• Less accuracy on larger distances
The distance calculation is based on the following equation:
Dist ( m ) =
f ( pix ) ⋅ base( m )
Disp ( pix )
in this equation f is the focal length of the lens in pixels. The
disparity is the difference in x-direction of the pixel
coordinates in both images
26-aug-14
3D Stereo camera Bumblebee
7
Introduction to stereo vision
•
Less accuracy on larger distances
When the distance according to each pixelvalue is plotted, the
following graph will appear
Disparity
-1,0033
y = 184,35x
2
R = 0,9972
16
14
Afstand(m)
12
10
Disparity
8
Macht (Disparity)
6
4
2
0
0
20
40
60
80
Pixelwaarde
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
8
4
Computer Vision: 2D Camera Calibration
26-aug-14
Introduction to stereo vision
To get a correct depth image, a few steps need to be taken:
26-aug-14
3D Stereo camera Bumblebee
9
Camera stereo parameters
• Mode
The BumbleBee camera’s have 7 different modes:
1: Raw Image
2: Rectified Color
3: Rectified
4: Disparity (this gives a depth image)
5: Disparity Color (this gives a depth image in false colors)
6: Disparity Validation (when certain area’s are not volidated in
the vision algorithm they get a certain color)
7: Absolute (in this mode the absolute world coordinates are
given)
• Pan
With this parameter the user can choose witch camera’s are
used to perform stereo vision algorithm’s
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
10
5
Computer Vision: 2D Camera Calibration
26-aug-14
Camera stereo parameters
• Disparity
The disparity range is the range of pixelvalues in wich the
stereo algorithm searches for a best match. A disparity value
of 0 means that an object is unlimited far away. A maximum
disparity value means that this is the closest distance that an
object can be measured
• Disparity Mapping
With disparity mapping the user can define a pixel range in
wich the result pixels will be shown. This is comparable with a
contrast stretch.
• Stereo Mask
The user can define the size of the mask that is used to
correlate both images.
• Edge Mask
The user can define the size of the edge mask that is used to
correlate both images.
26-aug-14
3D Stereo camera Bumblebee
11
Accuracy with different baselengths
• To achieve higher accuray a bigger baselength is needed.
When measuring the same distance with the same stereo
camera’s on a bigger base. The accuracy will be like the
following graph
Nauwkeurigheid (m)
Nauwkeurigheid
2,3
2,2
2,1
2
1,9
1,8
1,7
1,6
1,5
1,4
1,3
1,2
1,1
1
0,9
0,8
0,7
0,6
0,5
0,4
0,3
0,2
0,1
0
Nauwkeurigheid
0
1
2
3
4
5
6
Basis (m)
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
12
6
Computer Vision: 2D Camera Calibration
26-aug-14
Examples
• Rectified Color Image
26-aug-14
3D Stereo camera Bumblebee
13
Examples
• Disparity image (on 3m distance)
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
14
7
Computer Vision: 2D Camera Calibration
26-aug-14
Examples
• Disparity image (on 2m distance)
26-aug-14
3D Stereo camera Bumblebee
15
Examples
• Disparity image (on 1m distance)
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
16
8
Computer Vision: 2D Camera Calibration
26-aug-14
Advantages / Disadvantages
• Advantages
• High accuracy can be achieved by using the correct base
• Accuracy on different distances can be calculated
• When focal length of the lens is known, the needed base
can be calculated for good accuracy
• Disadvantages
• Not possible to calculate distance of every pixel in the
image
• Accuracy is not linear
26-aug-14
3D Stereo camera Bumblebee
Jaap van de Loosdrecht, NHL, vdLMV, [email protected]
17
9