What is decalibrating in 3D reconstruction with visual hull?

What is decalibrating in 3D reconstruction with visual hull?

by Noppawit Lertutsahakul -
Number of replies: 2

Hi,

I don't fully understand the slide 29-30-31 from shape from contour lecture.

What I understand is that we want to reconstruct a 3D shape from the contour (silhouette) image. Which can be done by taking a lot of photos around the object and reproject each silhouette to intersect each other. The more viewpoint (from a different angles) we have, the higher the detail of the convex hull.

For an example of reconstructing a cylinder. If we take a photo of this cylinder from 2 angle that is 90deg from each other, and reconstruct a 3D shape from that, you will get a cube, not a cylinder. As we increase the number of images taken around the object, eg: taking 1 image at every 5deg of rotation around the object, the edge of the reconstructed cube will get "cut away" by other images turning a cube into a cylinder.

To correctly reproject the silhouette into a 3D shape, you need to know the intrinsic and extrinsic parameter of the camera as shown in slide 28.

What I don't understand is in slide 29. If you have good calibration, then you have a good convex hull reconstruction as shown in the left figure, but what is "Decalibrating" on the right figure? Does it mean we do not have to calibrate the camera or what is it?

and what does it mean that "In practice, the silhouette of the visual hull is inside the original silhouettes"?

In reply to Noppawit Lertutsahakul

Re: What is decalibrating in 3D reconstruction with visual hull?

by Andrey Davydov -

Hi,

You understood the procedure correctly. Below I provide a few notes regarding your questions.


> what does it mean that "In practice, the silhouette of the visual hull is inside the original silhouettes"?

In theory the procedure works as you pointed out, but in practice cameras might not be calibrated properly. Further, I describe the logic. 

(1) "cameras are badly calibrated" -> (2) "Visual cones don't intersect" -> (3) "There are some parts of the object that have not been caught by a certain camera" ->

Finally, as the final hull (the predicted 3D body) is the intersection of all cones, (4) "The visual hull becomes smaller than the real shape of the body". 


> what is "Decalibrating" on the right figure? 

The term "decalibrate" literally means "to lose calibration" (see, (1) step above). Regarding the right figure, the hull reduction is not evident from the slide in pdf version, because originally it is an animation. You can catch it at timestamps 30:17-30:19 in this lecture video: https://tube.switch.ch/videos/6WW7NAct33. This is a practical illustration of what decalibration leads to.

> Does it mean we do not have to calibrate the camera or what is it?

The main message is the following. We might exploit this knowledge about decalibration and use it for the good. Assume cameras are decalibrated. We still might optimize cameras' parameters by maximizing the intersection of the silhouettes with the reprojections from the predicted hull. Because, as we know, "When the cameras are well calibrated the overlap of the reprojected visual hull and the original silhouettes is largest." (slide 31 with teapot animation).