Hello,
I took a picture of a flat-ish object from the top and I would like to fit it over the surface of the object's mesh model. I transformed the image in a cloud of points, one line for each pixel in the format x, y, z, r, g, b and I placed it over the mesh. Z is obviously constant because the image is 2D, I only know the center point of the picture
What I would like to do now is adjust the Z value of each point (pixel), in other words to "bend" the picture over the mesh.
Best Fit doesn't work because it applies a rigid transformation to the cloud, it doesn't "bend" it. Querying the CAD model asking for the Z value of each XY is doable but very painful.
Can CloudCompare simply take every point of the cloud and place it on the nearest surface point of the mesh? How?
Thank you very much for your help
Fit images to a mesh surface
Re: Fit images to a mesh surface
Interesting. The best way to achieve this would be to compute the (vertical) distances of your image points to the mesh.
This will give you a scalar field. Then you can also convert the 'Z' coordinates of the image points into a scalar field (equivalently, if the image points have a constant height you can directly create the corresponding constant scalar field). Then use the SF Arithmetic tool to substract the 'Distances' scalar field from the 'Height' scalar field. This should give you the new heights if I'm not mistaken. Eventually you can convert this scalar field back to the Z coordinates of the point.
The main issue will be to get the truly vertical distances! You can first try with the classical cloud/mesh distances but they might not be exactly vertical (as it's a nearest neighbor distance). But the qM3C2 plugin is able to compute signed vertical distances between two clouds. To use it you could simply sample (a lot of) points on the mesh first.
A very different solution would be to use the Rasterize tool on the mesh (once again after having sampled a lot of points on it). You could rasterize the sampled cloud with the exact same dimensions and resolution as your image (i.e. to get the same sampling pattern). If you manage to do this you could then save the resulting rasterized cloud, save it as ASCII and add the RGB information as 3 additional columns.
This will give you a scalar field. Then you can also convert the 'Z' coordinates of the image points into a scalar field (equivalently, if the image points have a constant height you can directly create the corresponding constant scalar field). Then use the SF Arithmetic tool to substract the 'Distances' scalar field from the 'Height' scalar field. This should give you the new heights if I'm not mistaken. Eventually you can convert this scalar field back to the Z coordinates of the point.
The main issue will be to get the truly vertical distances! You can first try with the classical cloud/mesh distances but they might not be exactly vertical (as it's a nearest neighbor distance). But the qM3C2 plugin is able to compute signed vertical distances between two clouds. To use it you could simply sample (a lot of) points on the mesh first.
A very different solution would be to use the Rasterize tool on the mesh (once again after having sampled a lot of points on it). You could rasterize the sampled cloud with the exact same dimensions and resolution as your image (i.e. to get the same sampling pattern). If you manage to do this you could then save the resulting rasterized cloud, save it as ASCII and add the RGB information as 3 additional columns.
Daniel, CloudCompare admin
Re: Fit images to a mesh surface
Thank you for your quick reply, that helps a lot!
I shouldn't have problems with the true vertical distance since everything is aligned along Z. Also, I don't need extreme precision, it's just a rough representation of how the object looks like. We're automatizing a manual inspection and I'd like to give the operator the "look and feel" of the real object, with the possibility of rotating, zooming etc.
If they only see the pictures out of context they can't thell what's what
I shouldn't have problems with the true vertical distance since everything is aligned along Z. Also, I don't need extreme precision, it's just a rough representation of how the object looks like. We're automatizing a manual inspection and I'd like to give the operator the "look and feel" of the real object, with the possibility of rotating, zooming etc.
If they only see the pictures out of context they can't thell what's what
Re: Fit images to a mesh surface
If I understand the problem correctly, a mesh with the photo applied to it is the end result?
I think CloudCompare is a very well thought and versatile tool, but this is something that might easily be done in another application that has texture or image mapping functions.
Meshlab can take an image and project it onto a point cloud to modify the vertex color, and I believe it can also project it onto a mesh to change the facet color.
Many 3D modeling apps can do texture mapping, such as Blender. Typically this is referred to as flat or planar mapping. The mesh color itself is not changed, so the image usually has to be included with the file if it is exported for use in another application. OBJ format also requires a material (.MTL) along with the image to use it in another application.
I think CloudCompare is a very well thought and versatile tool, but this is something that might easily be done in another application that has texture or image mapping functions.
Meshlab can take an image and project it onto a point cloud to modify the vertex color, and I believe it can also project it onto a mesh to change the facet color.
Many 3D modeling apps can do texture mapping, such as Blender. Typically this is referred to as flat or planar mapping. The mesh color itself is not changed, so the image usually has to be included with the file if it is exported for use in another application. OBJ format also requires a material (.MTL) along with the image to use it in another application.