[Done] Global shift add-ons

[Dimitri]

Hi Daniel,

some ideas pertaining to global shift:

1. when georeferencing a locally acquire point clouds 1 (TLS, drone) to cartographic coordinates by applying the correct transformation, CC correctly detect that there is a global shift to apply (GS1), but does not offer to specify it. This means that if you have other data with a different global shift (GS2) you cannot compare them in CC. At present you have 2 choices to modify GS1:
. either you make the transformation of cloud 1removing the X,Y,Z component of GS2 and subsequentely modify manually the global shift from 0,0,0 to GS2. It's ok for 1 cloud, but when you have 20 scans, as you cannot change the global shift for several scans at the same time, and there's no memory of the previous values used for the global shift, it's a long and cumbersome process.
. either apply the complete transformation to your clouds, then save it in ascii, reload them and apply the global shift you want.

A simple solution : when applying a large transformation and CC detects that a global shift is needed, an option would allow the user to specify a global shift (or use the last one, as when you load an ascii file).

2. Also, at present, when you edit the global shift (edit->global shift), this means that you are actually changing the true cartographic coordinates. There should be a tick box with "keep true cartographic position" such that if you change the global shift, the local coordinates would be adjusted such that the true coordinates are kept (i.e. if I add 10 m in each direction in my global shift, the local coordinates in CC are decreased by 10 m in each direction).

3. Having also the possibility to modify the global shift for several scans at once (as for the transformation) would be also useful

Cheers

Dim

[daniel]

1. Indeed, I agree with your simple solution (it was already planned to be as this but I was a bit lazy afterwards ;)
2. Indeed, Global shift edition is meant to change the absolute position of the cloud (and not to translate it). But the checkbox should do the trick.
3. Ok I'll add all this to the TODO list!

[jedfrechette]

What is the advantage of having Global Shift be a per-object setting?

Whenever, I've used this type of functionality it is to do what Dimitri describes in #1, georeference local data. In that case, I always want to apply the same global shift to all objects in a project so that their positions relative to each other are solely controlled by their local transformation matrices.

[daniel]

Well not sure exactly ;). Just to be sure to cover all cases I guess...

[jedfrechette]

I'll admit I haven't used the Global Shift functionality in CC much but coming from other software where it is a per-project setting I know that even just having one additional transform can cause confusion when moving between projects. Potentially having two transforms for every object seems like it would be really hard to manage for the user and would require a lot of careful attention to prevent errors on any decently sized project.

CloudCompare doesn't really seem to have the concept of a project, but perhaps it would make more sense to implement the global shift functionality at the level of groups rather than individual objects. If the user really did want a different global shift for each object they could still achieve that by putting each one in a separate group. However, for the more common case where they want to use the same shift for multiple objects that would be much easier to achieve by placing them all in the same group (Don't Repeat Yourself). Because each object (group or cloud) would only have a single transformation it would also be much easier for the user to understand the transformation hierarchy simply by looking at the tree view.

For reference here is how a few other applications handle these sort of issues:

Polyworks
------------

Per-project 'Huge Translation' setting with essentially the same use as Global Shift, enabling large coordinates to be stored as float32 without loss of precision. Each object within the project has a 4x4 transformation matrix. Objects can be placed inside logical groups, which cause all grouped objects to be transformed together, however, the actual transforms are still stored in each object.

RiScan
--------

Per project 'POP' matrix, which is a 4x4 transformation matrix applied to objects within the project. Individual objects have a 4x4 'SOP' matrix which stores their transformations. I don't remember if RiScan provides additional grouping functionality but like PolyWorks all local transforms are stored at the object 'SOP' level.

FARO Scene
--------------

Rotations and translations can be applied to individual objects and at the project level. In addition, an arbitrary number of nested groups can be created and each one assigned a rotation and translation. I don't believe it is possible for users to directly change scale. Final coordinates are determined by composing all relevant matrices moving back up the tree.

Cyclone
---------

Per-project 'User Coordinate System' (UCS) and per-object 4x4 transform matrixes. I believe the UCS only allows the user to adjust rotation and translation and is pretty similar to the UCS functionality you find in many CAD applications. You can see indications of this design in the dual matrices found in PTX headers.

[daniel]

As always: Impressive! I'll read this carefully and come back to you later.

[jedfrechette]

I don't know about impressive. So far you're the one doing all the hard work. :-)

[daniel]

Okay I've worked on this subject (too much) and here is the outcome:

• sadly all the solutions based on a "per branch" global shift setting are very interesting but can't be used in CC (this is a more global limitation: all the "scene graph" and equivalent hierarchical 3D positioning schemes are not compatible with fast comparison/processing of points - or it would cost too much memory to temporarily transform the clouds before processing them).
• what is closest to a "project" in CC is a BIN file (as you can store about anything inside - even labels and viewports - so you can basically save all your entities in a single file and reopen them later). As global shift & scale information are also saved in BIN files, we can say that we have almost a "per-project" setting (hum hum).
• the "global" once-and-for-all option is of course very tempting... but I'm still not sure I want to chop down the tree yet. So I've tried a (last?) attempt to improve the actual system. If it doesn't work better I'll draw the axe out...

The improvements consist mainly in two things:

• I've implemented the "bookmark" mechanism suggested by DImitri a while ago (a global_shift_list.txt file next to CC's executable with pre-defined and named shift/scale sets - a template file will be shipped with the next releases)
• a new dialog, hopefully clearer and that can works in both ways (i.e. either when loading a file or when editing the global shift/scale information). It can be used on one or several clouds at a time. And I even implemented the option to keep the "true cartographic coordinates" constant.

Here is the what the dialog looks like when loading a file (the combo-box let the user choose between the current suggested values, the previous one - if any - and the ones found in the 'global_shift_list.txt' file). When the user edit the shift/scale information, the 'original' and 'local' point coordinates are automatically updated so as to get an idea of the result:

cc_new_global_shift_dialog.jpg (65.09 KiB) Viewed 15690 times

You can get more information by clicking on the "?" icon by the way :

cc_new_global_shift_dialog_about.jpg (125.41 KiB) Viewed 15690 times

Here is what it looks like when editing the global/shift scale information afterwards (you'll notice the "keep original position fixed" checkbox). This dialog will also appear (with small differences) if the user applies a transformation that makes the cloud go out of bounds:

cc_new_global_shift_dialog_edit.jpg (56.18 KiB) Viewed 15690 times

[Dimitri]

This seems excellent. Having the possibility to load specific shifts for various projects stored on an external file is really great: I know that I tend to work with various sources of data (TLS, GPS, ALS, SRTM...) and being able to store a unique global shift for a give area is great. It will be to the user to be rigorous in keeping things clean and organized.

Thanks a lot Daniel.

[jedfrechette]

I like the shift/scale dialog. For the local coordinates could the number of digits after the decimal point be adjusted dynamically to indicate the approximate number of significant digits that can be expected? Or maybe show all of the digits, but highlight in red the ones that might be lost to rounding errors. It would be a good way to show users the problem without requiring they understand the intricacies of floating point precision. Also maybe there should be an indication that the local coordinates are the ones CC is storing on disk and working with.

How are the 'Suggested' shift values determined? Are they still per object? Would it be possible to have the suggested values for the first object be based on its coordinates, then reuse those suggested values for later objects? Essentially having a single suggested value per project.

I should mention that one of my main uses for the global shift mechanism is managing projects with data that moves between multiple programs that may or may not be designed for large coordinate geographic data. For that use case it is really convenient to only have two coordinate systems one local and one global that can be related by a single sidecar data file recording the shift. I'm not opposed to the option of per-object shifts if they're useful, but I think a user-friendly default would be to try and use a single set of offsets as much as possible.

The bookmarks idea is interesting, but isn't that list going to fill up really quickly? If you want to store UTM coordinates with mm level precision, a pretty common requirement for ground-based lidar collections then each of your bookmarks will only be good for a few kilometers.