Error opening/ saving bin files in 2.8.

[Aarie]

Hey Daniel,
i've stumbled again across the bug of not being able to save/ open large datasets in 2.8., where I get the error message: "Read error (corrupted file or no access right?)".
The dataset is approx. 2,5 billion points, split into 12 clouds. Furthermore CC crashes when I try to merge the single clouds.

Cheers,
Arie.

System specs:
Win 7 Pro 64bit
Xeon E-5 1650 V2
128 GB RAM
Nvidia Quadro M4000

[daniel]

Wow, 2.5 billion?!

I never tested CC with such a huge cloud, so it doesn't seems that strange that it crashes ;). At least for the merge method it's quite logical that it crashes (I think the theoretical limit for a single cloud is 2 billion points anyway). We should at least add some warning messages in this case...

Nevertheless the file issue is a lot more strange (especially since there are 12 clouds, so I think each cloud is not that big?). What is the generated file size? (in bytes)

[Aarie]

Unfortunately I deleted the "corrupted" file with all clouds loaded.

But I tried splitting the file into two single ones, each containing six pointclouds (where each cloud has between approx. 100 - 500 million points).
Again both bin files can't be opened.

File 1: 21.394.142.519 Bytes
File 2: 17.737.391.296 Bytes

Cheers.

[daniel]

If it's not too long, is it possible for you to repeat the process of saving 1, 2, 3, up to 6 (potentially) clouds and determine after which file size CC fails to load the file?

There was a 4 Gb limit at a time with older versions of Windows, but it doesn't exist anymore (I can safely read 6 Gb files on my side).

[Aarie]

Weird things happening here...

Following your suggestion, I imported the single clouds incrementally, each time saving it as a new file.
Now I can open the .bin file with six clouds loaded (File 1). But, if I import all the six pointclouds at once and save a bin file, it becomes corrupted.
What's going on?!

[daniel]

That is indeed weird!

Is the final file size exactly the same? And are the clouds organized the same way in the DB tree?

[Aarie]

Well, after importing all files at once, I place them all in the first folder, mark the folder and save as bin.
Importing the files seperately, I also just place the newly imported single cloud in the same folders as the previously imported clouds.

Regarding the filesize: I have to check on that, because I deleted the corrupted file. I'll get back to you on that in a moment.

[Aarie]

Filesize is identical.

[daniel]

Even more puzzling... It would be interesting to do a byte-to-byte comparison, but with files so huge it might get dirty. It's really strange.

[Aarie]

If you want, I can bring the dataset with me to VirtArch2017...