Reality Tiler V2 Improves Tiling Time and Memory Usage
Intro
In November 2023, we announced the Reality Tiler that replaced our then-current photogrammetry tiler as part of the 3D Tiling Pipelines included in Cesium ion and Self-Hosted products. We built the Reality Tiler with emphasis on scalability, tiling performance, and runtime efficiency. Today, we’re digging into Reality Tiler v2, released in July 2024. We’ll cover the complete refactor of the Reality Tiler, where we rebuilt the tiling pipeline from the ground up, bringing significant improvements to tiling time, tileset quality, and runtime performance. We’ll take a close look at memory and runtime performance metrics and end with next steps for further improvements and optimizations.
Before we dive in, note that the Reality Tiler primarily operates on photogrammetry models or models reconstructed from LiDAR scans, potentially massive in scale. These models typically consist of explicit geometry, e.g., a triangle mesh, and color texture information.
Why refactor?
The first version of the Reality Tiler operated by iteratively simplifying and subdividing the entire input model.
Reality Tiler v1 flow: simplification, subdivision, and tile generation.
Although the Tiler made use of out-of-core techniques to ensure that we didn’t run out of memory, it had two shortcomings. First, data access patterns were incoherent. When generating tiles or simplifying the model, the data needed for a series of operations were not coalesced into contiguous memory blocks, resulting in frequent cache misses and requiring unnecessary paging between disk and RAM. These types of access patterns can slow down processing, sometimes substantially.
The second shortcoming of that approach is that it created a fixed number of levels of detail (LODs) across the board. It is a reasonable approach for models that are relatively uniform, but some models aren’t uniform. In practice, many models have areas of higher detail and areas of lower detail. For example, parts of a model may have much higher triangle density than others or use higher resolution textures. A fixed number of LODs can create unbalanced tilesets, with some tiles containing too much information and other tiles containing too little. At runtime, this translates to inconsistent performance: Too many large tiles leads to lower framerates, and too many small tiles can result in excessive network requests and slow streaming.
Reality Tiler v1 created uniform tilesets. Notice how the tree is balanced but some tiles are larger than others.
From global to local
We considered several strategies to address the shortcomings of Reality Tiler v1, but we ultimately decided to flip the entire process on its head. Rather than taking a global approach—which is to say, operating on the entire model when producing tiles and performing simplification—we take a local approach and process only one part of the model at a time. More specifically, Reality Tiler v2 splits the model into leaf tiles and builds the tileset by iteratively merging and simplifying tiles until we are left with a single root tile. Because individual tiles are relatively small and mostly contiguous in memory, data access is much more coherent, and, consequently, the overall process is significantly faster.
Reality Tiler v2 flow: subdivision, mergers, simplification, and tile generation.
With Reality Tiler v2, we ensure that each leaf tile is approximately the same size (in terms of geometry and texture content, not extent in world-space). This means that certain parts of the input model with dense geometry or texture data may be tiled more deeply than others, and tiles further up the tree remain approximately uniform in size.
Reality Tiler v2 creates uniform tiles. Notice how the tree is unbalanced but the size of each tile is the same.
Reality Tiler Tiling Time Comparison
| Dataset | Triangle Count (Millions) | Prev. Tiling Time (h:mm:ss) | New Tiling Time (h:mm:ss) | Percent Reduction in Tiling Time |
|---|---|---|---|---|
| Carrick Hill (Australia) | 3.3 | 0:02:37 | 0:01:11 | 55% |
| Liverpool (UK) | 7.8 | 0:08:48 | 0:02:54 | 67% |
| Budapest (Hungary) | 43.1 | 0:59:05 | 0:14:08 | 76% |
| Melbourne (Australia) | 69.9 | 3:22:41 | 0:27:58 | 86% |
Reality Tiler Memory Usage Comparison
| Dataset | Triangle Count (Millions) | Prev. Peak Memory (GB) | New Peak Memory (GB) |
|---|---|---|---|
| Carrick Hill (Australia) | 3.3 | 13.5 | 14.9 |
| Liverpool (UK) | 7.8 | 22.1 | 22.7 |
| Budapest (Hungary) | 43.1 | 61.7 | 21.3 |
| Melbourne (Australia) | 69.9 | 134.5 | 26.9 |
Reality Tiler v2 tiling time is reduced across the board, about 2-7 times faster than Reality Tiler v1. Generally, the larger the input, the greater the speedup. Memory requirement trends have similar gains, with the largest datasets seeing the greatest reduction in peak memory usage (for smaller datasets, we see a slight increase in memory usage for these samples). Taken together, these improvements indicate substantially improved scalability for the Reality Tiler: It’s able to process more data in a shorter period of time, while typically using less memory.
Because of non-uniform tiling, we also see a reduction in the variation of file sizes for tiles. Melbourne, Australia, for example, is a dataset that does not have uniformly sized triangles, and therefore it particularly benefits from non-uniform tiling.
A wireframe view of a portion of Melbourne. Notice the variation in triangle size and density: The buildings (upper left) have small and densely packed triangles, while the triangles that make up the river (lower right) are much larger.
With Reality Tiler v1, which tiles uniformly, Melbourne’s average tile size was 1.28 MB, with a standard deviation of 706 KB. For v2, the average tile size is 395 KB, with a standard deviation of 192 KB.
Looking forward
We’ve come a long way since the v1 release of the Reality Tiler, and we still have a number of ideas for future enhancements. The two largest bottlenecks in our pipeline are now texture compression and texture coordinate generation. We will likely focus on both in the near future.
We will also work to bring down peak memory usage even further. Although mesh data is mostly handled out of core, the data structures that help us index into it reside in memory. We plan to adapt this structure to also not entirely reside in memory.
Interested in using the updated Reality Tiler? Sign in to your Cesium ion account or sign up and get started for free!