Invited talk at Hasso Plattner Institute.

1. Visualization Techniques for Massive Datasets
Dr. Matthias Trapp
Hasso Plattner Institute | Faculty of Digital Engineering | University of Potsdam

2. Massive Data: Extreme Growth in Data Volume
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 2

3. Simple Example:
In 2010 New York City added over 54 million metric tons
of carbon dioxide to the atmosphere; almost 2 tons every
second (approx. 75% from buildings)*
*From: Jonathan Dickinson and Andrea Tenorio, Sept. 2011, Inventory of Net York City Greeenhouse Gas
Emissions, Majors Office of Long-Term Plannung and Substainability, City of New York
Data Visualization
Chart 3
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets

4. Simple Example:
In 2010 New York City added over 54 million metric tons
of carbon dioxide to the atmosphere; almost 2 tons every
second (approx. 75% from buildings)*
*From: Jonathan Dickinson and Andrea Tenorio, Sept. 2011, Inventory of Net York City Greeenhouse Gas
Emissions, Majors Office of Long-Term Plannung and Substainability, City of New York
Data Visualization
Chart 4
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets

5. Simple Example:
In 2010 New York City added over 54 million metric tons
of carbon dioxide to the atmosphere; almost 2 tons every
second (approx. 75% from buildings)*
*From: Jonathan Dickinson and Andrea Tenorio, Sept. 2011, Inventory of Net York City Greeenhouse Gas
Emissions, Majors Office of Long-Term Plannung and Substainability, City of New York
Data Visualization
Chart 5
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets

6. Visualization
Chart 6
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets

7. Visual Analytics & Visual Data Mining
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 7
Visualization
Data
Models
Knowledge
Feedback Loop
Parameter Refinement
User Interaction
Mapping
Transformation
Model
Building
Model
Visualization
Massive
Data

8. The Visualization Pipeline
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 8
1.0978
0.541
1.40111
0.427
?????
0x4E
1.098
0.541
1.401
0.427
39.213
78.000
1.098
0.541
1.401
0.427
39.213
78.000

9. Overview first, zoom and filter, then details-on-demand.
[Shneiderman, 1996]
Requirements and constraints on interactive visualization techniques:
■ Dynamic configuration of filtering, mapping, and rendering stages
■ Real-time 3D rendering performance
■ Processing data with high update frequency
■ Low memory consumptions
Visual Information Seeking Mantra
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 9

10. Increased GPU Speed and Bus Throughput
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 10
http://docs.nvidia.com/cuda/cuda-c-programming-guide/#axzz4czYVdDYE

11. Increased GPU Speed and Bus Throughput
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 11
http://docs.nvidia.com/cuda/cuda-c-programming-guide/#axzz4czYVdDYE

12. Approach to GPU-based Visualization Techniques
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 12
Basic idea:
Implementation of visualization stages by GPU as extensive as possible:
■ Process data at different scales only if contributing to visualization
■ Reduce generation and storage of intermediate data representations

13. Compact data representations for GPU-based processing:
■ Low memory footprint
■ Suitable for GPU-based storage and fast access
■ Unified CPU/GPU representation for fast updates
Design of parallel algorithms and middleware systems:
■ Performing data processing, filtering, and mapping
■ Software architecture of (GPU-aligned) visualization middleware
■ Scalable interaction techniques
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 13
Design and Implementation Challenges

14. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 14
Results of Visualization Research Projects
Spatial Thematic Data Spatial Mobility Data
Software Systems Data
Spatial Movement Data

15. Spatial Thematic Data
▪ Evaluating the Perceptual Impact of Rendering Techniques on Thematic Color Mappings in 3D Virtual Environments
Engel, Juri and Semmo, Amir and Trapp, Matthias and Döllner, Jürgen
In Proceedings of 18th International Workshop on Vision, Modeling and Visualization (VMV 2013), pages 25-32, 9 2013
▪ Hierarchical Spatial Aggregation for Level-of-Detail Visualization of 3D Thematic Data
Vollmer, Jan and Trapp, Matthias and Döllner, Jürgen and Heidrun Schumann
ACM Transaction on Spatial Algorithms and Systems 4(3), 9:1-9:23, 2018

16. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 16
Spatial Thematic Data
Motivation
Surface related thematic data (results of solar potential analysis).
Aggregated “thermal breathing” of a house in a 24h cycle (winter & summer) [Energy3D].

17. Research question:
How to compute level-of-detail variants
of a thematic data set to support
overview, zooming, filtering, and the display of details-on-demand?
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 17
Spatial Thematic Data
Motivation
Surface related thematic data (results of solar potential analysis) at different level-of-detail.

18. ■ Heterogeneous representations of thematic data:
■ Dynamic data sets:
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 18
Spatial Thematic Data
Major Challenges
Different thematic data representations w.r.t. to polygonal modeled objects.
Example of dynamic 3D thematic data visualization of a pressure wave propagation in a virtual 3D city model.
ti-1 ti ti+1

19. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 19
Spatial Thematic Data
Data Aggregation Approach: Scene Voxelization
Single-pass Voxelization
Visualization of sparse-voxel-octree data at different octree levels.
Sparse-voxel-octree (SVO)

20. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 20
Spatial Thematic Data
Mapping Sparse Voxel-Octree to 3D Scene
Average mapping. Maximum mapping. Minimum mapping.

21. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 21
Spatial Thematic Data
Object-based Aggregation for Object Analysis
Average mapping to object features (roof and facades).

22. Approach
Spatial Mobility Data
▪ Interactive Rendering and Stylization of Transportation Networks Using Distance Fields
Trapp, Matthias and Semmo, Amir and Döllner, Jürgen
In Proceedings of the GRAPP 2015, pages 207-219, 2015
▪ Interactive Web-based Visualization for Accessibility Mapping of Transportation Networks
Schoedon, Alexander and Trapp, Matthias and Hollburg, Henning and Döllner, Jürgen
In Proceedings of EuroVis 2016, pages 79-83, 2016

23. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 23
Spatial Mobility Data
Reachability Maps based on Transportation Networks
OSM Data Set
of Berlin/Brandenburg:
2.9 million nodes
2.4 million line segments

24. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 24
Spatial Mobility Data
Mobility Analytics using Reachability Maps
Potential places of resident w.r.t. travel time; both work and university. Finding potential places for fast food restaurants.

25. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 25
Spatial Mobility Data
Overview of the Mapping and Rendering Approach

26. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 26
Spatial Mobility Data
Deriving Geometry from Compact Network Representations
Attributed Vertex Cloud
+
Geometry Shader
+
Vertex Pulling

27. ■ Adaptive level-of-detail visualization of massive graphs
■ GPGPU computation of shortest-path problems for mobility analysis
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 27
Spatial Mobility Data
Future Research
Reducing visual complexity based on reachability of street segments.

28. Spatial Movement Data
▪ Hardware-Accelerated Attribute Mapping for Interactive Visualization of Complex 3D Trajectories
Buschmann, Stefan and Trapp, Matthias and Lühne, Patrick and Döllner, Jürgen
In Proceedings of IVAPP 2014, pages 355-363, 2014
▪ Real-Time Animated Visualization of Massive Air-Traffic Trajectories
Buschmann, Stefan and Trapp, Matthias and Döllner, Jürgen
Proceedings of CyberWorlds 2014, pages 172-181, 2014
▪ Real-Time Visualization of Massive Movement Data in Digital Landscapes
Buschmann, Stefan and Trapp, Matthias and Döllner, Jürgen
In 16th Conference on Digital Landscape Architecture (DLA 2015), pages 213-220, 2015
▪ Animated visualization of spatial-temporal trajectory data for air-traffic analysis
Buschmann, Stefan and Trapp, Matthias and Döllner, Jürgen
The Visual Computer, vol. 32(3):371-381 2016

29. Major challenges for visualization techniques:
■ Representation of both spatial and temporal aspects of movements
■ Communication of additional data attributes (e.g., speed, type)
■ Handle data complexity (#trajectories, number and size of attributes)
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 29
Spatial Movement Data
Visual Analysis of 3D Trajectory Data
Visual clutter: 2D rendering of trajectories. 3D rendering of trajectories.
IFR Data Set
Frankfurt/Main:
12,500 trajectories
1.5 million sample points

30. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 30
Spatial Movement Data
Visual Analysis of Air-Traffic Data
Approaching and departing airplanes, with detailed information on selected trajectories (acceleration in color).

31. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 31
Spatial Movement Data
Computation and Rendering of Density Maps
Overall density of flights during a single week. Difference in density: arriving vs. departing flights.
Combined Visualization.

32. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 32
Spatial Movement Data
Space-time Cube Visualization Metaphor
Detailed visualization of a single trajectory over time. Temporal order of departing and arriving aircrafts.
space
time

33. Approach
Software Systems Data
▪ Interactive Rendering of Complex 3D-Treemaps
Trapp, Matthias and Hahn, Sebastian and Döllner, Jürgen
In Proceedings GRAPP 2013, pages 165-175, 2013
▪ Natural Phenomena as Metaphors for Visualization of Trend Data in Interactive Software Maps
Würfel, Hannes and Trapp, Matthias and Limberger, Daniel and Döllner, Jürgen
In Computer Graphics and Visual Computing (CGVC), 2015 The Eurographics Association
▪ Interactive Revision Exploration using Small Multiples of Software Maps
Scheibel, Willy and Trapp, Matthias and Döllner, Jürgen
In Proceedings IVAPP 2016, pages 133-140, 2016
▪ Evaluation of Sketchiness as a Visual Variable for 2.5D Treemaps
Limberger, Daniel and Fiedler, Carolin and Hahn, Sebastian and Trapp, Matthias and Döllner, Jürgen
In Proceedings of the 20th International Conference of Information Visualization (IV'16), pages 183-189, 2016
▪ Attributed Vertex Clouds
Scheibel, Willy and Buschmann, Stefan and Trapp, Matthias and Döllner, Jürgen
In book: GPU Zen, Publisher: Black Cat Publishing, Editors: Wolfgang Engel, 2018

34. “The fundamental cause of the software crisis is that massive, software-
intensive systems have become unmanageably complex.” Grady Booch 1994
Goal:
“Provisioning of communication artifacts
by interactive visualization of high-dimensional system information.” Diehl 2007
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 34
Software Systems Data
Software Visualization
Information
Visualization
Software
Engineering
Data
Analysis
Software Visualization

35. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 35
Software Systems Data
Software Visualization using Interactive Software Maps
Visual Variables Mapping:
■ Area: Lines-of-Code (RLOC)
■ Color: Number of Developers
■ Height: Number of Includes

36. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 36
Software Systems Data
Software Visualization using Interactive Software Maps
Visual Variables Mapping:
■ Area: Lines-of-Code (RLOC)
■ Color: Number of Changes
■ Height: Nesting Level
Challenges for visualization of interactive software maps:
■ Rendering of massive hierarchical data
■ Frequent changes to visual variable mappings
■ Fast spatialization (layout) of abstract system data

37. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 37
Software Systems Data
Massive Software System Data: 107 – 1012 Elements
http://www.informationisbeautiful.net/visualizations/million-lines-of-code/
Information Graphic:
≤ 102 Elements
Information Visualization:
~ 104 Elements
Visual Analytics:
≥ 106 Elements

38. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 38
Software Systems Data
Overview of the Mapping and Rendering Approach

39. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 39
Software Systems Data
Implementation of Primitive Transformation Shader
Geometry shader transforming an attributed 2D point into a cube by emitting a primitive template instance.

40. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 40
Software Systems Data
Rendering Break Down – Attributed Vertex Cloud

41. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 41
Software Systems Data
Rendering Break Down – Generated Primitives

42. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 42
Software Systems Data
Rendering Break Down – Generated Primitives

43. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 43
Software Systems Data
Rendering Break Down – Post Processing

44. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 44
Software Systems Data
View-frustum and Size Culling Implementation
bool passCulling(const in mat4 mvp, const in vec4 vertex, const in vec4 dimensions,
const in bool applyViewFrustumCulling, const in bool applySizeCulling)
{ float cPMaxX=-10000.0;float cPMinX=10000.0;float cPMaxY=-10000.0;float cPMinY=10000.0;
bool passCulling = true;
if(useViewFrustumCulling)
{ // 1. Do conservative culling and test only center of item
vec4 V = mvp * vertex;
passCulling=((-V.w<V.x)&&(V.x<V.w))&&((-V.w<V.y)&&(V.y<V.w))&&((-V.w<V.z)&&(V.z<V.w));
if(!passCulling)
{ // 2. Perform precise culling if item center is not in frustum
vec4 AABB[8];
for (int i = 0; i < 8; i++){
AABB[i] = mvp * (vertex + VERTEX[i] * dimensions);
vec4 p = AABB[i] / AABB[i].w;
p.xy = (p.xy + 1.0) * (viewport.zw * 0.5) + viewport.xy;
cPMaxX = max(cPMaxX, p.x); cPMinX = min(cPMinX, p.x);
cPMaxY = max(cPMaxY, p.y); cPMinY = min(cPMinY, p.y);
} //endfor
// 2. Perform precise culling if item center is not in frustum
int bounds[6] = int[6](0,0,0,0,0,0);
for(int i = 0; i < 8; i++){
if(AABB[i].x>AABB[i].w) bounds[0]++; if(AABB[i].x<-AABB[i].w) bounds[1]++;
if(AABB[i].y>AABB[i].w) bounds[2]++; if(AABB[i].y<-AABB[i].w) bounds[3]++;
if(AABB[i].z>AABB[i].w) bounds[4]++; if(AABB[i].z<-AABB[i].w) bounds[5]++;
}//endfor
for(int i = 0; i < 6; i++) if(bounds[i]==8) passCulling = false;
}//endif }//endif
// 3. Apply size culling if enable to every visible item
if(passCulling && applyViewFrustumCulling && useSizeCulling)
passCulling = ((abs(cPMaxX)-abs(cPMinX))*(abs(cPMaxY)-abs(cPMinY)))
> float(pixelSizeThreshold);
return passCulling;
}
Shader-based computation of view-frustum and size culling per data item instance.

45. Software Systems Data
Rendering Performance
Shape
Generation
Pseudo
Instancing
UBO
Instancing
TBO
Instancing
Indexed
VBO
Non-Indexed
VBO
Intermediate
Mode
13.884 0,38 1,68 1,47 1,49 0,63 1,37 5,12
98.858 1,75 6,28 4,01 4,01 3,42 8,67 35,45
365.645 6,15 27,49 15,91 14,93 12,57 32,35 133,05
614.920 14,76 60,88 31,12 31,51 28,46 54,71 220,01
0
50
100
150
200
250
milliseconds
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 45
NVIDIA GTX 460 GPU

46. Software Systems Data
Memory Footprint
CPU GPU CPU GPU CPU GPU CPU GPU
13.884 98.858 365.645 614.920
Shape Generation 444288 444368 3163456 3163536 11700640 11700640 19677440 19677520
Pseudo Instancing 444368 444368 3163536 3163536 11700640 11700640 19677520 19677520
UBO Instancing 444368 444368 3163536 3163536 11700640 11700640 19677520 19677520
TBO Instancing 444368 444368 3163536 3163536 11700640 11700640 19677520 19677520
Indexed VBO 1999296 1555008 14235552 11072096 52652880 40952240 88548480 68871040
Non-Indexed VBO 15439008 14994720 109930096 106766640 406597240 394896600 683791040 664113600
Intermediate Mode 444288 0 3163456 0 11700640 0 19677440 0
1
10
100
1000
10000
100000
1000000
10000000
100000000
1E+09
byte
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 46

47. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 47
Software Systems Data
Small Multiples of Software Maps
Project Revisions
Visual
Variable
Mappings

48. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 48
Software Systems Data
Outlook: Massive Tree-structured Data
Icicle Plot.
Tree Map.
Bundle View.
A Visual Survey of
Tree Visualization

49. ■ Techniques for the visualization of system evolution
■ Rendering techniques for GPU-based labelling and annotations
■ Research in layout stability and hybrid layout approaches
■ Design guidelines for software maps and software landscapes
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 49
Software Systems Data
Outlook: Software Maps and Landscapes
Evaluation of Sketchiness
as a Visual Variable
Metaphors for Visualization of
Trend Data
Labeling and Annotations

50. Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 50
Software Systems Data
Business Intelligence Maps
Visual Variables Mapping:
■ Height:
Business-Impact
■ Color:
Application Status
Green: Active
Orange: Inactive
Yellow: To Inactive
Violet: Historical
■ Data Set:
Meta data
IT landscape
of a multinational
company

51. Approach
Wrap-Up

52. Observations:
■ GPU-aligned concept can be implement for various visualization problems
and domains
■ Trade-off between on-the-fly data mapping and runtime performance
■ Complements out-of-core visualization approaches
Current limitations:
■ Concept is hardly applicable to all types of processing and mapping
■ Cache misses are currently a performance limiting factor
■ … Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 52
Observations and Limitations

53. ▪ Visualization of massive data – a key component of visual computing and analytics
▪ Visualization of massive data - frequently a starting point of new types of businesses
▪ Mobility Analytics
▪ Software Analytics
▪ Video Analytics
▪ …
▪ Effective visualization requires appropriate adoption of visualization metaphors
▪ Fusion of data processing and visualization from an implementations point-of-view
Matthias Trapp
04-05-2017
Visualization
Techniques for
Massive Datasets
Chart 53
Conclusions