3. CS 354 3
My Office Hours
Tuesday, before class
Painter (PAI) 5.35
8:45 a.m. to 9:15
Thursday, after class
ACE 6.302
11:00 a.m. to 12
Randy’s office hours
Monday & Wednesday
11 a.m. to 12:00
Painter (PAI) 5.33
4. CS 354 4
Last time, this time
Last lecture, we discussed
Graphics Performance Analysis
This lecture
Surfaces, programmable tessellation,
non-photorealistic graphics
Projects
Project 4 on ray tracing on Piazza
Due Thursday May 3, 2012 (extended)
5. CS 354 5
On a sheet of paper
Daily Quiz • Write your EID, name, and date
• Write #1, #2, #3 followed by its answer
Multiple choice: If a 3D application
renders no faster when the window is Multiple choice: A task is split into
resized smaller, the mostly likely to be two subtasks A and B consisting of
effective tuning listed below is
25% and 75% of the total work. If
a) enable mipmap texture filtering task B is optimized to be 5 times
faster, how much faster is the overall
b) optimize the efficiency of the task once B is optimized?
application’s C/C++ rendering code
c) defragment the hard drive a) 1.6x faster
d) enable vertex retrace b) 2.0x faster
synchronization for the buffer swaps
e) remove instructions from the c) 2.5x faster
fragment shaders
d) 3.1x faster
Name a type of ray that is easily traced
coherently.
e) 5.0x faster
6. CS 354 6
Modeling with Patches
Triangles
Good for rendering
Not as great for
modeling
No curvature
Assigning texture
coordinates is hard
Easier to model with
patches
7. CS 354 7
Types of Patches
Triangular patch Quadrilateral patch
Notice fractional tessellation scheme in wire-frame pattern
14. CS 354 14
Bi-cubic Patches Used in
Production Rendering
Production renderers for film and video
rely on surfaces
15. CS 354 15
Amplification of Standard
Triangle Meshes
Tessellation
results in triangle
amplification
with curvature
Original faceted mesh
Results of Phong
shading of curved
16. CS 354 16
Surfaces are Determined by
Their Control Points
Moving control
points displaces
the evaluated
surfaces
21. CS 354 21
Art Pipeline for Tessellation
Control
Cage
Smooth Surface Displaced Surface
Programmable
Tessellation
Generate LODs
…
GPU
Polygon Mesh
Pre-processing of patch content, off GPU
22. CS 354 22
Iso-line Tessellation
Intended for hair, particular animation and
combing
Scientific visualization applications too
23. CS 354 23
Visual Artifact Avoidance
Water-tightness of mesh is an issue
Similarly, gradient & normal continuity
Programmable tessellation has the
necessary programmability to avoid such
artifacts
Example of
unaddressed issue
causing cracking
24. CS 354 24
Tools Allow Authoring of 3D
Tessellation Content
3D modeling tools support generating content
from made of surface patches
Same model for
OpenGL 4.0
DirectX 11
sub-division surface modeling tool
sculpting tools
25. CS 354 25
OpenGL 2.1 vs.
OpenGL 3.1 with Geometry Shaders
primitive primitive
indices indices
topology & topology &
in-band vertex in-band vertex
attributes attributes
index to vertex index to vertex
attribute attribute
puller puller
vertex vertex
attributes attributes
vertex vertex
shader shader
assembled
assembled primitive
programmable primitive
geometry
fixed-function shader
primitive stream
clipping, clipping,
setup, & setup, &
rasterization rasterization
fragments fragments
fragment fragment
shader shader
OpenGL 3.1
OpenGL 2.1 supports geometry shaders
26. CS 354 26
OpenGL 3.1 with Geometry vs.
OpenGL 4.0 with Tessellation
indices primitive rectangular & indices primitive
topology & triangluar topology &
in-band vertex patch in-band vertex
attributes index to vertex attributes
index to vertex
attribute
attribute
vertex puller
puller vertex
attributes attributes
vertex
attributes
control point vertex
vertex shader
(vertex) shader
shader
control
assembled points assembled
primitive primitive
level-of-detail
parameters tessellation
programmable control (hull)
geometry geometry
shader shader shader
fixed-function
tessellation patch control
primitive stream primitive stream
generator points &
clipping, parameters clipping,
setup, & setup, &
rasterization tessellation rasterization
evaluation
fragments (domain) fragments
fine primitive
topology
shader
fragment fragment
(u, v) coordinates
shader shader
OpenGL 3.1 OpenGL 4.0
supports geometry shaders supports programmable tessellation
27. CS 354 27
DirectX 11:
Hull and Domain Shaders
Same hardware Input Assembler
functionality as in Vertex Shader
OpenGL 4.0
Hull Shader
Also exposed in
Direct3D 11 New for
Tessellator
D3D11
Shader stage
Domain Shader
naming differences
Hull = Control Geometry Shader Stream Output
Domain = Evaluation Rasterizer
Pixel Shader
Output Merger
28. CS 354 28
Provide High-level Context for
OpenGL 4.0’s Entire Dataflow
Show entire flow of various kinds of data within OpenGL
Vertices, fragments, pixels, filtered texels, buffer data
Include geometry shading & texture buffer objects
ARB extensions circa OpenGL 3.1
Ignores
Details of fixed-function & programmable operation within blocks
Evaluators
Display lists
Queries & errors
State changes
Accumulation
Feedback & selection
Uniform (bindable, parameter) buffer objects
29. CS 354 29
High-level Flow of Data within
OpenGL 3.1 + Geometry Shaders + Texture Buffer Objects
primitive topology,
transformed Legend
Vertex Vertex vertex data Geometric primitive
assembly & vertices
assembly primitive processing
batch transformed processing programmable
pixels
operations
type, vertex
fragments
vertex data attributes point, line,
and polygon fixed-function filtered texels
geometry operations
fragments
vertex Transform texture buffer data
transform feedback fetches
buffer
objects feedback
pixels in framebuffer object textures
buffer
objects
primitive batch type, vertex
texture stenciling, depth testing, blending
vertex indices,
vertex attributes texture fetches
buffer
buffer data, objects Texture Fragment Raster
unmap Framebuffer
mapping fragment processing operations
buffer texture
Command Buffer fetches
parser store pixel
map buffer, pack
get buffer buffer
data objects image and bitmap
texture
pixel fragments
image
pixel image or unpack Pixel specification
texture image buffer
specification packing
objects
pixels to pack image
rectangles,
bitmaps Image
Pixel Pixel
primitive
unpacking unpacked processing
pixels
processing copy pixels,
copy texture image
30. CS 354 30
Add Programmable Tessellation
Adds to the data flow
Control point processing
Similar to a vertex shader, but operates on control points
Patch assembly & control processing
Single Program Multiple Data execution
One thread of program execution per output control point
Allows patch basis conversions and level-of-detail computation
Level-of-detail requires computing reductions over all instances
Basis conversion example: reduce a Catmull-Clark subdivision patch to an
approximating Bezier patch
Patch tessellation generator
Rectangular and triangle patches
Continuous and discrete tessellation
Patch evaluation shading
Given patch’s transformed control points and a (u,v) position within the
patch domain, evaluate vertex position, gradients, and surface normal
Feeds evaluated vertex to conventional vertex shader
Added into previous diagram…
Clean fit: simply adds stages above vertex shading for tessellation
31. CS 354 31
Patch
Control point Patch tessellation Patch evaluation
assembly &
processing transformed transformed generation transformed processing
control points
processing patch patch, bivariate
patch
tessellation
control patch topology, evaluated patch vertex domain
texture
fetches points
primitive topology,
transformed Legend
Geometric primitive
Vertex Vertex vertex data patch data
assembly &
assembly primitive processing programmable vertices
batch transformed processing
operations pixels
type, vertex
vertex data attributes point, line, fragments
and polygon fixed-function
geometry operations
filtered texels
fragments
vertex Transform texture buffer data
transform feedback fetches
buffer
objects feedback
pixels in framebuffer object textures
buffer
objects
primitive batch type, vertex
texture stenciling, depth testing, blending
vertex indices,
vertex attributes texture fetches
buffer
buffer data, objects Texture Fragment Raster
unmap Framebuffer
mapping fragment processing operations
buffer texture
Command Buffer fetches
parser store pixel
map buffer, pack
get buffer buffer
data objects image and bitmap
texture
fragments
pixel image
pixel image or unpack Pixel specification
texture image buffer
specification packing
objects
pixels to pack image
rectangles,
bitmaps Image
Pixel Pixel
primitive
unpacking unpacked processing
pixels
processing copy pixels,
copy texture image
34. CS 354 34
Displacement Mapping
• Displacement Mapping
- Sample displacement value from a texture
- Displace vertex along its normal
• Watertight Normals
- cross product of a pair of tangent, bi-tangent vectors
- discontinuities occur at shared corners and edges
- define corner and edge ownership
35. CS 354 35
Tessellation Shader Example:
Starting Simple
Control point (vertex) shader transforms vertex positions to eye-
space
Simple 4x3 affine matrix transformation
Tessellation control shader accepts a 3 control points
Forms a standard triangle
Passes each control point on to tessellation valuation
Computes a level-of-detail scalar for each edge based on the scaled
window-space length of each edge
So the tessellation of the generated triangle patch adapts to the screen-
space size of the triangle
Tessellation evaluation shader runs at each tessellated vertex of
triangle patch
Gets 3 control points + (u,v,w) barycentric triangle weights
Weights control points to make a new position
Computes a surface normal
The normal is constant for the entire triangle in this case, but generally the
normal would vary
Outputs barycentric weights as RGB color (to visualize)
36. CS 354 36
Simple Example Rendered
Original single triangle
Visualization of barycentric
weights used by tessellation
evaluation shader
Adaptive tessellation automatically
tessellates distant, small triangle less
and close, large triangle more
37. 37
GLSL Control Point (Vertex)
CS 354
Shader
uniform vec2 wh; // scaled width&height
of screen
#version 400 compatibility out vec3 eye_space_pos;
out vec2 scaled_window_space_pos;
// Multiply XYZ vertex by 4x3 modelview out vec3 eye_normal;
// matrix (assumed non-projective)
vec3 transform(precise vec3 v) void main(void)
{ {
vec3 r; eye_space_pos =
// Remember: GLSL is oddly column-major transform(gl_Vertex.xyz);
// so [col][row]
for (int i=0; i<3; i++) { vec4 ndc_pos =
r[i] = v[0]*gl_ModelViewMatrix[0][i] + gl_ModelViewProjectionMatrix *
v[1]*gl_ModelViewMatrix[1][i] + vec4(gl_Vertex.xyz,1);
v[2]*gl_ModelViewMatrix[2][i] + scaled_window_space_pos =
gl_ModelViewMatrix[3][i]; wh*(ndc_pos.xy / ndc_pos.w);
}
return r; // Pass along two sets of texture
} coordinates...
gl_TexCoord[0] = gl_MultiTexCoord0;
gl_TexCoord[1] = gl_MultiTexCoord1;
mat3 nm = gl_NormalMatrix;
vec3 n = gl_Normal;
eye_normal = normalize(nm * n);
}
38. CS 354
GLSL Tessellation Control Shader 38
void main(void)
{
sharable_len[TID] =
#version 400 compatibility distance(scaled_window_space_pos[TID],
layout(vertices=3) out; scaled_window_space_pos[(TID+1)%3]);
barrier();
// thread ID
#define TID gl_InvocationID float len0 = sharable_len[0],
len1 = sharable_len[1],
out float sharable_len[]; len2 = sharable_len[2];
in vec3 eye_space_pos[]; eye_space_pos2[TID] = eye_space_pos[TID];
in vec2
scaled_window_space_pos[]; // Limit level-of-detail output to thread 0
out vec3 eye_space_pos2[]; if (TID == 0) {
// Outer LOD
gl_TessLevelOuter[0]=len1;//V1-to-V2 edge
gl_TessLevelOuter[1]=len2;//V2-to-V0 edge
gl_TessLevelOuter[2]=len0;//V0-to-V1 edge
// Inner LOD
gl_TessLevelInner[0] =
max(len0, max(len1, len2));
}
}
40. CS 354 40
More Complicated Example:
Adaptive PN Triangles
Idea: Send a triangle mesh with positions & normals
Have the tessellation unit tessellated a curved triangle that
matches the per-triangle positions & normals
Nice because existing triangle meshes can get extra triangle
detail
Details
Same control point (vertex) shader as before
Tessellation control shader
Input: 3 control points + normals
Output: 10 control points for a Bezier triangle patch
Output: 6 controls points for quadratic normal interpolation
Output: adaptive level-of-detail (LOD) based on edge lengths in
scaled window space
Tessellation evaluation shader
Does cubic interpolation of Bezier triangle to compute position
Does quadratic interpolation to compute surface normal
41. CS 354 41
Adaptive PN Triangles
Tessellation Rendered
Tessellated
& Phong
shaded
Original monkey head
faceted mesh
Visualization of barycentric
Wire frame of
weights used by tessellation
adaptive tessellation
evaluation shader
42. Adaptive PN Triangle Tessellation
CS 354 42
Control Shader // Compute triangle LOD parameters.
#version 400 compatibility
// interior control points are per-patch // Each of 3 threads computes each of 3
layout(vertices=3) out; // edge lengths.
gl_TessLevelOuter[TID] =
#define TID gl_InvocationID // thread ID distance(scaled_window_space_pos[TIDm1],
#define P eye_space_pos
#define P2 eye_space_pos2 scaled_window_space_pos[TID]);
#define N eye_normal
barrier();
in vec3 eye_space_pos[]; gl_TessLevelInner[0] = max(gl_TessLevelOuter[0],
in vec2 scaled_window_space_pos[];
in vec3 eye_normal[]; max(gl_TessLevelOuter[1],
out vec3 eye_space_pos2[]; gl_TessLevelOuter[2]));
out vec3 eye_space_normal[];
out vec3 ccw_cp[]; // ccw_cp[TID] is the control point immediate
out vec3 cw_cp[]; // counter-clockwise from P[TID]
out vec3 mid_normal[]; // cwc_cp[TID] is the control point immediate
patch out vec3 b111; // clockwise from P[TID]
ccw_cp[TID] = (2*P[TID] + P[TIDp1] –
void main(void) dot(P[TIDp1]-
{ P[TID],N[TID])*N[TID])/3.0;
// TIDp1 = counter-clockwise (plus 1) cw_cp[TID] = (2*P[TID] + P[TIDm1] –
// control point index from TID dot(P[TIDm1]-
int TIDp1 = TID<2 ? TID+1 : 0; P[TID],N[TID])*N[TID])/3.0;
// ^^ Means TIDp1 = (TID+1)%3;
// TIDm1 = clockwise (minus 1) control point P2[TID] = ccw_cp[TID] + cw_cp[TID];
// index from TID // (b210+b120+b021+b012+b102+b201)/6.0
int TIDm1 = TID>0 ? TID-1 : 2; vec3 E = (P2[0] + P2[1] + P2[2])/6.0,
// ^^ Means TIDp1 = (TID+2)%3; V = (P[0] + P[1] + P[3])/3.0;
b111 = E + (E-V)/2.0;
P2[TID] = P[TID];
eye_space_normal[TID] = N[TID];
float v = 2*dot(P[TIDp1]-P[TID],N[TID]+N[TIDp1])
/ dot(P[TIDp1]-P[TID],P[TIDp1]-P[TID]);
vec3 h = N[TID] + N[TIDp1] - v*(P[TIDp1]-P[TID]);
mid_normal[TID] = normalize(h);
}
43. CS 354 43
PN Triangle
1.
Control Shader
1. Compute window-space edge distances
Thread parallel
Scaled edge length = exterior (edge) LOD 2.
Maximum scaled edge length = interior LOD
2. Compute Bezier triangle edge control
points 3.
Thread parallel
3. Compute Bezier triangle central control
point
4.
4. Pass through eye-space position &
normal
Thread parallel
5. Compute quadratic normal edge control
points 5.
Thread parallel
44. CS 354 44
Adaptive PN Triangle Tessellation
Evaluation Shader (1 of 3)
#version 400 compatibility
layout(triangles) in;
layout(ccw) in; // 10 input control points for
layout(fractional_even_spacing) in; // Bezier triangle position
// Assumes matrix in gluPerspective form in precise vec3
// Intended to be cheaper than full 4x4 eye_space_pos2[];
// transform by projection matrix in precise vec3 ccw_cp[];
// Compiles to just 5 MAD ops in precise vec3 cw_cp[];
precise vec4 applyPerspective( patch in precise vec3 b111;
precise mat4 affine_matrix,
precise vec4 v) // 6 input control points for
{
precise vec4 r; // quadratic normal
interpolation
r[0] = affine_matrix[0][0] * v[0]; in precise vec3
r[1] = affine_matrix[1][1] * v[1]; eye_space_normal[];
r[2] = affine_matrix[2][2] * v[2] + in vec3 mid_normal[];
affine_matrix[3][2]*v[3];
r[3] = -v[2];
return r;
}
45. Adaptive PN Triangle Tessellation
CS 354 45
Evaluation Shader (2 of 3)
void main(void)
{
// Evaluate position by weighting triangle Weight the position control points with
// vertex positions with the generated // (cubic) Bezier triangle basis
//
vertex's precise vec4 www = w*w*w * vec4(b300,1),
// barycentric weights. uuu = u*u*u * vec4(b030,1),
vec3 barycentric_weights = gl_TessCoord; vvv = v*v*v * vec4(b003,1),
precise float u = barycentric_weights.x, wwu3 = 3*w*w*u * vec4(b210,1),
v = barycentric_weights.y, wuu3 = 3*w*u*u * vec4(b120,1),
w = barycentric_weights.z; wwv3 = 3*w*w*v * vec4(b201,1),
// w should be 1-u-v uuv3 = 3*u*u*v * vec4(b021,1),
vec3 triangle_vertex[3]; wvv3 = 3*w*v*v * vec4(b102,1),
for (int i = 0; i < 3; i++) { uvv3 = 3*u*v*v * vec4(b012,1),
triangle_vertex[i] = eye_space_pos2[i]; wuv6 = 6*w*u*v * vec4(b111,1),
} wuE = wwu3 + wuu3,
// 10 position control points of a Bezier uvE = uvv3 + uuv3,
triangle wvE = wwv3 + wvv3,
precise vec3 b300 = eye_space_pos2[0], E = wuE + uvE + wvE,
b030 = eye_space_pos2[1], C = www + uuu + vvv,
b003 = eye_space_pos2[2], p = C + E,
b210 = ccw_cp[0], clip_space_p =
b201 = cw_cp[0], applyPerspective(gl_ProjectionMatrix, p);
b021 = ccw_cp[1], gl_Position = clip_space_p;
b120 = cw_cp[1],
b102 = ccw_cp[2],
b012 = cw_cp[2];
46. CS 354 46
Adaptive PN Triangle Tessellation
Evaluation Shader (3 of 3)
// Weight the normal control points with a quadratic
basis
vec3 n200 = eye_space_normal[0],
n020 = eye_space_normal[1],
n002 = eye_space_normal[2],
n110 = mid_normal[0],
n011 = mid_normal[1],
n101 = mid_normal[2],
normal = n200*w*w + n020*u*u + n002*v*v
+ n110*w*u + n011*u*v + n101*w*v;
// Visualize barycentric weights as color.
vec4 color = vec4(barycentric_weights, 1.0);
gl_FrontColor = color;
// Output the normalized surface normal
gl_TexCoord[0] = vec4(normalize(normal),1);
// Output the (perspective-divided) eye-space view
vector
gl_TexCoord[1] = vec4(p.xyz/p.w, 1);
}
47. CS 354 47
PN Triangle Evaluation Shader
1. Bezier triangle position evaluation
• Cubic evaluation using 10 position control points
• ~80 Multiply-Add operations
2. Apply perspective transformation
• Multiply position by spare frustum matrix
• 5 Multiply-Add operations
3. Bezier triangle normal evaluation
• Quadratic evaluation using 6 normal control points
• 3×12 Multiply-Add operations
4. Output vectors
• Barycentric coordinates encoded as color
• Normalized surface normal and eye direction vectors
48. CS 354 48
Careful About Cracks at Patch
Seams artificially injected cracks
Patch seams need to be computed
very carefully
Otherwise you get cracks
Animation makes cracks obvious!
Be sure of the following
Your LOD parameters at the
seam of two patches are
computed bit-for-bit identical
The control points that influence
the evaluation along a patch edge
are bit-for-bit identical and use
symmetric evaluation
Influence means has “non-zero
weight”
A+B+C ≠ C+B+A in floating-point
precise keyword can help this
49. CS 354 49
Other Surface Representations
Polynomial functions
As seen
Subdivision schemes
Refine mesh, smooth mesh, repeat
Implicit surfaces
Also known as iso-surfaces
F(x,y,z) = 0
50. CS 354 50
Subdivision Schemes
Many schemes
Catmull-Clark
Basic idea
51. CS 354 51
Catmull-Clark Example
Works on any closed base mesh
52. CS 354 52
Meta-balls: Implicit Surfaces
Iso-surface of union of multiple implicit
functions
53. CS 354 53
Non-photorealistic Graphics
Motivation
Realism can obscure when you
want to elucidate or convey a
mood
medical illustration
three takes on a house
57. CS 354 57
Detecting Artistic Features
Silhouette
Contour lines
Highlights
Ridge and valley lines
58. CS 354 58
Finding Silhouette Edges
Silhouette = edge shared by both a front-
and back-facing surface w.r.t. the viewer
Recall: shadow volumes can be extruded from silhouettes edges
59. CS 354 59
Artistic Texturing
Collect or arrange marks in a specific
arrangement
half-toning
60. CS 354 60
Toon Shading
Quantized shading results
Short for cartoon shading
Instead of continuous fall off
61. CS 354 61
Next Class
Next lecture
Review for Final
In-class course evaluation
Reading
Chapter 10, 524-555
Finish Project 4
Simple ray tracer
Due Wednesday, now Thursday May 3, 2012