View
222
Download
0
Embed Size (px)
Citation preview
COMP9018 - Advanced Graphics
Advanced Graphics Part 3• What now?• So far:
– Revised OpenGL– Discussed performance optimisation
• Now: the OpenGL photorealism/special effects toolkit.
• Some revision + some new material
COMP9018 - Advanced Graphics
Reference• David Blythe• Advanced OpenGL programming• One of the SIGGRAPH lecture
courses.
COMP9018 - Advanced Graphics
The advanced OpenGL toolkit
• Most of the OpenGL special effects are based on five basic ideas or combinations of them.
• Textures (incl multitextures)• Blending (using alpha channel)• Accumulation buffer (adding images
together)• Stencil buffer ("the cookie cutter")• Fog/depth cues (colours affected by
distance from viewer)
COMP9018 - Advanced Graphics
Plan ... • Look at each of these individually,
then look at how they can be combined to do special effects, like shadows, reflection, caustics, etc.
COMP9018 - Advanced Graphics
Textures• Staple of current polygonal
graphics architectures.• Why? Allows you to add detail
easily, without additional geometry.
• Will quickly revise some aspects.• Quick demos: Nate's texture, VRML
demos.
COMP9018 - Advanced Graphics
Texture use• OpenGL supports 1D, 2D, 3D textures.• Most of the time, textures are images,
not procedural. OpenGL supports only images.
• Texel is a pixel in a texture image. Often used to avoid confusion with pixel. We'll be talking about texels mapping to pixels, so this is useful.
COMP9018 - Advanced Graphics
Using textures• Each vertex in a polygon is defined
as having texture coordinates.• Texture is stretched (i.e mapped)
onto each polygon. • Texturing is one thing that really
benefits from hardware acceleration.
COMP9018 - Advanced Graphics
Steps in using textures• Creating the texture• Indicate how texture is applied• Enable texture mapping• Draw scene with both glVertex and
glTexCoord calls.
COMP9018 - Advanced Graphics
Creating textures• Messy!• Involves loading stuff up into
memory and then telling system to accept it as texture.
COMP9018 - Advanced Graphics
glTexImage2D()• One ugly function• glTexImage2D(target, level,
internalFormat, width, height, border, format, type, data)
• target: ?• level: for mipmaps – more later• internal format: how OpenGL stores the
data internally. • width & height: obvious, but note: must
be of form 2m + 2b where b is ...
COMP9018 - Advanced Graphics
glTexImage2D cont'd• ... border. Must be either 0 or 1.
Specify borders on side? OpenGL must support at least 64x64, but may support more.
• format & type: how is image data stored in memory?
• Finally, the data.
COMP9018 - Advanced Graphics
Notes on format• Textures don't have to be RGB.• Can be RGB, RGBA, LUMINANCE or
ALPHA or INTENSITY.• What's the "A" in RGBA? What's
alpha? To do with blending. Talk about in a little while
COMP9018 - Advanced Graphics
What if image is not a power of 2?
• Use gluScaleImage() to correct. • Note: Textures can be, say 64x32.
COMP9018 - Advanced Graphics
Funky stuff• Can read current frame buffer
directly for texture map:• glCopyTexImage2D(target, level,
internalFormat, x, y, width, height, border).
• Can be useful sometimes ... many special effects use this as a hack.
COMP9018 - Advanced Graphics
Repetition and Clamping• Can either say want lots of repetitions
or just one copy.• Can control separately for each texture
dimension (e.g. can repeat vertically, but not horizontally).
• How to set glTexParameteri(GL_TEXTURE_2D, GL_WRAP_{ST}, GL_REPEAT | GL_CLAMP).
COMP9018 - Advanced Graphics
What's with the target parameter?
• Target is used for optimisation.• Why? Graphics card only has certain
amount of memory.• Remember: right place at right time
stuff? • target has two possible values:
GL_TEXTURE_2D, or GL_PROXY_TEXTURE_2D.
• Proxy is to make enquiries about space on graphics card, efficiency, etc.
COMP9018 - Advanced Graphics
1D and 3D textures• 1D textures useful, e.g. paintbrush
or something. • 3D textures often arise in medical
data (e.g. CAT scan, MRI etc). Examples of volume rendering.
• HUGE amounts of data.
COMP9018 - Advanced Graphics
Different texture modes• Replace: Overwrite previous values. • Modulate: Modify existing colour
using textures (eg for lighting)• Decal: Like a sticker. Anyone built
model cars? Attach sticker to outside, but transparent.
• Blended: Mixed in some way. • Example: Nate’s texture.
COMP9018 - Advanced Graphics
Different modes• How to think of them: existing value
of pixel is f and texture is t, c is the current texture colour
• Then replace is C = Ct, A = At (ie only texture matters).
• Modulate is C = Cf.Ct, A = Af.At• Decal is C=Cf.(1-At)+Ct.At, A=Af• Blend is C=Cf.(1-Ct)+Cc.Ct; A =
Af.At
COMP9018 - Advanced Graphics
Uses of different mode• Replace: In 2D. • Modulate: Typical for use with
lighted polygons. Most commonly, polygon used is white.
• Decal: Insignias etc. • Blend: Not very frequently used,
but can be used to mix in a "background colour". Kind of like "modulate".
COMP9018 - Advanced Graphics
Enable texture mapping• glEnable(GL_TEXTURE_*D)
COMP9018 - Advanced Graphics
Supplying coordinates• glTexCoord{1234}{sifd}{v}(coordinate).• 4? Yes, you can specify coordinates in
homogenous form as well. (s,t,r,q).• Options: Repeating vs clamped. • Can do this independently for each texture.• glTexParameter(target, pname, param)• target = GL_TEXTURE_*D, pname, value).
COMP9018 - Advanced Graphics
s=p1xƒ p2yƒ p3zƒ p4w
Automatic coord generation
• Can use OpenGL to work out texture coords for you.
• But quite complicated – and less efficient.
• How?• You supply a linear equation for each
texture coord, expressed in terms of a point's position coords of the form:
• (p1,p2,p3,p4) depend on application.€
s = p1x + p2y + p3z + p4w
COMP9018 - Advanced Graphics
Texture coord generation• Can specify in either object
coordinates (i.e. when glVertex() gets called), or eye coordinates.
• 99 per cent of time, you want object (pre-transform) coordinates.
• When would you need eye coordinates? Some effects depend on eye position, e.g. some textures.
COMP9018 - Advanced Graphics
The OpenGL code• glTexGen{ifd}(coord, pname, param)• coord is GL_S, GL_T, GL_R or GL_Q• pname is GL_TEXTURE_GEN_MODE –
then either GL_OBJECT_LINEAR, GL_EYE_LINEAR or GL_SPHERE_MAP
• pname is GL_OBJECT_PLANE or GL_EYE_PLANE, then the next parameter is the vector p1, p2, p3, p4
COMP9018 - Advanced Graphics
Demo• Have a look at texgen.c
COMP9018 - Advanced Graphics
So much to textures ...• Still to do:
– Texture filtering– Mipmaps– Multitexturing– Sphere maps/Environment Maps – Specular reflection and textures– Light maps– Bump maps
COMP9018 - Advanced Graphics
Texture management• Use texture objects.• Can speed things up. • Usage very similar to display lists.
COMP9018 - Advanced Graphics
Initialisation• Get some texture ids• glGenTextures(n, textureids)• n is number of textures you want,
textureids is an array of ints identifying the textures.
COMP9018 - Advanced Graphics
Usage• Usage is a little odd. Think of current
texture as part of the context. • glBind(type, textureid) sets the current
texture.• Any definitions affect the current
textureid. So to set up call glTexParameteri() and glTexImage2D().
• To change current texture, call glBind on another texture
• Look at texbind.c
COMP9018 - Advanced Graphics
Texture usage and hardware
• Textures can live in memory or on graphics card.
• If in memory have to be copied onto card whenever they need to be used. SLOW!
• Why not keep all textures on card? Because card has limited memory.
• Resident = texture lives on graphics card. • glAreTexturesResident(n, textureids,
residentstates) can be used to work
COMP9018 - Advanced Graphics
But I want CONTROL!• Not much use if you don't have control.• First of all, glDeleteTextures(n,
textureids) is useful.• glPrioritizeTexures(n, textureids,
priorities)• priority 1 = gotta be on the card, 0 =
who cares?• LRU algorithms are sometimes used if
textures have equal priority.
COMP9018 - Advanced Graphics
Filtering• General problem: Texels and pixels
are not going to be the same size.• Three possibilities:
– Magnification of texture: one texel maps to many pixels.
– Minification of texture: many texels map to one pixel.
– Exact match: Yeah right!
COMP9018 - Advanced Graphics
Magnification vs Minification
COMP9018 - Advanced Graphics
Solving the magnification problem
• Magnification leads to pixelation effects ... we can see the texels on the screen.
• Ugly!• How to fix? • Linearly interpolate: Each intensity value
is the centre of a pixel and then take a weighted average of surrounding four pixels.
• Called bilinear filtering.
COMP9018 - Advanced Graphics
Bilinear filtering (mag)
Texture Space (zoomed in to pixel level)
A B
CD
E
F
G
• Interpolate along AB to get intensity at E.
• Interpolate along DC to get intensity at F.
•Interpolate along EF to get intensity at G.
COMP9018 - Advanced Graphics
Comparison• On previous slide, if using point
sampling (GL_NEAREST), then it will be coloured black.
• If using bilinear filtering, it is a weighted sum of the four surrounding pixels, depending on distance.
• In this case, it would be coloured grey.
COMP9018 - Advanced Graphics
What about minification?• Ok, linear interpolation fixes problem with
close up, but what about far away?• Can also use linear interpolation, but what
use is that?• Point of using linear interpolation was to
"smooth" edges and pixels.• But real problem with min is the opposite.• Ideal solution: average values of pixels
covered.• Obviously not feasible
COMP9018 - Advanced Graphics
MIPMaps• A precomputation approach• Say we have a 64x64 texture, then we
calculate 32x32, 16x16, 8x8, 4x4, 2x2 and 1x1 ahead of time.
• Each is created by filtering higher resolution images down, so it is nicely rounded.
• Fits efficiently into memory.• Depending on distance to object, we
will use a different mipmap.
COMP9018 - Advanced Graphics
MIPMap idea
From OpenGL Programming Guide
COMP9018 - Advanced Graphics
MIPMap issues• Idea: At different distances use the
most appropriate MIPMap. • Generally, the most appropriate is
the one that gives the closest to 1:1 texel:pixel ratio.
• But can also interpolate between MIPMap levels.
• Called trilinear mipmapping.
COMP9018 - Advanced Graphics
How does this fix things?• Prevents aliasing. If we use point
sampling, then basically it picks a random texel as colour.
• This precomputes averages.
COMP9018 - Advanced Graphics
Mipmaps in OpenGL• Two options:
– Do it yourself: Rememeber glTexImage2D(target, level, internalFormat, width, height, border, format, type, data)? The level is the mipmap level.
– Get gluBuild2DMipmap() to do the work for you.
• Why not use gluBuild2Dmipmap? – Can use better filters– Might want to play tricks– Some textures should be mipmapped carefully
COMP9018 - Advanced Graphics
Filtering ...• How to handle the issue of
minification and magnification?• Have to apply a filter ... something
that will combine data from multiple texels.
• Note: Think of texel as providing value in CENTER of texel, not for whole texel.
COMP9018 - Advanced Graphics
Magnification• When magnifying we can do two ways:• Point sampling:
– Map pixel centre into texel space. Pick nearest texel as colour of pixel. glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST)
• Linear sampling:– Map pixel centre into texel space. Pick
neareast 4 texels and average using bilinear interpolation. glTexParameteri(", ", GL_LINEAR)
COMP9018 - Advanced Graphics
Minification• Point sampling and linear sampling
can be used as before. But glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST|GL_LINEAR)
• But can now use mipmaps!!
COMP9018 - Advanced Graphics
Minification types• GL_NEAREST_MIPMAP_NEAREST (choose
nearest mipmap level, then choose nearest pixel in that mipmap)
• GL_LINEAR_MIPMAP_NEAREST (choose nearest mipmap level, then do bilinear interpolation)
• GL_NEAREST_MIPMAP_LINEAR (interpolate between nearest point in two mipmaps)
• GL_LINEAR_MIPMAP_LINEAR (trilinear interp)
COMP9018 - Advanced Graphics
Environment mapping• Reminder: Environment mapping is a hack to
add reflection to polygon rendering.• Basic idea:
– Render/photograph the world around the object of interest onto an enclosing shape. Turn this into a texture map
– Reflect the ray from the viewer off the surface (like ray tracing)
– Use the reflected ray as an index for the texture map.
– Q: What “shape” is texture map.
COMP9018 - Advanced Graphics
Representation of scene
Object
Vector from viewer
ReflectedRay
Pixel incube mapthat is mapped on to object
N
Environment mapping
COMP9018 - Advanced Graphics
Environment mapping in practice
• Uses texture maps. • Is an example of view dependent
textures: depends on position on viewer.
• Coordinates are calculated whenever view changes.
• Means that every time viewer's position changes, so does texture coordinates.
COMP9018 - Advanced Graphics
Reflected ray• Calculations done in eye
coordinate space. (do on OHP).• Final equation:
€
r = u − 2(n • u )n
where
r = Reflected Ray
u = Vector from origin to point
n = Normal in eye coordinates
COMP9018 - Advanced Graphics
Representing surrounding world
• Different ways of representing the world around the object.
• Ideal solution would be to use a spherical texture.
• Problem: Representing a 3D surface like a sphere in 2D always leads to distortion.
• Some techniques:– Cube mapping– Sphere mapping– Dual paraboloid mapping (only in passing)
COMP9018 - Advanced Graphics
Cube map • Scene is represented as six sides of
a cube.• The cube is the unit cube ie from (-
1,-1,-1) to (1,1,1).
COMP9018 - Advanced Graphics
Cube map
Object
Vector from viewer
ReflectedRay
Pixel incube mapthat is mapped on to object
N
Cube map
COMP9018 - Advanced Graphics
Cube maps coords• How to work out coordinates in cube
map?• Two subproblems:
– Which face of the cube to look at?– What (s,t) coords of that face to use?
• Solution:– Use longest coordinate – Normalise by length, then adjust other
coordinates. Then get to usual texture coordinates.
COMP9018 - Advanced Graphics
What are coordinates?• Which side depends on normal
vector. • E.g. if vector is (0.7, 0.5, 0.5) then
this means it must intersect with the face on the positive x-axis
• The coordinates on the face will be s = 0.5/0.7, t = 0.5/0.7
COMP9018 - Advanced Graphics(-1,-1)(-1,-1)
(1,1)
(0,0)
(1,1)
?
Different representations• Texture maps are in [0,1]x[0,1]• Faces of cube are [-1,1]x[-1,1].• Question: How to map from cube face
to texture map?
COMP9018 - Advanced Graphics
Mapping from face to texture
• Simple
€
′ s =s
2+
1
2
′ t =t
2+
1
2
COMP9018 - Advanced Graphics
Making a cube map• How to make a cube map?• Can just take six pictures of a scene.• But what if we have an artificial
scene?• Even easier! Render six images one
for each face.• Hence can do real-time reflection
(except each will take 7 renders).
COMP9018 - Advanced Graphics
Using Cube Maps in OpenGL
• Added in version 1.3• Was an ARB extension before• Specify each of the textures (one
for each of the sides) • Basically a 2D texturing technique• How to calculate?
COMP9018 - Advanced Graphics
Making a cube map• Use the existing glTexImage2D() call• Uses a new target. Remember: Target is
usually GL_TEXTURE_2D• Usually something like:
GL_TEXTURE_CUBE_MAP_{SIGN}_{AXIS}• SIGN = POSITIVE|NEGATIVE• AXIS = X|Y|Z• glEnable(GL_TEXTURE_CUBE_MAP)
COMP9018 - Advanced Graphics
Coordinate generation• A new mode: GL_REFLECTION_MAP• glTexGenfv(GL_S,
GL_TEXTURE_GEN_MODE, GL_REFLECTION_MAP)
• Similarly for t and r!• Also support for GL_NORMAL_MAP
(talk about that later).
COMP9018 - Advanced Graphics
Manual definition• Don't HAVE to use either of these. Can
specify texture coordinate as glTexCoord3f(s,t,r). This will then be treated the same as the reflection vector.
• Hence can use for arbitrary vector lookup. • A very powerful capability ... can be used
for many special effects (look at bump mapping later).
COMP9018 - Advanced Graphics
Demos• Have a look at GLUT demos
– TexCyl– NewWave
COMP9018 - Advanced Graphics
Sphere maps• An alternative approach.• Older, more space efficient. • Easier to implement -- doesn't
involve choosing an individual side. • But ... harder to understand.
COMP9018 - Advanced Graphics
What is a sphere map?• Imagine taking perfect sphere
putting it in scene and taking picture
• See Kilgard's slides
COMP9018 - Advanced Graphics
Sphere maps in OpenGL • To generate coordinates, use
glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_SPHERE_MAP)
• Similarly for T (but not r!)• How are coordinates actually
generated?
COMP9018 - Advanced Graphics
Equations for sphere map• Proof: OHP
€
r = (rx,ry,rz)
m = 2 rx2 + ry
2 + (rz +1)2
s =rx
m+
1
2
t =ry
m+
1
2
COMP9018 - Advanced Graphics
Environment maps: In practice
• Unless surface is really reflective, lo-fi envmaps work well.
• In real-time applications (i.e. mostly games): – Envmap is only low res (even 64x64x6 for
cube maps)– Envmap is not updated every frame ... e.g.
every 10 frames - but doesn't work too well with moving viewer
• Sphere maps are old hat, cubes are in fashion, dual paraboloid still theoretical.
COMP9018 - Advanced Graphics
Environment maps: Limitations
• Sphere map's singularity is a problem.• Cubes still have problems in corners.• All only give one level of reflection (of
course, you can iterate process ...)• Bit of a performance hit, but can really
enhance the appearance. • Have a look at Spin.java
COMP9018 - Advanced Graphics
Multitexturing• Won't go into details too much• But some graphics hardware supports
applying more than one texture in a single pass.
• Can always accomplish same effect in multiple passes
• OpenGL supports multitexturing. • Why > 1 texture? Envmapping, bump
mapping, light maps