Modern OpenGL ES 3.0 and 3.1 Lighting and Materials

Modern OpenGL ES 3.0 and 3.1Lighting and MaterialsPresenters: Ron Fosner & Rudy Cazabón

OpenGL-ES 3.0 And Beyond Boston 2014OpenGL-ES 3.0 And Beyond Boston 201422

Overview

What is the Mission of the Modern OpenGL Graphics Pipeline?Definitions and TerminologyLights – Physical Motivation for IlluminationMaterials - Physical Motivation for Visible Surface Phenomena“Classic” OpenGL Materials and LightingModern OpenGL Implementations of Materials and Lights with GLSLNon-Photorealistic Rendering - Alternative Illumination and Shading Models


What is the Mission of the Modern OpenGL Graphics Pipeline?1. Present a modular

framework2. Describe sources of

illumination in a scene 3. Bind lighting functions

and data for visible surface phenomena to groups of geometries in a scene

Vertex Stage Rasterization Fragment Stage

Ldiffuse cos)ˆˆ,0max( LN

float NdotL,NdotHV; lightDir = vec3(gl_LightSource[0].position);

vec4 color = ambient;n = normalize(normal);NdotL = max(dot(n,lightDir),0.0);


Illumination - the transport of light from a source to a point via direct and indirect pathsIllumination Models:• Empirical - approximations to

observed light properties• Physically based - applying physics

properties of light and its interactions with matter

Lighting - computing the luminous intensity for a specified 3D point, given a viewpointMaterial – In OpenGL domain, refers to the surface properties that determines how it shall interact with N-lights in a sceneShading - assigning colors to pixels


Modern OpenGL with GLSLWhat We Know Thus Far

We already know how to render the world from a viewpoint.


Lights and Materials in the Modern OpenGL ContextThere are two components that go into the final color of a pixel

1. Lighting Model or Shading Model

How we calculate the intensity at a point on the surface

2. Surface Rendering Method –

How we calculate the intensity at each pixel


Simple OpenGL ES shader-based Pipeline

GPU Data FlowApplication

Framebuffer

VertexProcessing Rasterizer Fragment

Processing

VertexShader

FragmentShader

Shader Code

Shader Code


Lights – A Physical Motivation for Illumination

Lighting is one of the most complex aspects of the rendering process

• No-one has yet come up with a "perfect" simulation of rendering for use in real-time graphics - even non-real-time graphics haven't really solved the problem for every material

Lighting is required as it conveys the sense of surface roughness/smoothness and material visual appearance


…In the Beginning - Fiat LuxMaxwell Equations- Integral Form

Maxwell Equations - Differential Form

Rendering Equation

Simplifying Assumptions

𝑎2+𝑏2=𝑐2

Illumination

dtLtftLtL ireo )(),,,(),,,,(),,,(),,,( nxxxx


A Reasonable OpenGL Illumination Model

2210

1,1mindadaa

df

1

0

lights

l

shininessspecularldiffusellambientlfinal HNkILNkIdfkII

speculardiffuseambient

Illumination Model

Attenuation = Distance-based decay factor


Materials - Physical Motivation for Visible Surface Phenomena

“Materials” in CG refers to the group of properties that model the phenomena of reflection, refraction, absorption, and emission as it interacts with illuminationEqually hard as light to model!

𝑳 ( �⃗� , �⃗�𝑖𝑟 , ⟨𝒄𝒐𝒍𝒐𝒓 ⟩ ,𝒕𝒊𝒎𝒆 )= ∑❑

𝒔𝒂𝒎𝒑𝒍𝒆 𝒅𝒊𝒓(𝑮 . 𝑰 . 𝒍𝒊𝒈𝒉𝒕 )± ∑

❑

¿𝑳𝒊𝒈𝒉𝒕𝒔𝑹𝒆𝒇𝒍𝑭𝒖𝒏𝒄 (𝒅𝒊𝒇𝒇𝒖𝒔𝒆+𝒔𝒑𝒆𝒄𝒖𝒍𝒂𝒓 )


Reflection TermsAmbient

Iambient = RGB Colorvec3 ambient = Light.La * Material.Ka

Diffuse

Idiffuse = kdIlightcos = kdIlight(N . L) vec3 diffuse =

Light.Ld * Material.Kd * dot(N,L);

Specular

Ispecular = ksIlight (cos )shininess = ksIlight (V.R)shininess

vec3 spec = Light.Ls * Material.Ks * pow(max(dot(r,v),0.0),Material.Shininess );

NL R

V


Calculation of per-color Component

The contribution to the object’s color from each light source is given by:C = (att)(spot)[acm*acli + dcm*dcli*dot(L,N) + (att)srm scm*scli*dot(R,N)]respectively, and acm, acli, dcm, dcli, scm, and scli are the ambient material color, ambient light color, diffuse material color, diffuse light color, specular material color, and specular light color, respectively. Finally, att is the light source attenuation factor, spot is the spotlight contribution


Ambient + Diffuse + Specular


Fixed-function OpenGL Per-vertex Lighting

• “Classic” OpenGL had a fixed lighting model• Operated at the per-vertex level• Referred to as the Blinn-Phong lighting model

• Example usageglEnable(GL_LIGHTING);glEnable(GL_LIGHT0);glLightfv(GL_LIGHT0, GL_POSITION, position);glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuse);glLightfv(GL_LIGHT0, GL_SPECULAR, specular);

• Relied on• Per-vertex normal value sent with glNormal3f• And per-vertex material properties sent with glMaterialfv


Fixed-function OpenGL Materials

• “Classic” OpenGL material properties assigned to vertices• Lighting calculations were done at vertices only, and the

results of those calculations were interpolated to other points of the object

• Material properties are set using the glMaterial* family of commands, principally glMaterialfv and glMateriali.

• Example usage:

glMaterialfv(GL.GL_FRONT, GL.GL_SPECULAR, new float[] { 0.75f, 0.75f, 0.75f, 1 }, 0);

glMaterialfv(GL.GL_FRONT, GL.GL_AMBIENT_AND_DIFFUSE, new float[] { 0.75f, 0.75f, 0, 1 }, 0);

glMateriali(GL.GL_FRONT, GL.GL_SHININESS, i*16);


Modern OpenGL Implementations of Materials and Lights with GLSL

It is about programmable stages

• Vertex Shader• Operates on each

vertex of geometric primitives

• Passes through per-vertex color

• Transforms the vertex to match fixed-function processing

• Fragment Shader• Operates on each

fragment (think pixel)

• Outputs the fragment’s interpolated color to the framebuffer

void main(void){ gl_FrontColor = gl_Color; gl_Position = ftransform();}

void main(void){ gl_FragColor = gl_Color;}


Light Source PropertiesColor - We usually assume the light has one wavelength

struct LightInfo { vec4 Position; // Light position in eye coords. vec3 La; // Ambient light intensity vec3 Ld; // Diffuse light intensity vec3 Ls; // Specular light intensity

};uniform LightInfo Light;

Point Light - approximate the light source as a 3D point in space. Light rays emanate in all directions.

Directional - approximating the light source as coming from infinity. Light rays are parallel

Spotlights – approximates a nearby light at a specific point in space, a direction vector, a fall-off function from central direction vector and distance


AttenuationEnergy from a light source falls off proportional to 1/d2.Using only 1/d2, makes it difficult to correctly light things. Think if d=1 and d=2. Why?• We are approximating

things. Lighting model is too simple AND most lights are not point sources

2210

1,1mindadaa

df


Contributions from lights

Breakdown what a light does at a surface into three different components. This APPROXIMATES what a light does. • Illumination at a pixel from a light • Intensity = Ambient + Diffuse + Specular contributions.

Ilight = Iambient + Idiffuse + Ispecular

#version 300

Void main() {out vec3 LightIntensity;uniform vec3 Iambient;uniform vec3 Idiffuse;uniform vec3 Ispecular;

<stuff happens here>

LightIntensity = Iambient + Idiffuse + Ispecular;}


Contributions from materials

How a surface reflects into three different components.• Reflectance at a pixel from a reflected point• Intensity = Ambient + Diffuse + Specular contributions

struct MaterialInfo { vec3 Ka; // Ambient reflectivity vec3 Kd; // Diffuse reflectivity vec3 Ks; // Specular reflectivity float Shininess; // Specular shininess factor

}; uniform MaterialInfo Material;


Quick overview of textures

Say you have this triangle, with these texture coordinates.They go from 0 to 1 in X and Y*, with the origin being at the bottom left by convention.

* There can also be 1 or 3 dimensional textures, but the coordinates all go 0 - 1



And you have this texture loaded.Note that it’s texture cords also start in the lower left corner.



When you align the triangle’s and the texture’s coordinates, the align like this.



When the triangle is rendered, you just the part of the image that is aligned with the triangle’s texture coordinates. Textures are used to add details to a trianglewithout using a denserset of triangles.



If you multiply the vertex texture coordinates by 4, then you can wrap the images across the triangle multiple times.*

* Assuming texture coordinate wrapping is how the state is set!

4 images

4 images


Contributions from Textures

How a surface reflects into three different components.• Reflectance at a pixel from a reflected point• Intensity = Ambient + Diffuse + Specular contributions

uniform sampler2D uSourceTex;struct MaterialInfo { <previous stuf>

attribute vec2 aPosition;attribute vec2 aTexCoord;varying vec2 vTexCoord;

}; uniform MaterialInfo Material;


Shading

Shading is how we “color” a triangle.• Constant Shading• Gouraud Shading• Phong Shading

1

0

lights

l

shininessspecularldiffusellambientlfinal HNkILNkIdfkII

speculardiffuseambient


Constant Shading

Constant Intensity or Flat ShadingPer-triangle normalOne color for the entire triangleFastGood for some objectsWhat happens if triangles are small?Sudden intensity changes at borders


Gouraud Shading

Compute the color at each vertex firstCompute the color inside the polygon by interpolating the colors of the vertices composing the polygon


Gouraud ShadingRelatively fast, only do three calculationsNo sudden intensity changesProblem with Gouraud shading for larger triangles:• If light is near-middle of triangle it is possible to not

have lights/highlights in the middle as intended


Phong Shading

Phong lighting model is object-space illumination model

It considers four types of lightings: diffuse, specular, ambient and emissive.

Requires interpolation of the normal as it represents the “curvature”

Performed for each pixel, as during scan convert

This Calculates the lighting per pixel normal.


Comparison of Techniques

Flat shading is easy but results are bad

Gouraud shading is usually used for simple scenesPhong shading is better visually and with modern HW it is computationally feasible


Shading Terms Implemented in GLSLAmbient

Vertex shader Fragment Shader

Ambient

// Transforming The Vertexgl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

// Setting Each Pixel To Redgl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);


Shading Terms Implemented in GLSLDiffuse

Diffuse


// Transforming The Normal To ModelView-Spacenormal = gl_NormalMatrix * gl_Normal;

// Transforming The Vertex Position To ModelView-Spacevec4 vertex_in_modelview_space = gl_ModelViewMatrx * gl_Vertex;

// Calculating The Vector From The Vertex Position To The Light Positionvertex_to_light_vector = vec3(gl_LightSource[0].position �vertex_in_modelview_space);

// Defining The Material Colorsconst vec4 AmbientColor = vec4(0.1, 0.0, 0.0, 1.0);const vec4 DiffuseColor = vec4(1.0, 0.0, 0.0, 1.0);

// Normalizing Input Vectorvec3 normalized_normal = normalize(normal);vec3 normalized_vertex_to_light_vector = normalize(vertex_to_light_vector);

// Calculating The Diffuse Term And Clamping It To [0;1]float DiffuseTerm = clamp(dot(normal, vertex_to_light_vector), 0.0, 1.0);

// Calculating The Final Colorgl_FragColor = AmbientColor + DiffuseColor * DiffuseTerm;


Shading Terms Implemented in GLSLTexturing

Texturin

g


// Passing The Texture Coordinate Of Texture Unit 0 To The Fragment Shadertexture_coordinate = vec2(gl_MultiTexCoord0);

// Sampling The Texture And Passing// To The Frame Buffergl_FragColor = texture2D(my_color_texture, texture_coordinate);


Non-Photorealistic Rendering

Nearly all of the energy and efforts in computer graphics is towards achieving “photorealistic” or “real-world” representations for rendered objectsNon-Photorealistic Rendering (NPR) is an area of computer graphics that engages the mathematics and techniques of rendering to the goal of arriving at rendering methods that expressively stylized to painting, drawing, or draftingThe end-results of these NPR renderings may be artistic or have a “hard usage” such as technical illustrations


Examples of NPR

Gooch, Amy and Bruce, et. Al

Salesin, D. and Lee Markosian, et. Al

Cartoon/Cel Shading

Graftals

AnDevCon Boston 201439 AnDevCon Boston 2014

Back up slides

Documents

Modern OpenGL ES 3.0 and 3.1 Lighting and Materials