Further Programming for 3D applications CE00849-6 Introduction to Further Programming for 3D application Bob Hobbs Faculty of Computing, Engineering and

Further Programming for 3D applicationsCE00849-6

Introduction to Further Programming for 3D application

Bob Hobbs

Faculty of Computing, Engineering and Technology

Staffordshire University

OutlineOutline• Module DetailsModule Details

• What is 3D programming?What is 3D programming?

• Typical Processing StepsTypical Processing Steps

• Modelling and RenderingModelling and Rendering

• Windows and Direct XWindows and Direct X

• Programmable PipelineProgrammable Pipeline

• SummarySummary

Module DetailsModule Details

• Teaching TeamTeaching Team– Bob HobbsBob Hobbs [email protected]– Steve Foster Steve Foster [email protected]

• Semester 2Semester 2 15 cats15 cats• 3 Hours per week3 Hours per week

– 1 Hour Lecture 1 Hour Lecture Thu 4pmThu 4pm– 2 Hours Practical 2 Hours Practical Mon 9-11am& 2-4pm Mon 9-11am& 2-4pm

K106K106Fri 1-3pm & 3-5pm K106Fri 1-3pm & 3-5pm K106

mailto:[email protected]

mailto:[email protected]

Module DetailsModule Details

Course Handbook & Lecture NotesCourse Handbook & Lecture Notes

• BlackboardBlackboard

Assignment DetailsAssignment Details

• 50% assignment work50% assignment work

• 50% two hour exam50% two hour exam

Program of StudyProgram of Study• Week 01 Week 01 Introduction to Module and DirectX11Introduction to Module and DirectX11• Week 02 Week 02 Culling, Clipping and Line drawingCulling, Clipping and Line drawing• Week 03 Week 03 Lighting and Shading using Lighting and Shading using

Programmable Programmable pipelinepipeline• Week 04Week 04 Textures & Texture MappingTextures & Texture Mapping• Week 05Week 05 Blending, Stencilling and use of Blending, Stencilling and use of

Geometry Geometry ShaderShader• Week 06 Week 06 Meshing and InstancingMeshing and Instancing• Week 07Week 07 Data structures for graphics enginesData structures for graphics engines• Week 08Week 08 Picking, Interaction and Motion CuesPicking, Interaction and Motion Cues• Week 09Week 09 Tessellation and TerrainTessellation and Terrain• Week 10 Week 10 Cube, Normal and Bump mappingCube, Normal and Bump mapping• Week 11Week 11 Other SFXOther SFX• Week 12Week 12 Review and AssessmentReview and Assessment

What is 3D rendering?What is 3D rendering?

Object

Viewer

Generally deals with graphical display of 3D objects as seen by viewer

Synthetic image will vary according to: viewing direction, viewer position, illumination, object

properties, ...

Projection onto 2D surface

What is 3D Computer What is 3D Computer Graphics?Graphics?• 3D graphics: generation of 3D graphics: generation of

graphical display (graphical display (renderingrendering) ) of 3D object(s) from of 3D object(s) from specification (specification (model(s)model(s)))

Specification Modelling

Graphical display

Rendering

such as

Wireframe polygonal model Solid object

Vertices

Viewpoint

Typical Processing StepsTypical Processing Steps

Facets

Transformation

Hidden surface removal

Light source

Shading

Object model(s)

Illumination model

Viewing and projection

specification

Graphics engine

Graphical display

Rendering: operations that produce view of scene projected onto view surface

Modelling: numerical description of scene objects, illumination, and viewer


19 Apr 202319 Apr 2023 1010

2D pictureLight source

Surface material


xm

zm

ym

xm

zm

ym

3D world coordinates

3D modelling coordinates

2D device coordinates

3D viewing coordinates& Culling

xw

zw

ywxv zv

yv

Graphics Graphics pipelinepipeline:: generation of picture involves generation of picture involves sequence of sequence of coordinatecoordinate transformationstransformations, , hidden hidden surface removalsurface removal, , shading/texturingshading/texturing and and etc etc ......

19 Apr 202319 Apr 2023 1111Typical Typical ggraphics raphics ppipelineipeline


• Pos., orient. of display area

Geometry models

• Shapes• World pos., orient., sizes

Viewpoint

• Pos., orient.• Field of view• Projection type

Viewport

Object & back-face culling

View mapping onto display

device

Generation / configuration

of world

Mesh operationsGeometry processing: shape, world

geometry, viewpoint

2D p

ictu

re

Lights & materials

Shading, texturing, ..

.

• Light pos., orient. • Optical char.

Pixel operationsMaterial processing: colour,

texture, transparency/occlusion, …

Hidden surface

removal

Viewpoint

ModellingModelling

Human Head Model

1438 facets

ModellingModelling

Human Head Model

7258 facets

ModellingModelling

Teacher and Board Model

2074 facets

RenderingRendering

Shaded Human Head

1438 facets

RenderingRendering

Shaded Teacher and Board

19 Apr 202319 Apr 2023 2121

x

yRasterisation of filled triangleRasterisation of filled triangle

Point grid

(raster)Scan line

Point / Pixel

External point

RenderingRendering

Common approach: polygon interior filled Common approach: polygon interior filled line by lineline by line, , one point at a time one point at a time (rasterisation) (rasterisation)

• linearlinear interpolationinterpolation (i.e. line equation) often used for colour (i.e. line equation) often used for colour shadeshade, , depthdepth and other calculations and other calculations at each pixelat each pixel

Windows ProgrammingWindows Programming• Rendering takes place in a windowRendering takes place in a window

• Application must generate window in Application must generate window in which to render graphicswhich to render graphics

• IP3DA used C and GLUT to handle IP3DA used C and GLUT to handle windows. OpenGL can run in GLUT or other windows. OpenGL can run in GLUT or other windowing framework including MS windowing framework including MS windowswindows

• DirectX is integrated into windows DirectX is integrated into windows programming APIprogramming API

What is a “window?”What is a “window?”

• The most obvious windows are the The most obvious windows are the application windows on your desktop.application windows on your desktop.

• The term “window” also refers to any The term “window” also refers to any child window or control (such as a child window or control (such as a button) within a window.button) within a window.

• Windows are organized in a Windows are organized in a hierarchy.hierarchy.

Example - CalculatorExample - Calculator

Event Driven ProgrammingEvent Driven Programming

• Windows programming is Windows programming is event drivenevent driven..

• An event driven program sits and waits An event driven program sits and waits for an event to process.for an event to process.

• Events include moving or sizing a Events include moving or sizing a window, clicking the mouse button over a window, clicking the mouse button over a window, or typing keys on the keyboard.window, or typing keys on the keyboard.

• Windows programs are notified of events Windows programs are notified of events by messages sent from the OS.by messages sent from the OS.

Win32 APIWin32 API

• We will use the Win32 Platform API to We will use the Win32 Platform API to write a windows program.write a windows program.– Typically more tedious, but we only Typically more tedious, but we only

need to create one window.need to create one window.– Faster and Smaller executable.Faster and Smaller executable.– Written in C++Written in C++

• Other options include:Other options include:– MFC, Qt, wxWindows, SDL, .NET FormsMFC, Qt, wxWindows, SDL, .NET Forms

WinMainWinMain

• Instead of Instead of mainmain, windows programs use , windows programs use WinMainWinMain

• WinMainWinMain has different parameters: has different parameters:– HINSTANCE hInstanceHINSTANCE hInstance – a handle to the program – a handle to the program– HINSTANCE hPrevInstanceHINSTANCE hPrevInstance – no longer used. – no longer used.– LPSTR lpCmdLineLPSTR lpCmdLine – – unparsedunparsed command line. command line. Doesn’tDoesn’t include executable’s filename. include executable’s filename.

– int nCmdShowint nCmdShow – Specifies how the window should be – Specifies how the window should be initially shown (ie. Minimized, Maximized, Normal)initially shown (ie. Minimized, Maximized, Normal)

• WinMainWinMain must have a must have a WINAPIWINAPI modifier. modifier.– int WINAPI WinMain (…)int WINAPI WinMain (…)

• WinMainWinMain still returns an integer like main. still returns an integer like main.

HeadersHeaders

• To use the Win32 API we need to To use the Win32 API we need to #include windows.h#include windows.h

• If we #define If we #define WIN32_LEAN_AND_MEANWIN32_LEAN_AND_MEAN before #including windows.h, the before #including windows.h, the compiler will skip compiling the more compiler will skip compiling the more rarely used windows code.rarely used windows code.

Windows Program OutlineWindows Program Outline

• Create Window ClassCreate Window Class

• Create WindowCreate Window

• Show the WindowShow the Window

• Enter Message LoopEnter Message Loop

Window ClassesWindow Classes

• A A window classwindow class acts as a template for window acts as a template for window creation.creation.

• Window classes are Window classes are notnot classes in the C++ classes in the C++ sense.sense.

• We can set such properties as:We can set such properties as:– Name of the window class – Name of the window class – importantimportant for identifying it for identifying it

later!later!– Window style – How it looks and its default behaviors.Window style – How it looks and its default behaviors.– Window procedure – a pointer to function that handles Window procedure – a pointer to function that handles

messages for a window.messages for a window.– Default cursor, icon, and menu.Default cursor, icon, and menu.

Creating a Window ClassCreating a Window Class

• Fill out a Fill out a WNDCLASSWNDCLASS or or WNDCLASSEXWNDCLASSEX structure.structure.

• Use the structure to register the Use the structure to register the class with a call to class with a call to RegisterClassRegisterClass or or RegisterClassExRegisterClassEx, respectively., respectively.

Create the WindowCreate the Window

• Create a Window with a call to Create a Window with a call to CreateWindowCreateWindow or or CreateWindowExCreateWindowEx..

• Some of the parameters you will need to Some of the parameters you will need to enter include:enter include:– The name of a window class. The name of a window class. – The name of the window.The name of the window.– A handle to a parent window. (A handle to a parent window. (NULLNULL if no parent) if no parent)– Initial size and position of the window.Initial size and position of the window.

• CreateWindowCreateWindow and and CreateWindowExCreateWindowEx will will return a return a window handlewindow handle ( (HWNDHWND) if successful ) if successful or or NULLNULL if failed. if failed.

Showing the WindowShowing the Window

• When we create a window it starts off hidden.When we create a window it starts off hidden.

• To show the window you must call To show the window you must call ShowWindow (hWnd, nCmdShow)ShowWindow (hWnd, nCmdShow)– hWndhWnd – returned from – returned from CreateWindowCreateWindow//ExEx..– nCmdShownCmdShow – parameter from – parameter from WinMainWinMain..– This will send a message to the window telling it This will send a message to the window telling it

to show.to show.

• Call Call UpdateWindow (hWnd)UpdateWindow (hWnd) to force the to force the window to process the show message window to process the show message immediately. immediately.

Message LoopsMessage Loops• A A message loopmessage loop is needed to forward is needed to forward

messages to their windows.messages to their windows.

• There are different ways to make a message There are different ways to make a message loop but here is the most common one:loop but here is the most common one:

MSG msg = {0};

// returns FALSE when message is WM_QUITwhile (GetMessage (&msg, NULL, 0, 0)){ TranslateMessage (&msg); // translate keyboard messages. DispatchMessage (&msg); // send message to its window.}

Message Loop (Cont’d)Message Loop (Cont’d)• The loop uses:The loop uses:

– A A MSGMSG structure – Holds message information. structure – Holds message information.– GetMessageGetMessage – Retrieves the next message. Waits – Retrieves the next message. Waits

for a message if the message queue is empty.for a message if the message queue is empty.– TranslateMessageTranslateMessage – Translate some keyboard – Translate some keyboard

messages.messages.– DispatchMessageDispatchMessage – Dispatches the message to – Dispatches the message to

the appropriate windows procedure.the appropriate windows procedure.

Windows ProcedureWindows Procedure• Every window has an associate Every window has an associate

procedure for handling messages.procedure for handling messages.• Windows procedures have the Windows procedures have the

following format:following format:

– LRESULT CALLBACK ProcName (HWND LRESULT CALLBACK ProcName (HWND hwnd, UINT msg, WPARAM wp, LPARAM hwnd, UINT msg, WPARAM wp, LPARAM lp);lp);

Window Procedure ExampleWindow Procedure Example

• Here is an example procedure:Here is an example procedure:LRESULT CALLBACK WndProc (HWND hwnd, UINT msg, WPARAM wp, LPARAM lp){

switch (msg){case WM_DESTROY:

PostQuitMessage (0);return 0;

}

return DefWindowProc (hwnd, msg, wp, lp);}

Window Procedure Window Procedure ParametersParameters• The procedure has 4 parameters:The procedure has 4 parameters:

– HWND hwndHWND hwnd – handle to the window receiving – handle to the window receiving the message. This is useful when multiple the message. This is useful when multiple windows share the same procedure. You can windows share the same procedure. You can think of it sort of like a think of it sort of like a thisthis pointer. pointer.

– UINT msgUINT msg – The message. – The message.– WPARAM wp WPARAM wp andand LPARAM lp LPARAM lp – These are – These are

parameters to the message and may parameters to the message and may represent different things.represent different things.

Window Procedure ReturnWindow Procedure Return

• When the procedure completely When the procedure completely handles a message, it should return handles a message, it should return 00..

• The procedure should let the OS The procedure should let the OS handle all other messages by calling handle all other messages by calling DefWindowProcDefWindowProc and then return its and then return its result.result.

WM_DESTROY and WM_DESTROY and WM_QUITWM_QUIT

• Initially we only want to handle one Initially we only want to handle one message: message: WM_DESTROYWM_DESTROY..– This message is called when a window is This message is called when a window is

closed.closed.– Note that just because the window is Note that just because the window is

destroyed, the program will still run.destroyed, the program will still run.– To end the program we send To end the program we send WM_QUITWM_QUIT by by

calling calling PostQuitMessagePostQuitMessage..

WM_*WM_*• Other messages you may want to handle Other messages you may want to handle

include:include:– WM_CREATEWM_CREATE– WM_SIZEWM_SIZE– WM_LBUTTONDOWNWM_LBUTTONDOWN– WM_LBUTTONUPWM_LBUTTONUP– WM_MOUSEMOVEWM_MOUSEMOVE– WM_KEYDOWNWM_KEYDOWN– WM_KEYUPWM_KEYUP– WM_PAINTWM_PAINT

• Check MSDN literature for information Check MSDN literature for information about these messages: about these messages: http://msdn.microsoft.com/

http://msdn.microsoft.com/

Direct XDirect X• The window has to obtain a handle to a The window has to obtain a handle to a

Direct 3D and swap chain to render Direct 3D and swap chain to render graphics objects in the windows object graphics objects in the windows object created.created.

• In DirectX 11 this is based on the In DirectX 11 this is based on the Programmable Pipeline.Programmable Pipeline.

19 Apr 202319 Apr 2023 4545

• Shader: custom program segment Shader: custom program segment inserted into inserted into graphics pipelinegraphics pipeline and and executed on GPUexecuted on GPU– replaces replaces specific partsspecific parts of fixed-function pipeline of fixed-function pipeline

• comprises instructions that operate solely on: comprises instructions that operate solely on: – vertexvertex data ( data (vertex shadervertex shader))– or or pixelpixel data ( data (pixel shaderpixel shader))– geometric primitivegeometric primitive data ( data (geometry shadergeometry shader))

Introduction to PPLIntroduction to PPL

19 Apr 202319 Apr 2023 4747

• Shaders offer code development Shaders offer code development flexibility flexibility and and processingprocessing speedspeed: : – flexibility: shader implements flexibility: shader implements customised customised

vertex, polygon, or pixel processingvertex, polygon, or pixel processing – speed: shader runs on speed: shader runs on fast specialised chipsfast specialised chips

• many current graphics cards support shadersmany current graphics cards support shaders– programmable hardware pipelineprogrammable hardware pipeline on most modern on most modern

GPUsGPUs

• Shaders often used for better Shaders often used for better visual visual qualityquality in in real timereal time

Introduction to PPLIntroduction to PPL

19 Apr 202319 Apr 2023 4848

• Feature of DirectX Graphics and OpenGL: Feature of DirectX Graphics and OpenGL: – support for support for programmable pipelineprogrammable pipeline::

• API provides API provides high-levelhigh-level or or assembly language assembly language interfaceinterface to to graphicsgraphics processing hardware processing hardware

• specialisedspecialised programming programming languagelanguage for for vertex vertex and pixel processingand pixel processing

– developer can program special code for real-time developer can program special code for real-time graphics applications graphics applications

e.g. rich graphical effects such as simulation of cloth, e.g. rich graphical effects such as simulation of cloth, hair or furhair or fur

Programmable Graphics Programmable Graphics PipelinePipeline

19 Apr 202319 Apr 2023 5050

Layered architecture of fixed-function graphics system Layered architecture of fixed-function graphics system

D3DD3DD3DXD3DX

Device driver interfaceDevice driver interface

ApplicationApplication

HALHAL

HardwareHardware

Programmable Graphics Programmable Graphics HardwareHardware

DirectX GraphicsDirectX Graphics

19 Apr 202319 Apr 2023 5151

Graphics processing unit (GPU)Graphics processing unit (GPU)• Special purpose microprocessor(s)Special purpose microprocessor(s)

– multiplemultiple programmable processing units for programmable processing units for concurrent processingconcurrent processing on single chip on single chip• optimised for graphics processing, e.g. optimised for graphics processing, e.g. vectorvector / /

matrixmatrix manipulations manipulations

– used in used in computer graphicscomputer graphics (accelerator) card (accelerator) card• for manipulation and display of graphics datafor manipulation and display of graphics data

– can perform can perform some tasks of pipelinesome tasks of pipeline evolving hardware: evolving hardware: capability improvingcapability improving with time with time


19 Apr 202319 Apr 2023 5252

ATI Radeon HD ATI Radeon HD 5670 (supports 5670 (supports

DirectX 11)DirectX 11)

Source: Don Woligroski, The Radeon HD 5670 Architecture, Tom's Hardware, January 14, 2010 (http://www.tomshardware.com/reviews/radeon-hd-5670,2533-2.html)


19 Apr 202319 Apr 2023 5454

ApplicationVertex

ProcessorFragmentProcessor

Assem

bly &R

asterization

Frame buffer

Operations

Fram

e buffer

GPUCPU

Textures

GPU programming modelGPU programming model (Source: Bill Mark, “NVIDIA Programmable Graphics Technology”, Source: Bill Mark, “NVIDIA Programmable Graphics Technology”,

SIGGRAPH 2002 course)SIGGRAPH 2002 course)


19 Apr 202319 Apr 2023 5555

DirectX 10 graphics pipelineDirectX 10 graphics pipeline((see http://msdn.microsoft.com/en-us/library/windows/desktop/bb205123(v=vs.85).aspxsee http://msdn.microsoft.com/en-us/library/windows/desktop/bb205123(v=vs.85).aspx) )


19 Apr 202319 Apr 2023 6464

• Shader programming requires Shader programming requires languagelanguage specialised specialised for graphics for graphics – needs graphics-specific needs graphics-specific data typesdata types (e.g. vector, matrix) (e.g. vector, matrix)

and and operationsoperations (e.g. dot product, matrix multiply) (e.g. dot product, matrix multiply)– language language often tied to specific graphics APIoften tied to specific graphics API

• Actual Actual executableexecutable shader code is shader code is machine codemachine code – but high-level language can be compiled to machine but high-level language can be compiled to machine

code code • compilerscompilers available available

Shader LanguagesShader Languages

19 Apr 202319 Apr 2023 6767

HLSLHLSL• Sample Sample intrinsicintrinsic datadata typestypes::

– vectorvector types: contain from 1 to 4 components types: contain from 1 to 4 components (arranged as single row or column), e.g.(arranged as single row or column), e.g.• int1int1: vector containing 1 int; : vector containing 1 int; float3float3: vector containing 3 : vector containing 3

floats; floats; double4double4: vector containing 4 doubles : vector containing 4 doubles

– matrixmatrix types: contain rows and columns of same- types: contain rows and columns of same-type data, e.g.type data, e.g.• int2x1int2x1: integer matrix with 2 rows and 1 column; : integer matrix with 2 rows and 1 column;

float4x4float4x4: float matrix with 4 rows and 4 columns: float matrix with 4 rows and 4 columns


19 Apr 202319 Apr 2023 6868

HLSLHLSL• Sample Sample intrinsicintrinsic functionsfunctions::

– dotdot((aa, , bb): returns dot product of vectors ): returns dot product of vectors aa and and bb– mulmul((AA,, B B): returns product of matrices ): returns product of matrices AA and and BB– reflectreflect((ii,, n n): returns reflection vector): returns reflection vector given given

incident ray directionincident ray direction i i and surface normal and surface normal nn– roundround(x): returns x rounded to nearest integer(x): returns x rounded to nearest integer


19 Apr 202319 Apr 2023 6969

Sample code: simple HLSL shadersSample code: simple HLSL shaders

Example (from DirectX Graphics documentation Example (from DirectX Graphics documentation (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/HLSL_Shaders.asp))(http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/HLSL_Shaders.asp))

/* Vertex shader /* Vertex shader Functionality: Functionality: • transforms vertex position from object space to projection spacetransforms vertex position from object space to projection space• assigns per-vertex colour from vertex input */assigns per-vertex colour from vertex input */// Note: ‘: POSITION’ and ‘: COLOR0’ are semantics that tell compiler usage of variable// Note: ‘: POSITION’ and ‘: COLOR0’ are semantics that tell compiler usage of variablefloat4x4 mWorldViewProj; float4x4 mWorldViewProj; // World * View * Projection transformation // World * View * Projection transformation

// (global variable set by application)// (global variable set by application)// shader output structure// shader output structurestruct VS_OUTPUTstruct VS_OUTPUT{{ float4 Position : POSITION; // vertex position float4 Position : POSITION; // vertex position float4 Diffuse : COLOR0; // vertex diffuse colorfloat4 Diffuse : COLOR0; // vertex diffuse color};};


19 Apr 202319 Apr 2023 7070

Sample code: simple HLSL shadersSample code: simple HLSL shaders

Example (ctd.)Example (ctd.)

VS_OUTPUT Vertex_Shader_Transform( in float4 vPosition : POSITION, in float4 vColor : VS_OUTPUT Vertex_Shader_Transform( in float4 vPosition : POSITION, in float4 vColor : COLOR0)COLOR0)

{{ VS_OUTPUT Output;VS_OUTPUT Output;

// Transform the vertex into projection space. // Transform the vertex into projection space. Output.Position = mul( vPosition, mWorldViewProj );Output.Position = mul( vPosition, mWorldViewProj ); // Set the diffuse color for output // Set the diffuse color for output Output.Diffuse = vColor;Output.Diffuse = vColor; return Output;return Output;}}


19 Apr 202319 Apr 2023 7171

Sample code: simple HLSL shadersSample code: simple HLSL shadersExample (ctd.)Example (ctd.)

/* Pixel shader *//* Pixel shader */Functionality: copies interpolated vertex colour data, and outputs pixel colourFunctionality: copies interpolated vertex colour data, and outputs pixel colour

float4 Pixel_Shader( VS_OUTPUT in ) : COLOR0float4 Pixel_Shader( VS_OUTPUT in ) : COLOR0{ { float4 color = in. Diffuse;float4 color = in. Diffuse; // copy input colour// copy input colour

/* float4 color = in.Color; *//* float4 color = in.Color; */ // note error in original example code// note error in original example code

return color;return color;}}


Input parameter

Output semantic

19 Apr 202319 Apr 2023 7272

Suggested ReadingSuggested Reading

• http://msdn.microsoft.com/en-us/library/ee423531.aspx http://msdn.microsoft.com/en-us/library/ee423531.aspx

(Microsoft DirectX (Microsoft DirectX Graphics Documentation) Graphics Documentation)

• http://developer.nvidia.com/page/cg_main.html (Cg http://developer.nvidia.com/page/cg_main.html (Cg

homepage) homepage)

19 Apr 202319 Apr 2023 7373

• Programmable hardware pipeline on GPUProgrammable hardware pipeline on GPU– concurrent processing on single chipconcurrent processing on single chip

• Shaders offer code development flexibility and Shaders offer code development flexibility and processing speedprocessing speed

• Shader: program segment inserted into Shader: program segment inserted into graphics pipeline and executed on GPUgraphics pipeline and executed on GPU– replaces section of fixed-function pipeline replaces section of fixed-function pipeline

• vertex shader, pixel shader, geometry shadervertex shader, pixel shader, geometry shader

• Variety of shader languagesVariety of shader languages

SummarySummary

Documents

Further Programming for 3D applications CE00849-6 Introduction to Further Programming for 3D application Bob Hobbs Faculty of Computing, Engineering and