MPEG-4 Toward Solid Representation

MPEG-4 Toward Solid Representation

Alain Mignot and Pierre Garneau IEEE Trans. on Circuits and Systems for Video Tech., Vol. 14, NO. 7, JULY 2004, pp. 967-974.

Presented by: Reza Aghaee Multimedia Course(CMPT820) Simon Fraser University March.2005

2

Agenda

Introduction Future of 3D Standards Solid Representation: MPEG-4’s

Answer to 3D Challenge Implications of Rendering

Mechanisms Impacts on Applications Conclusion

3

Introduction

Traditional 3D Image Creation1. Tessellation

– Models are created of individual objects using link points that are made into a number of individual polygons

2. Geometry– Polygons are transformed in various

ways and lighting effects are applied

3. Rendering– Transformed images are rendered into

objects with very fine details

4

Introduction (cntd.)

Performance is the number of polygons

processed per second.

MPEG-4 is introducing a new and

different approach.

This method can be used in various 3D

applications. (Games/CAD/CAM/CAE)

5

Future of 3D Standards

3D Technology is at a Turning Point– 10 years old OpenGL is reaching its

limits.– It’s confirmed by user-programmable

parts of the pipeline.– This limited programming capability, in

the form of vertex and pixel shaders, appears to be the easiest solution.

6

Future of 3D Standards

Shader Programs May Not Be the Solution– It is actually a step backward, in the

opposite direction of the beneficial standardization process.

– Graphic APIs display 3D scenes on almost any device.

– Drivers take care of hardware differences other than performance.

7

Future of 3D Standards

Shader Programs May Not Be the Solution (cntd.)– A software driver can not simulate

most shader programs.– Developers should provide different

versions for different graphic boards.– Shader Programs do not bring any

additional information to the rendering process concerning the scene itself.

8

Future of 3D Standards

Historical Considerations– Higher performance was often

achieved by including parts of pipeline into dedicated hardware.

– First, later stages of the pipeline were performed in hardware and front portion in software driver.

– 3D technology should extend its domain on functionalities performed by application.

9

Future of 3D Standards

Limitations of the 3D Pipeline Common to OpenGL and DirectX– Algorithms designed to create the

illusion of depth and continuity.– 3D rendering is a set of techniques that

have nothing to do with geometric objects.

– In real 3D space, objects may influence each other even if invisible.

10

Future of 3D Standards

Limitations of the 3D Pipeline Common to OpenGL and DirectX (cntd.)– Two parallel data structures needed

• One for describing the scene ( objects with geometric attributes)

• One for the set of polygons (each object in its position relative to observer)

– First structure for managing interactions– Second structure only for rendering

11

Future of 3D Standards

Traditional 3D Game Production

12

Future of 3D Standards

Traditional CAD rendering

13

Projected Evolution of 3D standards

Data structure of geometric objects and their attributes is common in most applications.

Most applications share a common framework based on geometrical and physical properties of objects.

By including this framework in the rendering engine the whole process is well improved.

14

Solid Representation in MPEG-4

MPEG-4 introduces the purely mathematical definition of shapes.

These shapes are based on algebraic shapes combined with arithmetic shapes.

These shapes are completely independent of rendering process.

These functionalities are referred to as “Solid Representation”.

15

Solid Representation in MPEG-4

In MPEG-4, solid representation functionalities deal with:– Reducing the size of files to be transmitted or

shared by transmitting constructive commands rather than results.

– Increasing the geometrical precision of rendered objects by managing a polynomial representation of volumes.

– Manipulating complex objects combined by solid operations.

16

Solid Representation in MPEG-4

Solid Primitives– Any solid object has a

3-D form defined by a skin, which delimits the inside from the outside.

– If equation of surface is unknown the volume will be divided into simpler pieces until they have known forms.

– These primitives will then be assembled in order to constitute the original complex shape.

– Figure 3 is a complex shape made of a set of solid primitives.

Figure 3

17

Solid Representation in MPEG-4

There are two approaches to define a surface:– A Parametric Equation giving spatial

coordinates of each surface point.– Implicit Equation of algebraic surfaces.

Implicit Equation of a Sphere– (Px – Cx ) 2 + ( Py – Cy ) 2 + (Pz - Cz) 2 - R2 = 0

Eq=0 means point is on the surface,for Eq<0 point is inside and for Eq>0 point is outside the volume.

18

Solid Representation in MPEG-4 A quadratic equation (second-degree equation)

allows to define the entire quadrics family:

An equation of the fourth degree allows to define the quartics family:

19

Solid Representation in MPEG-4

Implicit second-degree equation defining quadrics is:

where for each point coordinates (X0…X3) the result is:

F(X0,…,X3) < 0 ; F(X0,…,X3) = 0 ; F(X0,…,X3) > 0

whether the point is inside, on the surface or outside.

0239328317306

225

2142032

12101200

XcXXcXXcXXcXc

XXcXXcXcXXcXc

20

Solid Representation in MPEG-4

The point coordinates are made homogeneous by adding X3 .

A surface of fourth-degree will have 35 coefficients.

Unbound surfaces should be bounded to be processed and displayed.

Cylinder equation is an example of unbound volumes.

21

Solid Representation in MPEG-4

In MPEG-4 the coefficients of the quadric (second-degree implicit surface) may be defined by six geometric control points.

P0,P1 : 2 contact points on the quadric.P2,P3 : 2 poles of the construction

tetrahedron.P4,P5 : 2 passing points of the quadric.

Each point is defined using homogeneouscoordinates allowing the point to be sent to infinity (affine geometry)

22

Solid Representation in MPEG-4

In MPEG-4 two Geometry nodes implement algebraic surfaces:

1. Implicit Node:Defines the surface by the coefficients of the

polynomial.

2. Quadric Node:Defines the surface by the six control points

explained in the previous slide.

23

Solid Representation in MPEG-4

“Arithmetic of Forms” is a logical modeling system for solid objects.

Description of a solid object takes the form of a solid tree made up of operators and operands.

Operands are primitives or more complex solid objects.

Operators are mostly union, intersection and logical subtraction.

24

Solid Representation in MPEG-4

Each basic geometric primitive splits space into three regions – external, boundary, and internal – coded by integers 0, 1, 2.

Constructive Solid Geometry (CGS) tree and corresponding model 3D

This ternary coding of space tells us the density of every point in space.

25

Solid Representation in MPEG-4

There are three basic sets of operators 1. General Arithmetic Operators on Densities:

– For instance, addition, multiplication, and difference of densities of two volumes.

– Below is multiplication of two forms F0 and F1.

26

Solid Representation in MPEG-4

Three basic Sets of Operators (cntd.)2. Arithmetic Operators with Ternary Logic:

– They use ternary logic only.– Examples are union and intersection.– Below is ternary intersection of F0 and F1.

27

Solid Representation in MPEG-4

Three basic Sets of Operators (cntd.)3. Densities Filtering

– In MPEG-4 a set of test functions is applied on the root of the solid tree to filter densities while keeping the filtering inside the tree.

– Examples can be “Equality Filter” (F0==F1)

Results of Solid Operations

28

Implications of Rendering Mechanisms

If the volumes must be displayed very precisely some techniques can be used to render the output.

If one accepts less geometric precision, it is possible to consider tessellation of implicit surfaces before or after applying solid operation and rendering.

29

Implications of Rendering Mechanisms

The functionalities of solid representation in MPEG-4 including Implicit, Quadric and SoldRep Nodes are completely independent from the rendering method.

All the solid operators are independent from the rendering process too.

30

Impacts on Applications

Compactness– Suitable for sophisticated web applications

including online gaming on small-constrained devices mobile phones and PDAs.

– Complete model of a historical castle with a level of detail from roof frame to door openers takes 50 Kb.

31

Impacts on Applications

The following example is a model solely based on SolidRep geometry.

Complete Solid Model of Leihorra villaDetails of the Leihorra villa’s model

32

Impacts on Applications

Exact Geometry– The exact geometry is preserved up to the

decoder.– It allows very precise scientific applications as

CAD/CAM or simulations.– Below is the simulation of the Canadian

Space arm from Canadian Space Agency.

33

Impacts on Applications

Embedded Topology and 3D Properties– There is a wrap-up of the topology and 3D

solid properties (e.g. constituent matter and physical properties)

– Local manipulation, exploration of the model and very accurate collision detection is allowed.

– The picture is an inside view of the villa as the result of a solid Operation.

34

Impacts on Applications

There are many games claiming that the player can destroy almost any object to find other hidden spaces.

This model can help implement these models without very heavy programming techniques.

Complete Original ModelModel cut with a laser beam

35

Conclusion

Polygonal pipelines can’t present the future of 3D standards.

Solid representation can transfer very complex models in their most exact geometry and in a very compact way.

Even without a dedicated hardware current CPUs are powerful enough to provide real-time processing.

Solid representation is now a part of the MPEG-4 part-16.

New APIs will be developed including object description to accelerate solid representation.