NURBS Streams

Frederick W. B. Li Rynson W. H. Lau Department of Computer Science, City University of Hong Kong, Hong Kong

{ borbor, rynson} @cs.cityu.edu. hk

Abstract

Deformable objects are important in modeling clothing, facial expression, animal characters and other soft-objects. However, they are rarely used in distributed virtual environ- ments. It is because both the transmission and the tessella- tion of deformable objects are too time consuming to run in real-time. Part ofthe problem can be solved by our ear- lier method on real-time rendering of deformable NURBS sugaces. In this paper, we present a technique to organize the data structures used to store the pre-computed polygon models and the deformation coeficients in supporting the rendering of deformable NURBS surfaces from hierarchi- cal form into a linear form, called NURBS streams. The NURBS streams will not only support progressive transmis- sion and rendering of deformable NURBS sugaces, but also simpliJj, sojiware and hardware implementation ofthe ren- dering method. To further speed up the rendering of these NURBS streams, we also introduce the idea of virtual ren- dering list to keep track of incremental node changes be- tween successive frames.

1 Introduction

Transmission of the object models is one of the hottest topic for distributed virtual environment systems [l]. De- spite the deformable objects can greatly enhance the real- ism and widen the applications of such systems, the object models used in most existing systems still tend to be non- deformable. A major problem is the high computational cost of tessellating deformable object models into polygon models for rendering as the objects deform. This may cause a great delay in rendering the deformable objects at the client. On the other hand, if we assign the tessellation task to the server, it will then generate extra network traffics due to the continuous transmission of the updated polygon mod- els to the client while the objects are deforming.

To solve this problem, we present a new linear data smc- ture, called NURBS streams, to support both efficient trans- mission and rendering of the deformable objects in dis- tributed environments. The surafce streams are developed as an extension of the real-time rendering of deformable

parametric free-form surfaces [2, 31. They allow progres- sive transmission and efficient rendering of deformable ob- jects modeled by parametric free-form surfaces. A NURBS stream organizes the data structures used to store the pre- computed polygon model, the deformation coefficients and the hierarchical surface from hierarchical form into a linear form. This linear data structure is encoded in such a way that partially transmitted models can be rendered and pro- gressively refined as more information is received. Hence, the client no longer needs to wait for the whole model to be transmitted before rendering and can thus provide a more immediate visual feedback to the user.

To allow efficient usage of the N U B S streams in a distributed environment, we make use of the servers to generate the NURBS stream for a deformable object, and the NURBS stream is then progressively transmitted to the client upon a request. This approach allocates the time con- suming NURBS streams generation process to the servers to make use of their processing power. It may also avoid the unnecessary task duplication at the clients for generating the NURBS streams for the same object. For instance. since the client is not responsible to both the generation of the sur- face streams and their construction methods. We can hence employ thin clients with limited processing power and sim- plified rendering engine in the distributed environment, to reduce both the cost of the hardware and the complexity of the software for the client machines.

The rest of paper is organized as follows. Section 2 in- troduces the NURBS streams. We will describe its internal data structure and show its packed format in supporting pro- gressive transmission. Section 3 describes the idea of pro- gressive rendering and introduces the virtual rendering list for optimizing the rendering performance of the NURBS streams. Section 4 presents some perfonnance results of our new method. Finally, Section 5 concludes the work pre- sented in this paper.

2 NURBS Streams

In our deformable NURBS rendering method, we maintain the polygon model of each deforming surface in a quadtree structure. The algorithms based on this pointer-based tree structure are most likely recursive in nature. In addition,

337 0-7695-1007-8/01 $10.000 2001 IEEE

the navigation methods associated with these algorithms are often restricted to preorder, inorder or postorder tree traver- sal. If an operation, such as the crack prevention process in our rendering method, requires information from neigh- bor nodes for the comparison of subdivision levels, an ad- ditional tree traversal operation will be needed to locate the neighbor node starting from the root node, unless ex- tra pointers are provided to link each node to its neighbor nodes. To overcome these limitations, we propose a new linear data structure, called NURBS streams, to maintain and transmit the polygon model. A NURBS stream is based on the linear quadtree structure proposed in [4] using the z- ordering indexing scheme [5]. It allows constant time nav- igation between any node pairs and supports progressive transmission. The simple linear structure of the NURBS streams also eases both software and hardware implemen- tation.

p:p"$ce L,s,

2.1 Indexing Scheme

Progressive Progressive Progressive Progressive RecordO Record, Record,, Record,

A linear quadtree is a pointerless scheme to store a generic quadtree in the form of a linear list of nodes. The quaduee nodes are ordered by both the z-ordering indexing scheme and their residing quadtree levels. Each node is assigned with a static and unique index regardless of its presence. For example, the index of the root node is "0' and a node residing at a deeper quadtree level has a greater index value. Given a particular nodep > 0, the node indices of its parent node and child nodes, and also its node level can be obtained by the following equations:

Parent node index:

Child node indices: 4p + i, i = 1.. .4 (2) Node level: LZog4 (3p + 1)J (3)

The node index of a neighbor node k is obtained by the following equations:

Westem neighbor: k = p - Ahd(n,) (4) Eastern neighbor: k = p + Ahd(n, + 1) (5)

Northern neighbor: k = p - Avd(n,) (6) Southem neighbor: k = p + Avd(n, + 1) (7)

wherep is the index of the current node. (n,, n,) is the 2D location of the node in the grid of nodes of level d. Ahd and Avd are the pre-computed lists, called distance vectors, that give the horizontal and vertical index differences for pairs of neighbornodes at level d, respectively. Ahd(s) and Awd(t) refer to the sth and tth elements of the horizontal and vertical distance vectors, respectively.

2.2 The Structure of a NURBS Stream

The main purpose of a NURBS stream is to store the quadtree nodes of the polygon model of a deforming NURES surface in a linear form for progressive transmis- sion. In order to represent a deformable NURBS surface by a single quadtree structure, we construct a hierarchical sur- face [61 for the NURBS surface. In the quadtree, each node contains four vertices and a set of deformation coefficients. The number of deformation coefficients depends on both the number of vertices in a node and the number of control points controlling the node. This number of control points is determined by the local modification property [7J of the NURBS surfaces. The vertices are used for rendering and the set of deformation coefficients are used in incremental polygon model updating.

Unlike the linear quadtree suggested by Balmelli et al. [5 ] , a NURBS stream does not consume extra spaces to hold empty nodes. This is very important since most quadtrees are likely unbalanced. Figure 1 shows the suuc- ture of a NURBS stream. It consists of a node presence list and a set of progressive records. The node presence list is an array of boolean values encoding the presence of nodes in the linear quadtree using Balmelli's index scheme. Each of the progressive records stores information of a sin- gle quadtree node. It consists a set of packed vertices and a set of deformation coeficients associated with the packed vertices. The packed vertices are the list of (z, y, z ) co- ordinates of the vertices in the quadtree node. The pro- gressive records are arranged in ascending order of node indices. When transmitting a NURBS stream to the client,

Order 01 Transmssion 1

Figure 1. The structure of a NLJRBS stream.

the server first send the node presence list to the client. The progressive records are then sent one by one to the client for rendering. Upon received each of the progressive records, the client may further refine the resolution of the polygon model of the deforming NURBS surface.

3 Rendering of the NURBS streams

3.1 Progressive Rendering

A NURBS stream is encoded in a way that the client may receive the polygon model of each deformable object pro-

338

Table 1. Iterative operations of progressive rendering with the corresponding conditions and actions.

gressively. In order to take advantage of this, the client should have the corresponding rendering method to support the progressive data. However, the original operations of our deformable NURBS rendering method, including the de Casteljau subdivision, incremental polygon model up- dating, resolution refinement, crack prevention and polygon model rendering, are recursive in nature. When polygon modcl is in the form of a quadtree, these operations often need to traverse the quadtree recursively from the root node to the appropriate nodes. Hence, instead of processing only the newly received nodes, they need to process most nodes in the partially received NURBS stream in each frame. This is very time consuming and does not utilize the progressive data from the NURBS streams.

In the new method, we have deveIoped a new se6 of it- crative operations to support the progressive rendering of the NURBS streams. The original recursive operations in the deformable NURBS rendering method in effect perform a level-by-level processing on the quadtree of the polygon model. ln the case of a NURBS stream, equation ( 3 ) shows that the level of a node is increascd with the index of the node. Hence, if we process the nodes of a NURBS stream in ascending order, we are implicitly performing the same level-by-level processing on the quadtree. Therefore, the progressive rendering of a NURBS stream can be carried out by a set of iterative operations with a generalized pseu- docode shown as follows: f o r e a c h ( n o d e n r e c e i v e d ) (

i f ( n - > c o n d i t i o n C h e c k ( G i v e n C o n d i r i o n . ~ ) == TXUE) n->performCorrespondingAction5(Given C o n d i t i o n s ) ;

I

Table 1 shows the non-recursive operations in our render- ing method together with their corresponding conditions for checking and the actions to perform. To implement the it- erative operations, we simply select a suitable condition I action pair from the table.

3.2 Virtual Rendering List

Since the operations in progressive rendering are non- recursive in nature, they can process the NURBS streams

Node Selecbon bst

Figure 2. The virtual rendering list.

efficiently and reduce the consumption of stack memory. However, in real situations, the polygon model of a deform- ing surface would change only slightly between consecutive frames. To future enhance the rendering performance, we may keep track of this incremental node changes to mini- mize the number of nodes need to be processed.

To do this, we may construct a list linking only those nodes for rendering in the current time frame. The list can either be implemented with a linked list linking all the nodes for rendering in the current time frame or an array to store the indices of these nodes.. However, in the case of a linked- li.st when, the resolution, of t h e polygon model changes, we need, to perEom. node. insert.ion or. deletion: toi generate the updated rendering Pst.. I,n the caqe of'a. no we need t o perform node insenionlandlsorti

Both) approaches consume extra processing ntradicts their original objective to improve

rendering performance. To solve this. problem. we use: a, virtira1~'rendering list in-

stead. The virtual rendering list is. constructed by two node indices and a navigatibmalgo~tRm:. First, instead of storing all the indices of the nodes used for rendering in the current time frame, we only store the minimum and maximum node indices, The relevant iterative operations then need to pro- cess nodes between these two nodes. When the resolution of the polygon model is changed, we update these two node indices accordingly. Second, due to the unbalanced nature of the underlying quadtree, some nodes between these two nodes may not exist. We therefore need to skip these nodes to avoid unnecessary checkings. This is done by a navi- gation algorithm. Figure 2 shows a quadtree segment of a polygon model corresponding to a deforming NURBS sur- face. Although the NURBS stream is in linear form, we show the quadtree segment in hierarchical form in Figure 2 for the sake of clarity. Assume that we are now processing the quadtree nodes sequentially from left to right at level n. If a node fulfills the given condition of the current operation, we apply the corresponding action on the node. Otherwise, we insert the index of its first child node in a node selection list. After finishing all nodes at level n, we move down to

339

level n + 1 and we process the quadtree nodes there. How- ever, instead of processing the nodes one by one, we extract the node indices from the node selection list and process the corresponding nodes only. The node selection list in effect helps us figure out and process only the nodes that exist at level n + 1 of the quadtree. This process runs repeatedly un- til we reach the leaf of the quadtree. Note that we only need to store the index of the first child node in the node selec- tion list, instead of storing the indices of all child nodes of the current node. This may reduce memory usage and pro- cessing overheads. Our assumption here is that if we need to process a child node, the other child nodes that belong to the same parent node also need to be processed. By carry- ing this virtual rendering list from one frame to the other, we effectively provide a rendering list for the iterative oper- ations and accelerate these operations by taking advantage of frame coherence. In addition, it does not require the use of extra pointers. node insertion, deletion and sorting pro- cesses.

in 1.897 milliseconds. A single progressive record is trans- mitted in 0.347 milliseconds. While receiving the NURBS stream, the polygon model of the deformable object at the client may be refined gradually. At the time of our experi- ments, the bandwidth available to us was about 2 Mbits. If we assume that the Intemet bandwidth is roughly 0.2 Mbits, the time needed to transmit the whole NURBS stream will be 1.44 seconds. However, since the information in the NURBS stream is arranged in progressive order, the client is expected to be able to visualize a coarse model of the deformable object within one or two seconds.

(with 409 progressive records) Node Presence Us1

0 053 ms 0 294 ms

Table 2. Record sizes and transmission time of different parts of the iVLJRB.5 stream.

4 Results and Discussions 5 Conclusion

We have implemented the new method in C++ with OpenGL and Openlnventor libraries. The server and the client modules communicate using TCPllP with the BSD Sockets Library. Figure 3 shows a human head model that we use in our experiments. It is a NURBS surface with

In this paper. we have presented the NURBS streams to support real-time rendering and progressive transmission of deformable NURBS surfaces. With this data structure, the client no longer needs to wait for the whole model to be transmitted before i t can visualize the deformable ob- jects. Besides supporting the rendering of the progressive data from the NURBS streams, the iterative operations in progressive rendering can both improve the rendering per- formance and reduce the consumption of stack memory at the client. Finally, we also introduce the virtual rendering list to speed up the rendering process by keeping track of incremental node changes between successive frames.

References Figure 3. Model used in our experiments.

400 control points. We apply knot insertion [7] to subdivide it into 400 BCzier surface patches and construct a hicrar- chical surface on these patches to form the base model for transmission. We run the experiments on a SGI Onyx’ ma- chine and a SGI Octane machine, each with one CPU acti- vated. They are physically connected to our university net- work through IOMbit Ethernet connections. The NURBS stream is transmitted in its native data types without com- pression. Table 2 shows the size and the transmission time of the NURBS stream. The total size of the NURBS stream is 122.76 Kbytes and is transmitted in 0.144 seconds. We also shows the breakdown of the size and the transmission time for the node presence list and each single progressive record. The node presence list is transmitted to the client

[11 M. Macedonia and M. Zyda, “A Tmononiy for Networked Virtual Environments,” IEEE Multimediu, vol. 4, no. 1, pp, 48-56, 1997.

[Z] E Li, R. Lau, and M. Green, “Interactive Rendering of Deforming NURBS Surfaces,” Proceedings of Eumgruphirs ’97, vol. 16, pp. 47- 56, September 1997.

[3] F. Li and R. Lau, “Incremental Polygonization of Deforming NURBS Surfaces,” Joumul of Gruphics Tools, vol. 4, no. 4, pp. 37-50, 1999.

[4] I. Gargantini, “An Effective Way to Represent Quadtrees,” Cvmmuni- cutions ofrhe ACM, vol. 25, pp. 905-910, December 1982.

[5] L. Balmelli, J. KovaEeviC, and M. Vetterli, ”Quadtrees for Embed- ded Surface Visualization: Constraints and Efficient Data Structures,” Pmceedings of IEEE lntemutionul Conference on I m g e Processing (ICIP)? vol. 2, pp. 487491, October 1999.

[6] F. Li and R. Lau, “Real-Time Rendering of Deformable Paranietric Free-Form Surfaces,” Pmreedings of ACM Symposium on Virtu01 Re- ulity Softwure und Technology, pp. 131-138, December 1999.

[7] L. Piegl and W. Tiller, The NURBS Book. Springer-Verlag, 1995.

340