CASE-BASED INTERACTIVE MULTIMEDIA SYSTEMSFOR RHEOLOGICAL SCIENCE

I. Deliyannis, J. HarveyComputer Science Department, University of Wales Swansea

Singleton Park, SA2 8PP UK{csdeli@swan.ac.uk, csjon@swan.ac.uk}

M. F. WebsterInstitute of Non-Newtonian Fluid Mechanics,

Computer Science Department, University of Wales SwanseaSingleton Park, SA2 8PP UK{m.f.webster@swan.ac.uk}

ABSTRACT

Multimedia environments are utilised to construct presentation systems, introducing some advanced interactive features.Some aspects of this research involve the organisation, presentation and interaction of complex industrial case-study data,arising from Computational Fluid Dynamic simulations and experimental trials (multimedia streams and static instances).Also, the development of rheological courseware is addressed. Modular interface constructs are employed to facilitaterapid and sound system-development. Object-oriented practices are deployed, based on an underlying graph structure.The multimedia nature of the implementation promotes interaction with synchronised animated flow-visualisation data(Motion-blur, static plots animated through settings, solid-modelling animations). This, in turn, enhances theunderstanding of the underlying data. The resulting implementation can be ported to a variety of computer-platforms, orstreamed over Internet connections without compromise in quality or interactivity. Such flexibility of distribution rendersthese systems ideal for publishing of scientific content between virtual research communities and industrialists, renderinge-Learning and e-Research widely accessible in a media-rich interactive form. It is shown how distinct individualmultimedia implementations are constructed and utilised, through a range of industrially-based and educational case-studies. In addition, the semantic linking of content is discussed and it is shown how this may be achieved through suchmerged (linked) multimedia systems (externally, locally - single computer, or over Internet communication channels),aiding presentation and detailed data interrogation.

KEYWORDS

Multimedia, Interaction-graphs, CFD, Experimentation, Industrial applications

1. INTRODUCTION

Four distinct multimedia systems (MMS) are chosen (Contraction-Flows [4,5], Dough-Kneading [7,8],History of Rheology [3,6] and Non-Newtonian Fluids [2]), to demonstrate the power of multimediaenvironments (MME) for research/industrial and educational-content distribution across virtual scientificcommunities. All four systems are developed under a single programming environment1, which supportsmultimedia objects, scripting, remote data-access, and stream-synchronisation. Different data-orientedinterfaces, novel to each case-study, enable customised data-interaction that reflects data-properties andcharacteristics. The underlying graph-structure mirrors the interface-organisation and data-connectivity. Inthis manner, distinct organisation is demonstrated across each implementation. Table 1 summarises the typeand volume of information contained within each case-study, and is ordered (top-to-bottom) in terms ofimplementation-complexity [10] (directly relating to data-complexity).

1Macromedia Director 8.5, Macintosh, www.macromedia.com

Table 1. MMS classification and complexity.

Name of MMSIndustriallyrelated

Data-instancesDelivery method

(medium)Contraction-Flows

(CF-MMS)Yes

O(150) video streams,O(150) slides

Internet/CD

Dough-Kneading(DK-MMS)

YesO(50) video streams,

O(100) slidesInternet/CD

History ofRheology

NoO(10) video / animated

streams, O(70) slidesInternet/CD/DVD/

VCR-TapeNon-Newtonian

FluidsNo

4 main streamsInternet/CD/DVD/

VCR-TapeCommon delivery methods for MMS are the CD and Internet media. Integrated implementations are not

easy to achieve using propriety software, such as Microsoft PowerPoint (PPT). Typical reasons for this maybe attributed to the large data quantities involved, the inability to detect data-duplication, and the defaultlinear-access of proprietary software. Partitioning of the data, into separate thematic entities is the favouredresolution, commonly adopted within such conventional implementations.

In the present study, a novel feature is the use of Multi-menus [1] to enable navigation between relateddata instances within each MMS. These menus are a concrete realisation of the underlying graph structure(representing sub-graphs). They facilitate data access, interrogation, and through direct interaction,interpretation. More-complicated systems include navigational aids, one being the pre-determined mode ofinteraction, termed “cruise-control” (cc). This permits a chosen route to be adopted, as a tour through asample of the data, in a particular order. Voiceover (VO) streams are provided for individual frames alongthe cc-path. This renders the MMS meaningful to audiences of wide knowledge levels, including non-experts. It also engenders flexibility in use, as a single MM implementation may meet the needs of manypresentations, each of different emphasis, without the need to create individual audience-related versions. Afurther advantage is that within any presentation instance, one may at will, digress from the cc-path.Alternatively, one may resume the tour at any point along its path, the links within each frame remainingactive all the while. Navigation and system-functionality are specified using the Scientific InteractiveMultimedia Model (SIMM) [1].

Multi-level linking and interaction [10 ], may be introduced through these graph-based “multi-menu”constructs. This permits direct interfacing with underlying content-structures, illustrated through typicalexamples involving parameter-adjustment (DK-MMS, CF-MMS). Higher-order linking and interactionacross MMS-sections is facilitated via direct frame-linking. Content-connectivity over various abstractionlevels, and particularly linking between external MMS, has been addressed within our earlier work [11].There, we dealt with the construction of super/master-MMS structures. Such structures permit disparatecontent to be linked effectively over various media, whilst system-functionality is preserved.

2. INTERACTION WITH THE DATA

Each case-study has different requirements with respect to interaction and navigation. This is determinedby the data-relationships and the target-audience. For straightforward case-studies, advanced navigation isnot essential. System design is based on sequential story-telling, and an ordered linear-path through thecontent is normally adequate. If a higher-level of interaction is required, then additional links may beprogrammed into an appendix, for example, or other related sections within the MMS.

2.1 “Non-Newtonian Fluids”

The “Non-Newtonian Fluids” MMS2 is split into four main sections: “Introduction to Non-NewtonianFluids”; “Rheometry”; “Viscometry”; and “Other Non-Newtonian Effects” (Figure 1). Each of these sectionscontains a number of sub-sections, all depicted with a characteristic icon within the multimedia menu. Here,

2 Currently distributed under the banner of Institute of Non-Newtonian Fluid Mechanics, University of Wales, UK.

interaction can be represented via a directional, fully-connected, five-node graph. Such a basic structure isconsidered sufficient to meet the needs of this educational presentation.

Passing from one node to the next is a fully-automated procedure. The MMS, in play-mode (Figure 2),runs without further user-communication, once a stream has initially been selected. At the same time linkingto other sections is permitted. A pre-rendered, single-clip approach is adopted in this case, as the content isrich in audio/visual material and requires precise timing in presentation-mode. Such a consideration isdifficult to achieve precisely within a MME. This level of timing accuracy (to one twenty-fifth of a second)may be achieved when high-end computers are employed. Nevertheless, here, the main concern is delivery ofthe MMS over a variety of platforms and to various hardware specifications.

Figure 1. Non-Newtonian Fluids MMS, Main Menu. Figure 2. “Rheometry” section instance.

2.2 “History of Rheology”

Passing next to a slightly more demanding case-study, allows us to demonstrate how MMEs handle highcontent-volume with ease. In terms of interaction, the History of Rheology MMS, uses a similar organisationto the foregoing, with a main-menu of six options: “Introduction”; “Some Highlights”; “Controversies”;“Friends and Disputes”; “International Meetings” and “Lessons from History” (Figure 3). This approach isframe-based, originating from a lecture presentation, constructed around static slides [6]. The underlying VOis inserted within each slide and the user can navigate forwards, backwards, or via access to the main menu.

Figure 3. History of Rheology MMS, Main Menu. Figure 4. History of Rheology, “Introduction” section.

Figure 4 highlights these features in a typical MMS screen-shot, displaying static and animated data. Thisapproach enables editing, in modular-fashion per-slide. Transitions and sprite movement is implemented atthe MME level. This is a computationally intensive process at runtime. Therefore, only a limited number of

such features, are incorporated per multimedia frame. Oncemore, presentation mode is automated, and theMME is programmed to detect VO termination, so as to proceed to the next frame directly. The associatedgraph structure commences from a cyclic-form graph, and is extended with links, to and from the mainmenu-node.

2.3 “Dough-Kneading DK-MMS”

Moving towards non-linear content-organisation, one faces the requirement for direct non-linearinteraction. This industrially-based case-study introduces a plethora of information, with such demands.Simulation and experimental data, are taken comparatively and evaluated using the MMS. On-demandinteraction is the default mode. The main-menu provides a number of features, including links to a studyoverview, animated introductory clip, access to viscous and viscoelastic sections and “cruise-control” modebuttons. Voiceover is included per slide throughout the MMS (bottom-left, slider-bar, media-controller). Thesame initial frame displays images relevant to the industrial process, including a model mixer, two states ofkneading, and the final product (Figure 5).

Figure 5. Dough-Kneading, Main Menu. Figure 6. Two-stirrer, inelastic, vessel rotating, Z2 plane,50rpm.

Figure 7. One-stirrer, inelastic, vessel rotating, Z2 plane,50rpm.

Figure 8. Two-stirrer, inelastic, vessel rotating, Z2 plane,through speeds; animation and 3D mode.

The multi-menu utilised is unique to this case study. It maps the geometric steps in modelling complexity(Figures 6,7). Starting from bottom-left, and rising upwards, increases the modelling complexity in stirrerpositioning and number: single-stirrer concentric; single-stirrer eccentric; two-stirrers eccentric; and two-

stirrers with baffles. Each multi-menu offers four basic aspects of comparison: below, through speed; left,through vessel depth; top, through material type; right, through rotation-type. Arrows, for each case, coverfull ranges. Horizontal-menus (lower-left of screen-shot), enable selection of mixer orientation for fully-filledor part-filled cases, relating to industrial settings for bread (vertical) and biscuit (horizontal) mixing. When aparticular stirrer-complexity setting is selected, the multi-menu displays satellite iconised options that relateto information presented on screen. The same can be used to switch between settings. Red indicates currentselection, black possible selection, and dimmed unavailable options. The multi-menu provides the “variablepriority” feature, that ensures the MMS retains the currently-selected option, (when, say, a fresh geometryselection is made). This aids visual comparison upon change in setting. A characteristic example is shownacross Figures 6 and 7, that represents two states of the MMS, for geometry switch between two-stirrers toone-stirrer eccentric instances, respectively. The information visualised is detailed in field variables (2Dprojections of streamlines, top-left; pressure, top-right; shear-rate, bottom-left; rate of work-done, bottom-right). Such a mode of advanced interaction enables effortless data evaluation, through visual comparison andfield adjustment in switch of setting. The static variable fields displayed are labelled, both on-screen and bypalette reference (bottom-right of screen). Views in 3D of this data may be accessed through thecorresponding button-icon.

The multimedia interface provides synchronisation options for animations. This enables tight-synchronisation and concurrent presentation of multiple data streams. Typically, Figure 8 contains acombined, synchronised presentation of four animated clips. Here, one is conveying change of mixing speed,from 12.5 to 100 revolutions per minute, within each animation clip. The fields represent Motion-blur (top-left), pressure (top-right), experimental laser flow visualisation (bottom-left), and 3D extension-rate (bottom-right).

The underlying structure used within this case-study is a multiply-connected graph. Commencing with atree, and multiple geometry options at the top node, lower-level additional links are added across branches,that relate experimental and simulation results. In addition, this structure includes sub-graphs of varioustypes: fully-connected (dense) for slide-sorters, or of cyclic-form for cruise-control.

Data organisation is optimised for the current case-study, allowing hierarchical access. So, for example,to access results for say the pressure-field, of a two-stirrer and vertical fully-filled case, the related optionsmust be selected from the appropriate multi-menu: second from top, left of screen-shot in Figure 6. Depthand speed comparatives would display such field data.

2.4 “Contraction Flows CF-MMS”

In contrast to DK-MMS, the introduction of content where pre-determined hierarchical access may belimiting is examined. The Contraction Flow case-study [4] is the most complicated considered, thus far, interms of interaction and underlying graph structure. An investigation tool-set emerges, based on simulationdata alone. Reference is made at the entry window, to a historical review sub-section (slide-sorter mode), andan animated short introductory sequence (flyer). Two connected, dual, graph menus are presented within themain-menu. One offers choice over model-fluid type, the other over geometry-type. Both aspects may befound in many common industrial processes. In contrast to Dough-Kneading, base-data units are uniform oftype (animated Motion-blur and static plots).

Two modes of presentation style are adopted, dynamic and static. Again, a number of data-combinationsare considered. Possible combinations include: all five fluids for any particular geometry (Figure 11, “All-Fluids” icon); all four geometries for any of the five fluids (“All-Geometries” icon); any valid two-fluidcombination (arrows); and any single-fluid for a single-geometry option (fluid/model icons). For the latterinstance, the space below the Motion-blur image (animation clip) is utilised to display additional relatedstatic information. These options are all available from the multi-menu instances shown in Figure 9. Interfaceinstructions, to aid user-selection, are provided below each menu. At lower system-levels, rheology and otherstatic results are accessible through iconised menus, which adjust dynamically, according to geometry andmodel selection (Figure 10). This is a two-stage process, where, if a more detailed view of static data isrequired, further slide-icon selection actions a zoomed, slide-sorter mode, departing from the animation-view.Selection is indicated by red colour and/or a bounding-box, about each slide icon.

Figure 9. Contraction Flows, Main Menu. Figure 10. Single dynamic geometry/fluid mode, withstatic material properties displayed.

Figure 11. Five-model combination for a single geometryselection.

Figure 12. Comparative, dynamic two-geometry/single-fluid Motion-blur-mode.

Figure 13. Comparative static two-geometry/single-fluid mode; streamline-data.

The programming approach to construct the multi-menu structure is modular. Each group of options (asdescribed above) has been programmed separately, and then superimposed onto the menu. This approachenables component re-use. System development is simplified when object-oriented techniques are employed.For example, each multi-menu component, when copied, preserves its links, icons and attributes. Icons canbe adjusted globally, with a single replacement edit. Links can be re-programmed using general conditionalstatements, identifying where and when to inter-link the MMS, as certain states are encountered (justified byuser-selection and current-data).

Motion-blur (MB) [9] visualisation is used to represent dynamic fluid-flow states, in a space-fillingmanner, covering a range of elasticity settings (Weissenberg Number, We). Use of MB gives an animatedgraphic-feel for fluid flow, but may not provide precise localised flow-representation (Figure 12). Thesimultaneous availability of streamline data (Figure 13) addresses this shortcoming, accessed through menuselection, via the y option-button, within top-level model- or geometry-menus (graphs).

Other modes of data-interaction may also be combined for base-level data-types. Three images aredisplayed bottom-screen, each representing a streamline plot, adopted at a particular We setting. This contextcombines a slide-sorter mode (as found in PPT build-mode), which utilises selectable iconised slides, hereover a brief slide-set. Upon selection, the corresponding slide is zoomed, centre-screen. De-selection isactioned by navigational progression, either within the slide-sorter, or to alternative multi-menu nodes.

Precise animation timing enables simultaneous display of a variety of non-uniformly constructedanimation clips. This enables automatic visual flow-variable comparison in a dynamic manner, utilisinganimation over material parameter incrementation (different fluids/flows at each animation increment). Useof this feature significantly reduces system development time (re-rendering of animations together forsynchronisation, becomes redundant), and thereby, enhances MMS usability. This is especially true, whenstreams are gathered from a variety of research sources and are not synchronised. Replication of thisimplementation feature, under proprietary software, would require rendering data-streams into a single,combined stream to ensure precise timing. Here, re-timings have been enforced, to match individualanimation frame-rates.

3. MMS CROSS-LINKING AND DELIVERY

To this point, frame-linking has been performed as standard, using the underlying graph structure tospecify data-relationships. The example of the Contraction-Flow static presentation, accessible from themain-menu of the case-study, demonstrates how different presentation-styles can be merged effectively. At ahigher organisational level, there is clearly merit in linking two or more individual MMS together. In thismanner, content with relevant context may be linked directly to a specific frame of interest, even fordistributed MMS [11]. This may arise when related content is to be accessed, and to avoid duplication, whencopying content from one MMS to another. Linking may be implemented at a higher level (MMS to MMS),ensuring data merges, without replication and may be achieved in a number of different ways. One approachis to merge presentations together under the same organisation environment. This is a time-consumingprocess, as each MMS must be individually merged within a single super-MMS file-structure. Practicalimpediments may be introduced in terms of file-space, and development time and effort, required to completethis procedure. Even after completion, the complexity of the new super-MMS may require a high-endcomputer to handle the vast amount of data involved. Utilisation of this technique is advisable only for smallmultimedia entities, to avoid system-overload.

An alternative strategy is to individually access the required MME, using either hyperlinks programmedin HTML, or batch files and shortcuts within the operating system. This is an efficient approach, but one notwithstanding its drawbacks. The MMS is accessed at the top-level and further user-interaction is required toreach particular items of data sought. This method is appropriate when a series of MMS are to be accessedsequentially.

A third method involves MMS-linking internally, employing a scripting language provided by the MME.MMS-connectivity is similar to HTML-type linking, with the added advantage that direct links can beprogrammed to specific frames within the target MMS. In this case, appropriate links, and a frame-basedHTML structure would be programmed, (where the menu-frame consists of an MMS designed to call otherMMS, on-demand). These links would appear in other frames. After such a fashion, it is feasible to integrate

this design under a common interface. An example of this implementation has been realised using twoframes, and is actively demonstrated at the Institute of Non-Newtonian Fluid Dynamics (INNFM) web-page[12]. Here, the left-frame (menu) reacts to user-choice and queries the underlying “Microsoft Access”database of multimedia objects, to retrieve data-fields or links to data, that are then displayed, centre-frame.The data of the frame are automatically compressed and transferred, using SHOCKWAVE streamingtechnology, reducing download time still further. One may export Java versions and set the image-compression to JPEG (trading quality for faster transfer rates), or to maximum compression without loss ofquality. This is particularly useful when mobile devices are to be used for data access, with limited visualcapabilities and bandwidth. One may take, as an example, the Nokia Communicator 9210 mobile telephone,with Internet access and SHOCKWAVE capabilities, that supports 4096 colours, but does not require full16.7M colour image-transfer.

The power of MM implementation is demonstrated when various delivery modes, through a range ofoperating systems, are demanded within a limited time-frame. The remit is a method of delivery. This mustbe consistent across various platforms, and utilise stream-compression to tackle content-delivery efficiently,transcending network bottlenecks. For example, various presentation versions may be exported in HTMLformat (static export-PowerPoint to HTML). In this manner, loss of functionality is introduced, aspresentations are downgraded to static slide-format. A further option is to stream the whole presentationfile(s) through the network using an Internet browser (again, PowerPoint file made available through HTML-link) or other file-transfer mechanisms. This will replicate functionality, but requires full-data content to bedownloaded, before presentation commences. The favoured option, proposed here, is to generate a client-application, which can be loaded locally to the user’s computer. This client-application would communicatewith the server, to access the data from the underlying multimedia-database. As data-streams are requestedfrom the server, they are transferred automatically, on-demand, in compressed format. Data instances, pre-delivered, may be re-used, providing a significant upgrade in speed and system-response. Again,SHOCKWAVE streaming technology is utilised to fulfil the above data-transfer and component re-userequirements. Certain data-instances may be pre-loaded, to reduce response times still further; in otherinstances, the same data may be unloaded (on-demand) to recover memory resource. Some characteristicexamples of such an implementation, where each has been exported using this automated process, areavailable over Internet connection and may be viewed at the INNFM web-site [12]. Such examples utilise thedatabase to dynamically request the content for each case-study (text, animation/image).

4. CONCLUSIONS

A data-presentation and development multimedia environment has been utilised to construct highly-interactive content-delivery mechanisms. Intelligent interaction and interrogation of the data is key. This hasbeen proposed and implemented at different levels: within a single presentation environment, and acrossmultiple instances. Desirable end-system characteristics include stream-compression, advanced-interaction,multi-platform support and multiple media delivery. A factor restricting the use of MME for the developmentof interactive multimedia presentations is the programmability aspects required to build a functional end-system. To aid in this direction, object-oriented techniques have been utilised to build component-basedinteractive (multi) menus. These techniques once invoked, may be re-used, reducing programming effort. Forexample, appealing aspects of these MMS include negation of data-duplication, and their design to handlelarge content volume. Overall, we believe that the advantages outcast the disadvantages, particularly as thevolume of content increases in size.

Furthermore, a major underlying theme throughout has been the development of graphs to invokeinteraction and guarantee link-integrity. The application of these technologies has been described, havingintroduced some of the MMS-capabilities, in terms of content-management, interaction, navigation anddeployment over various media. The end-systems have actively been used in over fifty instances worldwide[1], including conferences, industrial/academic presentations, and courses on Rheology, provoking bothcommendation and commercial interest. On-line delivery [12] is supported actively, often allowing MMS-updates to be viewed directly over Internet communication channels. Beyond intrinsic academic interest, andactive use of such MMSs in scientific research, one can envisage various other uses for these technologies,aiding various research and learning aspects of e-Society.

REFERENCES

Book1. Deliyannis I., 2002. Interactive Multi-Media Systems for Science and Rheology. Ph.D Thesis, University of Wales

Swansea.2. Barnes H. A., 2000. A Handbook of Elementary Rheology. Institute of Non-Newtonian Fluid Mechanics,

Aberystwyth.3. Tanner R. I. et al, 1998. Rheology: An Historical Perspective. Elsevier, Amsterdam.Journal4. Aboubacar M., et al, 2002. Highly Elastic Solutions for Oldroyd-B and Phan-Thien/Tanner Fluids with a Finite

Volume/Element Method: planar contraction flows; J. Non-Newt. Fluid Mech., 103, 65-103.5. K. Walters and M.F. Webster, 2001. The Distinctive CFD Challenges of Computational Rheology. To appear in

special issue Int. J. Num. Meth. Fluids (Keynote lecture: ECCOMAS Swansea 2001, available as CSR 6-2001).6. K. Walters, 1999. Lessons from History. Korea-Australia Rheology Journal, 11, 265-268.Conference paper or contributed volume7. D. Ding and M.F. Webster, 2000. Three-dimensional Numerical Simulation of Dough Kneading, "Rheology 2000 -

XIII Int. Congress on Rheology", eds. D.M. Binding, N.E. Hudson, J. Mewis, J-M. Piau, C.J.S. Petrie, P. Townsend,M.H. Wagner, and K.Walters, BSR, Glasgow, 2, 318-320.

8. M.F. Webster, D. Ding and K. S. Sujatha, 2001. The Modelling of Dough Mixing with Free Surfaces in Two andThree Dimensions, “Moving Boundaries VI – Computational Modelling of Free and Moving Boundary Problems”,Lemnos 2001, eds. B. Sarler and C.A. Brebbia, 101-107, WIT Press, Southampton.

9. N. L. Max and D. M. Lerner, 1985. A two-and-a-half-D motionblur algorithm. Computer Graphics (SIGGRAPH '85Proceedings), 19(3): 85-93.

10. I. Deliyannis and M. F. Webster, Multimedia environments and interactive scientific presentation of industrially-based data sets:“DSV-IS 2003, 10th International Workshop on the Design, Specification and Verification ofInteractive Systems, Funchal, Madeira Island, Portugal, June4-6. University of Wales, Swansea.

11. I. Deliyannis and M. F. Webster, 2002. WWW Delivery of Graph-based, Multi-level Multimedia Systems: Interactionover Scientific, Industrial and Educational data. IADIS 2002, Lisbon Portugal.

12. Institute of Non-Newtonian Fluid Dynamics (INNFM), 2003. WWW-site, University of Wales; Aberystwyth; Bangor;Swansea; Wales, UK, http://innfm.swan.ac.uk/.