Computer Graphics - Input & Interaction - Hanyang University Jong-Il Park

Computer GraphicsComputer Graphics- Input & Interaction- Input & Interaction--

Hanyang University

Jong-Il Park

Division of Electrical and Computer Engineering, Hanyang University

ObjectivesObjectives

Introduce the basic input devices Physical Devices Logical Devices Input Modes

Event-driven input Double buffering for smooth animations Programming event input with GLUT

Virtual reality: interaction + reality


Project SketchpadProject Sketchpad Ivan Sutherland (MIT 1963) established the basic

interactive paradigm that characterizes interactive computer graphics: User sees an object on the display User points to (picks) the object with an input device

(light pen, mouse, trackball) Object changes (moves, rotates, morphs) Repeat


Graphical InputGraphical Input Devices can be described either by

Physical properties Mouse Keyboard Trackball

Logical properties What is returned to program via API

• A position• An object identifier

Modes How and when input is obtained

Request or event


Physical DevicesPhysical Devices

mouse trackballlight pen

data tablet joy stick space ball


Incremental (Relative) DevicesIncremental (Relative) Devices Devices such as the data tablet return a position

directly to the operating system Devices such as the mouse, trackball, and joy stick

return incremental inputs (or velocities) to the operating system Must integrate these inputs to obtain an absolute

position Rotation of cylinders in mouse Roll of trackball Difficult to obtain absolute position Can get variable sensitivity


Logical DevicesLogical Devices Consider the C and C++ code

C++: cin >> x; C: scanf (“%d”, &x);

What is the input device? Can’t tell from the code Could be keyboard, file, output from another program

The code provides logical input A number (an int) is returned to the program

regardless of the physical device


Input ModesInput Modes

Input devices contain a trigger which can be used to send a signal to the operating system Button on mouse Pressing or releasing a key

When triggered, input devices return information (their measure) to the system Mouse returns position information Keyboard returns ASCII code


Request ModeRequest Mode

Input provided to program only when user triggers the device

Typical of keyboard input Can erase (backspace), edit, correct until enter (return)

key (the trigger) is depressed Eg. scanf()


Event ModeEvent Mode

Most systems have more than one input device, each of which can be triggered at an arbitrary time by a user

Each trigger generates an event whose measure is put in an event queue which can be examined by the user program


Event TypesEvent Types

Window: resize, expose, iconify Mouse: click one or more buttons Motion: move mouse Keyboard: press or release a key Idle: nonevent

Define what should be done if no other event is in queue


CallbacksCallbacks

Programming interface for event-driven input Define a callback function for each type of event the

graphics system recognizes This user-supplied function is executed when the

event occurs

GLUT example: glutMouseFunc(mymouse)


GLUT callbacksGLUT callbacks

GLUT recognizes a subset of the events recognized by any particular window system (Windows, X, Macintosh) glutDisplayFunc glutMouseFunc glutReshapeFunc glutKeyboardFunc glutIdleFunc glutMotionFunc, glutPassiveMotionFunc


GLUT Event LoopGLUT Event Loop

Recall that the last line in main.c for a program using GLUT must beglutMainLoop();

which puts the program in an infinite event loop In each pass through the event loop, GLUT

looks at the events in the queue for each event in the queue, GLUT executes the

appropriate callback function if one is defined if no callback is defined for the event, the event is

ignored


The display callbackThe display callback The display callback is executed whenever

GLUT determines that the window should be refreshed, for example When the window is first opened When the window is reshaped When a window is exposed When the user program decides it wants to change the

display In main.c

glutDisplayFunc(mydisplay) identifies the function to be executed

Every GLUT program must have a display callback


Posting redisplaysPosting redisplays

Many events may invoke the display callback function Can lead to multiple executions of the display callback on

a single pass through the event loop We can avoid this problem by instead using

glutPostRedisplay();

which sets a flag. GLUT checks to see if the flag is set at the end of

the event loop If set then the display callback function is

executed


Animating a DisplayAnimating a Display

When we redraw the display through the display callback, we usually start by clearing the window glClear()

then draw the altered display Problem: the drawing of information in the

frame buffer is decoupled from the display of its contents Graphics systems use dual ported memory

Hence we can see partially drawn display See the program single_double.c for an example

with a rotating cube


Double BufferingDouble Buffering

Instead of one color buffer, we use two Front Buffer: one that is displayed but not written to Back Buffer: one that is written to but not displayed

Program then requests a double buffer in main.c glutInitDisplayMode(GL_RGB | GL_DOUBLE) At the end of the display callback buffers are swapped

void mydisplay(){

glClear(GL_COLOR_BUFFER_BIT|….)./* draw graphics here */.

glutSwapBuffers()}


Using the idle callbackUsing the idle callback

The idle callback is executed whenever there are no events in the event queue glutIdleFunc(myidle) Useful for animations

void myidle() {/* change something */

t += dtglutPostRedisplay();

}

Void mydisplay() {glClear();

/* draw something that depends on t */glutSwapBuffers();

}


Using globalsUsing globals

The form of all GLUT callbacks is fixed void mydisplay() void mymouse(GLint button, GLint state, GLint x, GLint y)

Must use globals to pass information to callbacks

float t; /*global */

void mydisplay(){/* draw something that depends on t}


Virtual Reality?Virtual Reality?

Head-mounted Display (HMD)



BOOM (Binocular Omni-Orientation Monitor)



CAVE (Cave Automatic Virtual Environment)



Shared Virtual Environment


Virtual RealityVirtual Reality

Virtual 실제로는 존재하지 않지만 , 본질적으로 존재하는

것과 동등한 효과를 가지는 것 Reality 실제 일어난 일 , 현실감

Virtual Reality 실제로는 존재하지 않지만 , 실제로 존재하는 것과

동등한 사실이나 사건


Def. VRDef. VR

What is Virtual Reality? “A high-end user interface that involves real-time

simulation and interaction through multiple sensorial channels.” (vision, sound, touch, smell, taste)


VRVR 의 의 3 3 요소요소 Presence

사실적 reality, 몰입적 reality Interaction

조작적 reality Autonomy

일관성 , 자율성 . 행동의 reality

by D.Zelter (1992)


Three Three ““II””s of VRs of VR

Burdea, 1993.

Immersion

Interaction Imagination

I3


Similar concepts Similar concepts ((I)I)

Virtual reality (1989, J.Lanier, VPL) “ 계산기에 의해 합성된 인공적 세계”

Artificial reality(Mid-’70, M.Krueger) VIDEOPLACE: 실제처럼 느끼면서 컴퓨터가 만든

세계에 참여하여 상호작용하는 것 Cyberspace(1983, William Gibson)

“a single artificial reality that could be experienced simultaneously by thousands of people worldwide”


Similar concepts Similar concepts ((II)II)

Telepresence or Tele-existence 우주공간 , 심해 등 그대로는 체험하기 불가능한 곳의

상황을 현실감 넘치게 제시 현실의 복제라는 점에서 VR 과 차별화됨

Augmented reality - virtual objects in real world

Augmented virtuality - real objects in virtual world

Mixed reality 기타 : synthetic reality, virtual environment,

virtual world, artificial life


Augmented RealityAugmented Reality

(ARTEMIS, U.Toronto)


Augmented VirtualityAugmented Virtuality

(www.cyberworldcorp.com)


MR IncludesMR Includes……

virtual

AV

AR

real


Reality vs. VirtualityReality vs. Virtuality

Reality = Unmodelled Virtuality = Modelled

Reality-Virtuality ContinuumReality-Virtuality Continuum

RealEnvironment

VirtualEnvironment

Extent of World Knowledge ContinuumExtent of World Knowledge Continuum

WorldUnmodelled

World CompletelyModelled


History of VR(I)History of VR(I)

Sensorama Simulator (Morton Heilig, 1962) head-mounted television( “ , 1960)

Head-Mounted Display(Ivan Sutherland, 1966) Scene generator(Evans and Sutherland)

simple scene about 200-400 polygons at 20 scenes/sec (1973)

METAPLAY(1970) ,VIDEOPLACE(1974, Myron Krueger) Flight simulators etc. by military(70s to early 80s)


Sensorama SimulatorSensorama Simulator

Heilig, US Patent #3,050,870, 1962

Division of Electrical and Computer Engineering, Hanyang UniversityIvan Sutherland’s HMD (1966+)


Videoplace by M.KruegerVideoplace by M.Krueger


History of VR(II)History of VR(II) VIVED(Virtual Visual Environment Display) project (NASA)

LCD(Liquid Crystal Display)-based HMD (1981) Polhemus noncontact tracker – user’s head tracking Sensing glove (Fisher, 1985)

VIVED -> VIEW(Virtual Interface Environment Workstation) 3D virtual sound source (Fisher and Wenzel, 1988) image wireframe rendering with flat shaded surface


History of VR(III)History of VR(III) VPL, 1st commercial company selling VR products

(1987) DataGloves EyePhones - 1st HMD

“Vision”, 1st integrated commercial VR workstation (Division Ltd., UK, 1991)

“WorldToolKit”, 1st VR development software (Sense8, 1992)

Mixed Reality (ATR, 1994) Plenoptic modeling and image-based rendering

(McMillan and Bishop, 1995) Virtualized Reality (CMU, 1996)


The VPL DataGlove (1987)

Division of Electrical and Computer Engineering, Hanyang UniversityNASA VIEW system (1989)


VRML(Virtual Reality Modeling Language) VRML 1.0 (1995) VRML 2.0 (1996), VRML97: dynamic scene animation

Currently Java 3D API, VRTP(VR Transfer Protocol) … Mixed reality applications

History of VR(IV)History of VR(IV)


Recent Technologies in VR (I)Recent Technologies in VR (I)

Computer vision and computer graphics Image-based rendering Large scale visualization

mosaic Robust geometric registration

Motion/depth-keying Occlusion Image segmentation


Recent Technologies in VR (II)Recent Technologies in VR (II)

Devices See-through HMD (ST-HMD)

Optical, Video ST-HMD Wearable equipments New tracking sensors for outdoor


Recent Technologies in VR (III)Recent Technologies in VR (III)

New interfaces Gesture recognition Tangible interface Haptic devices Hybrid interfaces


Applications of VR (I) Applications of VR (I)

Mobile Outdoor navigation with wearable equipments Military training, sightseeing

Collaborative Simultaneous and interactive multi-users Shared virtual environment

Commercial Broadcasting: character service Entertainment Film industry


Applications of VR (II) Applications of VR (II)

Various simulator Flight, car driving Military training Building, factory, City modeling

Medical application Human body 3D modeling (CT, MR) Tele-surgery

Limitations of future VR Technology, interface, social acceptance


Relation of VR, CG, and CVRelation of VR, CG, and CV

VR


Major Issues in MRMajor Issues in MR

Accurate registration between virtual and real world Realistic display(vision, sound, touch, smell, taste) All should be in REAL-TIME

Analysis Time(TA) Synthesis Time(TS) Feedback Time(= TA + TS)

Should be small enough Should not annoy human’s feeling of reality


Critical Enabling TechnologyCritical Enabling Technology

Accurate and fast registration between RW and VW !!!

In more detail, a good tracker Fast Light and small Low power-consumption Accurate Immune to interference


Where Are We Heading?Where Are We Heading?

CG/CVComputers

HCI

MechanicsSensors

PsychologyHuman science

Art

??

Documents

Computer Graphics - Input & Interaction - Hanyang University Jong-Il Park