A camera-projector based remote sketching system

By

K. RAM SUNIL05A41A0453

LOYOLA INSITUTE OF TECHNOLOGY AND MANAGEMENTDHULIPALLA,SATTENAPALLI,(MANDAL),GUNTUR(D.T).

1

Abstract

Tele-Graffiti is a new invention in the field of Communication system. It is a remote

sketching system based on camera-projector. Systems combining video cameras and

LCD projectors are becoming more and more prevalent now-a-days.Tele-Graffiti is a

system allowing two or more users to communicate remotely with hand-drawn sketches.

A person writing at one site is captured using a video camera, transmitted to the other

site, and displayed there using an LCD projector. This is the way we can use this

Communication system. Such a system has a variety of applications in Teleconferencing

and Remote education. It also has a potential application in human-computer interaction,

using the paper as a combined display and input device. The advantage of our system

over other intelligent desktops and white-boards is that the users are free to move the

pieces of paper on which they are writing. In this paper, we describe the design,

implementation, and performance of Tele-Graffiti.

What's Tele-Graffiti?

Tele-Graffiti is a system allowing two or more users to communicate remotely with hand-

drawn sketches. A person writing at one site is captured using a video camera,

transmitted to the other site and displayed there using an LCD projector.

2

Such a system has a variety of applications in teleconferencing and remote education.

Example:

It also has potential applications in human-computer interaction, using the paper as a

combined display and input device.

Example:

There are some Demo movies introducing “Session Summarization and replay

application” featuring Tele-Graffiti's Hand-based User Interface

3

.

Design and implementation of the system

Tele-Graffiti is a system allowing two or more users to communicate remotely with hand-

drawn sketches. A person writing at one site is captured using a video camera,

transmitted to the other site and displayed there using an LCD projector.

4

Camera

Mirror

LCD projector

Paper

Such a system has a variety of applications in Tele-conferencing and Remote education.

It also has potential applications in human-computer interaction, using the paper as a

combined display and input device

Human computer interaction. A user provides input to a web page

In Tele-Graffiti, paper detection and tracking is based on real-time paper boundary

detection. In the applications mentioned above, this step needs to be robust to the

occlusions caused by the user’s hands and the distractions caused by printed or hand-

written material on the paper. As the paper may move during the time lag between image

projection and capture, we also need to disambiguate the paper from the projected image

at previous time-step. Finally, the two-fold ambiguity in the paper orientation has to be

resolved.

5

Web pages

Projected hand images

Inputs

Written comments

Paper detection

(1) The camera sees the paper and the projected image from the previous step.

(2) The orientation of the paper is ambiguous without the clipboard. To solve these

problems a paper detection algorithm had been developed that takes advantage of the

facts that

(a) The paper is bounded by four long edges roughly perpendicular to each other;

(b) we know the approximate location of the projected image from the previous time-step

and therefore its edges can be removed, and

(c) The asymmetry between the top and the bottom of the clipboard can be used to

resolve the ambiguity in the paper orientation.

Camera Simplified Tele-Graffiti system architecture.

Captured Desktop image Paper DetectionCompress &Transmit Corner Locations of Paper on Desktop Receive &

Compress

Corner Locations in Image to Project Texture mapping

Perspective Warped Image to Project

Projector

6

How It Works:

The system consists of a digital camera, a high resolution LCD projector to project the

received image, a mirror to reflect the project image onto a blank paper sheet, fixture to

hold these devices in position and a writing pad to fix the sheet of paper on the top of the

table at every participant’s station. We can write on a piece of paper with a pen. When we

impose our ideas or sketch on a paper, then our scribbling is captured by a camera and the

digitized image is transmitted to the receiving end through a data link. The image can be

transmitted to LAN or Internet. Data rate of more than 100 Mbps can be achieved

through the LAN. Up to 15 frames of image can be transmitted per second. If the users

were to communicate through the global network, delayed will be considerable. Data will

be transmitted as streaming video over the internet.

Uncompressed data streams at 2 to 3 frames per second can be transmitted over direct

Internet communications. The projector on other side handles the transmitted video. The

projector beams the image on recipient’s paper. The sender and recipient can now interact

as if they are both writing on the same piece of paper while sitting many miles apart.

When the system is turned off, the user sees only his marks on the paper.

Separating Final Viewed Images

We have an algorithm to separate the final viewed images into the two actually drawn

components (without imaging them directly). Two examples of this process are shown as

below

7

And

8

The Trick:

We can place the writing pad any where on the desk, rotate it and even pass around the

table without the loss of paper orientation to allow several people to write on it at each

side of the network. Inputs from both ends appear simultaneously at both ends. Now with

the help of tracking algorithms, the camera tracks the edges of the paper, so pen marks

maintain the same relative position on the paper at every side.

As we move the pad, the camera follows it and projector angle changes. The computer

ignores desktop clutter and works out a method on how to project what is read. The

image is projected sharply and exactly at the same position as it was at sender end. In

effect multiple users have the feeling of sharing the same piece of paper.

Advantages:

• As the inputs by the sender and the receiver appear simultaneously on the paper,

both of them can interact as if they are writing on the same piece of paper.

• Multiple users can participate and have the feeling of sharing the same piece of

paper.

• Architects can team up with experts abroad to work on designs simultaneously.

• Distance Education will be benefited as students and the teacher can interact in

their own handwritings and can get various doubts cleared.

• Telegraffiti can serve as a substitute to Internet Chat.

• Filling Web forms in our own handwriting.

Future Prospects:

The technology will be put to use commercially within two years. Researchers are trying

to add audio signals and face-tracking mechanism to develop a remote sketching

teleconferencing system. Efforts are also going on to shrink the size of the system to the

size of table lamp.

9

Conclusion

Many innovative reforms and technologies have developed in communication system.

Tele-Graffiti is one among them. There are many important advantages like Tele

conferencing and Remote education with this innovative communication system Tele-

Graffiti. Many researches are going on to develop most significant and advanced Remote

sketching Teleconferencing system. Many developmental steps are going on to reduce the

size of the system and to light weight the system. Finally the ultimate goal is to bring out

an advanced, splendid, innovative communication system into existence which would be

successful and most useful in the communication world.

Webliography

www.google.com

www.springerlink.com

www.ieeexplore.ieee.org

10