Screencasting and Gesture DetectionSUNDARAM PATEL, BRENDEN SMERBECK

ELE 408

DR. LI

April 24, 2017

OVERVIEWMa cros: a single compute r instruction tha t sta nds for a sequence of opera tions Project Idea :

• Record ma cros on the DE1-SOC boa rd

• Pla y sequence ba ck on loca l boa rd

• Send sequence for remote pla yba ck

Reasons for Change

1. Touch Driver• Not included in embedded Linux IMG

2. X11 a nd Ima ge Processing• Bitma p(B & W) to Pixma p(Color)• Pixma p libra ry not na tive ly supported

Project Overview

1. Dra wing● Simila r to “Pa int”

2. Gesture Detection● Dra wn Sha pes: Line or Circle

3. Screenca sting● Mirror the dra wing in rea l-time● Support unica st, multica st

1

2

3

TECHNICAL OVERVIEWBlock Diagram

Python: Controller ● Applica tion Logic● Networking (Socke t Progra mming)● Progra m Ca lls (C)

C: Hardware Control ● Seven-Segment Displa y● Memory Registe rs

TECHNICAL OVERVIEWDrawing

1. Pygame• Native graphical component• Game looping similar to sockets

while(1) loop

2. X,Y coordinates• Coords tracked during mouseEvents

mousebuttondown, mousemotion• If we’re drawing

Add coordinates to list

TECHNICAL OVERVIEWGesture Detection

1. True Detection• Dot product for theta, cross-

product for direction• Machine learning

2. Method• Vertical and Horizontal Lines

Relative vs absolute change• Normal Lines

Average variance in slopeError

TECHNICAL OVERVIEW“Server Application”

TECHNICAL OVERVIEWUnicast (TCP)

1. Pickle: easy object transmission

• pickle.dumps(data)accepts most data typesconverts to bytestream for TCPcompatible with 2.x/3.x

2. Method• Create a “packet” object

contains coordinates and status (mousedown, mouseup, motion)

• Pickle and send • Unpickle and draw

Server handles logic

TECHNICAL OVERVIEWMulticast (UDP)

1. Basics• Multiple clients at once• Connectionless transmission

2. Method• “hard coded” vs realistic approach• Generally-speaking, the server is clueless• Very similar to TCP

TECHNICAL OVERVIEWMulticast (UDP)

Practical Application

Educa tion• La rge lecture ha lls• Complica ted ma te ria l

Multica st• Students view work in rea l-time• Gesture Detection

Question ma rks on confusing problemsCircle a nswers for quizzes, e tc

Constra ints• La tency due to number of peers• Wire less vs Wired

Questions

Tha nk You