Stacking-block Game Teaching Media Using

Leap Motion TechnologyOne of the popular board game in the world is stacking-block game. One of the forerunners is LEGO. In this world, stacking-block game not only played at home, but also played in schools, even at the college level. Because stacking-block game develop many things. But the problem is the lack of digital media or application that can improve skill in children. With the Leap Motion Technology, we offer a new media to improve children’s skill to play a game.

Table of contents

01

Introduction

02

Basic Theory

03

System Design and Implementation

o Backgroundo Problemo Research purpose

The basic theories that support problem solving and similar research has been done by previous researchers

Detail explanation about the methodology of this research and the implementation of system, like interface design, hand assets, Leap Motion SDK for Unity

2

04

Experiments

Result of the experiments and the analysis of this experiments’ results.

IntroductionI01

Benefit of playing a stacking-block game

A B CLiteracy Communication Fine-motoric

Today, reading becomes one of the most important skills in modern culture.

Lisnawati RuhaenaLecturer of Psychology FacultyGadjah Mada University

o High self-confidento Emotional social intelligenceo Familial relationshipo Develop awareness of the social

environment

Ability of communicating is one of the main pillars to build the character of children

Fine motor movements prioritize the development process of imagination and right brain working system in children.

The activity aims to increase the accuracy, speed, imagination, and memory of children.

4

Problem

Stacking-block….

Lack of digital media or application that can stimulate fine-motoric skill in children.

5

Related works

2006

2007

2009

2013

Sony Hirawan, Djoni Haryadi Setiabudi, Gregorius Setiabudhi, Rolly Intan.Making the Software Games Stacking Blocks “Mega Bloks” 3 Dimensions For Children.Petra Christian University.

Yeni Risyani, Ledis Manullang.Designing Software “Word Tetris” GameHigh School of Informatics and Computers Management.

Rizkcy Ardiansyah, Nana Ramadijanti, Fernando Ardilla.Control of Tetris Game Using Hand Movement.Electronics Engineering Polytechnic Institute of Surabaya.

Heena Purkait, Miller Fawaz, Rojer Kavin Ram, Karthikeyan Subramanian.Leap Motion Controlled Robotic Arm.Wayne State of University.

o 3D game stacking-blocko In addition to fun, player

also get intellectually grinding.

o Delphi, OpenGL, and OpenAI

o “Word Tetris” Gameo To construct a wordo The purpose is to

entertainment

o To facilitate player playing a game without joystick or keyboard

o Using camerao HaarClassifier method,

C/C++, and OpenCV

6

InteractiveCreative

Skeletal Tracking

Research purposes

INFORMATIONDetailsTo develop an application that can improve literacy skills, communication, and motoric sensor by using a new technology. The technology used is a Leap Motion Technology.

o Motion captureo Hand model in real time according

to the movement of hand humano Identification of right or left

handednesso Reporting of grip factors

indication of each finger bone

o Moving the block by hando Two-way communication

o Using a new technologyo Have a unique pattern to

matchingo Explore an imagination of

user

7

Basic TheoryB02

About skeletal tracking in Leap Motion

Skeletal tracking is a method for tracking or read information about the position of the human body and/or fingers“ ”

9

How Leap Motion can detect the human hand?

1 2

43

In hardware perspective, Leap Motion device have two camera and three LED infrared

The camera and LED infrared will tracking the infrared with wavelength about 850 nm

Leap Motion device has interactive area with 2 feet wide on each side, by 2 feet on each side

Leap Motion device has a visibility about 60 cm above the device.

10

Captured image of Leap Motion’s camera11

Image from Leap Motion’s camera

“Grab” hand gesture “Put” hand gesture

System Design

S03

Design system

Input

Process

Output

o Leap Motion’s interaction

o Blocks/cubes

o Grab gestureo Pattern matchingo Put gesture

o Sample 3D for object preview

o Pattern 3D for example of stacking-block

13

Input of system

Invisible field

Split of interaction space

Hover Zone Touch ZoneUsed for aiming Used for creating

touch events on the screen

14

Type 1 Type 2

Calibration process in Leap Motion

ArmWrist

Position

Palm Position

Bones

Tip Position

Thumb

Index Middle

Ring

Distal

Intermediate

Proximal

MetacarpalThe main feature of the Leap Motion device is a positional tracking or track the position of the

hand that is represented by nodes

15

Put some block

Pattern matching

Process of system

Grab some block

01Grab

02

Match

03Put

Index finger and thumb close together. In other words, the nodes on the index finger would have close proximity to the node on the thumb

The process of pattern matching in this research using a symmetrical block adaptation of the popular game Tetris

Index finger and thumb away from each other. In other words, the nodes on the index finger would have long distance to the node on the thumb

16

Gesture representation17

Put Gesture in real life

Grab Gesture in real life

Output of the system (Dynamic Nodes)19

The results (main menu)

20

The results (put & take stage)

21

The results (construction stage)

22

ExperimentsE04

System

< 45 lx

Effective in

> 45 lx

Too much noise

Callibration

Performance testing24

Light Intensity

100 % 97 %

Put gesture

Grab gesture

30 cm

Best distance

Distance

Sensitivity

User testing

User is a children within range of age 6 to 15 years old

9

User testing25

With external device Without external device0

1

2

3

4

5

6

7

8

9

10Knowledge About Digital Game

Ever Never

Very Bad10% Bad

10%

Very Good70%

Excellent10%

Graphics

Very BadBadNot BadVery GoodExcellent

User testing25

Bad30%

Very Good60%

Excellent10%

Mechanics

Very BadBadNot BadVery GoodExcellent

Very Good90%

Excellent10%

Gameplay

Very BadBadNot BadVery GoodExcellent

User testing25

Addict Prefer to play a Leap Motion game0

1

2

3

4

5

6

7

User Feedback

Yes No

Thank YouDiscussion Time

Cititation271. Universitas Gadjah Mada. Raih Doktor Usai Teliti Literasi Awal Anak Dalam Keluarga. Universitas Gadjah Mada. [Online] 3 November 2014. [Cited: 8 January 2016.]

https://ugm.ac.id/id/berita/9430-raih.doktor.usai.teliti.literasi.awal.anak.dalam.keluarga.2. Hirawan, Sony, et al. Pembuatan Perangkat Lunak Permainan Susun Balok Mega Bloks 3 Dimensi Untuk Anak-anak. s.l. : Universitas Kristen Petra, 2006.3. Risyani, Yeni and Manullang, Ledis. Perancangan Perangkat Lunak Game Word Tetris. s.l. : STMIIK IBBI.4. Ardiansyah, Rizcky, Ramadijanti, Nana and Ardilla, Fernando. Kendali Game Tetris Menggunakan Gerakan Tangan. Surabaya : Politeknik Elektronika Negeri

Surabaya, 2009.5. Nugraha, Raditya, Setiawardhana and Ramadijanti, Nana. Game TIcTacToe Dengan Gerakan Jari Menggunakan Metode Viola And Jones. Surabaya : Politeknik

Elektronika Negeri Surabaya, 2011.6. Purkait, Heena, et al. Leap Motion Controlled Robotic Arm. s.l. : Wayne State University, 2013.7. Rybczynski, Witold . Looking Around: A Journey Through Architecture. 2006.8. Kurniasih, Dedeh. Susun Balok Untuk Latih Kemampuan Otak Anak. Jakarta : Kompas, 2010.9. Weichert, Frank, et al. Analysis of the Accuracy and Robustness of the Leap Motion Controller. Dortmund : Department of Computer Science VII, Technical University

Dortmunt, 2013. PMC3690061.10. Phung, Johnny. Leap Motion Teardown in Macro HD. Youtube. [Online] 27 July 2013. [Cited: 9 January 2016.] https://www.youtube.com/watch?v=uF0NSUmxFYA.11. Terdiman, Daniel. Leap Motion: 3D Hands-free Motion Control, Unbound. CNET. 2012.12. Buckwald, Michael, et al. Introducing the Skeletal Tracking Model. Leap Motion Developer Portal. [Online] Leap Motion, Inc. [Cited: 9 January 2016.]

https://developer.leapmotion.com/documentation/cpp/devguide/Intro_Skeleton_API.html/.13. —. How Does the Leap Motion Controller Work? Blog Leap Motion. [Online] Leap Motion, Inc, 9 August 2014. [Cited: 9 January 2016.]

http://blog.leapmotion.com/hardware-to-software-how-does-the-leap-motion-controller-work/.14. Leap Motion Hand Tracking Gets More Realistic To Solve Real World Problems. Kosner, Anthony Wing. s.l. : Forbes, 2014.15. Alexa. Apa Sih Bedanya JPM, JRE, JDK, sama SDK? UNIKOM Blog. [Online] UNIKOM, 25 February 2011. [Cited: 9 January 2016.] http://10108665.blog.unikom.ac.id/apa-

sih-bedanya.146.16. Buckwald, Michael, et al. Unity Plugin Overview. Leap Motion Developer Portal. [Online] Leap Motion, Inc. [Cited: 9 January 2016.]

https://developer.leapmotion.com/documentation/unity/unity/Unity_Overview.html.17. —. System Architeture. Leap Motion Developer Portal. [Online] Leap Motion, Inc. [Cited: 9 January 2016.]

https://developer.leapmotion.com/documentation/unity/devguide/Leap_Architecture.html.18. Hanselman, Scott. Leap Motion: Amazing, Revolutionary, Useless. Scott Hanselman. [Online] 14 August 2013. [Cited: 10 January 2016.]

http://www.hanselman.com/blog/LeapMotionAmazingRevolutionaryUseless.aspx.19. Audy, Aurélien. Leap Motion: le test complet! Clubic. [Online] 29 July 2013. [Cited: 10 January 2016.] http://www.clubic.com/technologies-d-avenir/article-575170-1-

leap-motion-test.html.20. JCGM. International Vocabulary of Metrology - Basic and General Concepts and Associated Terms (VIM). 2008.21. Buckwald, Michael, et al. Hand Assets. Leap Motion Developer Portal. [Online] Leap Motion, Inc. [Cited: 11 January 2016.]

https://developer.leapmotion.com/documentation/unity/unity/Unity_Hand_Assets.html.