JSARToolKit / LiveChromaKey / LivePointers - Next gen of AR

http://www.kawa.net/http://www.kawa.net/ 11

JSARToolKit / LiveChromaKey

2009.05.27Yusuke Kawasaki

川﨑有亮 (kawanet)http://www.kawa.net/

Spark project 勉強会 #09


Yusuke Kawasaki（川﨑有亮）

• Perl monger– XML::TreePP, XML::FeedPP, ...

• JavaScript addict– Wiimote over HTTP, Gainer, ...

• ActionScript 3.0 newbie– Today!

• Media Technology Labs– Recruit Co., Ltd. in Japan

http://www.kawa.net/http://twitter.com/kawa0117


§§1 1 -- JSARToolKitJSARToolKit

JavaScript Augmented RealityJavaScript Augmented Reality


JSARToolKit

• JSAR（ジャバスクリプト拡張現実）– JavaScript Augmented Reality

• JSARToolKit– My first work using ActionScript 3.0– Powered by SWFObject! – A library to run AR by JavaScript– A proxy wrapper for a Flash app using FLARToolKit– Debuted at OSDC.TW 2009 in Taipei

• Genealogy– 2008.03 – NyARToolKit for Java - by A⻁@nyatla.jp– 2008.05 – FLARToolKit for ActionScript3 by Saqoosha– 2009.04 – JSARToolKit for JavaScript by Kawanet

• Saqoosha さん、本当にありがとうございます！！_o_


JSARToolKit Synopsis

It needs just several lines of JavaScript code to run AR.When a marker found, it calls onDetect callback function.

< s c r i p t t y p e = " t e x t / j a v a s c r i p t " s r c = " s w f o b j e c t . j s " > < / s c r i p t >< s c r i p t t y p e = " t e x t / j a v a s c r i p t " s r c = " j s a r . j s " > < / s c r i p t >< s c r i p t t y p e = " t e x t / j a v a s c r i p t " > < ! - -

v a r p a t s = [ ' j s a r l o g o . p a t ' ] ;v a r j s a r = n e w J S A R ( ' j s a r _ h e r e ' ) ;j s a r . d r a w M a r k e r R e c t = t r u e ;j s a r . o n D e t e c t e d = f u n c t i o n ( r e s u l t ) { . . . } ;j s a r . i n i t ( ) ;j s a r . s e t M a r k e r ( p a t s ) ;

- - > < / s c r i p t >< d i v i d = " j s a r _ h e r e " > < / d i v >


JSAR returns a JavaScript Object

{" d e t e c t e d " : t r u e ," m a r k e r s " : [{

" c o d e i d " : 0 ," d i r e c t i o n " : 2 ," c o n f i d e n c e " : 0 . 5 1 2 6 3 7 ," v e r t e x " : [{ " x " : 3 4 6 , " y " : 2 0 1 } ,{ " x " : 2 7 7 , " y " : 1 9 1 } ,{ " x " : 3 5 3 , " y " : 1 3 0 } ,{ " x " : 3 9 5 , " y " : 1 5 4 }

]} ,

] ,}


JSARToolKit Layers

Flash Layer jsar.swf

HTML Canvas Layer

HTML Div Layer


AiR Xiaolongbao（エア⼩籠包）

2009.04.19 @ Taipei (OSDC.TW)


§§2 2 -- LiveChromaKeyLiveChromaKey

BluescreenBluescreen--less augmented less augmented ININ realityreality


LiveChromaKey

• A new image synthesizing engine for AR– 2009.05 released by Kawanet (me!)

• Pure ActionScripe 3.0– No library dependencies– No JavaScript

• Only webcam needed– No blue back screen– No need to print

AR marker PDF


Step 1/3 - Stationary background

At first, connect you webcam and run LiveChromaKey.Never move until it recognizes stationary background.


Step 2/3 – Bluescreen on the fly

It generates bluescreen automatically.Live Video – Background = Bluescreen

－ ⇒


Step 3/3 - Transparent foreground

Get the foreground image as a translarent Sprite.Live Video – Bluescreen = Foreground

－ ⇒


LiveChromaKey Synopsis

LiveChromaKey provides four kinds of Sprites.

v a r c h r o m a k e y : L C K _ C o r e = n e w L C K _ C o r e ( ) ;c h r o m a k e y . i n i t ( ) ;

v a r s p L i v e : S p r i t e = c h r o m a k e y . g e t L i v e ( ) ;v a r s p B a c k : S p r i t e = c h r o m a k e y . g e t B a c k g r o u n d ( ) ;v a r s p M a s k : S p r i t e = c h r o m a k e y . g e t M a s k ( ) ;v a r s p F o r e : S p r i t e = c h r o m a k e y . g e t F o r e g r o u n d ( ) ;

t h i s . a d d C h i l d ( s p L i v e ) ;t h i s . a d d C h i l d ( s p B a c k ) ;t h i s . a d d C h i l d ( s p M a s k ) ;t h i s . a d d C h i l d ( s p F o r e ) ;


Example:Example: Travelling in EgyptTravelling in Egypt

egypt.swfegypt.swf


Sprites augmented IN reality

The key feature of LiveChromaKey is to insert Sprites augmented between background and foreground images.

Sprite


Example:Example: Slideshow in realitySlideshow in reality

minority.swfminority.swf


MMultiarmed ultiarmed ddeityeity (or (or 美川健⼀美川健⼀))

exile.swfexile.swf


ひとりひとりおそ松くんおそ松くん

exile.swfexile.swf


ひとりひとり ExileExile

exile.swfexile.swf


§§3 3 -- LivePointersLivePointers

Live Color Pointer Detection LibraryLive Color Pointer Detection Library


LivePointers

• Live Color Pointer Detection Library–will be released soon!

• Pure ActionScripe 3.0– No library dependencies– No JavaScript

• Webcam as a 3D human interface– No need to print AR marker PDF


Pointer = Pixel Cluster

• Ex.–Cluster #0–250 pixels–RGB: 25D985–H: 152–S: 0.83–V: 0.85


Labeled Pointer Detection

• Labeled pointer needs a guide color.

• ex. fingercap– 0x2197A9 (uint)– ￥105（キャン・ドゥ）

• Fingercap is definitelyimportant user interfacedevice of the future.


LivePointers（⽇本語のみスマソ）

• 最短距離法 (Nearest Neighbor Clustering) をベースにして、ActionScript 3.0 でリアルタイムに表⽰できる程度に単純化したクラスタ分析処理を⾏う。

• 画⾯内の単⾊のカラーマーカーを検出します。– ⽩・⿊・グレー・淡⾊は検出しません。– （S・V値が⾼い）くっきりした⾊が得意です。– 模様は検出しません。（⾊のみを認識する）– 通常は、画⾯内の全ての検出⽤ガイドとしてRGBで

⾊指定した「ラベル付マーカー」を利⽤する。

• 3次元⼊⼒のポインティングデバイスとして利⽤可能。


Now on the Spark!

JSARToolKithttp://www.libspark.org/svn/js/JSARToolKit/

LiveChromaKeyhttp://www.libspark.org/svn/as3/LiveChromaKey/

LivePointers... coming soon!


Thank you!

Yusuke Kawasaki川﨑有亮 (kawanet)

http://www.kawa.net/