Meme Media Architecture for the Reediting and Redistribution of Web Resources Yuzuru Tanaka and Kimihito Itoh Meme Media Laboratory Hokkaido University

Meme Media Architecturefor

the Reediting and Redistribution ofWeb Resources

Yuzuru Tanaka and Kimihito Itoh

Meme Media Laboratory

Hokkaido University

Current Web Technologies

Publicationinfrastructure

server

client

Browse + dowmload/upload

server

compound documentwith embedded toolsand services

Meme Media & Meme Pool

reediting

redistribution

meme pool

drag & drop

drag & drop

userA

userC

userBcompound document

not only MM contents, but also tools and services to make them interoperate with each other

Required Systems

• Meme Media for people– to externalize, and to re-edit memes

• Meme Pool for people – to publish, to reuse, and to redistribute memes

• Meme Market for people– to introduce business activities in a meme pool

• Meme Management for people– to organize and access memes

IntelligentPad as Meme Media

Compound Document

P2 P3

P1Containers and their Connections

P2 P3

P1

Extraction of a Web Page Portionas a View Document

Client Document (HTML)

HTML view

View-Definition Code

Retrieval Code

View-Editing Code

Description of the editing process

doc = getHTML(“http://www.abc.com/”,null)doc = getHTML(“http://www.abc.com/”,null)

view = doc　　　　　 .EXTRACT( editing location )　　　　　 .REMOVE( editing location );

view = doc　　　　　 .EXTRACT( editing location )　　　　　 .REMOVE( editing location );

Delete (Remove)Cut out (Extract)

Extracted

information

Description of view definition

Node Identification

/HTML[0]/BODY[0]/TABLE[0]/TR[1]/TD[1]

Node identifier 　：　 to identify any object in a document

[2]

<HTML>

<BODY>

<TR> <TR> <TR>

<TD><TD> <TD> <TD><TD><TD>

[1]

<B><B><B>

[12,969.05][1,300.98]

[TOPIX] [日経平均 ]

・・・

<HEAD>

・・

・・

DOM Tree

<TABLE>

[0]

/ node-name [ index ] / node-name [ index1- index2 ] / node-name [*] / node-name / text() / node-name / attr( attr-name )

Extraction of a Web Page Portion

HTMLviewPad

<HTML>

<BODY>

<TABLE>

<TR> <TR> <TR>

<TD><TD> <TD> <TD><TD><TD>

<B><B><B>

・・・

<HEAD>

・・

・・

CLIP(/HTML[0]/BODY[0]/TABLE[0]/TR[1])

http://weather.yahoo.co.jp/weather/

Editing code

URL

Wrapper Object(HTMLviewPad)

Extraction

Drag

Editing Operators

EXTRACT( node identifier )

REMOVE( node identifier )

INSERT( node identifier, document , relation)

2nd parameter ： document to insert3rd parameter ： where to insert < CHILD | BEFORE | AFTER | PARENT >

<BODY>

<HTML>

<BODY>

<HTML>

EXTRACT （●）

<BODY>

<HTML>

<BODY>

<HTML>

REMOVE （●）

<BODY>

<HTML>

doc

INSERT （● , doc , X ）

CHILD BEFOREAFTER PARENT

<BODY> <BODY> <BODY><BODY>

Code Description and Evaluation

Web Server

HTTP Access

Parse

r

EditingProcessor

Client Document

doc = getHTML(“http://www.abc.com/”,null)

view = doc .EXTRACT(“/HTML/BODY/…/TABLE[0]”) .REMOVE(“/TABLE[0]/TR[1]”);


view = doc .EXTRACT(“/HTML/BODY/…/TABLE[0]”) .REMOVE(“/TABLE[0]/TR[1]”);

EXTRACT “/HTML/BODY/…/TABLE[0]”⇒REMOVE “/TABLE[0]/TR[1]”⇒EXTRACT “/HTML/BODY/…/TABLE[0]”⇒REMOVE “/TABLE[0]/TR[1]”⇒

http://www.abc.com/http://www.abc.com/

HTML view

View-Definition Code

View Evaluator

HTML

Automatic Code Geneation

Move to parent Back to the child

Selected element name

Node selection tool

HTMLviewPad

getHTML(“http://www.abc.com/”,null)getHTML(“http://www.abc.com/”,null)

Step1 document selection

Step2 specification of a editing location node identification

Step3 execution of the editing

generation of a view editing code

…/TABLE[0]/TR[0]/TD[1]…/TABLE[0]/TR[0]/TD[1]

Editing Operation １ (Live-Cut)

HTMLviewtPad

HTMLviewPad

Drag Out


view = doc .EXTRACT(“/HTML/BODY/…/ 　　　　　　　　　 ;.EXTRACT(“/HTML/BODY/…/TABLE[0]”).EXTRACT(“/HTML/BODY/…/TABLE[0]”)

A copy of the code

Automatically generated code

Editing Operation 2

　　

　 EDIT 　　

CLIP

REMOVE

Menu Selection


view = doc　　　　　 .EXTRACT(“/HTML/BODY/…/TD[2]/…/TABLE[0]/”)　　　　　　　　　　　　　　　　　　　　　　　　　　　 ;　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　

.REMOVE(“/TABLE[0]/TR[1]/”).REMOVE(“/TABLE[0]/TR[1]/”)

Generated code

HTMLviewPad HTMLviewPad

extract & delete

Editing Operation 3

　　BEFORE

AFTER

CHILD

PARENT

Menu Selection( Insert Type)

Drop In

　　　　 doc = getHTML(“http://www.abc.com/”,null)A::doc = getHTML(“http://www.abc.com/”,null)

A::view = A::doc　　　　　 .EXTRACT(“/HTML/BODY/…/TD[1]/…/TABLE[0]/”)　　　　　 .REMOVE(“/TABLE[0]/TR[1]/”);

　　　　 view = doc 　　　　 .EXTRACT(“/HTML/BODY/…/TD[0]/…/TABLE[0]/”) 　　　　 .REMOVE(“/TABLE[0]/TR[1]/”) 　　　　 .INSERT(“/TABLE[0]”,A::view,AFTER);

A::doc = getHTML(“http://www.abc.com/”,null)


A::doc = getHTML(“http://www.abc.com/”,null)


　　　　　　　 .INSERT(“/TABLE[0]”,A::view,AFTER)　　　　　　　　　　 .INSERT(“/TABLE[0]”,A::view,AFTER)　　　

Combination of codes

HTMLviewPad HTMLviewPad

Generated code

insert

Data Mapping for Defining Slots

MAP( node-identifier , slot-name)

MappingEngine

RenderingEngine

View-Document

Event

View rendering

#MappingCode Mapping 　 DATA

Mapping-definition code

Mapping view information to slots

<BODY>

<HTML>

Mapping of node values

anchor object/HTML/…/a

Slot Mapping ：mechanism for reusing Web resources

Textual object

Wrapper (HTMLviewPad) Slot

12,969.0512,969.05

Numerical value 　/HTML/…/content

Table object

/HTML/…/table

Form object Button form 　 [ false ]

/HTML/…/form

Table data

Tag descriptor format

TOPIX 1,311.49日経平均 12969.05ジャスダック 57.94

○× 株式会社

Submit event [ http://xxx/xxx.cgi?a=○× ・・ ]

Yahoo JapanYahoo JapanCharacter data [ Yahoo Japan ]

OnClick event 　 [ http://www.yahoo.co.jp ]

PadPad

InputField form 　 [ ○×Inc. ]

Node Mapping Rules （１）

MAP( “/HTML/BODY/…/P/ txt()” ,” Value1”)

Value1::Text12,969.05

Numerical value type

MAP( “/HTML/BODY/…/TABLE[0]” ,“ Stock”)

Stock::Data

TOPIX 　， 1,311.49日経平均　， 12969.05ジャスダック　， 57.94

Node mapping rules

ViewportPad

<OUT><P>12,969.05</P>

<TABLE><TR><TD>TOPIX</TD><TD>1,311.49</TD></TR><TR><TD> 日経平均 </TD><TD>12,969.05</TD></TR> <TR><TD> ジャスダック </TD><TD>57.94</TD></TR> </TABLE>

Character string type (CSV)

<OUT>

HTML Source Slot name

Slot

Mapping value

(Slot value)

Mapping Type

Slot Type

Different mapping for different node types


reediting

redistribution

meme pool

drag & drop

drag & drop

userA

userC

userB

Node Mapping Rule (2)

MAP(“ /HTML/BODY/…/A[0] “,” Link1”)

Link1::Text

Next Page<OUT>

On jumping Link1::refURL

./next.html

Link1::jumpURL

http://www.abc.com/next.html

MAP(“ /HTML/BODY/…/FORM[0] “, “SearchForm”)

SearchForm::Input_text_keyword

映画<IN,OUT>Character string type

SearchForm ::submit

true<EventFire>

Boolean type

SearchForm ::request

http://www.abc.com/search?keyword= 映画<EventListener>

Character string type

Query issuing event

On issuing a query


<OUT>


<EventListener>Character string type

<A href = ./next.html>Next Page</A>

<FORM action=“./search”><INPUT Type=txt name=keyword ><INPUT Type=submit value=“ 検索”></FORM>


reediting

redistribution

meme pool

drag & drop

drag & drop

userA

userC

userB

Estimate

Live Document Framework

日経　 \12,841.76 ダウ　 $9791.09Reuse of live

knowledge on the Web

　 Integration of live knowledge resources from different web pages

\12,841.76\12,841.76 $9791.09

$9791.09

Extraction of live knowledge resources to make them work as pads

total in $

currency rate

total in yen

Nikkei average Dow Jones average

Explicit Slot Definition

reediting

redistribution

meme pool(2D & 3D)

drag & drop

drag & drop

userA

userC

userB

HTML and Web Applications

Internet

Webserver

<a>　日本語を入

力してくださ

い。 <b></b>　翻訳結果 <c></c></a>

人工知能①

HTML

<a>　日本語を入力してください。 <b> 人工知能 </b>　翻訳結果 <c>Artificial

Intelligence </c></a>

③② 「人工知能」

HTML

Web Browser and DOM

HTML ： text

<a>　日本語を入力してください。 <b></b>　翻訳結果 <c></c></a>

• HTML– Text file for a Web page

• DOM– Document 　 Object 　 Model– Internal tree representation of a Web page

a

b c日本

語を

入力

して

くだ

さい

翻訳

結果

DOM ： treeDisplay by a browser

Input a Japanese word.

translated English word

Input a Japanese word.

translated

English word

Wrapping a Web Application

人工

知能

a

b日本

語を

入力

して

くだ

さい

翻訳

結果

c

「人工知能」

Artifi

cial

Inte

lligence

Making some DOM nodes work as slots

IntelligentPad

Japanese slot

English slot

Result HTML

DOM

a[1]/b[1]

Identifier on the DOM ：　a[1]/c[1]

日英翻訳サービス

Internet

⇒Users can easily wrap any services to define pads.

Federation of more than one serviceover the Web

Two pads are pasted together to compose a new service.

Japanese EnglishHTTP

keyword URL

slot connectionInternet

HTTP

Japanese to English Translation Service

Search engine service

Japanese to English Translation pad

Search engine pad

An Example of Required Interoperabilityamong Bioinfomatic Web-Sevices

tggctggcaa tggccttgct gacctcgagc cgggcccacg tggggacctt tggagcacagcctacgatcc tggtgcaagg ccggtggatg cagaggccag tccatatacc acccaggcct gcgaggagcg tggtccccac ccatccagcc catatgtgca agtgcccttg acagagaggc

tggctggcaa tggccttgct gacctcgagc cgggcccacg tggggacctt tggagcacagcctacgatcc tggtgcaagg ccggtggatg cagaggccag tccatatacc acccaggcct gcgaggagcg tggtccccac ccatccagcc catatgtgca agtgcccttg acagagaggc

Motif search Result list

Homology search Result list Details of the first

Paper title Paper retrieval

Composition of an integrated tool

Paper abstract

GSCOPE

PDB+3D

LEXICON

intermediate

①distribution

②reediting

③redistribution

Reediting of Live-Documents and Redistribution of Edited Results as HTML Documents

Information provider

userCGIWeb server

Live Cut

Live Cut

IntelligentPad

HTMLXHTML

i-mode HTMLCustomized portfolio

Flattening: Redistribution of a composed pad as a Web page

reediting

redistribution

meme pool= the Web

drag & drop

drag & drop

userA

userC

userB

A JavaScript program to define slots in an HTML view

Use of a JavaScript program for an HTML-view composition

Alternative man-machine interface:C3W (Clipping, Connecting and Cloning for the Web)

CloningClipping

Connecting= B

Web browser composite pad

FAQ

• Web service?– Web service requires API on each server side, and

programming on client sides.– No need.– Furthermore, any Web service can be wrapped to work

as a pad, and hence can be pasted together with other pads.

• Copyright violation?– New consensus are now growing.– New technologies and systems like Creative Commons

for new ways of license management.

Our R&D Efforts since 1987

• Meme Media – IntelligentPad (2D, 1989), IntelligentBox (3D, 1995)– Meme media over the Web (2D, 2002)

• Meme Pool– Piazza (1998, 2000)– the Web as a meme pool (2002)

• Meme Market – request module & account module (1998)

• Meme Organization and Access– Topica (2000), Relational Annotation (2003)

• Application memes as intellectual assets (1989~ )

Recent Book on Meme Media Technologies（ John Wiley: IEEE Press, July 2003, 500 ページ）

To get our systems:

• http://www.pads.or.jp/english/instance/index.html

• http://ca.meme.hokudai.ac.jp

• For more information:– [email protected]

Some Other Topics on Meme Media Technologies

IntelligentBox as Meme Media(1995)

Component Boxes anda Composition with them

Unified Treatment of Animation and Tools

３ D

cDNA DB Visualization (1 day work)

DB Visualization Environment(Visual query components ）

Application to cDNA DB Visualization (10 min. work)

Integrated Visualizationof DB Records and Web Contents

UNESCO Web pageson World Heritage

Local DB records(GNP of countries)

DB+Web Content

Virtual Lab

IntelligentBox AVS

slot connection

AVS

#integer #string

#Geom

solver

InputFilter

OutputFilter

solver controller

Virtual Lab.

Virtual Lab.

From Multimedia to Meme Media

• Multimedia Contents– Multimedia document with embedded multimedia

contents

– Multimedia document architecture for the editing and distribution of multimedia documents

• Knowledge resources– Compound document with embedded multimedia

contents, tools, and services

– Meme media architectures for the editing and distribution of knowledge resources (, including functional composition)

Knowledge Media

Mark Stefik (1986)The Next Knowledge MediumThe AI Magazine, 7(1): 34-46

“In addition to the study of expert systems, we must study knowledge media for the exchange of knowledge.”

“Such knowledge media may work as memes.”

Node Identification

横浜3 - 1ヤクルト

中日9 - 5巨人

広島1 - 4阪神

横浜3 - 2中日

ヤクルト4 - 5広島

巨人1 - 2阪神/HTML[0]/…/TABLE[0]/TR[“巨人” :0]

/HTML[0]/…/TABLE[0]/TR[1]

更新

August 21st

　 node identification with indices

　 node identification with pattern matching

August 20th

[0]

[1]

[2]

[0]

[1]

[2]

/ node-name [ matching-pattern : index ]

Node identification with string matching pattern

Users’ Operation on a Web page

人工知能

人工

知能

a

b日本

語を

入力

して

くだ

さい

翻訳

結果

c

人工

知能

a

b日本

語を

入力

して

くだ

さい

翻訳

結果

c

「人工知能」

Artifi

cial

Inte

lligence

Update of the DOM

ArtificialIntelligen

ce

例：日英翻訳　　サービス

Internet