Modeling Malaysian Public Opinion by Monitoring the Malaysian
BlogosphereBrian Ulicny
VIStology, Inc.Framingham, MA
IBlogs GoalsCluster Posts by Event, and Rank using new metrics for • Relevance• Timeliness (& Locality)• Specificity• CredibilityApplied to Malaysian Blogosphere as a Case StudyAFOSR funded; Distributed Intelligence Program
– Automated Ontologically-Based Link Analysis of International Web Logs for the Timely Discovery of Relevant and Credible Information
– Ends November, 2008.
4/2/08 VIStology, Inc. 2
Commercial State of Art
4/2/08 VIStology, Inc.
1. Technorati Blog Search 2. Organized by Date and
Authority.3. Authority = Global Link
Popularity.4. No way to filter blogs
by location currently.5. So must probe via
keywords.6. No aggregation of
topics or links locally.
3
4 VIStology, Inc.
The Malaysian BlogosphereFormer British colony.
English 2nd Most Common Language.Complex ethnic and religious social structure.Press tightly controlled
RSF: 124th of 169.But: Internet growth encouraged.Over 150K Malaysian profiles on Blogger.com.
310K profiles on Facebook.comBlogs have exposed alleged government corruption
and organized protests.Government has intimidated and arrested bloggers.
4/2/08
Malaysian SoPo Blogosphere
4/2/08 5VIStology, Inc.
Malaysia SoPo Blogs:385 blogs in crawl seed yields 220.3K urls, 4693 Sites
@ depth 4• ~2K Blogs
– 1,059 Blogspot Blogs, 801 Wordpress, 16 Typepad• 272 .org, 45 .gov.my • Of the Blogspot blogs (structured profiles)
– 659 Malaysia, 166 other, 234 unspecified• 122 Kuala Lumpur• 88 Selangor (State)• 31 Johor (State), 10 Subang Jaya, 11 Klang, 6 Johor Bahr
– 10 list Koran as favorite book; 4 Bible
4/2/08 VIStology, Inc. 6
Fragmentation of References:Diversity of Links for Bersih Rally Posts
4/2/08 VIStology, Inc. 7
Posting and Linking PatternsMalaysian SoPo Bloggers, Blogging Unfree Press
4/2/08 VIStology, Inc. 8
Cf. 50% of US “A-list”Blog posts contain a news link(Adamic, Glance. 2005)
Clustering by Event:Nov. 10, Kuala Lumpur, Malaysia, police
4/2/08 VIStology, Inc.
StarNST
CNN
BBC
9
Blog Post RelevanceAboutness Determined by Term Frequency
in Post Text plus Linked Text
4/2/08 VIStology, Inc.
IHT article Jeff Ooi PostLinks To
10
Blog Post SpecificitySpecificity Determined by # Unique NPs
in Post Text and Linked Text
4/2/08 VIStology, Inc.
IHT article (16 UNPs) Jeff Ooi Pos(26 UNPs)Links To11
Blog Post Timeliness & Locality:Timeliness is Difference between Post Time and
Dateline
4/2/08 VIStology, Inc. 12
Published: November 11, 2007 Posted by Jeff Ooi on November 11, 2007 10:54 AMTimeliness = 0 days
Dateline: Kuala Lumpur, Malaysia Profile: MalaysiaLocality = 0 miles
Centrality vs. CredibilityBlog EDCGoldTruths
(7.8)Rocky’s Bru(14.8)
ForeverFriends(3.8)
Centrality .546 .472 .012Real* Name Ina Stanley Ahiruddin Attan (Orchid)
Provides Location Durham, NC Kuala Lumpur, MY Banjarmasin, ID
Provides Email No Yes No
Average Posts/Day 0.25 1.9 _
Linking Sites 44 38 1
Avg. Non-Spam Comments
1.2 35 0
Blogging for n Months
5 16 4
4/2/08 VIStology, Inc. 13
Conclusions and Future Work
• 1. It is possible to geolocate many Malaysian bloggers explicitly, particularly Blogspot bloggers.
• 2. System can now extract (nearly all) the features we need automatically now.
• 3. Integration efforts ongoing to tie all the pieces together.
• 4. This will allow us to do a complete, fully automated analysis of important events in the Malaysian (sopo)blogosphere, such as the Bersih Rally example that was repeatedly used in the slides.
4/2/08 VIStology, Inc. 14
16 VIStology, Inc.
IBlogs Architecture
BlogOntology
Event ontology
4/2/08
BaseVISor and Ontology Reasoning
4/2/08 VIStology, Inc. 17
Alias: Pak Lah?
Abdullah BadawiPak LahAbdullah Ahmad Badawi
Bersih Rally in Global Blogosphere:1432 Posts in Week 1
4/2/08 VIStology, Inc. 18
Bersih Rally Posts
4/2/08 VIStology, Inc.
Bersih Rally Hindraf Rally
19