Web Intelligence, World Knowledge and Fuzzy Logic
Lotfi A. Zadeh Computer Science Division Department of EECS UC Berkeley
September 14, 2004
URL: http://www-bisc.cs.berkeley.eduURL: http://zadeh.cs.berkeley.edu/
Email: [email protected]
In moving further into the age of machine intelligence and automated reasoning, we have reached a point where we can speak, without exaggeration, of systems which have a high machine IQ (MIQ). The Web, and especially search engineswith Google at the topfall into this category. In the context of the Web, MIQ becomes Web IQ, or WIQ, for short. PREAMBLE
WEB INTELLIGENCE (WIQ)Principal objectivesImprovement of quality of searchImprovement in assessment of relevance
Upgrading a search engine to a question-answering system
Upgrading a search engine to a question-answering system requires a quantum jump in WIQ
QUANTUM JUMP IN WIQCan a quantum jump in WIQ be achieved through the use of existing tools such as the Semantic Web and ontology-centered systems--tools which are based on bivalent logic and bivalent-logic-based probability theory?
CONTINUEDIt is beyond question that, in recent years, very impressive progress has been made through the use of such tools. But, a view which is advanced in the following is that bivalent-logic- based methods have intrinsically limited capability to address complex problems which arise in deduction from information which is pervasively ill-structured, uncertain and imprecise.
COMPUTING AND REASONING WITH PERCEPTION-BASED INFORMATION?Perceptions are dealt with not directly but through their descriptions in a natural language
Note: A natural language is a system for describing perceptions
A perception is equated to its descriptor in a natural languageKEY IDEAS
CONTINUEDThe meaning of a proposition, p, in a natural language, NL, is represented as a generalized constraint
X: constrained variable, implicit in pR: constraining relation, implicit in pr: index of modality, defines the modality of the constraint, implicit in pX isr R: Generalized Constraint Form of p, GC(p)pX isr Rrepresentationprecisiation
RELEVANCEThe concept of relevance has a position of centrality in search and question-answeringAnd yet, there is no definition of relevanceRelevance is a matter of degreeRelevance cannot be defined within the conceptual structure of bivalent logicInformally, p is relevant to a query q, X isr ?R, if p constrains XExampleq: How old is Ray? p: Ray has two children
CONTINUED?qp = (p1, , pn)
If p is relevant to q then any superset of p is relevant to q
A subset of p may or may not be relevant to q
TEST QUERYThe Mountains of California, by John Muir (1894) - John Muir ... Number of Black Bears in the San Gabriel Mountains AllFusion Harvest Change Manager Helps DesertDocs Manage Mountains ... Launching of the International Year of Mountains (IYM), 11 ... Preliminary Meeting Notice and Agenda Santa Monica Mountains ...
WebResults 1 - 10 of about 1,040,000 for number of mountains in CA. (0.31 seconds) NUMBER OF MOUNTAINS IN CALIFORNIAGoogle
TEST QUERYDefinition of Switzerland - wordIQ Dictionary & Encyclopedia Culture of Switzerland General information about Switzerland Hike the mountains of Switzerland Fragmented Ecosystems: People and Forests in the Mountains of ...WebResults 1 - 10 of about 249,000 for number of mountains in Switzerland. (0.30 seconds) NUMBER OF MOUNTAINS IN SWITZERLANDGoogle
TEST QUERYNumber of Ph.D.s in computer science produced by European universities in 1996Google:For Job Hunters in Academe, 1999 Offers Signs of an Upturn Fifth Inter-American Workshop on Science and Engineering ...
TEST QUERY (GOOGLE)largest port in Switzerland: failure
Searched the web for largest port Switzerland. Results 1 - 10 of about 215,000. Search took 0.18 seconds.
THE CONSULATE GENERAL OF SWITZERLAND IN CHINA - SHANGHAI FLASH N ... EMBASSY OF SWITZERLAND IN CHINA - CHINESE BUSINESS BRIEFING N ... Andermatt, Switzerland Discount Hotels - Cheap hotel and motel ... Port Washington personals online dating post
TEST QUERY (GOOGLE)smallest port in Canada: failure
Searched the web for smallest port Canada. Results 1 - 10 of about 77,100. Search took 0.43 seconds.
Canadas Smallest Satellite: The Canadian Advanced Nanospace ... Bw Poco Inn And Suites in Port Coquitlam, Canada
TEST QUERY distance between largest city in Spain and largest city in Portugal: failurelargest city in Spain: Madrid (success)largest city in Portugal: Lisbon (success)distance between Madrid and Lisbon (success)Google
TEST QUERY (GOOGLE)population of largest city in Spain: failure
largest city in Spain: Madrid, success
population of Madrid: success
HISTORICAL NOTE1970-1980 was a period of intense interest in question-answering and expert systemsThere was no discussion of search enginesExample: L.S. Coles, Techniques for Information Retrieval Using an Inferential Question-answering System with Natural Language Input, SRI Report, 1972Example: PHLIQA, Philips 1972-1979Today, search engines are a reality and occupy the center of the stageQuestion-answering systems are a goal rather than reality
RELEVANCEThe concept of relevance has a position of centrality in summarization, search and question-answeringThere is no formal, cointensive definition of relevanceReason:Relevance is not a bivalent conceptA cointensive definitive of relevance cannot be formalized within the conceptual structure of bivalent logic
DIGRESSION: COINTENSIONChuman perception of Cp(C)definition of Cd(C)intension of p(C)intension of d(C)CONCEPTcointension: coincidence of intensions of p(C) and d(C)
RELEVANCErelevancequery relevancetopic relevanceexamples
query: How old is Ray proposition: Ray has three grown up children topic: numerical analysis topic: differential equations
QUERY RELEVANCEExampleq: How old is Carol?p1: Carol is several years older than Rayp2: Ray has two sons; the younger is in his middle twenties and the older is in his middle thirties
This example cannot be dealt with through the use of standard probability theory PT, or through the use of techniques used in existing search enginesWhat is needed is perception-based probability theory, PTp
PTpBASED SOLUTION(a) Describe your perception of Rays age in a natural language(b) Precisiate your description through the use of PNL (Precisiated Natural Language)
Result: Bimodal distribution of Age (Ray)unlikely \\ Age(Ray) < 65* +likely \\ 55* Age Ray 65* +unlikely \\ Age(Ray) > 65*
Age(Carol) = Age(Ray) + several
CONTINUEDMore generallyq is represented as a generalized constraint
q: X isr ?R
Informal definitionp is relevant to q if knowledge of p constrains Xdegree of relevance is covariant with the degree to which p constrains Xconstraining relationmodality of constraintconstrained variable
CONTINUEDProblem with relevance
q: How old is Ray?p1: Rays age is about the same as Alansp1: does not constrain Rays age
p2: Ray is about forty years oldp2: does not constrain Rays age
(p1, p2) constrains Rays age
RELEVANCE, REDUNDANCE AND DELETABILITYDECISION TABLEAj: j th symptom
aij: value of j th symptom of Name
REDUNDANCEDELETABILITYAj is conditionally redundant for Namer, A, is ar1, An is arnIf D is ds for all possible values of Aj in *Aj is redundant if it is conditionally redundant for all values of Name compactification algorithm (Zadeh, 1976); Quine-McCluskey algorithm
RELEVANCEAj is irrelevant if it Aj is uniformative for all arjD is ?dif Aj is arjconstraint on Aj induces a constraint on Dexample: (blood pressure is high) constrains D(Aj is arj)is uniformative if D is unconstrainedirrelevance deletability
IRRELEVANCE (UNINFORMATIVENESS)(Aj is aij) is irrelevant (uninformative)
EXAMPLEA1 and A2 are irrelevant (uninformative) but not deletableA2 is redundant (deletable)0A1A1A2A20D: black or whiteD: black or white
THE MAJOR OBSTACLEExisting methods do not have the capability to operate on world knowledgeTo operate on world knowledge, what is needed is the machinery of fuzzy logic and perception-based probability theoryWORLD KNOWLEDGE
WORLD KNOWLEDGE?World knowledge is the knowledge acquired through experience, education and communicationWorld knowledge has a position of centrality in human cognitionCentrality of world knowledge in human cognition entails its centrality in web intelligence and, especially, in assessment of relevance, summarization, knowledge organization, ontology, search and deduction
EXAMPLES OF WORLD KNOWLEDGEParis is the capital of France (specific, crisp)California has a temperate climate (perception-based)Robert is tall (specific, perception-based)It is hard to find parking near the campus between 9am and 5pm (specific, perception-based)Usually Robert returns from work at about 6pm (specific, perception-based)Child-bearing age is from about 16 to about 42 (perception-based)
VERAS AGEq: How old is Vera?
p1: Vera has a son, in mid-twenties
p2: Vera has a daughter, in mid-thirties
wk: the child-bearing age ranges from about 16 to about 42WORLD KNOWLEDGETHE AGE EXAMPLE
CONTINUED16*16*16*41*42*42*42*51*51*67*67*77*range 1range 2p1:p2:(p1, p2)00(p1, p2): a= 51* 67*timelinesa*: approximately aHow is a* defined?
PRECISIATION OF approx