Explanation Aware Design And Computing 2009 09 11

Thomas Roth-Berghofer

Explanation-aware System Design and Computing

Thomas Roth-BerghoferSenior researcher, [email protected]

German Research Center for Artificial Intelligence DFKI GmbH, Kaiserslautern, Germany

1Freitag, 11. September 2009

mailto:[email protected]







Traditional view on software systems

U

I

User

Software System



„Trust me. I know what I am

doing!“

„Traditional“ behaviour of software systems

SLEDGE HAMMER



Overview

• Explanation-aware view on software design: communication scenario

• Aspects of explanation-aware design

• Example: coTag — Code tagging and similarity-based retrieval



Communication participants

U

I

Explainer

Originator

User Software System



U

I

Explainer

Originator

The user communicates by way

of a user interface (UI) with the whole software system and

is the recipient of explanations.



Originator

U

I

Explainer

Originator

The originator is the tool the user works with to perform tasks and solve problems.



Explainer

U

I

Explainer

Originator

The explainer can be seen as another tool that helps understanding how the originator works and what knowledge the originator uses.



Explanation knowledge

U

I

Explainer

Originator

• concept explanations• templates• …



Problem solving knowledge

U

I

Explainer

Originator

• results• concepts• workflows• …



Reasoning information

U

I

Explainer

Originator

• intermediate results• context snapshots• …



What are explanations?



What are explanations?Explanations are answers to questions.



Cognitive aspects of explanations

• „Explanations are the most common method used by humans to support decision making.“ (Roger Schank, 1986)

• Main purpose:

• Explain a solution.

• How was the solution derived?

• How does a system work?

• How to handle a system

• Explain failures.



Computational aspects of explanations

• Backward explanations:

• Explain result and how it was obtained.

• Forward explanations:

• Explain (indirectly) by showing different ways to further optimise a given result.

• Open up possibilities for exploratory use.



Code-tagging and similarity-based retrieval with myCBR

EXAMPLE:

Roth-Berghofer, Th. and Bahls, D. (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Bramer, M., editors, Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag.



Programmer‘s dilemma



Typical questions of programmers

• Where is the code fragment I used to solve a similar problem with in the past?

• Is this piece of code still available?

• Is it worth the effort to search for it?

• If so, what would be the right search term?



Personalised approach

• Personal vocabulary: tags

• Linking tags



Linking tags

GridBag

PatternLayout

Similar!



Personalised approach

• Personal vocabulary: tags

• Linking tags

• Work context

• Social dimension: tag exchange

• Similarity-based retrieval



Case-BasedReasoning cycle

Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI Communications, 7(1):39–59, 1994.



Design decisions / constraints

• Integration in IDE eclipse

• Storage of code snippets and tags separately from code

• Queries = Search text plus work context

• Community repository for experience exchange



Code snippet & context

Work context

• java.net.URL

• java.net.URLConnection

• java.io.InputStream

• java.lang.StringBuffer

• java.io.BufferedReader

• java.lang.String

• java.lang.Exception

Java code snippet



Case structureAttribute Value type category

Tags String (multiple) Problem description

Context items String (multiple) Problem description

Code snippet String Solution

Document type String Provenance

Project name String Provenance

File path String Provenance

Author ID String Provenance

Creation date Long Provenance

Rating Float Maintenance

Rating count Integer Maintenance



Case structureAttribute Value type category

Tags String (multiple) Problem description

Context items String (multiple) Problem description

Code snippet String Solution

Document type String Provenance

Project name String Provenance

File path String Provenance

Author ID String Provenance

Creation date Long Provenance

Rating Float Maintenance

Rating count Integer Maintenance

Set by user

Set by coTag



Acquiring case



Query view

• Search for tags: init, logging config

• Include context => regard currently selected code



Retrieval

• Result for: init, logging, config

• Ranked list of code snippets



Presentation of cases



Situations in which explanations play a role

• Instructing explanations:• Novice users want to know about how tagging and (similarity-based)

retrieval works.

• Convincing explanations:• Regular users want to check when the retrieval does not meet their

expectations.

• Improving explanations• Regular users want to correct coTag‘s behaviour.



Explanation of matching

• Search terms: • init, logging, config

• Case tags: • init, Logger



Graphical explanation of trigram matching

• Syntactical similarity• Typos

• Stemming



Similarity customisation

• Tag similarities:

• Updates personal and community similarity measure

unsimilar 0%

partly similar 25%

similar 50%

very similar 75%

identical 100%



Three levels of similarity calculation

Personal

Imported

Trigram



Customised (personal) and imported similarity



Client-side architecture









Tag and exchange code snippets







• Re-finding information is a quite typical task in knowledge-work.

• Tagging is a helpful and well-known technique.

• Similarity-based retrieval can improve searches.

• Explanation-aware design and development of applications helps dealing with increased complexity of similarity-based retrieval.

Summary



Explaining Semantic Search Results of Medical Images in MEDICO

Forcher, B., Möller, M., Sintek, M., and Roth-Berghofer, Th. Explanation of semantic search results of medical images in Medico. In Th. Roth-Berghofer, N. Tintarev, and D. B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt 2009), pages 13–24, 2009.



Goal of Medico Project

Development of

• intelligent

• robust and

• scalable

semantic search engine for medical images



U

I

Explainer

Originator

Reconstructive explanations

line of reasoning

line of explanation



• Exploration interface with concept explanations support domain understanding.

• Justification interface provides action explanations, which counteract encapsulation and information hiding.

RadSem



Explainer

• Concept• Why• How• Action

explanations

• Relevant• Innovative• Convincing• Short and easy to overlook• Provide different perspectives

and follow-up questions

Good explanations

Kinds Goals

• Transparency• Justification• Relevance• Conceptualisation• Learning

OPEN QUESTION: OPERATIONALISATION OF THOSE CRITERIA

Originator



Kinds of explanations

• Concept explanations

• Action explanations

• Why- and How-explanations



Concept Explanations

• The goal of concept explanations is to build links between unknown and known concepts.

• Variations:• Definition: “What is a bicycle?” – “A bicycle is a land vehicle

with two wheels in line. Bicycles are a form of human powered vehicle.”

• Functional mapping: “What is a bicycle?” – “A bicycle serves as a means of transport.”

• Prototypical usage of individual things or actions: “What is a bicycle?” – “The thing, that man over there just crashed with.”

• …



Action explanations: “Why was this seat post selected?” – “For the given price, only one other seat post was available. But this was too short.

Action explanations

• Action explanations explain the activities of the respective system (originator).

• In RadSem: Reconstructive explanations based on search concepts and found concepts.



Why-explanations

• Why-explanations provide causes or justifications for facts or events.

• Examples:• Justification: “Why does the universe expand?” – “Because we

can observe a red shift of the light emitted by other galaxies.”

• Cause: “Because the whole matter was concentrated at one point of the universe and because the whole matter moves away from each other


Transparency

Justification

Relevance

Conceptualisation

Learning

Sørmo, F., Cassens, J., Aamodt, A.: Explanation in Case-Based Reasoning – Perspectives and Goals, 2005.

How did the system reach an answer?

Why is the answer a good answer?

Why is the question relevant?

What is the meaning of a concept?

Teach the user about the given domain.

Explanation goals



When are explanations good explanations?

• Short and easy to overlook

• Innovative

• Relevant

• Convincing

• Different perspectives and follow-up questions

• NaturalW. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag.



Take home messages

U

I

Explainer

Originator

• The ability to explain reasoning processes and results can substantially affect the usability and acceptance of a software system.

• Basic explanation scenario helps identifying communication partners and knowledge bases.

• Explanation goals and kinds further help structuring knowledge acquisition and use in software design and computing.



Explanation-aware System Design and Computing

Thomas Roth-BerghoferSenior researcher, [email protected]

Thank you!




Publications of ExaCt research group

2009

[Roth-Berghofer, Tintarev, Leake, 2009] Roth-Berghofer, Th., Tintarev, N., and Leake, D.B., editors. Proceedings of the IJCAI-09 workshop on Explanation-aware Computing (ExaCt 2009), July 2009.

[Adrian et al., 2009] Adrian, B., Forcher, B., Roth-Berghofer, Th., and Dengel, A. Explaining ontology-based information extraction in the NEPOMUK semantic desktop. In Thomas R. Roth-Berghofer, Nava Tintarev, and David B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt 2009), pages 94–101, 2009.

[Forcher et al., 2009] Forcher, B., Möller, M., Sintek, M., and Roth-Berghofer, Th. Explanation of semantic search results of medical images in medico. In Thomas R. Roth-Berghofer, Nava Tintarev, and David B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt 2009), pages 13–24, 2009.

[Stahl, Roth-Berghofer, 2009] Stahl, A. and Roth-Berghofer, Th. Rapid Prototyping of CBR applications with the Open Source Tool myCBR. Künstliche Intelligenz, 23(1):34–37, March 2009.

2008

[Roth-Berghofer et al., 2008] Roth-Berghofer, Th., Schulz, S., Bahls, D., and Leake, D.B., editors. Proceedings of the ECAI-08 workshop on Explanation-aware Computing ExaCt2008. University of Patras, July 2008. http://ceur- ws.org/Vol- 391.

[Forcher, Adrian, Roth-Berghofer, 2008] Forcher, B., Adrian, B., and Roth-Berghofer, Th.. Explanation styles in iDocument. ExaCt 2008, ECAI-08 Workshop.

[Bahls, 2008] Bahls, D. (2008). Explanation support for the case-based reasoning tool myCBR. Project thesis, University of Kaiserslautern.

[Forcher et al., 2008] Forcher, B., Adrian, B., and Roth-Berghofer, T. (2008). Explanations in the information extraction system iDocument. Künstliche Intelligenz, 22(2).

[Mittag, 2008] Mittag, F. (2008). ReduxExp: A justification-based explanation-support server. Project thesis, University of Kaiserslautern.

[Roth-Berghofer and Bahls, 2008] Roth-Berghofer, T. R. and Bahls, D. (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Bramer, M., editors, Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag.

[Roth-Berghofer and Mittag, 2008] Roth-Berghofer, T. R. and Mittag, F. (2008). ReduxExp: A justification-based explanation-support server. Proceedings of AI-2008. the twenty-eighth SGAI international conference on artificial intelligence. In Petridis, M., Coenen, F., and Bramer, M., editors, Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag.

[Roth-Berghofer and Richter, 2008a] Roth-Berghofer, T. R. and Richter, M. M., editors (2008a). Künstliche Intelligenz—Topic: Explanation, volume 22, Bremen. BöttcherIT Verlag.

[Roth-Berghofer and Richter, 2008b] Roth-Berghofer, T. R. and Richter, M. M. (2008b). On explanation. Künstliche Intelligenz, 22(2):5–7.

2007

[Bahls and Roth-Berghofer, 2007] Bahls, D. and Roth-Berghofer, T. (2007). Explanation support for the case-based reasoning tool myCBR. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence. July 22–26, 2007, Vancouver, British Columbia, Canada., pages 1844–1845. The AAAI Press, Menlo Park, California.

[Eppert, 2007] Eppert, M. (2007). Generating provenance explanations for the gnowsis rebirth machine - a first pass. Pro ject thesis, University of Kaiserslautern.

[Roth-Berghofer et al., 2007] Roth-Berghofer, T. R., Schulz, S., and Leake, D. B., editors (2007). Proceedings of the AAAI-07 workshop on Explanation-aware Computing ExaCt2007. AAAI Press. Technical Report WS-07-06.


http://ceur-

http://ceur-

Publications of ExaCt research group

2006

[Richter et al., 2006] Richter, M. M., Roth-Berghofer, T., and Schulz, S., editors (2006). Explanation-aware Computing, volume 25. SAP - Slovak Academic Press Ltd., Bratislava.

2005

[Roth-Berghofer et al., 2005a] Roth-Berghofer, T., Cassens, J., and Sørmo, F. (2005a). Goals and kinds of explanations in case-based reasoning. In Althoff, K.-D., Dengel, A., Bergmann, R., Nick, M., and Roth-Berghofer, T., editors, WM 2005: Professional Knowledge Management, pages 264–268, Kaiserslautern, Germany. DFKI GmbH.

[Roth-Berghofer and Cassens, 2005] Roth-Berghofer, T. R. and Cassens, J. (2005). Mapping goals and kinds of explanations to the knowledge containers of case-based reasoning systems. In Muñoz-Avila, H. and Ricci, F., editors, Case-Based Reasoning Research and Developmen, pages 451–464, Heidelberg. Springer Verlag.

[Roth-Berghofer et al., 2005b] Roth-Berghofer, T. R., Schulz, S., and Woody, A., editors (2005b). Proceedings of the AAAI Fal l Symposium on Explanation-aware Computing ExaCt2005. AAAI Press. Technical Report FS-05-04.

2004

[Roth-Berghofer, 2004] Roth-Berghofer, T. R. (2004). Explanations and Case-Based Reasoning: Foundational issues. In Funk, P. and González-Calero, P. A., editors, Advances in Case-Based Reasoning, pages 389–403. Springer-Verlag.

[Memmel, Roth-Berghofer, 2004] Memmel, M. and Roth-Berghofer, Th. Explanation and e-learning: A first pass. In Klaus-Peter F¨ahnrich, Klaus P. Jantke, and Wolfgang S. Wittig, editors, Von e-Learning bis e-Payment. Das Internet als sicherer Marktplatz, pages 255–263. Akademische Verlagsgesellschaft Aka, 2004.


Invitation to participate

• Manifesto

• Mailing list

• Workshop series

• …

HTTP://ON-EXPLANATION.NET


http://on-explanation.net

http://on-explanation.net

Technology

Explanation Aware Design And Computing 2009 09 11