Invariant Based Invariant Based Programming Programming in Educationin Education

Tutorial, FM’08Linda Mannila

26.5.2008

BackgroundBackgroundChallenges at CS departments

◦ First year students have weak skills in logic, formal reasoning, writing proofs etc.

◦ Students do not see a connection between theoretical and practical courses

Goal◦ Giving students hands-on skills in

constructing proofs and reasoning about correctness early on

◦ Linking theory with practice

IBP in EducationIBP in EducationCourse for PhD students in 200515 case study sessions with

faculty, students, researchersUndergraduate course, given

twice, starting in spring 2007

Undergraduate CourseUndergraduate CourseElective (~15 students/course)Prerequisities:

◦ Introductory programming◦Logic

17 interactive, example-driven sessions ◦4 (7) sessions with SOCOS

Addition, ..., linear search, ..., sortingEvaluation

◦questionnaires, interviews, observations, assignments, exam

Experienced benefits (+)Experienced benefits (+)Introduction to program correctness and formal verification

Good introduction to formal verification and how tools can be used in that context

Helps remove errors in the algorithm that could lead to bugsA practical method for introducing program correctness

IBP summarizes the proof conditions in a good way IBP is intuitive

Introduction to a more abstract view of programming Learning to think about how a program works in general,

without a given programming language Learned to think about a program as states and transitions

instead of merely as transitions as is usually the caseMore tangible overview of the program structure

Learning to draw a program makes it easier to see its structure

Makes it easy to keep the track of the various parts of a program

New and useful contents

Experienced difficulties (-)Experienced difficulties (-)Syntax and notation

◦ It’s difficult to formulate one’s programs according to the standard

◦Since I have programmed previously e.g. the Java way of expressing things is quite ingrained

Proofs◦Proving programs by hand is very work

intense◦Proving complex programs is difficult

Finding the correct invariant and postcondition

Using SOCOS

Preliminary error analysisPreliminary error analysisOnly minor problems related to the

◦ Method (e.g. the syntax)◦ Algorithm

Main problem◦ Weak logical background◦ Interpreting and constructing quantified

expressions◦ Understanding when to use quantifiers◦ Understanding which one to use

Problems with logic affect other aspects ◦ Unclear invariants (not necessarily weak)◦ Unclear proofs

ConclusionsConclusionsLearning IBP is not difficult

◦Students get the big picture rather quickly◦Appreciate the diagrammatic notation

Main difficulties related to predicate logic◦More focus on using logic in practice◦Students need more practice in reading

and constructing quantified expressionsRemedy?

◦Split the course into two parts: 1) manual and 2) with tool support

More time for practice

Assignment Results Assignment Results (2007)(2007)Scores

◦ Average 25.5 (max 40) ◦ Seven students > 30 points

Most common errors◦ Syntax related (e.g. using Java like syntax

or not following the given proof format)Other errors

◦ Incomplete invariants ◦ Problems with the algorithm◦ Using undeclared variables◦ One “off by one” error

Error analysis (2008)Error analysis (2008)Jonas Erikson, Master’s ThesisAnalyzed student solutions to four

assignments (all in all ~50 solutions)

Three main error categories:◦Syntax (related to the diagram

representation)◦Reasoning (related to misconceptions

and weak understanding)◦Logical background (related to having a

weak background in expressing logical statements)

Total number of errorsTotal number of errors

Nested loops

Related to syntaxRelated to syntax

Related to reasoningRelated to reasoning

Nested loops

Related to logical backgroundRelated to logical background

Define own predicates,use quantifiers to a larger extent

DiscussionDiscussionCould IBP be introduced at high

school level?◦As a course in mathematics?◦Would the method need to be

simplified? How?