A Pen-Based Statics Tutoring System

A Pen-Based Statics Tutoring System

A Pen-Based Statics Tutoring SystemJack LeeThomas StahovichRobert Calfee

Presented by: James Herold

Hi,

We will be talking about a statics tutoring system.1Demo

We will start out with a demo of our system.

Traditional Mouse and keyboard based tutoring method ask student to select answers among a set of choices. Ours ask the student to construct solutions from scratch, just as they would on pen and paper.

*demo

Now that we have seen a demo of how the program functions, we can look at the tutorial aspect of the system and a little bit on how the system works.2OutlineIntroductionSolving a problemSystem BoundariesDrawing/Critiquing ForcesWriting/Critiquing EquationsUser StudyResultsConclusionWe will first look at a demo of the program, then how systems and FBDs are constructed. Next we look at writing equations and we conclude with results.3Textbook instructions for FBDsStep 1: Decide what system to isolate.

Step 2: Next isolate the chosen system by drawing a diagram which represents its complete external boundary.

Step 3: Identify all forces which act on the isolated system

-- J.L. Meriam and L.G. Kraige. Engineering Mechanics - Statics. John Wiley & Sons, Inc.,6th edition, 2007.The first step in solving statics problems is the construction of FBDS.

This is how the students are taught in a typical textbook. They are asked to isolate the system, draw a diagram representing the boundary, and then draw the forces.4Typical Student Work

However, typical student work can be in the form of abstract stick figures or5Typical Student Work

They simply redraw the entire system. There is no notion of a system boundary in either of these cases. Student have trouble with this concept. To assist students in understanding this concept, our system explicitly ask them to trace a boundary.6Solving a problemCreate BoundaryCritiqueBoundaryDrawForcesCritiqueForcesWriteEquationsCritiqueEquationsDone!!Creating a Boundary

Critiquing Boundary:Statically indeterminate

Critiquing Boundary:Cutting through rigid bodies

Critiquing Boundary:Does not contain the force of interestSolving a problemCreate BoundaryCritiqueBoundaryDrawForcesCritiqueForcesWriteEquationsCritiqueEquationsDone!!Drawing a force

+All interactions between the system and objects outside the system boundary must be represented by forces.Free body Diagram ErrorsHierarchical Feedback++All interactions between the system and objects outside the system boundary must be represented by forces.The missing force is at a roller support.Free body Diagram ErrorsHierarchical Feedback15---All interactions between the system and objects outside the system boundary must be represented by forces.The missing force is at a roller support.The missing force is at the highlighted location.Free body Diagram ErrorsHierarchical Feedback

Video Explanations

Buggy Rules

Student FBDForce in Wrong direction.Missing 2 Forces.Low-Level Errors

CorrectConceptual Diagnosis1. It appears that a body has been incorrectly modeled as a two-force member.After identifying all the forces, the system will critique the free body diagram.

This is an example of a student FBD.

We can give them very low-level feedback on their error by saying theyre missing 2 forces, and have a force in the wrong direction.

However, what is the student really thinking?

Conceptually, they might have mistaken this as a two force member due to the rod-like shape. Our system tries to explain at this conceptual level.18

Buggy RulesWWCorrectStudent FBDForces are in the wrong direction.Low-Level ErrorsConceptual DiagnosisHaving a load does not cause all the forces to be in the opposite direction of it.(60 rules)

These are two other examples

Another example is when the student draws all the forces opposite of the weight. Again, we can just tell them that the forces are in the wrong direction.But our system can identify that the student have mistakenly thought that all forces point opposite to the load.19Solving a problemCreate BoundaryCritiqueBoundaryDrawForcesCritiqueForcesWriteEquationsCritiqueEquationsDone!!

Writing an Equation

Writing an Equation

Type: Fx=0Lever Writing an Equation

Bx D cos (u) = 023Matching Equations- T COS(U) + F SIN(U) = 0- F COS(U) + P SIN(U) = 0Student EquationCorrect Equation?When we have the students equation, how do we know if its correct. We need to match it with the correct equation. Its easy if the student equation is correct. However, if the equation is different, how do we know what errors the student has made? Such as this example, how do we know which term the student was trying to write.

To figure out this, we use a weighting factor for each type of term.We believe that sign error and trigonometric error are much more common than using a wrong force.So, we use this to determine which terms should be paired up.

How do we know if the students equation is correct? We match like terms together. To do this, we break the user and correct equation down into individual terms (T1, T2, P).Compare each term to all the terms in the correct equation and maps the user term to the closest one.If the term matches completely, the value will be 1 and that term will be correct in the user equation.

24Matching Equations- T COS(U) + F SIN(U) = 0- F COS(U) + P SIN(U) = 0Student EquationCorrect Equation?When we have the students equation, how do we know if its correct. We need to match it with the correct equation. Its easy if the student equation is correct. However, if the equation is different, how do we know what errors the student has made? Such as this example, how do we know which term the student was trying to write.



25Matching Equations- T COS(U) + F SIN(U) = 0- F COS(U) + P SIN(U) = 0Student EquationCorrect Equation?When we have the students equation, how do we know if its correct. We need to match it with the correct equation. Its easy if the student equation is correct. However, if the equation is different, how do we know what errors the student has made? Such as this example, how do we know which term the student was trying to write.



26OutlineIntroductionSolving a problemUser StudyResultsConclusionWe will first look at a demo of the program, then how systems and FBDs are constructed. Next we look at writing equations and we conclude with results.27

PretestNP2 Tutorial

PosttestUser Study

Participants: 101 students, working in pairsUC Riverside statics courseSessions:4 through out the quarterStudy in 3rd Session -- 30 min with NP2

We had the system deployed in a statics course of over 100 students. Weve had them use the system for a total of 4 sessions. The user study was performed on the 3rd session, right after they were introduced to machines and frames in class. They did not use the software to learn machines and frames previously.

We performed a user study by giving a brief introduction, then we hand students a pretest for 10 minutes. They are then allowed to use the NP2 tutorial system for 30 minutes, and then posttest for 10 minutes. There are 2 different problems, and the students are divided into 2 groups, with one set have problem A first, and the other have problem B. Due to time constraints, we only asked the students to do the free body diagram.28

User Study: Problems

Problem AProblem BThese are the two problems we used (A,B respectively).29Improved Solution Time

Pretest

Posttest7.2 minutes4.9 minutes

Pretest

Posttest7.3 minutes

5.6 minutes

Problem AProblem B32% Reduction24% ReductionWe measured the amount of time it took the students to do the problems before and after using the tutorial system. We compared the same problem instead of the pretest a to posttest b and vice versa because there is varying difficulty between the problems. Specifically, B is a harder problem.

We see a reduction in the amount of time spent for pre to post test for both problems.30Solution Correctness: Problem AWeve also evaluated the correctness of tests by dividing the errors into a set of categories as shown here. Again, we compared the same problem to each other. We can see that in most cases, there are less errors in the post test than pre test.

Optional FBD are FBDs that are not wrong, but are not needed to solve the problem.

*Talk about it when someone asked: We speculate that there is a large difference in the Other extra forces because the landing gear problem made the student think that rollers should have a normal force, but the landing gear is not actually on the ground.31Solution Correctness: Problem BThe results for problem B we also see improvement in post test in most cases.32Ease of DrawingLabelsArrowsBoundaryEasyHard33Weve also asked the students to fill out a survey. The survey questions are ranked from 0-10, with 10 being highest.

In general,

The response is better than average. We speculate that the survey suffered because the program was not complete when this program was first rolled out. We had incremental changes throughout the use of the session. As a result, the constant change of user interface and bugs contributes to the lower scores. Weve also noticed some of the surveys had consistent 1s throughout his survey.

We plan to perform another user study with a more robust version of the program in the near future.

*talk about it if someone asks:In terms of the ease of drawing, the worst score is Labels. We believe this is because many students label forces with a letter and subscript. Since our program wasnt designed to interpret subscripts, the subscript character gets merged with the first character. We have resolved this issue in the newer versions by incorporating a gesture based text input.

Arrows are low because of the single tail to head arrow. We noticed users draw arrows in various different fashions. Were thinking of using a stroke order independent approach for arrow recognition.

Usefulness for LearningEquilibrium ProblemsTwo-Force MemberNewton's 3rd LawForces

BoundarySelectionNot UsefulVery Useful

This ProgramHas Usable FeedbackIs preferableto WIMPIs easy to learnHighLow

Overall ReactionStimulatingDull SatisfyingFrustrating EasyDifficultWonderfulTerribleHighLowConclusionPossible to scaffold handwritten workSketch recognition enables natural work environment Creating a solution rather than selecting the answerBuggy-rules enable diagnosis of conceptual misunderstandings

NP2 is an effective tutorial systemStudents like the interfaceMeasurable learning gains are quickly achieved

This work is provided by the National Science Foundation via award No. 0735695. They also gratefully acknowledge support from HP, who provided the tablet PCs used in this project.

In conclusion,

We have constructed a novel pen-based statics tutoring system.37Chart36.97.16.876.5

Boundary SelectionForce IdentificationNewton's 3rd LawTwo Force MemberEquilibrium Problems

Sheet1(Trace) BoundaryArrowsLabels1236.86.25.62.22.42.5Boundary SelectionForce IdentificationNewton's 3rd LawTwo-Force MemberEquilibrium Problems123456.97.16.876.52.12.12.222.1Is easy to learnIs perferable to WIMPHas Usable Feedback1237.16.56.82.332.2Terrible:WonderfulDifficult:EasyFrustrating:SatisfyingDull:Stimulating12346.26.75.46.42.12.42.42.4

Sheet1

(Trace) BoundaryArrowsLabelsLow HighEase of Drawing

Sheet2

Boundary SelectionForce IdentificationNewton's 3rd LawTwo Force MemberEquilibrium ProblemsLow HighUsefulness for Learning

Sheet3

Is Easy To learnIs Preferable to WIMPHas Usable FeedbackLow HighThis Program

Terrible-WonderfulDifficult-EasyFrustrating-SatisfyingDull-StimulatingLow HighOverall Reaction

Chart47.16.56.8

Is Easy To learnIs Preferable to WIMPHas Usable Feedback


Sheet1


Sheet2


Sheet3



Chart56.26.75.46.4

Terrible-WonderfulDifficult-EasyFrustrating-SatisfyingDull-Stimulating


Sheet1


Sheet2


Sheet3



Documents

A Pen-Based Statics Tutoring System