C.D. Thomson, A. Corbett, M. Holcombe

University of Sheffield, Department of Computer Science and the Institute of Work Psychology 1

This work was supported by an EPSRC grant: EP/D031516 – the Sheffield Software Engineering Observatory.

Teaching Overview20 years of experience teaching three modules.

[Parker 1999 A & B, Holcombe 1998, Thomson 2007 A]

The introduction of standardised technology over the last 7 years.

Based in the computer science department.In each module the student’s develop a piece of

software based on requirements that they capture.In each successive module the students are given

more autonomy and harder technical challenges.A high percentage of the marks are awarded on

the process and evidence of it.

2

Software Hut moduleIn the second year, teams of 4-6 students, 15

hours per week, over 12 weeks.3-4 teams compete to deliver a product to a

external customer.Problems:

Students would produce great software but not record how they did it.

Students did not plan their work formally, occasionally this meant that a team would not deliver.

3

Week Date Deliverable Items1 14-15/2 THURSDAY – initial briefing and lecture on XP, FRIDAY – meeting clients2 21-22/2 THURSDAY – lecture on Unit tests, FRIDAY – Management meeting

3 28-29/2 THURSDAY – lecture on System testing and modelling, FRIDAY - Release 1: Demo to client and manager. Place code and binary in release directory.

4 26/2 THURSDAY – lecture on how to package and deliver software, FRIDAY – Management and client meeting – deliver requirements document

5 13-14/3 THURSDAY – lecture on debugging, refactoring, library development, Place code and binary in release directory. FRIDAY – Management and client meeting

Easter 16/3-7/46 10-11/4 THURSDAY – lecture and XP review, FRIDAY – Management and client meeting

7 17-18/4 FRIDAY - Release 3: Demo to client and manager. Place code and binary in release directory.

8 24-25/4 FRIDAY – Management meeting

9 1-2/5 FRIDAY - Release 4: Demo to client and manager. Place code and binary in release directory.

10 8-9/5 FRIDAY – Extensive systems testing and Demonstration to client.

11 15-16/5MONDAY – Final release to client. Please hand in a CD and printed documentation to be set to the client, via reception.WEDNESDAY – QA exercise email to Mike and the other team.

12 22-23/5

MONDAY – Final release and QA hand in, to be checked by the managers, place in team directory. (NB you may correct any errors found by the client or after the QA exercise).FRIDAY – Poster presentations in the Lewin Lab and Personal Evaluations via reception (posters also from Crossover and Genesys teams – external visitors will be present.. 4

Software Hut mark schemeThe following will be assessed for the team as a whole:

5 A set of weekly meetings, entered into the management tool (automatically copied to wiki).

5 One or more well documented code libraries.15 XP Compliance as measured by details on the Wiki:

Story cards; Pair working/programming (via tasks on story card pages); Automated and Manual tests; Small releases.

The following will be assessed for each team member:20 XP Compliance as above, but for individual

entries: Tasks on story card pages; Time sheets.

5 Personal: Personal Evaluation, in the final week of the semester (5).

5

Timeline

6

CrossoverCrossover

Software HutSoftware Hut

GenesysGenesys

20042002 20032001 2005 2006 2007 2008

Current management tool introduced

Linked wiki

First wiki

Web management tool

Formalised templates

Test server

Legacy systemStudents were asked to record two aspects of their

process:Their meetings (in the form of timesheets, actions,

agendas and minutes)And their time spent on a timesheet.

Our first step in 2003 was to formalise these (in software hut) so that each team used the same template, thus making them easier to compare.

The quality was very variable, often a document would be present but little information was contained in it.

Meetings were often not minuted in the second half of the project.

7

Typical minutesTeam: Group LL (XP)Date/Time: 07/03/03 09:06---10:55(9:00 am) 11:15---11:35(with client) Agenda: Discuss the structure of the software Chair: S1Present: Every team memberApologies: None Details of Discussion: 9:00 am meeting:1. Produced some story cards;2. Made the Power Point, which will be shown to the client, to simulate the software that will be produced;3. Do the research for the life cycles that we are going to show in the software  With Client:A few points from the client: Give us CD for the relevant information; Show students the opportunities of this path; Do not put so many complex stuff into the system, because that will scare the students away

(consider the difficulties’ level);  Action Points: Have a meeting with client on 18/03 10:00---12:00;Finish story cards and test sets which are needed for the requirements documents by next Friday;Do the estimation for the project;The client will be free every Monday and Wednesday(off normal working hours) Get the CD from Dr. M George  Next Meeting: 07/03/03 Mapping Building LT03 with client

8

Another exampleTeam: LL (XP)Date/Time: 14/02/03 13:00---13:53 Agenda: Discussed Software Hut preferences and Roles Chair: S1Present: Every team memberApologies: None Details of Discussion: Discussed Software Hut preferences;Discussed Software Hut Roles;Looked at what each person had to accomplish, and decide each member’s job.  Action Points: 1 Print the stuff for the archivist;2 Decide the role for each member: S1 is chair, S2 is leader of the team, S3 is the

secretary and S4 the archivist;3 Look into what XP is;4 Wait to find out which client we are working with. Next Meeting: 17/02/03 Lewin Lab

9

Directory structureIn order to capture the process followed we

depended on taking an image of the teams’ directory structure weekly.

In some cases this worked well with teams updating their files and arranging them into predefined locations.

Other teams ignored this advice, and appeared to work only sporadically or at the end of the project.

10

Formalising process captureTo address these issues we set out to:

Make clear exactly what was required.Place a grade bearing requirement on the

students to provide evidence that they followed a process.

Provide tools that would guide the students to produce evidence.

11

Wiki mk 1With the Genesys students we introduced a

wiki (mediaWiki) where they could document their projects.

We hoped this would help to provide continuity from one year to the next.

We found that it required a committed student to ensure the wiki was updated and was uniform across the various projects.

Often the wiki was well maintained at the start of the project but not at the end – when the documentation is most needed.

12

Code management systemsTeams of developers working on software often use

tools to manage the files they create.These can be used to record the work of an individual.Popular tools include:

CVSSubversion

We found that using such a system could get in the way of the main teaching aims with the students in years 1 and 2.

However the fourth year students find them easier to use perhaps because they are more familiar with the task.

13

Problems with code management systems for evidence We found that students did not fully understand

how to use these systems, which resulted in their frustration and time not spent on the core problem.

Two specific classes of problem are worth mentioning [Thomson 2008]:

The non-use of the system (0.16 Errors Per File). This resulted in limited information about the roles of the

team members and their working practice. Direct manipulation of the repository.

Data corruption due to files deposited without using the CVS tool.

Data corruption due the re-initialisation of the repository (0.03 EPF).

14

Web management toolThis tool was written in PHP and accessed

via a web browser [Thomson 2007 B].The aim was to implement the formalised

documents so that the students would have to fill in the required fields.

However the editing was rudimentary, particularly in the first year of use. This was refined in the second year.

15

16

Week of project

1 2 3 4 5 6 7 8 9 10 11 12

Pag

e hi

ts

0

2000

4000

6000

8000

10000

Meetings Story cards Project planning Time sheets

Week of project

1 2 3 4 5 6 7 8 9 10 11 12

Pag

e hi

ts

0

2000

4000

6000

8000

Meetings Requirements (incl. story cards) Project planning Time sheets

Figure 2: Software Hut 2003-4 usage.Figure 1: Software Hut 2004-5 usage.

17

Web tool feedbackIn the first version of this tool the students

had to manually associate the different forms of data, they found this difficult to do.In the second run we automated this process,

but they still did not associate items, other than meetings and agendas.

The students commented that it was default to enter the text, as often boxes were the wrong size and laid out awkwardly.

18

HackystatHackystat is a tool (Uni. Of Hawaii) that

automatically records process information from software engineering tools. [Johnson 2007]

We set this up but the students were not keen on using it as it felt like it was snooping too much.

The information collected could be queried by the students using a web interface, but it was not visually appealing.

19

Management toolThe server is implemented in PHP/MySQL.The client is implemented in Java and can run

on most computers.The Java interface allows a richer experience

for the client, which is easier to use.We can customise the view to allow different

student groups to see different functionality.It is possible to add entries directly into

teams diaries to remind them of important deadlines.

20

Tool screens

21

Tool screens

22

Tool screens

23

Useage Data

24

Figure 3. Software Hut 2007-8 Figure 3. Software Hut 2006-7

WeekWeek

Num

ber

of h

its

Num

ber

of h

its

Tool minutes example1) Present

1.1) Members 53:;78:;59:;57: 1.2) Chair 59:1.3) Secretary 53:

2) Team problemOnly remaining work is on the research supervision

changes the client identified on last Friday's client meeting. Other work is testing, which we are far ahead on already.

3) Team problemDocumentation must be completed by Monday 12th

May. User Guide (already started), Maintenance Document and Installation Guide - start on these as soon as possible this week.

25

Tool minutes example 1) Present

Work is likely to be slow this week, as all members of the group also need to dedicate a large portion of time towards the HCI assignment, however progress should be made in a number of area. This week we will also focus on getting all of the past information into the team directory.

1.1) Members 32:;64:;34:;554: 2) Story

The contact form is to be reimplemented. Chris will provide code for the javascript/css based popup which will be used, which the form will be added to.

2.1) Identifier or creator c: Client 2.2) Story card 2026: Contact Fo...

3) Story This needs to be implemented. 3.1) Identifier or creator 32: 3.2) Story card 3004: Newsletter...

4) Story News is to be changed to be an adaptation of pages rather than it's own specific entity. Mat will make the Page

code more robust to allow this to be done. 4.1) Identifier or creator 64: 4.2) Story card 3011: Add News ;3012: Edit News ;3013: Remove New... ;3014: Show News ... ;3015: Show

News ... 5) Story

The client has decided they would like an upload area and file manager within the partner's zone and admin area. Indi will attempt to address this during the week by using php_file_tree() however this is unlikely to be completed for the next meeting.

5.1) Identifier or creator c: Client 5.2) Story card 3008: Partner Zo...

6) Story Overall, while progress has been slow in area, We feel that we will be able to complete the project with time to

spare, allowing us to rework the design after presenting a working model should the client choose so, and provide substantial documentation.

6.1) Identifier or creator 32

26

Management tool feedbackIn the first iteration the tool was built into Eclipse,

this meant that it took a while to load.We now provide a standalone version.

The minute editor was initially a little clumsy.We implemented a word processor like editor.

The Genesys students did not see the relationship between the tool and their wiki.

Sometimes the tool was slow to run.We were able to optimise the server component

which helped a lot.Some students wanted a web based tool.

27

Test serverThis was an application that the students

submitted their tests to (one of the deliverables).

It ran the tests written by the students and returned the results to them.

It also made a log of the tests, and when they were made as evidence for testing.

Submission was via FTP.

28

Test server feedbackThe system proved to be difficult to use, as

the students were struggling with the test methodology that we had proposed (Although they had apparently used in previous years!)

The server showed that typically teams did not manage to write tests that would run before the final weeks of the project.

The students also reported that they found the FTP procedure hard to use.

The tool was enhanced this year and we will try to use it again next year.

29

Wiki mk 2Sometimes the students found it hard to decide

what to put on the standard wiki.On this occasion we used mediaWiki again but

added a plugin, which integrates into the management tool.

Students can now click a button in the tool and the details of that screen are added to the wiki.

Crucially the wiki allows students to annotate the required parts with their own information.

30

Wiki useage (modify)

31

Week (2007-8)

Num

ber

of m

odifi

catio

ns

Wiki screens

32

Wiki mk 2 feedbackIf was found useful to add extra information,

and record things in their own way.Still unclear about how to get started on the

wiki.We provided an example wiki, but it maybe

worth while pre-populating this info into each teams wiki individually as an outline template.

33

ConclusionsStudents are looking for both guidance and

freedom to run their projects.Tool support should provide both, guidance

through structured forms, freedom through the ability to annotate or provide additional information.

Students also want to be rewarded for their work so mark schemes must be realistic in rewarding the time spent ‘running’ a project.

By providing this support we have seen the student

34

The futureThe idea of networked working, especially

when software developers are not in the same place is becoming important.

As is the idea of having contact throughout the customers organisation.

We are considering extending the tool to support these features.

35

References Johnson, P. M. (2007). Automated software process and product

measurement with Hackystat. Dr. Dobbs Journal. Holcombe, M. and Stratton, A., Fincher, S., Griffiths, G., (eds) (1998).

“Projects in the computing curriculum”, Proceedings of the Project98 workshop, Sheffield, Springer.

Parker, H.E.D. and Holcombe, W.M.L. (1999 A) “Campus based industrial software projects: risks and rewards”. Manuscript.

Parker, H.E.D. and Holcombe, W.M.L. (1999 B) “Keeping our clients happy: myths and management issues in ‘client led; student software projects”, computer science education, 9 (3), pp 230-241,.

Thomson, C. and Holcombe, M. (2007 A). 20 years of teaching and 7 years of research: Research when you teach. In 3rd South-East European Workshop on Formal Methods, Thessaloniki, Greece. South-East European Research Centre.

Thomson, C. D. (2007 B). Defining and Describing Change Events in Software Development Projects. PhD thesis, Department of Computer Science, University of Sheffield.

Thomson, C. and Holcombe, M. (2008). “Correctness of Data Mined from CVS”, In proceedings of 5th Working Conference on Mining Software Repositories, Leipzig, Germany, May 10-11, ACM,

36