Programming Techniques Lecture 3 Agile Development Based on: Software Engineering, A Practitioner’s Approach, 6/e, R.S. Pressman School of Software

Programming Techniques

Lecture 3 Agile Development

Based on: Software Engineering, A Practitioner’s Approach, 6/e, R.S. Pressman

School of Software


Manifesto for Agile Software Development

Proposes that it may be better to value:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan


Agile Software Engineering

• Represents a reasonable compromise between the conventional software engineering for certain classes of software and certain types of software projects;

• Embraces a philosophy that encourages customer satisfaction, incremental software delivery, small project teams (composed of software engineers and stakeholders), informal methods, and minimal software engineering work products;

• Guidelines stress on-time delivery of an operational software increment over analysis and design.


Agile Development

• Can deliver successful systems quickly;

• Stresses continuous communication and collaboration among developers and customers.


What is “Agility”?

• Effective (rapid and adaptive) response to change

• Effective communication among all stakeholders

• Drawing the customer onto the team

• Organizing a team so that it is in control of the work performed

Yielding …

• Rapid, incremental delivery of software


Agility

• An agile team is able to respond to changes during project development

• Agile development recognizes that project plans must be flexible

• Agility encourages team structures and attitudes that make communication among developers and customers more facile


Agility

• Agility eliminates the separation between customers and developers

• Agility emphasizes the importance of rapid delivery of operational software and de-emphasizes importance of intermediate work products

• Agility can be applied to any software process as long as the project team is allowed to streamline tasks and conduct planning in way that eliminate non-essential work products


Agility Principles

1. Highest priority is to satisfy customer through early and continuous delivery of valuable software.

2. Welcome changing requirements even late in development, accommodating change is viewed as increasing the customer's competitive advantage.

3. Delivering working software frequently with a preference for shorter delivery schedules (e.g., every 2 or 3 weeks).

4. Business people and developers must work together daily during the project.


Agility Principles

5. Build projects around motivated individuals, given them the environment and support they need, trust them to get the job done.

6. Face-to-face communication is the most effective method of conveying information within the development team.

7. Working software is the primary measure of progress. 8. Agile processes support sustainable development,

developers and customers should be able to continue development indefinitely.


Agility Principles

9. Continuous attention to technical excellence and good design enhances agility

10. Simplicity (defined as maximizing the work not done) is essential.

11. The best architectures, requirements, and design emerge from self-organizing teams

12. At regular intervals teams reflects how to become more effective and adjusts its behavior accordingly.


An Agile Process

Are based on three key assumptions:

1. It is difficult to predict in advance which requirements or customer priorities will change and which will not. It is also difficult to predict how customer priorities will change as a project proceeds.

2. For many types of software, design and construction activities are interleaved (construction is used to prove the design). It is difficult to predict how customer priorities will change as a project proceeds.

3. Analysis, design, and testing are not as predictable from a planning perspective as one might like them to be.


An Agile Process

• An agile process, must be adaptable to manage unpredictability

• Agile processes must be adapted incrementally to progress

• Incremental adaptation requires customer feedback based on evaluation of delivered software increments (executable prototypes) over short time periods


An Agile Process

• Is driven by customer descriptions of what is required (scenarios)

• Recognizes that plans are short-lived

• Develops software iteratively with a heavy emphasis on construction activities

• Delivers multiple ‘software increments’

• Adapts as changes occur


Human Factors

• Agile development focuses on the talents and skills of individuals, molding the process to specific people and teams

• The process moulds to the needs of people and team, not the other way around.


Agile Team Member Traits

• Competence – innate talent, specific software related skill, knowledge of the process applied.

• Common focus – to deliver a working software increment to the customer within the time promised.

• Collaboration – with one another, with the customer and with business managers.

• Decision-making ability – for both technical and project issues.

• Fuzzy-problem solving ability - have to deal with ambiguity. • Mutual trust and respect • Team self-organization:

- organises itself for work to be done.- organises the process to best accommodate its local environment- organises the work schedule to best achieve delivery of the software increment.


Agile Process Models

• Extreme Programming (XP)

• Adaptive Software Development (ASD)

• Dynamic Systems Development Method (DSDM)

• Scrum

• Crystal

• Feature Driven Development (FDD)

• Agile Modeling (AM)


What is Extreme Programming

• The most widely used agile process, originally proposed by Kent Beck

• “Extreme Programming (XP) is a lightweight, efficient, low-risk, flexible, predictable, scientific, and fun way to develop software" -- Kent Beck, Extreme Programming Explained

• Relies on object-oriented approach


XP Values

• Communication: – developer and customer work closely together

– pair programming

• Simplicity: just satisfy current design.

• Feedback: from development process, e.g. prototype.

• Courage: do the right thing, e.g. throw away code.


XP Key Activities

• Planning

• Design

• Coding

• Testing


XP Planning

• Begins with the creation of “user stories” – placed on an index card

• Customer assigns a value (i.e. a priority) to the story

• XP team assesses each story and assigns a cost – measured in development weeks

• If the story will require more than 3 weeks, the customer is asked to split the story into smaller stories – and assign new value etc…


XP Planning

• Stories are grouped for a deliverable increment

• A commitment is made on delivery date

• After the first software increment, “project velocity” – number of user stories implemented during the first release, is used to help define subsequent delivery dates for other increments

• Customer can add stories, change the value of an existing story, split stories or eliminate them.

• The XP team then reconsider all remaining releases and modifies its plans accordingly


XP Design

• Follows the keep it simple (KIS) principle

• Encourage the use of Class-Responsibility-Collaborator (CRC) cards

• CRC cards are only design work produced

• For difficult design problems, suggests the creation of “spike solutions”

• Spike solution – helps to lower risk when true implementation starts and validate the original estimates.


XP Design

• Encourages “refactoring” - an iterative refinement of the internal program design

• “Refactoring is a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behaviour.”[Martin Fowler]


XP Design

• XP design is viewed as a transient artefact that can and should be continually modified as construction proceeds

• Refactoring is to control these modifications by suggesting small design changes that ‘can radically improve the design


XP Coding

• Recommends the construction of a unit test for a store before coding commences

• Encourages “pair programming” – two people work together at one workstation to create code for a story

• Continuous integration and smoke testing is utilized


XP Coding – Smoke Testing

• A common approach for creating “daily builds” for product software

• Smoke testing steps:– Software components that have been translated into code

are integrated into a “build.”

– A series of tests is designed to expose errors that will keep the build from properly performing its function.

– The build is integrated with other builds and the entire product (in its current form) is smoke tested daily.


XP Testing

• Unit tests created before coding are implemented using an automated testing framework to encourage use of regression testing;

• All unit tests are executed daily• Integration and validation testing done on daily basis • “Acceptance tests” - customer tests, are defined by the

customer and executed to assess customer visible functionality


XP Principles

• Provide rapid feedback: feedback improves design.

• Assume Simplicity: only produce requirement.

• Incremental changes: reduces the risk.

• Embrace change: being an iterative approach change can be accommodated .

• Do Quality work. Using frequent testing and customer feedback. Writing the tests for each scenario provided by the users before writing the programs and keeping all of the tests running at all times. Writing acceptances tests when new functionality is identified. Tests grow with system;


XP core practices

1. Planning2. Small Releases - putting a minimal system into production

quickly and growing it in whatever directions that prove most valuable

3. Simple Design - starting projects with a simple design that constantly evolves to add needed flexibility and to remove unneeded complexity;

4. Continuous Testing - integrating and testing the whole system several times a day; writing tests for new function before writing function helps make code testable also it decouples form its environment; i.e. a method/class to be tested is independently callable. Tests act as documentation.


XP core practices

5. Refactoring - improving design of existing software;

6. Pair Programming - producing all software in pairs, two programmers at one screen;

7. Collective Code Ownership

8. Continuous Integration

9. 40-Hour Working Week

10. On-site Customer

11. Coding Standards


Extreme Programming (XP)

unit test continuous integration

acceptance testing

pair programming

Release

user stories values acceptance test criteria iteration plan

simple design CRC cards

spike solutions prototypes

refactoring

software incrementproject velocity computed


XP Issues

• Extreme Programming is somewhat controversial, for example:- team members are not forced to be specialists such as analysts or integrators;- documentation separate from the code is discouraged as communication is done face to face or through carefully written code and its tests.

• XP is a method favored for small to medium-sized teams developing software in the face of vague or rapidly changing requirements.


XP Issues

• Customers generally do not want have time or resources to be continuously involved in a project


Summary

• It seems natural to introduce XP to an upper level course in software engineering.

• Less of an Computer Science focus more Software Engineering approach.

• Some of the practices of XP should be introduced earlier in the curriculum.

• XP is not a ‘silver bullet’, it should be compared and contrasted with other approaches.


Adaptive Software Development

• Originally proposed by Jim Highsmith

• Emphasizes self-organizing teams, interpersonal collaboration, and both individual and team learning.


ASD Features

• Mission-driven planning

• Component-based focus

• Uses “time-boxing” - the practice of deciding a priori the fixed amount of time that can be spent on each task. When the task's time limit is exceeded, development moves on to the next task (with the hope that a majority of the critical work was completed before time ran out).

• Explicit consideration of risks

• Emphasizes collaboration for requirements gathering

• Emphasizes “learning” throughout the process


ASD Phases

• Speculation

• Collaboration

• Learning


ASD Phases - Speculation

• The project is initiated and adaptive cycle planning is conducted.

• Adaptive cycle planning uses:

- project initiation information – the customer’s mission statement,

- project constraints (e.g. delivery dates or user descriptions)

- basic requirements – to define the set of release cycles (software increments).


ASD Phases - Collaboration

• Requires teamwork from a jelled team, joint application development is preferred

• People working together must trust one another to:

- criticise without animosity;

- to assist without resentment;

- work as hard or harder as they do;

- have the skill set;

- communicate problems in a way that leads to effective action.


ASD Phases - Learning

• Focus groups – the customer and/or end-users provide feedback on software increments

• Formal technical reviews – ASD team members review the software components, improving quality and learning as they proceed.

• Postmortems – addressing its own performance and the process with intent of learning and then improving its approach.


Adaptive Software Development

adaptive cycle planning uses mission statement project constraints basic requirements time-boxed release plan

Requirements gathering JAD mini-specs

components implemented/ tested focus groups for feedback formal technical reviews postmortems

software incrementadjustments for subsequent cycles

Release


Dynamic Systems Development Method (DSDM)

• Promoted by the DSDM Consortium (www.dsdm.org)

• Provides a framework for building and maintaining systems which meet tight time constraints using incremental prototyping in a controlled environment

• Uses Pareto principle (80% of project can be delivered in 20% required to deliver the entire project)


Dynamic Systems Development Method (DSDM)

• Each increment only delivers enough functionality to move to the next increment

• Uses time-boxes to fix time and resources to determine how much functionality will be delivered in each increment


DSDM Features

• Similar in most respects to XP and/or ASD• Guiding principles

» Active user involvement is imperative. » Teams must be empowered to make decisions.» The focus is on frequent delivery of products. » Fitness for business purpose is the essential criterion for

acceptance of deliverables.» Iterative and incremental development is necessary to converge

on an accurate business solution.» All changes during development are reversible.» Requirements are baselined at a high level» Testing is integrated throughout the life-cycle.


DSDM Life Cycle Activities

• Feasibility study (establishes requirements and constraints);

• Business study (establishes functional and information requirements needed to provide business value);

• Functional model iteration (produces set of incremental prototypes to demonstrate functionality to customer);

• Design and build iteration (revisits prototypes to ensure they provide business value for end users, may occur concurrently with functional model iteration);

• Implementation (latest iteration placed in operational environment);


Dynamic Systems Development Method


Scrum

• Originally proposed by Schwaber and Beedle

• Scrum—distinguishing features– Development work is partitioned into “packets”

– Testing and documentation are on-going as the product is constructed

– Work occurs in “sprints” and is derived from a “backlog” of existing requirements

– Meetings are very short and sometimes conducted without chairs (stand-up meetings)

– “demos” are delivered to the customer with the time-box allocated


Scrum principles

• Small working teamed used to maximize communication, minimize overhead, and maximize sharing of informal knowledge.

• Process must be adaptable to both technical and business challenges to ensure bets product produced.

• Process yields frequent increments that can be inspected, adjusted, tested, documented and built on.

• Development work and people performing it are partitioned into clean, low coupling partitions.

• Testing and documentation is performed as the product is built. • Provides the ability to declare the product done whenever

required.


Scrum Activities

• Backlog - a prioritized list of requirements or features the provide business value to customer, items can be added at any time;

• Sprints - work units required to achieve one of the backlog items, must fit into a predefined time-box, affected backlog items frozen;

• Scrum meetings - (15 minute daily meetings) addressing these questions: What was done since last meeting? What obstacles were encountered? What will be done by the next meeting?

• Demos - deliver software increment to customer for evaluation).


Scrum

Scrum Process Flow (used with permission)


Crystal

• Proposed by Cockburn and Highsmith

• Crystal—distinguishing features– actually a family of process models that allow

“maneuverability” based on problem characteristics

– face-to-face communication is emphasized

– suggests the use of “reflection workshops” to review the work habits of the team


Crystal principles

• Its always cheaper and faster to communicate face-to-face

• As methodologies become more formal teams become weighed down and have trouble adapting to project work vagaries

• As projects grow in size, teams become larger and methodologies become heavier

• As projects grow in criticality some degree of formality will need to be introduced in parts of the methodology


Crystal principles

• As feedback and communication become more efficient the need for intermediate work products is reduced

• Discipline, skills, and understanding counter process, formality, and documentation

• Team members not on the critical project path can spend their excess time improving the product or helping people who are on the critical path


Crystal

• Incremental development strategy used with 1 to 3 month time lines

• Reflection workshops conducted before project begins, during increment development activity, and after increment is delivered to review the work habits of the team


Feature Driven Development

• Originally proposed by Peter Coad et al

• Practical process model for object-oriented software engineering

• FDD—distinguishing features– Emphasis is on defining “features”

» a feature “is a client-valued function that can be implemented in two weeks or less.”


FDD Philosophy

• Emphasizes collaboration among team members • Manages problem and project complexity using feature-

based decomposition followed integration of software increments

• Technical communication using verbal, graphical, and textual means

• Software quality encouraged by using incremental development, design and code inspections, SQA audits, metric collection, and use of patterns (analysis, design, construction)


FDD Framework Activities

• Develop an Overall Model - contains set of classes depicting business model of application to be built

• Build a Features List - features extracted from domain model, features are categorized and prioritized, work is broken up into two week chunks

• Plan By Feature - features assessed based on priority, effort, technical issues, schedule dependencies


FDD Framework Activities

• Design By Feature - classes relevant to feature are chosen, class and method prologs are written, preliminary design detail developed, owner assigned to each class, owner responsible for maintaining design document for his or her own work packages;

• Build By Feature -- class owner translates design into source code and performs unit testing, integration performed by chief programmer


Feature Driven Development


Agile Modeling

• Practice-based methodology for effective modeling and documentation of software systems in a light-weight manner

• Originally proposed by Scott Ambler

• Modeling is essential for all systems, but that the complexity, type and size of the model must be tuned to the software to be built.


Agile Modeling Principles

• Model with a purpose – have a specific goal

• Use multiple models – that provide value to their intended audience

• Travel light (only keep models with long-term value)

• Content is more important than representation

• Know the models and tools you use to create them

• Adapt locally – the modeling approach should be adapted to the needs of the agile team


Agile Philosophy

• The importance of self-organising teams that have control over the work they perform;

• Communication and collaboration between team members and between practitioners and their customers;

• A recognition that change represents an opportunity;

• Emphasis on rapid delivery of software that satisfies the customer.


Refactoring

• “Behaviour-preserving transformations are known as refactorings. Refactorings are changes whose purpose is to make a program more reusable and easier to understand, rather than to add behaviour” William Opdyke[Opd92].

• XP encourages refactoring – a construction technique that is also a design technique.


Refactoring

• “Refactoring is the process of taking a running program and adding to its value not by changing its behaviour but by giving it more of these qualities that enable us to keep developing at speed” [Kent Beck]. Where the qualities Beck refers to are easy to read and understand code, no duplicate logic, sensible hierarchies.


Refactoring

• Reduce redundancy, eliminate unused functionality and simplify design.

• With appropriate tools, it removes fear of change and builds confidence.

• Nothing is set in stone, refactoring facilitates change.


Refactoring

• It can be considered as a technique that does not produce new functionality but incorporates best practice into evolving designs.

• RF segments development into two stages:

1) adding or designing functions and

2) refactoring those designs.

• RF considers code as a manifestation of design, so code and design are related.


Refactoring

• Making software flexible also makes it more complex. The excessive flexibility may never be used, so another approach is to build a system that meets the immediate requirements and can then be refactored to meet future requirements.

• Design goal is to build systems that can fairly easily be refactored.

• Refactoring can lead to simpler design without sacrificing flexibility.


Refactoring

• A reasonable pool of knowledge exists.

• Many common and well known refactorings are catalogued:

–rename variables, methods;

–extract portions of methods into separate methods;

–push methods, variables up and down the hierarchy;

–move methods into other classes through references.


Refactoring

• Also there is a set of guidelines that help identify when existing design/code need refactoring:

- duplicate code,

- long methods,

- large classes,

- long parameter lists, often called the ‘smell’ of bad code which often indicates bad design.


Refactoring

• RF, like testing, should be part of the process, there is no ‘refactoring stage’

• Guidelines when to refactor:

- when a function is being added.,

- when a bug is fixed,

- after code review (code review help spread knowledge throughout the team).


Refactoring

• Refactoring is a key technique in ‘agile-development’ processes in contrast with complete up front design.

• RF does not replace design, must do some design, as usual ‘no silver bullet’. Trick is to find the correct balance between design and refactoring.


Refactoring

• Large and small scale refactoring (LF,SF) differ in both a technical and management perspective. Technically is not possible (or desirable) to completely refactor a large system.

• From a management point of view LF requires agreement between entire developers and manager.

• Currently most tools and catalogues address SF.


Advantages of RF

• RF permits change of design in the light of experience and in light of changing requirements. Maybe a past decision was wrong and almost certainly there will be new requirements.


• With appropriate tools, it removes fear of change and builds confidence.

• Nothing is set in stone, refactoring facilitates change.


Advantages of RF

• A reasonable pool of knowledge exists.

• The assumption is that each automatic refactoring is safe and fast.


Advantages of RF

• Refactoring is relatively easy and natural in an OO environment like Java.

• You can refactor for performance reason, however RF differs from pure performance enhancement in that is should reduce complexity.

• Without refactoring, as changes are made, the design will decay.

• Helps eliminate duplicate code.


Advantages of RF

• RF makes software easier to understand. If RF is part of the process it could be included as part of the code review.

• The refactoring catalogues can be considered as a set of ‘best practice’

• RF can make design insight more explicit by using the catalogue as design guideline.

• RF can reduce/remove the need to attempt to predict every possible the way in which system might change in the future



Disadvantages of RF

• Systems not just program code a persistent data aspects need to be considered.

• Some refactorings change interface. If the interface is published, then the two interfaces must be maintained.

• Refactoring catalogues and tools maturing, no universally accepted standards.

• Refactoring difficult where code ownership is held by individuals rather team based.


Disadvantages of RF

• Can there be designs that cannot be refactored? Open issue.

• Sometimes the existing code base is so poorly understood or in such a mess that it is better to re-write from scratch rather than refactor.

• Buggy code should not be refactored. Refactoring works on the assumption that the existing system works.


Disadvantages of RF

• When not to refactor:

- if the team/programmer do not understand the technique;

- if the benefits are too vague and long term;

- refactoring is an overhead activity , programmers are paid to write new code;

- refactoring might break a new program.


Disadvantages of RF

• How does refactoring fit in with configuration and change management where several lines and/or versions of software exist.

• Cost of refactoring increases in non-OO systems (e.g. COBOL). Then we must focus more heavily on design prior to coding.

• Fairly new technique, RF needs to be proved on large system in order to gain wider industry support.


Disadvantages of RF

• Refactoring is not a standalone activity, must integrate with process and tools.

• Good software tools are needed for testing, debugging, code management, deployment (e.g. JBuilder, JUnit, Ant, Cactus, VisualWorks, ENVY etc). How will the tools be chosen? Will the tools work well together?


References

• William F. Opdyke. Refactoring Object-Oriented Frameworks. PhD thesis, University of Illinois, 1992. Available at: ftp://st.cs.uiuc.edu/pub/papers/refactoring/opdykethesis.ps.Z

• Extreme Programming Explained: Embrace Change By Kent Beck, ISBN 0201616416, Addison-Wesley 1999


2005F

Instructor

John D Lin - [email protected]

Lectures

Tues & Thurs 10:00-11:30 RM. 100, Lower Block

Documents

Programming Techniques Lecture 3 Agile Development Based on: Software Engineering, A Practitioner’s Approach, 6/e, R.S. Pressman School of Software