The Professional Programmer

What is Perl

The
Professional
Programmer

Dave CrossMagnum Solutions [email protected]

The Professional Programmer

Looking beyond programming skills

What else do you need to be a successful programmer?

The Pragmatic Programmer

The Pragmatic ProgrammerAndrew Hunt

David Thomas

What We Will Cover

Unit testing

Source code control

Continuous integration

Coding Standards

Agile techniques

Unit Testing

Unit Testing

Never program without a safety net

Does your code do what it is supposed to do?

Will your code continue to do what it is supposed to do?

Write unit tests

Run those tests all the time

What is Unit Testing?

Code that exercises an individual element (unit) of your code

Provide known inputs

Look for expected outputs

Bottom-up testing

Typical Unit Tests

Create an object

Call methods with known inputs

Look for expected outputs

Non-OO Unit Tests

Possible to test non-OO code

Call functions rather than methods

Even call programs and examine their outputOutput files

Effect on databases

Etc

Unit Testing Advantages

Confidence in the building blocks of your program

Modular code is easier to test

Well-tested code is good

Easy to test code is better

Unit Testing in Perl

Perl has a culture of unit testing

All Perl modules ship with test suites

Lots of work in this area since 2001

Perl makes it easy to write test suites

Simple Test Program

use Test::More tests => 4;

BEGIN { use_ok('My::Object'); }

ok(my $obj = My::Object->new);

isa_ok($obj, 'My::Object');

$obj->set_foo('Foo');

is($obj->get_foo, 'Foo');

Simple Test Output

$ prove -v test.t
test....
1..4
ok 1 - use My::Object;
ok 2
ok 3 - The object isa My::Object
ok 4
ok
All tests successful.
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.05 cusr 0.00 csys = 0.07 CPU)
Result: PASS

How Good Are Your Tests?

How much of your code is exercised by your tests?

Devel::Cover can help you to find out

Deep internal magic

Draws pretty charts./Build testcover

Devel::Cover Output

Devel::Cover Output

Devel::Cover Output

Other Advantages

A test suite gives you confidence in your code

But it's also a specification of your code

Your code works if it passes the tests

Once the tests pass you can stop writing code

See Test-Driven Development later

Further Information

Perl Testing
A Developer's NotebookIan Langworth

chromatic

Source Code Control

Source Code Control

Source code is a programmer's most valuable asset

If you lose your source code you are in trouble

Ensure your source code is always safe

Other Advantages

Rewind to previous versions of codeEncourages experimentation

Mark particular versions as importantReleased versions

Branching and merging

Brief History

SCCS

RCS

CVS

Subversion

Git

Typical Workflow

Network-available centralised repository of source codeLAN or internet

Developer checks out local copy of code

Makes changesEnsures unit tests still pass

Uploads new versions

Locking

Older systems locked code whilst a developer worked on it

Only one developer working on code at a time

No chance of conflicts

Development bottlenecks

Non-Locking

Optimistic concurrency

Anyone can work on any file at any time

Conflicts detected at check-in

Most of the time people don't work on the same filesOr, at least, on the same parts of the same file

More effective use of time

Branching & Merging

Multiple versions of code available simultaneouslyReleased version

Development version

Experimental version

Merge changes back into older branches

Merging easier with more modern systems

Distributed Version Control

Network model allows many people to work on same codebase

Single master repository on the internet

Need internet access to check in

Distributed model has no master repository

Many equal peers

Check into local repository

Merge with peers when online

Git

Git is currently the most popular source code control system

Written by Linus Torvalds for the Linux kernel

Distributed model

Strong support for branching and merging

GitHub

GitHub offers free Git hosting to Open Source projects

Paid hosting also available

Very popular choice

No excuse for not having public git hosting for your project

Also wiki and ticket tracker

Hooks

Most systems support some kind of hooks

Code that is run when something is checked inOr at other times

Send email for every check-in

Ensure all checked-in code compiles

Run perltidy

Etc...

Conclusions

Source code control is a problem that all programmers have

No excuse for not being familiar with source code control systems

At some point in your career, source code control will save you from an embarrassing mistake

The file you accidentally delete is never under source code control

Continuous Integration

Continuous Integration

Continuously monitoring the health of your codebase

Giving visibility to the changes that are being made

What Does It Do?

Monitors source code repository

Every time it sees a check-in

Run unit tests

Produce a web site

See if your build is broken

Advantages

Instant feedback on the quality of codebase

Instantly know when tests are failing

Instantly know which tests are failing

Instantly know who broke the build

Software

Easy to build something to do this

Perl would be an excellent choice

Plenty of ready-built softwareMuch of it Open Source

Most popular choice is currently CruiseControl

Enhancements

Don't just run tests

Measure test coverage

Generate documentation

Code complexity statistics

Coding standards checksPerl Critic

Etc...

Build Radiator

Standard practice is to build an internal web site

But people have to choose to visit it

Large monitor on the wall of the development office

Rolling dashboard of important stats

No excuse for not seeing the data

Coding Standards

Coding Standards

Perl is a very loosely-structured language

Anything goes

Should probably tighten up on that

perldoc perlstyleHow Larry Wall writes Perl

Perl Best Practices

Perl Best PracticesDamian Conway

Perl Best Practices

Damian Conway's ideas on what makes a good Perl program

Naming conventions

Code layout

Disallowed constructs

Recommended CPAN modules

Disagreeing With PBP

You don't need to agree with everything that Damian says

But you should consider his ideas

And have a good reason for any disagreements

Code Layout

Perl compiler doesn't care about your code layout

But the maintenance programmer will

Make your code as easy as possible to follow

Consistent formatting is a big help

Maintenance Programmers

Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live


Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live


Bear that image in mind whilst writing code

I find it helps me focus

Enforcing Code Layout

Different programmers have different ideas about good code layout

No right answer

Brace positioning

Cuddled else

Size of indent

Maximum line length

perltidy

Perl tidy automatically reformats code according to various rules

Decide on a project standard

Create a perltidy config file with those rules

Everyone can use their own local rules

But run standard config before check-inOr even during check-in

Enforcing Best Practices

Many suggestions in Perl Best Practices are well worth following

Enforce them automatically

Perl::Critic does this

Command line program perlcritic

$ perlcritic your_program.pl

Very depressing output

perlcritic

Five levels of criticism available

Gentle, stern, harsh, cruel, brutal

Gentle is defaultReports only most severe violations

Change that on command line

$ perlcritic brutal your_program.pl

Configuring perlcritic

Not everyone agrees with all perlcritic rules

Configure behaviour with .perlcriticrc

Turn off certain policies

Or invent your ownBased on PPI

Have a project-wide .perlcriticrc

Criticism Statistics

For continuous integration it's useful to get statistics from a file

$ perlcritic statistics your_program.pl

$ perlcritic statistics-only your_program.pl

Criticism Statistics

$ perlcritic --statistics-only lib/Calendar/Simple.pm
1 files.
4 subroutines/methods.
128 statements.
269 lines.

Average McCabe score of subroutines was 8.50.

1 violations.
Violations per file was 1.000.
Violations per statement was 0.008.
Violations per line of code was 0.004.

1 severity 5 violations.

1 violations of BuiltinFunctions::ProhibitStringyEval.

Coding Standards

Many valid options in Perl

Choose one set of options

Enforce with perltidy and perlcritic

Agile Techniques

Systems Analysis and Design

In the bad old days, Systems Analysis and Design was a huge task

Analysts would spend months analysing a problem

Huge specification documents would be written

All before a line of code was written

System Design Diagrams

The specs were always full of diagrams

How the system looks now

How the system should look

Data flow diagrams

Entity-Relationship diagrams

Jackson Structured Programming

The Flaw

This all took too long

Months before spec was complete

Then more time whilst system was coded, tested and deployed

By the time the system was complete, the problem had changed

Back to the drawing board

Agile Programming

Analysis and design needed to be more agile

Able to respond to demands more quickly

Turn around projects in less time

Deliver value sooner

Agile Methodologies

Many lightweight methods arose in mid 1990s

Extreme programming

Scrum

Meeting Utah Feb 2001

Agile Manifesto

Agile Manifesto

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Individual & Interactions

People are more important than processes

Development teams know how to organise their work

Business says what they need

Development team works out how to achieve that

Giving people more control and responsibility

Working Software

Documentation is often out of date before it is finished

Mismatch between business language and technical language

Business often don't know what they want until they see the options

Build prototypes and discuss alternatives

Customer Collaboration

Don't just involve users when the project is specified

Have user representatives as a part of the development team

Instant response to developer questions

More involvement leads to more business ownership of project

Responding to Change

Requirements will change during the project lifetime

Be flexible

Plan for change

Scrum

Common agile process for software development

In use at many companies

Scrum Roles

Scrum Master: manages the process (like project manager)

Product Owner: represents the stakeholders

Team: people responsible for analysis, design, implementation and testing

Stakeholders: people who are requesting the work

Scrum Process

Team decides on the length of a sprint (between one and four weeks)

Stakeholders build up a product backlog

Stakeholders (with product owner) prioritise most important work

Team work out how much they can do in one sprint

Team carry out work

Team demonstrate progress at end of sprint

Scrum Meetings

Sprint planning: start of sprintConverts product backlog to sprint backlog

Daily scrum: short meeting where team members report on progress and plan day

Sprint demo: end of sprintDemo work to stakeholders

Sprint retrospective: end of sprintTeam discusses what went well and what can be improved

Scrum Artifacts

Product backlogLong list of things to do

High level

Sprint backlogMore detailed descriptions of current projects

Broken down to tasks of about a day

Burn down chartLists work planned for current sprint

Current progress plotted

YAGNI

You Ain't Gonna Need It

Do not plan for the future

Only implement what is required now

The simplest thing that could possibly work

Refactor and add functionality later

Refactoring

Changing internal structure without modifying external behaviour

Changes made to make next steps easier

Run test plan

Refactor

Run test plan again

Refactoring

RefactoringMartin Fowler

Test Driven Development

Test plans give you confidence that you code does what it is supposed to do

And that your refactoring hasn't broken anything

They can also act as a specification for a change

If the code passes this test then this feature has been implemented


Run existing test plan

Write test for new feature

Run new test plan

Code/refactor until new tests pass

Run complete test plan


Test-Driven DevelopmentKent Beck

Pair Programming

Code review is useful

Constant code review is better

Two people working together on a problem

Two heads are better than one

Better initial design

Fewer logic and syntax errors

Benefits outweigh costs

Agile Development

Agile methods are becoming more popular

I see many clients using versions of Scrum or Extreme Programming

More common in smaller companies

Larger companies of still prefer their huge specifications

This is changing slowly



A few more things to consider

Operating system

Editor/IDE

Language

Business

That's All Folks

Any Questions?

London Perl Workshop5th December 2009

Technology

The Professional Programmer