What is Perl
The
Professional
Programmer
Dave CrossMagnum Solutions [email protected]
The Professional Programmer
Looking beyond programming skills
What else do you need to be a successful programmer?
The Pragmatic Programmer
The Pragmatic ProgrammerAndrew Hunt
David Thomas
What We Will Cover
Unit testing
Source code control
Continuous integration
Coding Standards
Agile techniques
Unit Testing
Unit Testing
Never program without a safety net
Does your code do what it is supposed to do?
Will your code continue to do what it is supposed to do?
Write unit tests
Run those tests all the time
What is Unit Testing?
Code that exercises an individual element (unit) of your code
Provide known inputs
Look for expected outputs
Bottom-up testing
Typical Unit Tests
Create an object
Call methods with known inputs
Look for expected outputs
Non-OO Unit Tests
Possible to test non-OO code
Call functions rather than methods
Even call programs and examine their outputOutput files
Effect on databases
Etc
Unit Testing Advantages
Confidence in the building blocks of your program
Modular code is easier to test
Well-tested code is good
Easy to test code is better
Unit Testing in Perl
Perl has a culture of unit testing
All Perl modules ship with test suites
Lots of work in this area since 2001
Perl makes it easy to write test suites
Simple Test Program
use Test::More tests => 4;
BEGIN { use_ok('My::Object'); }
ok(my $obj = My::Object->new);
isa_ok($obj, 'My::Object');
$obj->set_foo('Foo');
is($obj->get_foo, 'Foo');
Simple Test Output
$ prove -v test.t
test....
1..4
ok 1 - use My::Object;
ok 2
ok 3 - The object isa My::Object
ok 4
ok
All tests successful.
Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.05 cusr
0.00 csys = 0.07 CPU)
Result: PASS
How Good Are Your Tests?
How much of your code is exercised by your tests?
Devel::Cover can help you to find out
Deep internal magic
Draws pretty charts./Build testcover
Devel::Cover Output
Devel::Cover Output
Devel::Cover Output
Other Advantages
A test suite gives you confidence in your code
But it's also a specification of your code
Your code works if it passes the tests
Once the tests pass you can stop writing code
See Test-Driven Development later
Further Information
Perl Testing
A Developer's NotebookIan Langworth
chromatic
Source Code Control
Source Code Control
Source code is a programmer's most valuable asset
If you lose your source code you are in trouble
Ensure your source code is always safe
Other Advantages
Rewind to previous versions of codeEncourages experimentation
Mark particular versions as importantReleased versions
Branching and merging
Brief History
SCCS
RCS
CVS
Subversion
Git
Typical Workflow
Network-available centralised repository of source codeLAN or internet
Developer checks out local copy of code
Makes changesEnsures unit tests still pass
Uploads new versions
Locking
Older systems locked code whilst a developer worked on it
Only one developer working on code at a time
No chance of conflicts
Development bottlenecks
Non-Locking
Optimistic concurrency
Anyone can work on any file at any time
Conflicts detected at check-in
Most of the time people don't work on the same filesOr, at least, on the same parts of the same file
More effective use of time
Branching & Merging
Multiple versions of code available simultaneouslyReleased version
Development version
Experimental version
Merge changes back into older branches
Merging easier with more modern systems
Distributed Version Control
Network model allows many people to work on same codebase
Single master repository on the internet
Need internet access to check in
Distributed model has no master repository
Many equal peers
Check into local repository
Merge with peers when online
Git
Git is currently the most popular source code control system
Written by Linus Torvalds for the Linux kernel
Distributed model
Strong support for branching and merging
GitHub
GitHub offers free Git hosting to Open Source projects
Paid hosting also available
Very popular choice
No excuse for not having public git hosting for your project
Also wiki and ticket tracker
Hooks
Most systems support some kind of hooks
Code that is run when something is checked inOr at other times
Send email for every check-in
Ensure all checked-in code compiles
Run perltidy
Etc...
Conclusions
Source code control is a problem that all programmers have
No excuse for not being familiar with source code control systems
At some point in your career, source code control will save you from an embarrassing mistake
The file you accidentally delete is never under source code control
Continuous Integration
Continuous Integration
Continuously monitoring the health of your codebase
Giving visibility to the changes that are being made
What Does It Do?
Monitors source code repository
Every time it sees a check-in
Run unit tests
Produce a web site
See if your build is broken
Advantages
Instant feedback on the quality of codebase
Instantly know when tests are failing
Instantly know which tests are failing
Instantly know who broke the build
Software
Easy to build something to do this
Perl would be an excellent choice
Plenty of ready-built softwareMuch of it Open Source
Most popular choice is currently CruiseControl
Enhancements
Don't just run tests
Measure test coverage
Generate documentation
Code complexity statistics
Coding standards checksPerl Critic
Etc...
Build Radiator
Standard practice is to build an internal web site
But people have to choose to visit it
Large monitor on the wall of the development office
Rolling dashboard of important stats
No excuse for not seeing the data
Coding Standards
Coding Standards
Perl is a very loosely-structured language
Anything goes
Should probably tighten up on that
perldoc perlstyleHow Larry Wall writes Perl
Perl Best Practices
Perl Best PracticesDamian Conway
Perl Best Practices
Damian Conway's ideas on what makes a good Perl program
Naming conventions
Code layout
Disallowed constructs
Recommended CPAN modules
Disagreeing With PBP
You don't need to agree with everything that Damian says
But you should consider his ideas
And have a good reason for any disagreements
Code Layout
Perl compiler doesn't care about your code layout
But the maintenance programmer will
Make your code as easy as possible to follow
Consistent formatting is a big help
Maintenance Programmers
Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live
Maintenance Programmers
Always write code as though it will be maintained by a homicidal, axe-wielding maniac who knows where you live
Maintenance Programmers
Bear that image in mind whilst writing code
I find it helps me focus
Enforcing Code Layout
Different programmers have different ideas about good code layout
No right answer
Brace positioning
Cuddled else
Size of indent
Maximum line length
perltidy
Perl tidy automatically reformats code according to various rules
Decide on a project standard
Create a perltidy config file with those rules
Everyone can use their own local rules
But run standard config before check-inOr even during check-in
Enforcing Best Practices
Many suggestions in Perl Best Practices are well worth following
Enforce them automatically
Perl::Critic does this
Command line program perlcritic
$ perlcritic your_program.pl
Very depressing output
perlcritic
Five levels of criticism available
Gentle, stern, harsh, cruel, brutal
Gentle is defaultReports only most severe violations
Change that on command line
$ perlcritic brutal your_program.pl
Configuring perlcritic
Not everyone agrees with all perlcritic rules
Configure behaviour with .perlcriticrc
Turn off certain policies
Or invent your ownBased on PPI
Have a project-wide .perlcriticrc
Criticism Statistics
For continuous integration it's useful to get statistics from a file
$ perlcritic statistics your_program.pl
$ perlcritic statistics-only your_program.pl
Criticism Statistics
$ perlcritic --statistics-only lib/Calendar/Simple.pm
1 files.
4 subroutines/methods.
128 statements.
269 lines.
Average McCabe score of subroutines was 8.50.
1 violations.
Violations per file was 1.000.
Violations per statement was 0.008.
Violations per line of code was 0.004.
1 severity 5 violations.
1 violations of BuiltinFunctions::ProhibitStringyEval.
Coding Standards
Many valid options in Perl
Choose one set of options
Enforce with perltidy and perlcritic
Agile Techniques
Systems Analysis and Design
In the bad old days, Systems Analysis and Design was a huge task
Analysts would spend months analysing a problem
Huge specification documents would be written
All before a line of code was written
System Design Diagrams
The specs were always full of diagrams
How the system looks now
How the system should look
Data flow diagrams
Entity-Relationship diagrams
Jackson Structured Programming
The Flaw
This all took too long
Months before spec was complete
Then more time whilst system was coded, tested and deployed
By the time the system was complete, the problem had changed
Back to the drawing board
Agile Programming
Analysis and design needed to be more agile
Able to respond to demands more quickly
Turn around projects in less time
Deliver value sooner
Agile Methodologies
Many lightweight methods arose in mid 1990s
Extreme programming
Scrum
Meeting Utah Feb 2001
Agile Manifesto
Agile Manifesto
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Individual & Interactions
People are more important than processes
Development teams know how to organise their work
Business says what they need
Development team works out how to achieve that
Giving people more control and responsibility
Working Software
Documentation is often out of date before it is finished
Mismatch between business language and technical language
Business often don't know what they want until they see the options
Build prototypes and discuss alternatives
Customer Collaboration
Don't just involve users when the project is specified
Have user representatives as a part of the development team
Instant response to developer questions
More involvement leads to more business ownership of project
Responding to Change
Requirements will change during the project lifetime
Be flexible
Plan for change
Scrum
Common agile process for software development
In use at many companies
Scrum Roles
Scrum Master: manages the process (like project manager)
Product Owner: represents the stakeholders
Team: people responsible for analysis, design, implementation and testing
Stakeholders: people who are requesting the work
Scrum Process
Team decides on the length of a sprint (between one and four weeks)
Stakeholders build up a product backlog
Stakeholders (with product owner) prioritise most important work
Team work out how much they can do in one sprint
Team carry out work
Team demonstrate progress at end of sprint
Scrum Meetings
Sprint planning: start of sprintConverts product backlog to sprint backlog
Daily scrum: short meeting where team members report on progress and plan day
Sprint demo: end of sprintDemo work to stakeholders
Sprint retrospective: end of sprintTeam discusses what went well and what can be improved
Scrum Artifacts
Product backlogLong list of things to do
High level
Sprint backlogMore detailed descriptions of current projects
Broken down to tasks of about a day
Burn down chartLists work planned for current sprint
Current progress plotted
YAGNI
You Ain't Gonna Need It
Do not plan for the future
Only implement what is required now
The simplest thing that could possibly work
Refactor and add functionality later
Refactoring
Changing internal structure without modifying external behaviour
Changes made to make next steps easier
Run test plan
Refactor
Run test plan again
Refactoring
RefactoringMartin Fowler
Test Driven Development
Test plans give you confidence that you code does what it is supposed to do
And that your refactoring hasn't broken anything
They can also act as a specification for a change
If the code passes this test then this feature has been implemented
Test Driven Development
Run existing test plan
Write test for new feature
Run new test plan
Code/refactor until new tests pass
Run complete test plan
Test Driven Development
Test-Driven DevelopmentKent Beck
Pair Programming
Code review is useful
Constant code review is better
Two people working together on a problem
Two heads are better than one
Better initial design
Fewer logic and syntax errors
Benefits outweigh costs
Agile Development
Agile methods are becoming more popular
I see many clients using versions of Scrum or Extreme Programming
More common in smaller companies
Larger companies of still prefer their huge specifications
This is changing slowly
The Professional Programmer
The Professional Programmer
A few more things to consider
Operating system
Editor/IDE
Language
Business
That's All Folks
Any Questions?
London Perl Workshop5th December 2009