Portable perl How to write code that will work everywhere

Portable perl

How to write code that will work everywhere

Some common fallacies

• Perl is portable, my code is written in perl

– Therefore my code is portable.



– Therefore my code is portable. • WRONG!



– Therefore my code is portable.• WRONG!

• My code uses no XS, calls no XS,

– Therefore portable.



– Therefore my code is portable.• WRONG!

• My code uses no XS, calls no XS,

– Therefore portable.• WRONG!

Who cares about portability

• Portability is only an issue if you genuinely want your code to work everywhere.

• Portability is not a requirement for ad-hoc scripts written for a single environment

Who cares about portability

• Portability is only an issue if you genuinely want your code to work everywhere.

• Portability is not a requirement for ad-hoc scripts written for a single environment

• But… Think of CPAN.– You publish your module– You want it to work on as many platforms as

possible

Nearly all of Perl is portable

• Perl tries to bridge the gaps between different platforms

• It is usually worth the mileage to make modules portable.

• Module authors should be interested in making/keeping their modules portable

Operating system platforms

• Unix


• Unix

• MS Win32


• Unix

• MS Win32

• Apple Mac


• Unix

• MS Win32

• Apple Mac

• VMS

• Other (see perldoc perlport for the full list)

What are the main issues

• File names and paths

• Shell commands

• Environment variables

• User interaction

• Communications

• Multitasking

• Internationalisation

File names

• Unix– /home/me/my_dir/my_file

• Windows– C:\My documents\my_dir\My file.txt

• VMS– MY_DEVICE:[ME.MY_DIR]MYFILE.TXT;1

POSIX filenames

• A platform independent standard/foo/bar/module.pm

lib/Foo/Bar/module.pm

POSIX filenames

• A platform independent standard/foo/bar/module.pm

lib/Foo/Bar/module.pm

• Works on Windows:C:/perl/lib/test.pm

• Even works on VMS/my_device/me/my_dir/myfile.txt

Problems with POSIX filenames

• No provision for a 'volume' or 'device' – (in Unix terminology, a mount point)

• Variations in character set– E.g. Windows allows spaces– Is underscore allowed? Dash? Dollar? More than 1 dot?

Punctuation characters?

• Case sensitivity– On Unix, FOO, Foo and foo are different files– Accented letters

• Problems if you mix native and POSIX syntaxC:/mydir\myfile.txt OK sometimesMYDEV:[MYDIR]SUBDIR/FILE.TXT NO!!

The alternative

• Use native syntax throughout

• Test $Ô for the operating system• Do what is necessary to construct a filename

The alternative

• Use native syntax throughout

• Test $Ô for the operating system• Do what is necessary to construct a filename

• Fortunately, this has been done for you

use File::Spec;

my $file = File::Spec->catfile( qw( mydev mydir file.txt));

File::Spec and File::Spec::Functions

• File::Spec provides a class methods API

• File::Spec::Functions provides equivalent functions for export.

• The following are exported by default:– canonpath catdir catfile curdir rootdir updir

no_upwards file_name_is_absolute path

• There are also other functions available:– devnull tmpdir splitpath splitdir catpath abs2rel

rel2abs case_tolerant

A word of warning

• File::Spec merely manipulates strings– curdir returns ‘.’ on Unix– canonpath merely removes /. and /foo/.. etc.

• use Cwd to handle real paths– cwd gives you what curdir really is– abs_path does what you would expect

canonpath to do

Other things to watch out for with files

• Symbolic links– Are Unix specific– Many O/S don't even support hard links

• Security and permissions– (rwx rwx rwx) is a Unix convention– Some O/S don’t have the same concept of a

user and a group– VMS ends directories with a .DIR extension

Calling the shell

• Don't do it!– (if you want your code to be portable)

• Thus, avoid pipe files, backticks and system– Unless providing a mechanism for your

interactive user to type in commands.

• If you absolutely must– Stick to commands with a common syntax– Test $Ô and provide variants

Shell globbing

• Not all command interpreters expand wildcards– (i.e. not all shells glob). Unix shells do glob.

perl myprog.pl *.txt

• Perl on MS-DOS does not glob• Perl on VMS does glob!

– Though the VMS command line interpreter does not

Environment Variables

• Unix scripters are used to having standard ones:• HOME TERM SHELL USER etc.• These are all Unix specific.• $ENV{PATH} is common to most O/S's

– But not all, VMS for example.

– VMS does not need PATH, as you are required to enter the command "RUN" in order to run a program, or @ for a script.

Interacting with the user

Problem:

• You really need to talk to the user.

• stdin, stdout and stderr could be redirected

• You want to prompt him for a password.

• You don't want to proceed until you have received the password.

Non-portable solutionopen TTYIN,'/dev/tty' or die "Failed to open terminal";open TTYOUT,'>/dev/tty' or die "Failed to open terminal";

# Autoflush on for TTYOUTselect TTYOUT;$| = 1;

# Echo offsystem "stty -echo";print "Enter password:";my $pass = <TTYIN>;chomp $pass;

# Echo on againsystem "stty echo";

validate_pass($pass);

What was non-portable?

• Use of "system" to control echoing

• /dev/tty - the terminal will not be called this

• A minor point: echoing might already be off

A better way

• Refer to perlfaq8:– How do I ask the user for a password?

• Unfortunately, the POD for Term::ReadKey says:

– ReadLine MODE [, Filehandle]• This call is currently not available under Windows.

use Term::ReadKey;

ReadMode('noecho'); my $password = ReadLine(0);

Combining Term::ReadKey and Term::ReadLine

use Term::ReadKey; use Term::ReadLine;

my $term = Term::ReadLine->new;ReadMode('noecho'); my $password = $term->readline ("Enter Password:");ReadMode('restore');

Other nice features of Term::ReadLine

• Allows command recall from the history if the O/S supports it

• Can operate inside Tk

Communications

• File sharing with a different architecture

• Exchanging packets over a network

Communications

• File sharing with a different architecture

• Exchanging packets over a network

Look to standards• Each side changes the data to comply to the

standard format

Line termination

• How exactly is "\n" stored or transmitted?– On Unix, a single ASCII \012 (line feed)– On Windows, \015 \012 (carriage return, line feed)

– On Mac OS, just plain \015 (carriage return)

Line termination

• How exactly is "\n" stored or transmitted?– On Unix, a single ASCII \012 (line feed)– On Windows, \015 \012 (carriage return, line feed)

– On Mac OS, just plain \015 (carriage return)• This means that "\n" can stand for two characters

– This is why chomp is better than chop

• This is important when reading foreign files• This is also important when sending and receiving

packets

Line termination

• Remember: "\n" ne "\012"

• perldoc perlport goes into more detail.

| Unix | DOS | Mac | --------------------------- \n | LF | LF | CR | \r | CR | CR | LF | \n * | LF | CRLF | CR | \r * | CR | CR | LF | --------------------------- * text-mode STDIO

Use binmode on non-standard files(i.e. other than local text files)

• binmode makes no difference on Unix ASCII files.

• Other O/S need to differentiate between text files and binary files.

• binmode is needed for UTF-8 locale differences

• "seek" and "tell" will produce spurious results if binmode is not correctly set.

Number representation

• You will often be unable to read another system's binary numbers– Differing endianness– Different floating point representations

• Use a "network" standard– pack and unpack format 'n' and 'N'– IEEE floats are standard and portable

Multitasking

• Beware forks or threads– If you want to be portable, you are best off avoiding

fork altogether.

– But you can probably use a multitasking module if it has been written with portability in mind.

– Threads support is far from brilliant on many platforms.

• Not all O/S are that good at non-blocking I/O

Portability and XS modules

• For the module writer, this is a question of writing portable C code, not portable perl.

• XS code can be a pain for the installer as it needs a C compiler.

• Not only does it need a C compiler; it needs the same one (and the same options) that the perl you are running was built with.

Choices for the perl admin

• Download and install a pre-built binary

• Roll your own

Choices for the perl admin

• Download and install a pre-built binary• Roll your own

• Rolling your own is usually better, but you always need a working ANSI C compiler

• Even with pre-built binaries, you need a C compiler to extend perl (i.e. add XS modules) - the same one perl was built with.

Choices for Win32

• Download ActiveState perl– Module binaries available through ppm

– MVC can be used to build CPAN modules

• Roll your own with MVC and DOS– Good luck!

– But it's do-able. Use nmake to make

• Roll your own with Cygwin and MingW– You might need to spend time getting the C compiler to

work first

Choices for most Unix O/S

• O/S may come with perl pre-installed– But it's probably a really old version

• Roll your own, to get the perl build you want– You may need to build and install gcc first– Or pay for a licensed ANSI C compiler

C compilers on Unix

• HP-UX: /bin/cc not suitable for much apart from building kernels. gcc works fine.

• Solaris: native cc is fine. Pre-installed perls were built with gcc though.

C compiler for VMS

• Buy a licence for DECC - tough.

• Get a test drive account and do all your building there, for free :)

• gcc not recommended on this platform according to the README.VMS

Reporting portability problems in other people's code

• Core modules (are meant to work everywhere)– Port problems in the core are bugs– Report to p5p via perlbug– Discuss on platform mailing lists e.g. vmsperl

• Other CPAN modules– Consider whether the module was intended to be

portable– Check reports from the CPAN testers– Use RT (rt.cpan.org)– Or mail the module author

Documents

Portable perl How to write code that will work everywhere