Aust. N. Z. J. Stat. 2012 doi: 10.1111/j.1467-842X.2012.00653.x

BOOK REVIEW

Permutation Tests for Complex Data: Theory, Applications and Software. By F. Pesarin andL. Salmaso. Chichester: Wiley. 2010. 448 pages. AU$150.00 (hardback). ISBN: 978-0-470-51641-6

The idea of permutation testing is that, under a certain null hypothesis, trial residuals are exchangeable,either as a whole or within various ‘exchangeable subsets’. Then the null distribution of any plausibletest statistic, conditional on the set of residual values, can be enumerated by carrying out permutationsof residuals within exchangeable subsets. Alternatively, convenient normal approximations to this nullpermutation distribution are readily available. The technique is conceptually simple and can be appliedto a wide variety of standard and unusual statistical problems.

An early indication of the importance of permutation testing was the understanding that theclassical F-tests of ANOVA are approximations to exact permutation tests.

With this background, any book on permutation testing should be useful, and welcome. However,reading the present book raises a number of concerns. One of them is a kind of over-arching question,which takes a little explaining.

In multiple regression with covariates whose regression parameters are not of interest – nuisanceparameters – the presence of the covariates ‘gets in the way’ of the permutation argument, unlessthere are exchangeable subsets, as will often occur in designed experiments. This ‘nuisance parameterproblem’ – a significant drawback to the wider applicability of permutation testing – is not addressedin the present book, which concentrates on multivariate data and ‘multiple tests’ of the form H0 : θi =0, i = 1, 2, . . . , p, against general alternatives. The general nuisance parameter problem appears to bedeliberately avoided. Or is it?

A feature of the book is the so-called NPC (non-parametric combination) methodology, claimedto combine p tests of the form H0i : θi = 0, a kind of generalization of multiple comparisons. But suchtests clearly involve nuisance parameters, so how can the nuisance parameter problem be avoided?However, it appears that the p tests are not as claimed, but all have the same H0, and differ in being testsagainst different alternatives H1i : θi �= 0, other θ j = 0. So no nuisance parameters are involved, buthow useful are these tests, and how can they claim to be a useful generalization of multiple comparisons?

Fast forward to near the end of the book. Here, in a final case study example involving logisticregression, the claimed NPC P-values are systematically smaller than the corresponding ones in con-ventional logistic regression, which are tests of H0i . Most statisticians would claim that the conventionallogistic regression tests are useful, but the NPC tests are not. So the material is not incorrect, but is ofdoubtful statistical relevance. At the very least, the statistical meaning of the procedures needs to beexamined.

The lack of clarity that the reader encounters is exacerbated by some other features. The authorsare involved in two other recent books (Pesarin 2001 and Basso et al. 2009) on permutation testing.It would be useful to know how the content of the present book relates to those books, but this is notdiscussed.

The concentration on multivariate data and multiple tests makes the material complex, inevitably.Understanding the detail is made no easier by the practice of using the most abstract possible mathemati-cal notation, with levels of abstraction well beyond what is needed for specific applications. Line-by-linethe text is hard to follow, for the writing style is highly discursive and wordy, with an unsettling indi-rectness. Definitions and explanations often begin obliquely, meander, and fade away before resolution.Often, the effort needed to elicit meaning is far more than is reasonable.

There are interesting points made from time to time, usually without elaboration or illustration.Indeed, the examples presented are scarce compared with the density of opaque theory. However, thebook may appeal to some readers interested in permutation testing who have a patient frame of mindand no shortage of time.

C© 2012 Australian Statistical Publishing Association Inc. Published by Blackwell Publishing Asia Pty Ltd.

Australian & New Zealand Journal of Statistics

2 BOOK REVIEW

Positive aspects are the inclusion of software, and some topics whose presence alone indicates abroad horizon for permutation testing, such as missing data, survival analysis, and isotonic regression.

BRUCE BROWN

University of New South Wales, Australiae-mail: bruce.brown@unsw.edu.au

References

PESARIN, F. (2001). Multivariate Permutation Tests: with applications in Biostatistics. Chichester: JohnWiley.BASSO, D., PESARIN, F., SALMASO, L. & SOLARI, A. (2009). Permutation Tests for Stochastic Ordering and

ANOVA: Theory and Applications in R. New York: Springer.

C© 2012 Australian Statistical Publishing Association Inc.