rev-panu

8/6/2019 rev-panu

1/5

992 NOTICES OF THE AMS VOLUME 48, NUMBER 9

Book Review

Exploring RandomnessandThe UnknowableReviewed by Panu Raatikainen

Exploring Randomness

Gregory ChaitinSpringer-Verlag, 2000ISBN 1-85233-417-7176 pages, $34.95

The UnknowableGregory ChaitinSpringer-Verlag, 1999ISBN 9-814-02172-5

122 pages, $29.00In the early twentieth century two extremely in-

fluential research programs aimed to establishsolid foundations for mathematics with the helpof new formal logic. The logicism of Gottlob Fregeand Bertrand Russell claimed that all mathemat-ics can be shown to be reducible to logic. DavidHilbert and his school in turn intended to demon-strate, using logical formalization, that the use ofinfinistic, set-theoretical methods in mathemat-icsviewed with suspicion by manycan neverlead to finitistically meaningful but false state-ments and is thus safe. This came to be known asHilberts program.

These grand aims were shown to be impossibleby applying the exact methods of logic to itself:the limitative results of Kurt Gdel, AlonzoChurch, and Alan Turing in the 1930s revolution-ized the whole understanding of logic andmathematics (the key papers are reprinted in [5]).

What Gdel proved in 1931 is that in any finitelypresented system of mathematical axioms there aresentences that are true but that cannot be proved

to be true in the system. Church showed in 1936that there is no general mechanical method fordeciding whether a given sentence is logically valid

or not and, similarly, that there is no method fordeciding whether a given sentence is a theorem ofa given axiomatized mathematical theory. Such an

impossibility proof required an exact mathemati-cal substitute for the informal, intuitive notion ofa mechanical procedure; Church used his own -

definable functions. Turing arrived independentlyat the same results at the same time. Moreover, hegave a superior philosophical explication of the con-

cept of mechanical procedure in terms of abstractimaginary machines, known today as Turing ma-chines; this advance made it possible to prove ab-

solute unsolvability results and to develop Gdelsincompleteness theorem in its full generality. Thisidentification of the intuitive notion of mechanical

method and an exact mathematical notion is usu-ally called Churchs thesis or, more properly, theChurch-Turing thesis. It is the fundamental basis

of all proofs of absolute unsolvability.One of the greatest achievements of modern

mathematical logic was certainly the proof by Yuri

Matiyasevich in 1970, based on earlier work byJulia Robinson, Martin Davis, and Hilary Putnam,that the tenth problem of Hilberts famous list of

open mathematical problems from 1900 is in factunsolvable; i.e., there is no general method for de-ciding whether a given Diophantine equation has

a solution or not [11]. This result implies that inany axiomatized theory there exist Diophantine

Panu Raatikainen is a fellow in the Helsinki Collegium for

Advanced Study and a docent of theoretical philosophyat the University of Helsinki. His e-mail address is

[email protected] .

8/6/2019 rev-panu

2/5

OCTOBER 2001 NOTICES OF THE AMS 993

equations that have no solution but cannot beproved in the theory to have no solution.

However, it is now a widespread view, espe-

cially in computer science circles, that certainvariants of incompleteness and unsolvabilityresults by the American computer scientist GregoryChaitin are the last word in this field. These vari-

ants are claimed to both explain the true reasonfor Gdels incompleteness theorem and to be theultimate, or the strongest possible, incomplete-ness results. Chaitins results emerge from thetheory of algorithmic complexity or program-size

complexity (also known as Kolmogorov com-plexity); Chaitin himself was, in fact, one of thefounders of the theory.

The classical work on unsolvability dealt solelywith solvability in principle: one abstracted from

the practical limits of space and time and requiredonly finiteness. From the late 1950s onward, how-ever, more and more attention has been paid to dif-ferent kinds of complexity questionsat least in

part because of the emergence of computing ma-chines and the practical resource problems that ac-companied them. In logic and computer sciencevarious different notions of complexity have beenstudied intensively. First, computational complex-

itymeasures the complexity of a problem in termsof resources, such as space and time, required tosolve the problem relative to a given machinemodel of computation. Second, descriptive com-

plexityanalyzes the complexity of a problem interms of logical resources, such as the number ofvariables, the kinds of quantifiers, or the length ofa formula required to define the problem. And fi-nally, by the algorithmic complexity, or the pro-

gram-size complexity(or Kolmogorov complexity),of a number or a string, one means the size of theshortest program that computes as output thatnumber or string.

Theory of Algorithmic Complexity

The basic idea of the theory of algorithmic com-plexity was suggested in the 1960s independentlyby Ray J. Solomonoff, Andrei N. Kolmogorov, andGregory Chaitin. Solomonoff used it in his com-putational approach to scientific inference,

Kolmogorov aimed initially to give a satisfactorydefinition for the problematic notion of a random

sequence in probability theory, and Chaitinfirst studied the program-size complexity ofTuring machines for its own sake. Kolmogorov

went on to suggest that this notion also providesa good explication of the concept of the informa-tion content of a string of symbols. Later Chaitinfollowed him in this interpretation. Consequently,

the name algorithmic information content hasfrequently been used for program-size complex-ity, and the whole field of study is very oftencalled algorithmic information theory ([10] is

a comprehensive survey of the theory and itsapplications).

Chaitin was active in developing this approachinto a systematic theory (although one shouldnot ignore the important contributions by manyothers). From the 1970s onwards Chaitins inter-est has focused more and more on incompleteness

and unsolvability phenomena related to the notionof program-size complexity. Indeed, he nowsays that the most fundamental application ofthe theory is in the new light that it sheds onthe incompleteness phenomenon (The Unknow-able, pp. 867).

It was known from the beginning that program-size complexity is unsolvable. Chaitin, however,made in the early 1970s an interesting observation:Although there are strings with arbitrarily largeprogram-size complexity, for any mathematicalaxiom system there is a finite limit c such that inthat system one cannot prove that any particularstring has a program-size complexity larger than

c [1]. Later Chaitin attempted to extend his in-formation-theoretic approach to incompletenesstheorems in order to obtain the strongest possi-ble version of Gdels incompleteness theorem ([3],p. v). For this purpose he has defined a specificinfinite random sequence .

As was noted, one of the major sources thatoriginally motivated the development of thetheory of program-size complexity, especially inKolmogorovs case, was a problem in the theoryof probability, viz. that of giving a precise andplausible definition for the notion of a randomstring. The problem is related to the paradox ofrandomness, which may be explained as follows:

Assume we are given two binary strings of 20digits each, and we are informed that they wereboth obtained by flipping a coin. Let these twostrings be:

x = 00000000000000000000

and

y = 01001110100111101000.

Now according to the standard theory ofprobability, these strings are equally probable.And yet intuitively one tends to think that xcannot possibly be a randomly generated string

there is too much regularity in itwhereas yappears to be genuinely irregular and randomand may well be the result of a toss of a coin. Thealgorithmic theory of randomness explicates thisidea of regularity with the help of Turing machineprograms. One considers a finite string as regular,or nonrandom, if it can be generated by a simpleprogram, i.e., if its program-size complexity isconsiderably smaller than its length. Accordingly,a finite string is defined to be random if itsprogram-size complexity is roughly equal to its

8/6/2019 rev-panu

3/5


length, i.e., if it cannot be compressed to a shorterprogram. (Note that this notion is relative to achosen programming language or coding system;a finite string may be random in one but nonran-dom in another.)

Extending this approach to infinite stringsturned out to be, however, more difficult than was

thought. Kolmogorovs firstidea was that an infinite stringbe considered random if all ofits finite initial segments arerandom. But Per Martin-Lfshowed that this definition doesnot work and then gave a moresatisfactory definition in mea-sure-theoretic terms. In 1975Chaitin presented a definition(which is equivalent to Martin-Lfs definition) in terms ofprogram-size complexity: Aninfinite string is defined to be

random if the program-sizecomplexity of an initial segmentof length n does not drop arbi-trarily far below n [2]. Oneshould add that it is not indis-putable that this really providesin all respects an unproblematic

explication of the notion of randomness.Also in 1975 Chaitin presented for the first

time his (since then much-advertised) number:it is the halting probability of the universalTuring machine U, i.e., the probability that Uhaltswhen its binary input is chosen randomly bit bybit, such as by flipping a coin. The infinite string

is, according to the above definition, random [2].Somewhat analogously to his earlier incomplete-ness result, Chaitin has demonstrated that noaxiomatic mathematical theory enables one todetermine infinitely many digits of ([3], [4];cf. [6], [8]).

Chaitins New Approach via LISP Programs

This is the general theoretical background of thebooks under review. How do these two new booksby Chaitin relate to these older works? In terms ofresults there is hardly anything new compared tothe older work by Chaitin and others reviewedabove. Rather, these books aim to popularize that

work. In The Unknowable the emphasis is on theincompleteness phenomena related to program-size complexity. Exploring Randomness aims toexplain the program-size complexity approachto randomness.

What is new is Chaitins approach via LISP-programming; he has translated his ownearlier work, which was in terms of abstract,idealized Turing machines, into actual programsin the LISP computer language. Well, not exactly:according to Chaitin, no existing programming

language provides exactly what is needed, so heinvented a new version of LISP. Chaitin begins TheUnknowableby saying that what is new in the bookis the following: I compare and contrast Gdels,Turings and my work in a very simple and straight-forward manner using LISP (p. v). According toChaitin this book is a prequel to his previous

book, The Limits of Mathematics(Springer-Verlag,1998), and is an easier introduction to his workon incompleteness. In Exploring RandomnessChaitin in turn writes that [t]he purpose of thisbook is to show how to program the proofs of themain theorems about program-size complexity,so that we can see the algorithms in these proofsrunning on the computer (p. 29).

Such an approach may be attractive for pro-gramming enthusiasts and engineers, but for therest of us its value is less clear. It is quite doubt-ful whether it manages to increase the under-standing of the basic notions and results andwhether it really makes the fundamental issues,

which are rather theoretical and conceptual, moreaccessible. All these can be, and have been, ex-plained quite easily and elegantly in terms of sim-ply describable Turing machines, and it is ques-tionable that it is really easier to understand themby first learning Chaitins specially modified LISPand then programming them. And after all, the keypoint here is that there is no program for decid-ing the basic propertiesthat they are not pro-grammable. In The Unknowable (p. 27) Chaitinsays that [r]eaders who hate computer program-ming should skip directly to Chapter VIthat is,should skip half of the book (and similarly with Ex-ploring Randomness). What is left is two popular

surveys of the field.What is totally missing from Chaitins accounts

is the link between his particular programminglanguage and the intuitive notion of mechanicalprocedure, that is, an analogue of the Church-Turing thesis. Turings conceptual analysis of whata mechanical procedure is and the resultingChurch-Turing thesis are indispensable for theproper understanding of the fundamentalunsolvability results: only the thesis gives themtheir absolute character (in contradistinction tounsolvability by some fixed, restricted methods).Consequently, without some extra knowledge,the theoretical relevance of the limitations of the

LISP programs that Chaitin demonstrates mayremain unclear to the reader: one may wonderwhether perhaps some other programminglanguage would do better. This is a seriousweakness of these presentations as first intro-ductions to the unsolvability phenomena.

Problematic Philosophical Conclusions

The most controversial parts of Chaitins workare certainly the highly ambitious philosophicalconclusions he has drawn from his mathematical

8/6/2019 rev-panu

4/5

OCTOBER 2001 NOTICES OF THE AMS 995

work. Recall that according to Chaitin the most

fundamental application of the theory is in thenew light that it sheds on the incompleteness

phenomenon. He writes: Gdel and Turing wereonly the tip of the iceberg. AIT [algorithmic infor-

mation theory] provides a much deeper analysisof the limits of the formal axiomatic method. It

provides a deeper source of incompleteness, amore natural explanation for the reason that nofinite set of axioms is complete (Exploring

Randomness, p. 163).But why does Chaitin think so? It is because

he interprets his own variants of incompletenesstheorems as follows: The general flavor of my

work is like this. You compare the complexity of

the axioms with the complexity of the result youretrying to derive, and if the result is more complex

than the axioms, then you cant get it from thoseaxioms (The Unknowable, p. 24). Or, in other

words: my approach makes incompleteness more

natural, because you see how what you can do

depends on the axioms. The more complex theaxioms, the better you can do (The Unknowable,p. 26).

But appearances notwithstanding, this is sim-ply wrong. In fact, there is no direct dependence

between the complexity of an axiom system and

its power to prove theorems. On the one hand,there are extremely complex systems of axioms

that are very weak and enable one to prove onlytrivial theorems. Consider, for example, an enor-

mously complex finite collection of axioms with theform n < n + 1; even the simple theory consisting

of the single generalization for all x, x < x + 1 canprove more. On the other hand, there exist very sim-

ple and compact axiom systems that are

sufficient for the development of all knownmathematics (e.g., the axioms of set theory) and

that can in particular decide many more cases ofprogram-size complexity than some extremely

complex but weak axiom systems (such as the oneabove). Moreover, it is possible for two theories to

differ considerably in strength or complexity but

nevertheless be able to decide exactly the samefacts about program-size complexity and have the

same Chaitinian finite limit c [12]. Analogously,Chaitins claim with respect to that an N-bit

formal axiomatic system can determine at most

Nbits of (The Unknowable, p. 90) is again nottrue for related reasons [13].

It has been shown conclusively (see [9], [12], [13])that Chaitins philosophical interpretations of his

work are unfounded and false; they are based onvarious fatal confusions. And thus we have all the

more reason for doubting the claim that hisapproach can explain the true source of the

incompleteness and unsolvability theorems. As

his philosophical interpretations fall, so does thisclaim. Chaitins findings are not without interest,

but their relevance for the foundations of mathe-matics has been greatly exaggerated.

Further, Chaitin has often stated that he hasshown that mathematical truth is random: But thebits of this number , whether theyre 0 or 1, aremathematical truths that are true by accident!theyre true by no reasonthere is no reason that

individual bits are 0 or 1! (Ex-ploring Randomness, pp. 234)This is false too. The individualbits of are 0 or 1 dependingon whether certain Turing ma-chines halt or notthat is thereason. It is an objective matterof fact; the truth here is com-pletely determined, andChaitins interpretation of thesituation is quite misleading.

Chaitin also claims that ismaximally unknowable andthat in his setting one gets in-

completeness and unsolvabilityin the worst possible way (Ex-ploring Randomness, p. 19). Butcontrary to what Chaitins owninterpretations suggest, his re-sults can in fact be derived asquite easy corollaries of Tur-ings classical unsolvability result and are not es-sentially stronger than it. Moreover, there are manyincompleteness and unsolvability results in theliterature of mathematical logic that are in variousways stronger than Chaitins results; many of themalso have a much more natural mathematicalcontent [13].

Chaitin, however, seems to be quite indifferentto all such criticism. Instead of trying to seriouslyanswer it in any way, he sweeps all such problemsunder the carpet with rather cheap rhetoric: AITis tremendously revolutionary; it is a major para-digm shift, which is why so many people find thephilosophical conclusions that I draw from mytheory to be either incomprehensible or unpalat-able (Exploring Randomness, p. 161). Or: AIT isa drastic paradigm shift, and as such, obeys MaxPlancks dictum that major new scientific ideasnever convince their opponents, but instead areadopted naturally by a new generation that growsup with them and takes them for granted and that

have no personal stake nor have built careers onolder, obsolete viewpoints (Exploring Random-ness, p. 163).

But wouldnt it be conceivable that the true rea-son for some resistance is not dogmatic prejudicebut that his conclusions are untenable becausethey are very weakly justified and even contradictvarious logico-mathematical facts? Creative andoriginal as Chaitin has been, it is sometimes quitedisturbing that he seems totally ignorant of largeparts of mathematical logic relevant to the issues

8/6/2019 rev-panu

5/5


he is dealing with. It is regrettable that Chaitin does

not respond to criticism of his work but simplyevades difficult questions and keeps on writing as

if they did not exist. Chaitins own attitude beginsto resemble the dogmatism he accuses his oppo-

nents of.

At worst, Chaitins claims are nearly megalo-

maniacal. What else can one think of statementssuch as the following?: AIT will lead to the majorbreakthrough of 21st century mathematics, which

will be information-theoretic and complexity based

characterizations and analyses of what is life, whatis mind, what is intelligence, what is consciousness,

of why life has to appear spontaneously and thento evolve (Exploring Randomness, p. 163).

Problems of Popularization

The historical surveys of the theory of program-

size complexity that Chaitin gives are sometimes

rather distorted. Some have even complained thatChaitin is rewriting the history of the field and

presenting himself as the sole inventor of its mainconcepts and results [7]. This complaint also fits

to a considerable degree the present books. They

are quite idiosyncratic.The style of these books is very loose and pop-

ular. Large parts of the text are directly transcribedfrom oral lectures and include asides like Thanks

very much, Manuel! Its a great pleasure to be

here! It is perhaps a matter of taste whether onefinds this entertaining or annoying. Personally, I

dont think that this is a proper style for books ina scientific series. It certainly does not decrease the

sloppiness of the text.

The books bring together popular and intro-ductory talks given on different occasions, and

some of this material has already appeared else-where. There is also a lot of redundancy and over-

lap between the books. Both books (as well as the

earlier The Limits of Mathematics) start with aloose and not very reliable historical survey

Chaitin himself calls it a cartoon summarybeginning with Cantors set theory, going through

Gdels and Turings path-breaking results, and

culminating, unsurprisingly, in Chaitins own work.All three books then present an introduction to

LISP and Chaitins modification of it. Both The Un-knowable and Exploring Randomness contain a

more theoretical section on algorithmic informa-tion theory and randomness. And finally, bothbooks end with rather speculative remarks on the

future of mathematics. Would it have been better

to do some editing and publish just one bookinstead of three?

To a considerable degree, Exploring Randomnessand The Unknowable just recycle the same old

ideas. Consequently, for those with some knowl-

edge of this field, these books do not offer anythingreally new. For those with no previous knowledge

of these matters, it is questionable whether thesebooks are really a good place to start.

(There is an errata for Exploring Randomnesson Chaitins home page: http://www.cs.auckland.ac.nz/CDMTCS/chaitin/).

References

[1] GREGORY J. CHAITIN, Randomness and mathematicalproof, Sci. Amer. 232:5 (1975), 4752.

[2] , A theory of program size complexity for-mally identical to information theory,J. Assoc. Com-put. Mach. 22 (1975), 329340.

[3] , Algorithmic Information Theory, CambridgeUniversity Press, Cambridge, 1987.

[4] , Randomness in arithmetic, Sci. Amer. 259:1(1988), 8085.

[5] MARTIN DAVIS (ed.), The Undecidable, Raven Press,

New York, 1965.[6] JEAN-PAUL DELAHAYE, Chaitins equation: An exten-

sion of Gdels theorem, Notices Amer. Math. Soc.36:8 (1989), 984987.

[7] PETER GACS, Review of Gregory J. Chaitin, Algorith-mic Information Theory,J. Symbolic Logic54 (1989),

624627.[8] MARTIN GARDNER, Mathematical games: The random

number Omega bids fair to hold the mysteries of the

universe, Sci. Amer. 241:5 (1979), 2034.[9] MICHIEL VAN LAMBALGEN, Algorithmic information the-

ory,J. Symbolic Logic54 (1989), 13891400.[10] MING LI and PAUL VITANYI, An Introduction to Kol-

mogorov Complexity and Its Applications, Springer-

Verlag, New York, 1993.[11] YURI V. MATIYASEVICH, Hilberts Tenth Problem, MIT

Press, Cambridge, MA, 1993.[12] PANU RAATIKAINEN, On interpreting Chaitins incom-

pleteness theorem, J. Philos. Logic27 (1998),

269586.[13] , Algorithmic information theory and unde-

cidability, Synthese 123 (2000), 217225.
http://www.cs.auckland.ac.nz/CDMTCS/chaitin/http://www.cs.auckland.ac.nz/CDMTCS/chaitin/http://www.cs.auckland.ac.nz/CDMTCS/chaitin/http://www.cs.auckland.ac.nz/CDMTCS/chaitin/http://www.cs.auckland.ac.nz/CDMTCS/chaitin/

Documents

rev-panu