Computational Complexity and the Existence of Complexity Gapsbor/Papers/complexity-gaps.pdf · Computational Complexity and Existence o.f Complexity Gaps 159 digit of the sequence

Computational Complexity and the Existence of Complexity Gaps

A. BORODIN

University of Toronto,* Toronto, Ontario, Canada

ABSTRACT. Some consequences of the Blum axioms for step counting functions are inves- tigated. Complexity classes of recursive functions are introduced analogous to the Hartmanis- Stearns classes of recursive sequences. Arbitrarily large "gaps" are shown to occur throughout any complexity hierarchy.

KEY WORDS AND PHRASES: computational complexity, measures of complexity, recursive functions, tape complexity, step counting functions, axiomatic complexity theory

CR CATEGORIES" 5 .21 , 5 .22

1. Introduction

During the 1930's, a great deal of research concerned the question of "what functions are computable." Subsequent understanding of the concept of computability gave rise to a new question, namely, "how difficult to compute is a given function." Our purpose is not to analyze the complexity of any specific functions but rather to continue the development of a theory of complexity.

In this paper, we shall study further the properties of complexity classification schemes, in order to develop a framework within which concepts of computational complexity (such as "difficulty" of a function) can be formulated. Our classification scheme evolves directly from the one introduced by Hartmanis and Stearns [11]. In [11, 22], "time" and "memory" are viewed as resources, which any efficient algorithm will try to conserve. Thereafter, a complexity class is formed by placing a bound (a function of the input value) on the amount of a particular resource that an algorithm may use. A function is in a given class if there is an algorithm for computing the function, in which the amount of resource does not exceed the bound.

In [11], every computable monotone-increasing recursive 1 function t induces a class of recursive sequences by limiting the number of steps for computing each

Copyright © 1972, Association for Computing Machinery, Inc. General permission to republish, but not for profit, all or part of this material is granted

provided that reference is made to this publication, to its date of issue, and to the fact t h a t reprinting privileges were granted by permission of the Association for Computing Machinery. * Department of Computer Science. This paper embodies the results contained in the author's Ph.D dissertation (Cornell University, 1969). A preliminary version of this paper appeared in the ACM Symposium on Theory of Computing, May 1969. The research has been supported in part by National Science Foundation Grants G J96, GP6426 and GP7701, and National Research Council Grant A7631.

A number theoretic function is one whose domain and range is contained in the natural numbers N. We will henceforth use the standard terminology of recursive function to mean computable total number theoretic function and partial recursive to mean computable p a r t i a l

number theoretic function.

Journal of the Association for Computing Machinery, Vol. 19, No. I, January 1972, pp. 158-174.

Computational Complexity and Existence o.f Complexity Gaps 159

digit of the sequence. Specifically, a sequence a -- a~la2 • "" is in St if and only if there is a multitape T.M. which outputs the nth digit a . within t (n) computation steps. We may say that the number of computation steps (= time) is a resource and we measure the complexity of a computation by the amount of resource ex- pended.

Following Hartmanis and Stearns, similar schemes for classifying recursive sets were studied in [22, 8]. Here the classes are derived by placing a bound on the number of tape squares (tape reversals, respectively) in a computation, the bound being a recursive function of the input length. Again, membership in a class is determined by the existence of some "efficient" (relative to the measure) algorithm.

Are any of the results and concepts formulated in the previously mentioned papers valid with respect to other (abstract) models of computation? The concepts are valid. It is the work of Blum [1] which shows how to develop a theory of complexity in a formulation which is machine (model) independent.

5Iotivated by the work of Rabin [18], Blum chooses two axioms which provide a broad characterization for the intuitive notion of a resource or a measure of complexity. A partial function ¢~, the step counting function, 2 is associated with each (algorithm) ~i. Axiom 1 merely says that a measure is well defined in the sense that we get a value for the measure if and only if the computation converges (i.e. produces a value for the function being computed). Axiom 2 captures the notion of being able to determine if a particular computation exceeds a bound on the measure.

All previously studied measures of complexity (time, tape, reversals) can be viewed as examples of step counting functions. Our goal will be to introduce and investigate classification of recursive functions based on bounding (by a reeursive function) the step counting function ¢~ associated with a computation (algorithm) ¢~. Although we will be talking about classes of functions, we note that we could have chosen to study recursive sequences or recursive sets.

In the next section, we define our notion of resource bounded complexity classes with respect to axiomatically defined resources (complexity measures). Some immediate observations are then made by generalizing results in Hartmanis and Stearns.

Section 3 concerns itself with hierarchy theorems. In particular, the gap theorem shows how essential are the restrictions in known hierarchy theorems (i.e. the use of tape constructable functions and real time countable functions). More precisely, the gap theorem shows that there is no uniform way to increase a resource bound to guarantee an increase in computing power.

We then begin to view classes of function (the set of permissible bounding functions) as names for complexity classes. In this context, it is established that no naming class can have all the properties one might desire. Specifically, it is shown that there is no class of functions (bounds) 8 for which (1) there is a uniform way [composition with some fixed function h(x) << x] to decrease the computing power and (2) $ names all the classes.

5linimal growth rates are studied in Section 4, as an example of a "gaplike phenomena in the low end of the complexity structure." Section 5, the conclusion, reviews the distinction between "class or structural properties" as opposed to "naming properties." This distinction is made as a possible starting point for attack- ing the problem of "how to classify measures."

Many other names for ¢~ appear in the literature, including complexity function, ¢ con- structible function, run-time function, difficulty function.

Journal of the Association for Computing Machinery, Vol. 19, No. I, January 1972

160 A. B O R O D I N

2. Axiomatically Defined Complexity Classes

2.1. BASIC DEFINITIONS. We restate the Blum formulation precisely. An effective list (~b0, ~1, ~b~, • • • ) of all partial recursive functions is postulated, such tha t this list satisfies the ST theorem and the universal Turing machine theorem. 3

The list {$~} should be thought of as a list of algorithms or devices for computing the partial recursive functions. A partial function ~ is associated with each algo- r i thm ~b~. The set of functions {~} satisfies the following axioms.

Axiom 1: (~(x ) is defined *=~ ~bi(x) is defined. Axiom 2: The function

M ( i , x , m ) = {~ i f ~ ( x ) = m otherwise

is total recursive. Henceforth, we will use the s tandard notat ion 0k (x) J, to mean ~b~(x) defined at

x (x in domain of ~ ) and ~ ( x ) T to mean ~i undefined a t x.. As an immediate consequence of Axiom 2, we note tha t the following predicates

are recursive for all k, i, and x:~

¢,(x) = k, ¢ , (x) < k, ¢~(x) > k.

Example 2.1: Let {Z~) be the class of Turing machines with a read only input tape, a write only output tape, and one "work" tape. Let ~ be the function computed by Z~. We fix our input and output convention to be the binary representa- tion of integers and allow an arbi t rary but finite number of work symbols. Define • ~ (x) = m iff Z~ with input x stops and uses precisely m squares of the work tape for the computat ion. {~i} has been defined to satisfy Axiom 1. To see tha t {O~} satisfies Axiom 2, we note tha t either the number of tape squares used in the computat ion increases as the computat ion proceeds or the machine is in a loop. Since we can effectively determine how many machine operations may occur without entering a loop or using a new square on the work tape, Axiom 2 holds for {¢~}.

Definition 2.2: Let (~0, ~bl, ~b2, • . . ) and (~0, ~ i , ~2, • • ") be as stated for the Blum axioms. We will call (I) = ({ ~b~), {~i} ) a dynamic complexity measure or resource (and for brevity, we will often refer to • as a measure). Informally, this notat ion allows us to denote both the measure and the "devices" upon which the measure is imposed. For notational convenience, the set of step counting functions {¢~} would be associated with the measure ~. • is dynamic in the sense tha t it describes the behavior of the computat ions of an algorithm; a computat ion being the execution of an algorithm when applied to an input. This is in contrast to a static or definitional measure such as the size [2] or "s t ruc ture" of the i th algorithm.

Rogers [20] shows tha t acceptable indexings are recursively isomorphic. T h a t is, if {~i*} and {~i) are acceptable then there exists a recursive 1-to-1 onto function f

The S, ~ theorem asserts the existence of a total recursive function a such that 4~,~i.~)(n) = V~(n~, n). The universal Turing machine theorem states that for any recursive 1-to-1 onto map J : _~t X N --* N, there exists a universal machine Z, with the property that 4~i(J(x, y)) = 4~(y) for all x and y. Indexings, satisfying the above theorems, are called acceptable by Rogers [21]. It should be clear that any effective enumeration of Turing machines would yield an acceptable indexing.

¢~(x) > k iff it does not hold that (¢~(x) < k or ¢~(x) = k). Specifically, (~k) ¢~(x) > k when ¢~(x) ]" [equivalently, ~(x) ~ ].

Journal of the Association for Computing Machinery, Vol. 19, No. 1, January 1972

Computational Complexity and Existence of Complexity Gaps 161

such that 4)~ "~ 4)7,) (and 4)~* ~ 4)i-1(~)).5 A complexity measure ~ = ({4)i}, {~}) may be viewed as a complexity measure (I)* = ({4)~*}, {~*}) with ~ * ~-~ ¢I-~(~) •

and 4 " determine the same set of step counting functions, and in particular, for any function g, {(I)~ I 4)~ --~ g} = {OJ*l 4)i* "~ g}. Informally, we may say that the existence of a complexity measure with certain properties will not be unique to a particular list {4)i}.

The framework for our investigation will be classes of recursive functions. We generalize the Hartmanis-Stearns concept of classes of recursive sequences S~ as follows:

Definition 2.8: Let ~ be a complexity measure, t a recursive function.

R~ ~ = {recursive functions f l (3i)[4)i ~'~ f h O~(x) <_ t (x) for almost all x]}. 8

Informally, f is in Rt a if f is recursive and there exists an algorithm for computing f such that the difficulty (relative to the measure or recource ~ ) is bounded almost everywhere by the function t.

We call Rt ~ a • resource bounded class or more briefly a complexity class. Regarding Definition 2.3, one may wonder why we did not require 4)~(x) _< t (x)

for all x. The "almost everywhere" criterion is consistent with the concept of a "bounding function" (e.g. Ackerman's function bounds the primitive recursive functions in this "for almost all sense"). I t is also consistent with the Blum speedup and compression results, which are discussed later in this paper. Moreover, the distinction between "for almost all x" and (Vx) disappears for a complexity measure if both

(1) the computation "device" has some sort of "finite control," capable of mini- mizing a function's complexity at a finite number of argument values;

(2) input-output conventions do not "contribute" to the complexity measure. The measure of Example 2.1 satisfies these properties. "Time" on a Turing ma-

chine would not satisfy the second condition since, the number of steps needed merely to read the input or write the output contributes to the total number of steps (= time). Thus if we used the (Vx) criteria for " t ime," no class Rt * could contain all constant functions, which we intuitively feel are easy to compute.

Henceforth, we will abbreviate statements such as ~ ( x ) < t (x) for almost all x by ~,. _< t almost everywhere (a.e.) or ~ ( x ) ~ t (x) a.e.x. Similarly, ~ ( x ) _< t (x) for infinitely many x will appear as ~ ~ t infinitely often (i.o.).

We will sometimes have need to denote the class of algorithms (more precisely the indices) which "behave" within some bound.

Definition 2.4: Let ¢ be a complexity measure, t a recursive function.

It ~ = {il qh < t a.e. and ( V x ) ~ ( x ) $ }.

I t should be apparent that I , ~ I ~ = ,, implies that Rt ~ = R~t, but not necessarily conversely. When it is clear from the context, we will sometimes omit the superscript q~, and write It and Rt .

2.2. ANALOGS TO tIARTMANIS-STEARNS. I t is easy to show that many of the properties of the St classes proved in [11], generalize to our R, ¢ classification. The proofs have been omitted in the following straightforward development of results pertaining to the enumerability of R, a classes.

6 _~ represents Strong equality of partial functions. That is, ~ ~_ ~t, iff (Vx)[4~(x) I ¢=~ ~(x) ~ h ¢(~) ~ ~ ~(~) = ~(~)]. t"For almost all x" P(x) =- (3xo)(Vx ~ xo)P(x).

Journal of the Association for Computing Machinery, Vol. 19, No. 1, J anua ry 1972

162 A. BORODIN

Definition 2.5: A class C of [partial] recursive functions is recursively enumer- able (r.e.) ¢~ :3 [partial] recursive h(i, m): C = {kxh(i, x) I i >_ 0}. 7

In the context of a listing {~b~l of partial reeursive functions, it is equivalent to say that C is r.e. ¢=~ 3 recursive h': C = {~bh,¢o (x) I i ~ 0]. We will say that h' enu- merates C. We may think of h' as enumerating a list of machines and the set of functions computed by these machines is exactly C. Note that if C is a class of recursive functions, then each ~bh,(~) is a reeursive function.

Example 2.6: The class C of functions of finite support is r.e. Tha t is, C = {fl f recursive and f ( x ) = 0 a .e .x . I .

PROPOSITION 2.7. Let C be an r.e. class of recursive functions. Let • be a complexity measure. Then 3 recursive functions bl and b2 :

f in C ~ f < _ b x a.e.,

f in C yinR 2. THEOREM 2.8. Let ¢ be a complexity measure. Using Proposition 2.7, let b be such

that Rb ¢ contains all functions of .finite support. Then for any recursive function t,

t >__ b a . e . ~ R , # i sr .e .

In [13, 14], it is shown that there are measures ~ for which some of the "small"

classes (i.e. Rh $, h(x) - 0) are not r.e. Following suggestions in [11] and [25], we introduce a property of some measures which allows Theorem 2.8 to follow for any nonempty class R, ®.

Definition 2.9: • is finitely invariant iff

If in Rt ~ h f ' total h f = f ' a.e.] ~ f in Rt ~.

T h a t is, total functions that have the same value almost everywhere are in the same complexity class. Many familiar measures are finitely invariant (e.g. time, modified so that reading the input is not counted, or tape on a T.M.) . However, Definition 2.9 is independent of the Blum axioms. 8

THEOREM 2.10. Let • be finitely invariant. Then for all recursive t, Rt ¢ is r.e. 2.3. COMBINING ALGORITH.~IS. In order to achieve the generality desired, we

have chosen to study complexity classification with respect to axiomatically defined resources. On the other hand, there is an obvious advantage in studying specific measures such as time and tape; namely, our familiarity with how these measures "behave ." The purpose of this section is to show tha t in many ways, the "behavior" of an arbitrary Blum measure is not "radically" different from the well-studied time

or tape. In particular, Blum shows that "measures (resources) are recursively related." PROPOSITION 2.11 (Blum). Let • and ~ be measures with respect to the same

indexing. Then 3 recursive h: (Vt)It ¢ _ I ~ t , , and It $ C_ I, , and thus

Rt ~ c_ R~, and Rt $ C_ R~,,

where t ~ (x) = h (x, t (x)).

7 x notation denoting h as a function of x. 8 In fact, there exists a measure ~' such that there are arbitrarily large classes Rt ¢' (i.e. t can be made arbitrarily large) such that R~' is not closed under finite changes to functions which are members of R~'.



PROOF. L e t

p(i , x, m) = {O '(x) "{- ~,(x) ifotherwise.~b,(x) _< m or ~, (x ) <_ m

Set.

Then, for all x >_ i,

h (x, m) = max p (i, x, m) t~_z

¢ , (x ) < h(x, ~ , (x ) ) ,

4~,(x) < h(x, ¢ , ( z ) ) .

The proposition follows by noting that h (x, m) is nondecreasing in both variables. QED

As a direct application of the previous proof technique, we can show that for any measure, if one combines algorithmic devices in well-determined ways to form new devices, then the complexity of the new device will be recursively related to the complexity of the original devices. For example, when the measure is time or tape, we can often "combine computations by running them in parallel" and the resulting complexity does not exceed the maximum of the individual complexities.

PROPOSITION 2.12. Let (~ be any complexity measure. Let Ch(~,...~,) (x) be a partial function such that

[¢,(x) ~ , i = i l , / 2 , . . . , in] ~ ~(,,....,,.) (~) $.

Then there exists a recursive g (x, yl , y2, "'" , yn) such that

[¢^(~,....~.)(x) _< g(x, ¢~,(x), ¢i2(x), " " , ¢,~(x))] a.e.

PROOF. L e t

p( i l , "'" , in , x, y, ,

Then

g(x, yl , " ' ' , Y n ) =

satisfies the proposition. Example 2.13: Let

. . . , yn) = {~h(~,'".~n ) (x) if ~ j ( x ) = y~', 1 _< j _< n, otherwise.

max p( i l , . . . , in, x, y, , . . . , yn) i j~z

~<T_<n

{ ~ (x) if ¢~ (x) _< ¢i (z) Ch(~,,,(x) = ~¢),(x) if ~ , ( x ) > ~;'(x)

(T if ¢, (x ) T h ¢~.(x)T.

If ¢^(~.~.)(x) = min ( ~ ( x ) , (I)j(x)) holds, then we say that ~) has the "parallel computation" property.

8. Hierarchy Theorems and the Existence of Complexity Gaps

3.1. A REVIEW OF SOME KNOWN HIERARCHY THEOREMS. Let us examine some existing complexity hierarchy results.


164 A. BORODIN

Example 3.1: C~ is the class of all recursive sets acceptable in tape L (n) on an off-line T.M.

(n = input length) L is tape construetable iff L is total and there exists an off-line T.M. Z~ such

tha t

L ( n ) = max {tape used by Z~ on input x}, z: I z l - "

where I xl = input length of x. Then

/4(n) - - , inf L - - ~ 0 and l-a tape constructable ~ CL, -- Cz.t ~ /2~.

See [22]. Example 3.2:

St -- class of all t computable binary sequences where t is increasing. Then

inf t l(n) log t l(n) --+ 0 and t~ real time countable =~ S,, - Sit r ~ ~ . ~(n)

See [11] and [12]. The Blum compression theorem is a hierarchy result which pertains to all com-

plexity measures. An essential concept in this theorem is that of a "measured set of functions."

Definition 3.3: A measured set of functions 9 = {7~} is an r.e. se~ of partial recursive functions such that there exists a total reeursive function

M (i, x, m ) = {~ if T , (x) f

That is, a measured set of functions satisfies Axiom 2 for a set of step counting functions. Thus the set {¢~} is always a measured set. Whether or not a set g is a measured set does not depend on the choice of formalism {¢~}.

Suppose $ is any set of recursive functions which contains arbitrarily large functions (i.e. for every total recursive function f, their existsan h in $ such that h > f a.e. ). Then from elementary recursion theory we know that $ cannot be r.e. The next observation is then immediate.

PROPOSITION 3.4. There is a measured set 9 which contains the set of tape constructable (real time countable)functions. However, the set of tape constructable (real time countable)functions cannot be r.e. and therefore cannot be a measured set.

THEOREM 3.5. (Blum Compression Theorem Restricted to Total Functions). Let dp be a complexity measure and {7;} a measured set. Then there exist recursive functions g and h such that for every total 7~ , 3 a total recursive function f satisfying:

(1) ~ y ~ - - f ~ " > "fl a.e., (2) dpg(~) ~--f and ~o(o(x) <_ h(x, 71(x)) a .e .x . We can easily restate this theorem in terms of our complexity classes. THEOREM 3.6. (Weak Compression Theorem). Let • be a complexity measure

and {'),~} a measured set. Then "4 a total h' such that for every "su.fftciently large"

Journal of the Amociation for Computing Machinery, Vol. 19. No. 1 , J a n u a r y 1972


total ~ , o R~, c Ra,oT,.

PROOF. Let b (x) be any increasing reeursive function. ~ sufficiently large will mean ~ ~_ b a.e. Let h be as in Theorem 3.5. Then

h' (u) ffi max h(z, y) ,<b-1 (y)

satisfies the theorem where b -l (y) = ~[b(z) _~ y]. QED Theorem 3.6 is a weak version of Theorem 3.5 in the sense that f in R~,. ~ -

R ~ only guarantees that ~j ~ f ~ &~ ) ~ i.o. rather than ~s ) ~ a.e. We also note that Theorem 3.6 does not express the fact that an f in ® ® Rh,.~ -- R ~ can be found effectively from the index i.

We are now in a position to make a very basic observation. Namely, a salient fea- ture of all known complexity hierarchy results is that some restriction has been placed on a function which bounds the resource. In Example 3.1, it was necessary to have L2 he tape constructable. Example 3.2 had the requirement that ta be real time countable. And finally, in Theorem 3.6, the bounding functions were restricted to a measured set. In this chapter, we shall make precise just how essential these con- ditions are.

I t is appropriate to comment briefly on the distinction between hierarchy theorems as in Example 3.1 or 3.2, and the general hierarchy result expressed in Theo- rem 3.6.

Using the terminology of Constable [7], the hierarchies in the former case are examples of "downward diagonalization." We may think of Examples 3.1 and 3.2 to be of the form:

infh°t- '-~O~ R~, -- R, ~

and thus

lira h o t - - --* O ~ R t c R ~ ,

where ~,~ indicates a total function in some measured set. Intuitively, we see how much below ~,~ we need to be in order to induce a class smaller than R ~ . Theorem 3.6 represents an "upward diagonalization" in that we show that increasing ~ by some h insures that R~, c Rho~.

This distinction is" studied further in [7]. For our purposes we need only be aware that the distinction exists. I t is more important for us to note than any measured set induces a uniform (in some h) upward hierarchy. That is, given any ~,~ there is a uniform way (composition by h) to extend T~ so as to induce a larger class of func- • tions.

3.2. THE GAP THEOREM. Informally, Theorem 3.6 guarantees that "in certain complexity ranges, we need only increase the computing power by some fixed function in order to achieve a genuine increase in computing capability." In contrast, Theorem 3.7 will show that "there exists arbitrarily large gaps in some complexity. ranges where no new computation is performed." The following theorem was first proven by Trachtenbrot [23] and independently by this author.

0 h ' o ~ d e n o t e s c o m p o s i t i o n of f u n c t i o n h ' w i t h f u n c t i o n ~ ; t h a t is , (h~ o ~ ) ( x ) ffi h'(7~(x)). C d e n o t e s p r o p e r i nc lu s ion .

Journal of the As~clatlon for Computing Machinery, Vol. 19, No. 1, January 1972

166 A. BORODIN

THEOREM 3.7. (Gap Theorem). Let • be a complexity measure, g a nondecreasing recursive function such that (Vx) g( x ) >_ x. Then "4 an increasing recursive function t such that

PROOF. Define t as follows: t (O) = 1, t (n) = #Ic > t(n -- 1){(Vi ~ n)([(~,(n) < k] or [~,(n) > g(k)] )}] °

(1) (Vn) k exists, since (Vi _< n) if ¢~(n)T then (Vk) ~ ( n ) > g(]c) and if ~ ( n ) ~ then (3k) ¢~(n) < k.

(2) k can be found recursively, since • is a complexity measure and thus [el(n) < k] and [ ~ ( n ) > g(k)] are recursive predicates.

(3) t satisfies the theorem, since n >_ i implies that either ~ ( n ) < t (n) or ~, (n ) > got (n) . QED

We observe tha t an arbitrarily large t can be found to satisfy Theorem 3.7. Sup- pose we want t (n) > r(n) , then define

t(0) = r(0) + z,

t (n) = #k > max(t(n -- 1), r ( n ) ) { . . . } .

COROLLARY 3.8. (Weak Gap Theorem). Let ~ be a complexity measure and let g be any recursive function such that (Vx) g (x) > x. Then there exists an arbitrarily large, increasing, recursive function t:

I t v = I~.~ and consequently R, ~ = R~.t .

PROOF. Immediate from Definitions 2.3, 2.4, Theorem 3.7 and the preceding observation.

COROLLARY 3.9. zl Let ~P and ~ be any two measures using the same indexing. Then there exists an arbitrarily large increasing recursive t:

^ A

It a = I t * and thus Rt a = Rt ~ •

PROOF. Let h(x, m) be as in Proposition 2.10. Choose a monotone g such that (Vx) got(x) >_ h(x, t ( x ) ) for all sufficiently large t [i.e. t (x) >__ x]. Now use Corol- lary 3.8 to choose arbitrarily large t': I~, = I~.,, = Io°o.t, (that is, construct a ! I

t , gogo t gap). Then

Io°o°e Io.r I o . , , c _ = , ^

Thus, t = got' satisfies the corollary. QED Suppose • = ({¢~}, {~}) and ~ = ({$i}, {~,}) refer to the " t ime" measure

of two radically different computers. Recall the isomorphism between acceptable indexing. Then, using the technique of Proposition 2.10 and Corollary 3.9, it is easy to show that no matter how much "bet ter" one computer may seem compared to

z0 ~k > t ( n - - 1){P(k)} denotes the least k such that k > t(n -- 1) and P(k) holds. n Also proven independently by E. McCreight in his thesis.

Journal of the Association for Computing l~fachinery, Vol. 19, No. 1, January 1972

Computational Complexi ty and Existence of Complexity Gaps 167

the other, there will be arbitrarily large t such that the sets of functions computable in time t is the same for both computers.

Corollary 3.8 is not a "speedup" theorem. Rather, it states that there is no computation which "behaves" in the range (t, got). Corollary 3.8 is a weak version of Theorem 3.7 in the sense that the condition [~i > t i.o. ~ 4,~ > g o t i.o.] would suffice to have I , ~ = I~.t. Using an intricate priority argument, Constable [6] has shown that a "weak operator gap" holds for all measures. An observation due to Blum [1] shows tha t this "weak" gap is the best possible for many operators.

I t is interesting to note how "small" (relative to g) an increasing recursive t can be found satisfying Theorem 3.7.

COROLLARY 3.10. Let ~, g, and t be as in Theorem 3.7. Let f ( y ) = g (y "-k 1 ). Then ( V x ) t (x) < b (x) where b (0) = 1 and b (x) = f(~+l) o b (x - 1 ) for x ~ 1.

PROOF. Review the construction of t in Theorem 3.7. Look at the n q- 1 intervals.

{ t (n - 1) q- 1, g o [t(n - 1) -F 1]},

{g o [t(n - 1) + 1] -b 1, g o [g o [t(n - 1) ~- 1] -t- 1]} . . . ,

{f(') o t (n - 1) + 1, g o [ f (~)o t ( n - 1) ~- 1]}.

f U ) o t ( n - 1) + 1 ~ ~ ( n ) < f u + ~ ) o t ( n - 1)

o r

( V i <_ n ) ~ ( n ) <_ f('+l) o t (n -- 1).

The corollary then follows by the construction of t. Example 3.11: Let • be any measure and le tg(x) = x -/- 1. Then, by the pre-

ceding corollary, letting f ( x ) = x -I- 2, there exists t ( x ) < b (x ) such that R~ ~ = R~+l where

b(0) = 1,

b ( x ) = b(x - 1) q- 2(x Jr 1) _< x ~ -F 3x q- 1.

Corollary 3.10 provides an estimate of the growth rate of t relative to g. In particular, if g is primitive recursive in Grzegorczyk [9] class ~ , then t can be bounded by a function in ~n+,. With a judicious choice of encoding instantaneous descriptions we know that M is primitive recursive (elementary) if ~ refers to the usual meaning of computation steps, tape or tape reversals. In general M need not be primitive recursive. I t should be apparent that if M and g are in ~', then the t of Theorem 3,7 is in ~n+l.

3.3. T H E H O N E S T Y T H E O R E M A N D N A M E I N V A R I A N T D O W N W A R D G A P S . After the gap theorem was announced, McCreight and 5{eyer proved a powerful theorem about "ways to name complexity classes." Their theorem, interesting in its own right, takes on added significance when viewed in contrast ~-ith the Blum compression theorem and the gap theorem.

THEORE.~I 3.12. (SIcCreight-SIeyer Honesty Theorem). Let ~ be any complexity measure. Then there exists a measured set ~ such that for every total reeursive funct ion t, there is a total t' in g: It ~ -- 1~, and .'. Rt ¢ = R~, .

Let ~ be as in Theorem 3.12. Using the compression theorem, let h be such that for all sufficiently large total t' in 9, R~, ~ R~°t , . Then, even though we may find a large "gap t" such that R, ~ = R~ot, :3 a total t' in ~. Rt a = Raaot = R~, C ROb.t,.

Journal of the Assoclation for Computing Machinery, Vol. 19, No. 1, J anua ry 1972

168 A. BORODIN

This leads McCreight and Meyer to suggest that "theorems like the gap theorem which appears to be making statements about the structure of t-computability classes may only be indicating the properties of badly chosen names for the classes."

This remark has a good deal of merit. Indeed, we shall return to this distinction between "structural" and "naming" properties. However, we should not infer that one type of property is more important than the other. We begin to view a set of functions as a "coordinate system" for the complexity classes or as a set of names for the classes. A modified gap argument will show that for some purposes, bad names are unavoidable (or equivalently, that certain naming properties hold in any "coordinate system").

We first formalize the ability of some classes to induce hierarchies. Definition 3.18: A class of partial functions 9 is upward extendible (by ha w.r.t.

~) if for every sufficiently large total g in g, Ro ~ C Rh¢.°o • The relationship between measured sets, sets of "g-honest functions [17, 15]"

and "upward extension" has been well studied [1, 3, 15]. These studies (and in particular, the compression theorem) allow us to characterize a measured set (or a set of g-honest functions) as a good naming class for the purpose of extending upward in the hierarchy. Furthermore, the honesty theorem says that for each ¢ we can choose such a class and still name all the • resource bounded complexity classes.

Definition 8.14: A class of partial reeursive functions 9 is downward extendible (by h~ w.r.t. ~ ) iff 3 recursive increasing hd: for every sufficiently large total g in g,

R~I° a c Ro ~ where again h-~(y) ffi ~[h(z ) >__ y].

Example 8.15: By the compression theorem, for any measure ~, "4 reeursive h' @ P

and g, such that for all sufficiently large ~ , R ~ c R¢0<~I where ~g,) _< h o ~ a.e. Thus {~g<~)} is a downward extendible class (by h' w.r.t ¢) .

Example 8.16: In [15], a measure is called proper if for all total @~, ~ in R ~ . Let • be a proper and let ¢ have the "parallel computation property" in the sense of Example 2.13. (The tape measure of Example 2.1 is such a measure.) Constable [7] notes that for all such measures, the set {~} is downward extendible.

We need one "technical" lemma, which is a slight modification of the compression theorem. This lemma is used twice: first for the observation that downward extendible classes are upward extendible and then in the proof of Theorem 3.20 (the existence of name invariant downward gaps).

LEMMA 3.17. (Modification of Compression Theorem). Let • be any measure, a measured set, and hi any recursive function. Then "4 a recursive function h2 (x, y)

such that for all total g in 9, "4 f:

~ f = ~ > hlog a.e.

" 4 d ~ ' ~ f and chi(x) <_ h2(x ,g (x ) ) a.e.x.

As in Theorem 3.6, we can replace h2 by a function of one variable h2 p if we re- strict the Lemma to all sufficiently large g in 9. Obviously, h2 t > h~ a.e.

PROOF. An essential lemma to the compression theorem is Theorem 7 of [1]. We make a simple modification of that theorem and show:

(1) To every partial recursive function g, there corresponds a 0-1 valued partial recursive function f with the same domain as g such that i f d~ ~-- f, then

• ~ h l o g a.e.


Compuh~tional Complexity and Existence of Complexity Gaps 169

(2) There exists a total reeursive function r which takes indices for g and hi into an index for the corresponding f.

Let ~bz ~ h~. Holding l fixed, we can go (via r) effectively from an index for g to an index for f. The Lemma then follows exactly as in the proof of the compression theorem.

We used the Lemma for the next easy observation. PROPOSITION 3.18. Let 9 be downward extendible (by ha w.r.t. ~ ), then 9 is up-

ward extendible (by some h= w.r.t. ~ ). PaOOF. Let g in 9 be sufficiently large (assume at least that g (x) >__ x a.c.x.)

R ¢ h T . o c R o ® ~ 3 4 k : h~ -log < ¢ k i.o.

~ k < g a.e.

Setting h~ = ha in Lemma 3.17, we get that 3f:

~ f = ~ 4 ~ > h~o4k a.e.

= , , ~ > haoh~og >_ g i.o.

and 3~b~ ~ f with

¢~.(x) _~ h2(x, ¢~(x)) a.e.x.

~_ h2(x, g(x)) a.e.x.

~_ h~(g(x), g(x)) a.e.x.

The proposition is proved when we set

h=(y) = h~(y, y).

QED Definition 3.19: A class of partial functions 9 is class determining w.r.t. 4 iff for

. ¢ every total recursive function t, there exists a total function g m 9 such that Rt = Ra ®.

We may think of any class determining 9 as a coordinate system for 4. That is, the total functions in 9 name all the 4 complexity classes. As an immediate consequence of the following theorem, due to R. Constable and the author, we see that a class 9 cannot be both class determining and downward extendible w.r.t, any 4.

THEOREM 3.20. (The Existence of NameInvariant Downward Gaps). Let 4 be any measure. Let ha be any recursive increasing function. Then there exist arbitrarily large, increasing, recursive t such that:

Ro = R~ ~ R ~ l . o = Rg.

That is, it is not possible to go down from Rt by ha no matter what name (g ) we use. PROOF. Letting 9 = {4~}, apply Lemma 3.17 with hi = ha to yield an appro-

priate h2. That is, for every 4k, :3 recursive f:

"4~b~__~f and ¢i(x) ~ h2(x,¢k(x)) a.e.x.

Construct arbitrarily large increasing recursive "gaplike" t:

4~(x) <_ t(x) ~ h~(x, 4~(x)) ~ t(x) for almost all x.


170 ~ . BORODIN

It is easy to verify that such a t can be constructed. Suppose 3 total g such that Ro = Rt and R ~ ° g c Rg. Then, 3 total ¢k such that

(i) hd~og < Ck i.o. which implies g < hdo@k i.o. (ii) Ok _< t a.e. which implies h2(x, ~k(x)) ~_ t(x) a.e.x.

But then there exists f:

~ b ~ _ _ ~ f ~ > h ~ o ~ a.e.

¢~ > g i.o.

and 3 ~i ~ f with ¢j(x) _< h2(x, &k(x)) _< t(x) a .e .x . . ' , f i n R, - Ro which is a contradiction. QED

Intuitively, the proof of Theorem 3.20 consists of forming a (ha -~ o t, t) gap. We suggest that Rt may be thought of as a "limit" of a way of extending classes. That is, {f~.} an r.e. class of reeursive functions and for all j , Rfj c Rh~.f; C Rhs~.I; "'" reflects that h2 is a means by which we can extend classes. It is obvious that this way is limited since there are functions which are much greater than h ~ ofj for any i and j (h2 recursive implies that {h2~ofi} is also an r.e. class of recursive functions and is therefore bounded.).

Viewing Rt = Ro as a limiting class for h~ extension makes the theorem quite intuitive. If we could go downward by hi applied to the name g for R t, then a total

• &~ function would lie "within hi" of g and hence we could "exceed R~" by h2 extension (that is, Ro was not a limiting class for hs extension). Consequently, we might say that this naming exclusion holds because "downward extendible implies upward extendible." No such exclusion holds for class determining and upward extension because "upward extension does not necessarily imply downward extension."

COROLLARY 3.21. I f • is proper and has a parallel computation capability, then {~} is not class determining.

PROOF. Immediate from Example 3.16 and Theorem 3.20. In fact, :il arbitrarily large t: (~i) R, ¢ ~ R~, .

A stronger version of this corollary was first proven by McCreight and Meyer. Namely, they showed that ¢ proper ~ {~b~} not class determir, ing.

4. Minimal Growth Rates

4.1. MONOTONE I~IINIMAL GROWTH RATES. In [22], the concept of "minimal tape function growths" was introduced. It was shown that for on- and off-line Turing machines, the amount of work tape used in a computation must "grow" as a function of input length in order to recognize a nonregular set. The essence of that proof was that if the amount of tape used in a computation is not bounded by some constant for all inputs, then for infinitely many inputs the computation requires an amount of tape at least proportional to log n (n = input length) for the on-line model and log log n for the off-line model.

We now abstract this concept as follows. Definition 4.1: We say a complexity measure • has a monotone minimal growth

rate if there exists a nondecreasing unbounded recursive function t:

(1) ~ total, unbounded, and nondecreasing almost everywhere ~ ~ ( x ) >__ t(x) a . e . x .

(2) ~ total, and unbounded ~ ~ ( x ) >_ t(x) i.o.x.

Journal of the Asaoclation for Computing Machinery, Vo|. 19, No. 1. January 1972


A minimal growth rate corresponds to a "gap" in the "low end" of the complexity hierarchy. We shall show in Example 4.4 that not all complexity measures have a monotone minimal growth rate. This contrasts with an observation by McCreight and Meyer that all measures have a "minimal growth rate" if we do not insist on monotonicity.

One reason to desire monotonieity is that in the context of sequence generation it is natural to insist that all bounding functions be nondecreasing. In Theorem 4.3, we will give a sufficient condition for when a measure has a monotone minimal growth rate.

Example ~.2: Let • be as in Example 2.1. Let P( i , k) be the predicate (Vx)~i(x) <_ k. I t is easy to show that P(i, k) is decidable for all i and k. Given i and k, we may view the Turing machine as a finite automaton and construct its set of "permissible" states. We can then effectively test if there is a sequence of moves (i.e. there is an input) which causes the machine to leave the set of permissible states (i.e. the bound is exceeded).

THEORE.~I 4.3. Let • be any complexity measure. Let P~ (i, k) be the predicate as in Example ~.2 [i.e. (Vx)~i(x) _< k]. Then P~ (i, k) decidable for all i and I¢ ~ ~ lw, s a monotone minimal growth rate.

PROOF. Define t as follows: t(0) = 0;

if (Vi < t(x)){(Vz)[¢,(z) < x] or (:3y

then

else

t(x -{- 1) = t(x) -{- 1

_< x)[~,(y) > t(x)]}

t (x + 1) = t (x). (1) t(x) is obviously nondecreasing. (2) t(x) is unbounded; for, given any k,

( 3 x ) N i < k){ N z ) [ V , ( z ) < x] or (3y < x) [~ , (y) > k]}.

(3) Suppose ¢i total and Unbounded. Given any n: i <_ t(n), let

b = max (~)~(x)) and x0 = ~ [ t ( x ) = k].

In order for t to be unbounded, ~x > ~ :

(~y <_ x)[¢,(y) > t(x) = t(Xo)]. Let

Yo = Izy[O,(y) > t(y) = t(xo)].

By definition of k, yo >_ n and by definition of t, Oi(y0) > t(yo). (4) Suppose ¢~ total, unbounded and nondecreasing almost everywhere. Choose

n: i <_ t(n), (Vy >_ n ) ~ ( y ) <_ ¢~(y -k 1), and q~(n) _< t(n). Then

(Vy >_ n)t(y -k 1) = t(y) + 1 ~ O,(y) > t(y) ~ O,(y + 1 ) _ > t ( y + 1 ) .

QED

I t is easy to exhibit a measure which does not have a monotone minimal growth rate.

Example ~.~: I t is known that R 1 = class of primitive recursive functions con-

Journal of the Association for Computing Machinery, Vol. 1O, No. 1, January 1072

172 A. BORODIN

talns "arbitrarily slow growing" nondeereasing functions. Let q~ be any complexity measure, S an infinite recursive set: i E S ~ q'~ total.

Let

if ~,~ (x) = ( f . (x) if i is the j th largest number in S,

where {f~} is an enumeration of R ~. Then ~ cannot have a nondecreasing minimal

growth rate, and consequently P$ (i, k) is not decidable for all i and k. Definition 4.5: • has a weak minimal growth rate if there exists a recursive func-

tion t with lim inf t --* ¢0 satisfying: ¢~ unbounded on dom cI,~ ~ ~ >_ t i.o. THEOREM 4.6. (McCreight and Meyer [15]). Every measure q~ has a weak mini-

mal growth rate. Example 4.4 and the preceding theorem provide an interesting result about the

recursive functions. COROLLARY 4.7. There exists a recursive function t such that lira inf t ~ oo but

there does not exist any unbounded, nondecreasing recursive function b: b <_ t a.e. PROOf. Let ¢ be the measure of Example 4.4. Let t be a weak minimal growth

rate function for ¢. That is, lim inf t --+ oo and ~ unbounded ~ ¢~ > t i.o. Now suppose =1 an unbounded recursive b < t a.e. Then =l a nondecreasing unbounded elementary function f~. such that fj < b a.e.. '. ¢~ = f~ which implies ~ unbounded and 6~ < b < t a.e. contradicting the definition of t as a weak minimal growth rate function. QED

!

5. Conclusion

We want to attempt to clarify the distinction between "properties about the structure of t computability classes" and "properties about names for the classes."

Important structural properties concern the partially ordered set ({Rt*}, c ) . For example, consider the following interesting fact about the ({R~¢}, c ) structure [11, 15]:

(V sufficiently large classes R~ ®) (3 a class Rt ¢, ) (V classes R/v)[ Rt ¢ U Rot, r~ Rf¢].

On the other hand, there are properties which can not be expressed using only classes for variables; these properties require that we allow functions (names for the classes) as variables. We have seen many examples of such properties (the compression, gap, and honesty theorems).

We shall call such properties naming properties. It is obvious that these properties play an important role in the study of complexity classifications. In general, we can investigate naming properties w.r.t, the total functions of any class determining set 9. That is, the function variables are restricted to the total functions of 9. In particular,

might be (the set of all partial recursive functions) and then a function variable could be any total recursive function.

A further distinction should be made concerning naming properties. Some properties not only must be expressed using function (name) variables but, moreover, hold precisely because of the existence of certain names. The gap theorem reflects such a property. This contrasts with Theorem 3.25, which shows that the existence of downward gaps is in some sense name invariant.

We can call a naming property P name-dependent if there exist class determining

Journal of the Association for Computlns Machinery, VoL 19, No. 1, January 1972


sets 9~ and 9~ such that P holds w.r.t, the total functions of ~ but does not hold w.r.t. the total functions of g2. A naming property P is name-invariant if P holds w.r.t, the total functions of any class determining set.

The Blum axiomization allowed us to characterize computing resources or complexity measures and consequently significant advances have been made in the study of resource bounded classification. By intent, the Blum axioms are very weak. Tha t is, every conceivable measure should satisfy these axioms but many measures satisfy the axioms which we would agree are "pathological." A significant problem remaining in "axiomatic complexity theory" relates to the question of how to classify measures, and, in particular, how to refine the Blum axioms so as to characterize precisely the "natural" measures (i.e. exclude pathological measures).

Given a measure ~), let us view any class-determining set of names for • as a possible set of names to use. What other properties should the class have? We are suggesting that searching for a good naming class is analogous to the development of good coordinate systems in geometry. We seek those properties which are invariant for any good naming class. I t is our hope tha t these "coordinate invariant properties" and "structural properties" will provide the basis needed for classifying measures.

ACKNOWLEDGMENTS. I would like to express my sincere gratitude to Professors R. Constable, J. Hartmanis, and J. Hopcrof t for their encouragement, insight and continual guidance, Professor P. C. Fischer for the beginning of this research, and Professors A. Meyer, P. Young, E. Robertson and Dr. E. McCreight for numerous suggestions. I also want to thank the referees for their excellent comments and especially for the present proof of Theorem 4.3.

REFERENCES

1. BLUM, M. A machine-independent theory of the complexity of recursive functions. J. ACM 15 (1967), 322-336.

2. BLUM, M. On the size of machines. Inform. Cont. 11 (1967), 257-265. 3. BORODIN, A. Computational complexity and the existence of complexity gaps. Dep. of

Comp. Sci. Tec. Rep. No. 69--41, Cornell U. (Aug. 1969). 4. BORODIN, A., CONSTABLE, R., AND HOPCROFT, J. Dense and non-dense families of com-

plexity classes. IEEE Tenth Annual Syrup. on Switching and Automata Theory (Oct. 1969), pp. 7-19.

5. CONSTABLE, R. Extending and refining hierarchies of computable functions. Comp. Sci. Tech. Rep. No. 25, U. of Wisconsin (June 1968).

6. CONSTABLE, R. The operator gap. IEEE Tenth Annual Syrup. on Switching and Automata Theory (Oct. 1969), 20-26.

7. CONSTABLE, R. Upward and downward diagonalization over axiomatic complexity classes. Dep. of Comp. Sci. Teeh. Rep. No. 69-32, Cornell U. (March 1969).

8. FISCHER, R., HARrMANIS, J., AND BLUM, M. Tape reversal complexity hierarchies. IEEE Ninth Annual Symposium on Switching and Automata Theory (Oct. 1968), 373-382.

9. GRZE~ORCZYK, A. Some classes of recursive functions. Rozprawy Matematyezne (Warsaw) $ (1953), 1-46.

10. I-IARTMANIS, J. Computational complexity of one-tape Turing machine computations. J. ACM 15 (1968), 325-339.

11. HARTMANIS, J., AND STEARNS, R.E. On the computational complexity of algorithms. Trans. Amer. Math. Soc. 117 (May 1965), 28,5-306.

12. HE~NIE, F., AND STEARNS, R.E. Two-tape simulation of multitape Turing machines. J. ACM 13 (Oct. 1966), 533-546.

13. LANDWEBER, L. H., AND ROBEBrSON, E. R. ' Recursive properties of abstract complexity classes. ACM Symposium on Theory of Complexity (May 1970), pp. 31-36.

Journal of the Assoclation for Computing Machinery, Vol. ID No. 1, January 1972

174 A. BORODIN

14. LEWIS, F . D . Decision problems for complexity classes of computable functions. ACM Symposium on Theory of Computing (May 1970), pp. 22-30.

15. McCREIGHT, E. M., AND MEYER, A. R. Classes of computable functions defined by bounds on computation. ACM Symposium on Theory of Computing (May 1969), pp. 79- 88.

16. MEYER, A., AND FISCHER, P. On computational speedup. IEEE Ninth Annual Sympo- sium on Switching and Automata Theory (Oct. 1968), pp. 351-355.

17. MEYER, A., AND RITCmE, D. A classification of functions by computational complexity. Proc. Hawaii International Conf. on System Sciences (1968), pp. 17-19.

18. RABIN, M. Degree of difficulty of computing a function and a partial ordering of recur- side sets. Tech. Rep. No. 2, Hebrew U., Israel (1960).

19. RITCHIE, R. Classes of predictably computable functions. Trans. Amer. Math. Soc. 108, 1 (1963), 139-173.

20. ROGERS, H., JR. GSdel numberings of partial recursive functions. Y. Symbolic Logic ~S, 3 (Sept. 1958), 331-341.

21. ROGERS, H., JR. Theory of Recursive Functions and Effective Computability. McGraw- Hill, New York, 1967.

22. STEARNS, R., HARVMANIS, J., AND LEWIS, P., II . Hierarchiesof memorylimited computations. IEEE Sixth Annual ~ymposium on Switching Circuit Theory and Logical Desi~a (1965), pp. 179-190.

23. TRACHTENBROT, B . A . Complexity of Algorithm8 and Computation [in Russian]. Novo- sibrisk, 1967.

24. YAMADA, H. Real-time computation and recursive functions not real-time computable. IRE Trans. EC-11 (1962), 753-760.

25. YOUNG, P. Toward a theory of enumerations. IEEE Ninth Annual Symposium on Switch- ing and Automata Theory (Oct. 1968), pp. 334-350.

26. YOUNG, P. A note on dense and nondense families of complexity classes. Comp. Sci. Tech. Rep. CSD TR 40, Purdue U. (August I969).

RECEIVED JUNE 1970; REVISED JUNE 1971

Journal of the As~oclation for Computing Machinery, Vol. 19, No. 1, January 1972

Documents

Computational Complexity and the Existence of Complexity Gapsbor/Papers/complexity-gaps.pdf · Computational Complexity and Existence o.f Complexity Gaps 159 digit of the sequence