• View

  • Download

Embed Size (px)




    S. Yu. Yakovenko and L. A. Kontorer

    §1. Introduction

    1.1. Economics. In this paper we make an attempt to discuss an approach to infinite horizon stationary optimization problems, based on the notion of (nonlinear) Bellman operators.

    These problems are not merely of an academic interest, since they constantly appear in the normative theory of growth and capital accumulation in mathemati- cal economics. Speaking rather vaguely, the normative theories describe economic dynamics as follows. There is defined (either explicitly or implicitly) a family of feasible paths, or feasible trajectories, describing a possible dynamics of a system. Usually this is done using the so called technological restrictions (e.g., balance equations and capacity limitations are to be satisfied independently of the adopted economic policy). Next, there comes a description of a criterion, or economic goals pursued by the system under consideration. Finally there is assumed (also in a more or less explicit form) that the system behaves rationally, or optimally. This means that the true trajectory provides an optimum to the criterion over all feasible paths starting at the same initial point.

    The criterion is a scalar valued functional defined on all feasible paths (we do not consider here the multicriterial case). Usually it takes the form of a sum or an integral of a certain expression (referred to as utility) over the trajectory. But such a form immediately brings under consideration an important parameter which has been in shadow up to now. This parameter is the time horizon (otherwise referred to as the planning horizon).

    If this parameter is assumed to take a certain finite value, than no additional mathematical difficulties arise, since the finite sums/integrals are quite well defined, yielding thus the correct criterion of optimization. But economic difficulties appear. Their nature can be explained using the consumption/investment dichotomy.

    The latter is the model in which the profit obtained on each step has to be divided into the immediate consumption part and the one to be invested for increase in the productivity capacities of the system which results in the increase of the future profits. The criterion is the integral consumption.

    Given a certain finite planning horizon, the optimal trajectories of the model (under reasonable assumptions) behave as follows. When on earlier stages, usually

    Preprint. The final version appeared in Nonlinear semigroups and infinite horizon

    optimization. Idempotent analysis, 167–210, Adv. Soviet Math., 13, Amer. Math. Soc., Providence, RI, 1992. MR 120379.

    Typeset by AMS-TEX 1


    there is an optimal proportion between consumption and investment which is more or less precisely maintained along the optimal paths. But this proportionality holds no more when the model is close to the beginning or the end of the time interval. The starting segment is greatly influenced by the initial state, and since this state is prescribed, there is no problem in such a deviation, at least from the point of view of methodology. Completely different is the other extremity, towards the end of the planning horizon. The system behavior on final steps follows the ‘aprés nous le déluge’ pattern: the investment almost vanishes.

    Clearly, such a property of the model is quite reasonable within the framework of the finite horizon optimization, since the decrease in investments would result in shrinking future profits, but this future is beyond the planning horizon. Apparently, such features of the model have to be avoided somehow. There is a series of tricks for this. The simplest one is to disregard the model as soon as its behavior begins to be influenced by this terminal effect, but this is not so easy in the multidimensional case with more than two alternatives. Some other approaches were developed in different particular cases, but they rely heavily on certain properties of the model (convexity etc.), and require a thorough preliminary investigation of properties of all the finite extremals; the most illustrating example is the turnpike theory.

    Instead of finite horizon problems, one might speak about infinite horizon ex- tremals. But, on the contrast to the finite case, the infinite sums and improper integrals sometimes (and even very often) happen to diverge. Using a kind of regu- larization procedures, one can try to transform the criteria to something convergent, but the justification of such manipulations hardly goes beyond the heuristic level.

    An alternative strategy consists in introducing certain partial orders on all the possible cost flow paths corresponding to all feasible paths (the cost flow path is the sequence of scalars cT ∈ R, T ∈ N or T ∈ R, where cT is the value taken by the criterion on the initial T -step segment of the trajectory). Unfortunately, such partial orders (or, more correctly, binary relations) are not very naturally introduced,1 and the most popular among them are nontransitive (a short survey of this theory is given below).

    As an immediate consequence of this nontransitivity there arises the existence problem: whether there exist the ‘best’ elements with respect to those orders (here ‘best’ means either majorizing or nonmajorizable). It often happens that some ad- ditional assumptions are necessary to prove the existence theorems. The convexity (or strict convexity) assumption is most important among them, and sometimes the analysis follows the lines of the turnpike theory.

    Therefore the economic problem as it was just stated is to develop a dynamic optimization theory which would be independent on the choice of the planning hori- zon.

    1.2. Mathematics. Stationary dynamic optimization problems in mathematics usually come in one of the following frameworks.

    1for example, try to compare the following three cost flow paths:

    0, 2, 0, 2, 0, 2, . . .

    2, 0, 2, 0, 2, 0, . . .

    1, 1, 1, 1, 1, 1, . . .


    The discrete time problem. Let X be a phase state of the model. It could be either a subset of a Euclidean n-space, or a discrete set, or even an abstract topological space. The points of this set represent different states of the system. We assume that the model is deterministic, so the information about the states is complete and precise.

    A trajectory or path is a sequence of states x = {xt ∈ X : t = 0, 1, 2, . . . }. Therefore the time variable t is assumed to be taking only nonnegative integer values.

    The feasible paths are usually defined recursively: there exists a multivalued map F : X → 2X and x is feasible if and only if xt+1 ∈ F (xt) for all t (the system is stationary, so F is independent of t).

    The criterion of optimization in the most general form is the sum

    B(x) = ∑ t

    b(xt, xt+1),


    b(·, ·) : X ×X → R

    is the utility function: the value b(x, y) is associated with the utility of transition from the state x to the state y in one step (this function is also time independent). From now on we will call b(·, ·) the transition function.

    These data allow to pose the stationary dynamic optimization problem in discrete time,

    B(x)→ max, x0 = a ∈ X,

    where a is a prescribed initial state, and the maximum is taken over all feasible paths starting at a.

    In order to simplify the construction we will incorporate the feasibility restric- tions into the transition function by setting b(x, y) = −∞ if y /∈ F (x) (the infinite penalty). Clearly, we shall pay for such a simplification by imposing certain condi- tions of semicontinuity on b, but all the constructions will become more transparent. Without further mentioning it we made the

    Semicontinuity Assumption. The phase state is a separable topological space, and the transition function b(·, ·) is upper semicontinuous, that is, its hypograph {(x, y, u) ∈ X ×X × R : u 6 b(x, y)} is a closed subset.

    In fact, the functional B is still undefined, since we did not point out the upper limit for the summation. If N ∈ N is a finite integer, then one can introduce the functional

    BN (x) = N−1∑ t=0

    b(xt, xt+1).

    Replacing B by BN , we obtain a correct finite horizon optimization problem of a common type.

    Proposition. Under the semicontinuity assumption if the phase state is compact and the horizon N finite, then the optimization problem

    BN (x)→ max, x0 = a


    always possesses a solution.

    The difficulties arise when defining something like B∞. The above finite horizon problem admits a natural generalization. Let f : X → R

    be any upper semicontinuous function. Then we may add to the criterion the terminal term f(xN ). The problem thus obtained will be of the form (we write the unabridged expression)

    N−1∑ t=0

    b(xt, xt+1) + f(xN )→ max(1.1)

    xt ∈ X, t = 0, 1, . . . , N, x0 = a ∈ X.(1.2)

    A solution to the problem is an N + 1-tuple from XN+1. The set of all solutions will be denoted by extrN (b, f). The above proposition holds also for this case.

    This generalization points out another peculiarity of the infinite horizon case. Indeed, the latter does not admit any terminal terms besides almost meaningless ones like f( lim

    t→∞ xt). At the same time if we will think of the infinite horizon

    optimization problem as a limit (in a sense) of regular finite horizon problems, the terminants may occur in the latter one


View more >