Three-Dimensional Air Combat:Numerical Solution of Complex Di�erential GamesR. Lachner�, M. H. Breitnery, and H. J. PeschzMay 16, 1996AbstractComplex pursuit-evasion games with complete information under statevariable inequality constraints are investigated. By exploitation of Isaacs'minimax principle, necessary conditions of �rst and second order are de-rived for the optimal trajectories. These conditions give rise to multipointboundary-value problems, which yield open-loop representations of the op-timal strategies along the optimal trajectories. The multipoint boundary-value problems are accurately solved by the multiple shooting method.The computed open-loop representations can thereafter be used to syn-thesize the optimal strategies globally.As an illustrative example, the evasion of an aircraft from a pursuingmissile is investigated. The ight of the aircraft is restricted by variouscontrol variable inequality contraints and by a state variable inequalityconstraint for the dynamic pressure. The optimal trajectories exhibitboundary arcs with regular and singular constrained controls. The in- uence of various singular surfaces in the state space including a low-dimensional universal surface is discussed.Key Words. Di�erential games, pursuit-evasion games, singular sur-faces, multipoint boundary-value problems, multiple shooting method.1 IntroductionDi�erential games are useful to describe many aspects of human interactions.Unlike optimal control problems, they deal with the competitive character ofthose interactions. Even unpredictable disturbances can be investigated, if\nature" is taken into account as an additional player. The solution of dif-ferential games is much more involved than the solution of optimal controlproblems. This is due to the fact that only optimal feedback-type strategies�Assistant Professor. E-mail: lachner@math.tu-clausthal.de.yAssistant Professor. E-mail: breitner@math.tu-clausthal.de.zProfessor of Mathematics. E-mail: pesch@math.tu-clausthal.de.All authors' address: Institute of Mathematics, Clausthal University of Technology, Erzstr. 1,D{38678 Clausthal-Zellerfeld, Germany.Formerly, Department of Mathematics, Munich University of Technology, D{80290 Munich,Germany. 1

are suitable to describe the global behavior of the players, especially if one ofthem acts nonoptimally. A global computation of these optimal strategies isgenerally impossible, since the state space is split up by singular surfaces evenfor simple di�erential games. Another reason is that direct optimization meth-ods, which are easy to deal with, do not exist so far for di�erential games. Directoptimization methods for optimal control problems (see, e. g., von Stryk [30])are incompatible with the information structures usually associated with di�er-ential games. Moreover, feedback control schemes that can be applied in realtime, have been developed even for complicated optimal control problems (seethe survey in [25]) whereas only a few real-time feasible and generally applicablemethods can be found which have already been applied to di�erential games;see [3], [4], [5], and [27].In the present paper, optimal evasive maneuvers of an aircraft against anoptimally guided missile are investigated. The results of [21], where the missile-versus-aircraft scenario has been restricted to a common vertical plane, are ex-tended here to pursuit-evasion maneuvers in three space dimensions. Thereby,the maneuverability of the aircraft is restricted by several control and state vari-able inequality constraints. Various initial conditions are investigated includingthe head-on shoot.The application of Isaacs' minimax principle yields complicated optimal con-trol histories including constrained and singular subarcs. These control historiescoincide with the optimal strategies along the optimal trajectories. A combin-ation of all the necessary conditions from the minimax principle gives rise tomultipoint boundary-value problems with interior point conditions and jumpconditions for the state and the adjoint variables. These multipoint boundary-value problems are then solved by means of the multiple shooting method; seeBulirsch [9], Stoer and Bulirsch [29], and the references cited in [25]. For themost recent code used in this paper, see Hiltmann et. al. [15].By means of homotopy techniques, the optimal trajectories and the associ-ated open-loop representations of the optimal strategies can be computed in theentire capture zone. These open-loop representations provide the informationby which optimal strategies can be approximated globally, e. g., by a successivesolution of neighboring multipoint boundary-value problems (see [2]), by Taylorseries expansions around many optimal trajectories (see [3], [4]), or by neuralnetworks (see [27]). Therefore, this work can be considered as a �rst step to-wards the real-time computation of optimal strategies for realistic di�erentialgames of pursuit-evasion type.2 Pursuit-Evasion GameThe following pursuit-evasion game arises as a subproblem for an air combatscenario between two aircraft. Similar problems have been treated recently withvarious solution methods; see, e. g., Fink [10], Greenwood [11], Guelman, Shinar,2

Green [13], Gutman, Katz [14], and the papers [6], [7], [19], [20], and [26] of theauthors. At time t = 0 the pursuing aircraft launches a medium-range air-to-airmissile P , which carries on pursuing the evading aircraft E. The missile P ismuch more maneuverable then the aircraft E, but can accelerate only a shorttime due to its limited fuel supply. As a result, E can always escape if theinitial distance between the opponents is large enough. Therefore, only initialconstellations for which P can enforce capture against all possible maneuversof the aircraft are considered here. This aerial scenario can be formulated asa pursuit-evasion game with the missile P identi�ed with the pursuer and theaircraft E with the evader.Note that the restriction of the ight paths to a common vertical plane asin [6], [7], [19], [20], [21], and [26] is discarded here. Therefore, the computationof optimal ight paths for almost arbitrary initial constellations including theso-called head-on shoot can be performed. As a by-product, the two-dimensionaloptimal trajectories computed in the above-mentioned papers, will turn out tobe optimal in the three-dimensional space, too, if the initial velocity vectors liein a common vertical plane.The state of P and E is described by their position vectors and their veloctityvectors. The equations of motion for the missile P and the aircraft E then readas follows (see, e. g., Miele [24]),_xP = fxP = vP cos P cos�P ;_xE = fxE = vE cos E cos�E ;_yP = fyP = vP cos P sin�P ;_yE = fyE = vE cos E sin�E ;_hP = fhP = vP sin P ;_hE = fhE = vE sin E ;_vP = fvP = TP;max(t)�D0;P (hP ; vP )� n 2P DI;P (t; hP ; vP )mP (t)�g sin P ; (1)_vE = fvE = �E TE;max(hE ; vE)�D0;E(hE ; vE)� n 2E DI;E(hE ; vE)mE�g sin E ;_ P = f P = gvP (nP cos�P � cos P ) ;_ E = f E = gvE (nE cos�E � cos E) ;_�P = f�P = gvP nP sin�Pcos P ;_�E = f�E = gvE nE sin�Ecos E : 3

Subscripts P and E refer here to the missile P and the aircraft E. The statevariables x, y, h, v, , and � denote the two coordinates of the horizontalposition, the altitude, the velocity, and the vertical and horizontal ight pathangle of P and E, respectively. The thrust TE = �E TE;max of the aircraftis controlled by the throttle �E 2 [�E;min; �E;max]. The thrust TP = TP;maxof the missile is not controllable. It is assumed to be piecewise constant withtwo discontinuities at t = 3 [s] and t = 15 [s] indicating the transitions fromthe boost phase to the sustain phase and from the sustain phase to the coastphase. The control variables nP and nE denote the load factors, which gov-ern the ight path angles. The bank angles �P and �E denote control vari-ables peculiar to the three-dimensionality of the ight scenario. They determ-ine the radii of curvature. Since only ight times less then 90 seconds occur,the mass mE of the aircraft is assumed to be constant neglecting the e�ectsof fuel consumption (�mE < 0:1mE). The mass mP of the missile decreaseslinearly during the boost and the sustain phase and is constant during the coastphase. The constant g denotes the gravitational acceleration. The complic-ated functions TE;max, D0;P , D0;E, DI;P , and DI;E are based on data for theF15E-Strike Eagle multi-role combat aircraft and the medium-range air-to-airmissile AMRAAM AIM-120A. These data provide realistic approximations forthe maximum thrust of the aircraft, the zero-lift drag, and the induced drag.The coe�cients for the approximations have been determined by the method ofleast squares.At time t = 0 the game starts with the initial conditionsxP (0) = 0 [m] ; xE(0) = xE;0 [m] ;yP (0) = 0 [m] ; yE(0) = yE;0 [m] ;hP (0) = hP;0 [m] ; hE(0) = hP;0 [m] ;vP (0) = vP;0 [m/sec] ; vE(0) = vE;0 [m/sec] ; P (0) = 0 [deg] ; E(0) = E0 [deg] ;�P (0) = 0 [deg] ; �E(0) = �E;0 [deg] : (2)These initial conditions are abbreviated by z(0) = z0 withz = (xP ; xE ; yP ; yE ; hP ; hE ; vP ; vE ; P ; E ; �P ; �E)> :Note that xP (0) = 0, yP (0) = 0 and �P (0) = 0 can be prescribed without loss ofgenerality, since these initial conditions can always be obtained by a translationand rotation of the coordinate system. The dimension of the state space can bereduced from 12 to 9 de�ning the new variables xE�xP , yE�yP , and �E��P .The condition P (0) = 0 is due to the technical reason that the launch of amissile is dangerous for the carrying aircraft when 6= 0.By choosing admissible values for the control variables of both players inEqs. (1), a trajectory is determined uniquely in the state space. The terminaltime tf of the game is determined by the capture condition�xP (tf )� xE(tf )�2 + �yP (tf )� yE(tf )�2 + �hP (tf )� hE(tf )�2 � d2 = 0 ; (3)4

where the capture radius d is chosen as 50 [m]. Equation (3) de�nes a hyper-manifold in the state space called the terminal manifold. If P is not able toenforce capture against all possible maneuvers of E, the outcome of the gameis \E not captured" and tf is set to in�nity.The players choose their strategies �P and �E under the aspect of minim-aximizing the objective functionalJ(�P ;�E ; t = 0; z = z0) = tf : (4)The pursuer P engages in driving the state z from the initial state z0 to theterminal manifold in minimal time, whereas the evader E tries to avoid captureor, if this is impossible, tries to maximize the capture time tf .For a realistic modelling, several constraints have to be taken into account.The most important constraint for the evader, which shall particularly be dis-cussed here, is the limit of the dynamic pressure qE ,Q(hE ; vE) = qE(hE ; vE)� qE;max � 0 (5)with qE = 12�(hE) vE2, qE;max = 80 [kPa], and � denoting the air density. This�rst-order state variable inequality constraint keeps the aircraft away from the utter boundary and limits the static load. For certain initial conditions of thestate, it is advantageous for the aircraft to descent, in order to maximize thrust,to gain additional kinetic energy, and to force the missile to follow into regionsof high drag. In this case the altitude constrainthE � 0 ; (6)a second-order state variable inequality constraint, may become active. How-ever, optimal trajectories with altitude-constrained subarcs will not be presentedhere; see [19] instead. Finally, the control variables are bounded bynP 2 [nP;min; nP;max] = [0; 20] ;nE 2 [nE;min; nE;max] = [0; 7] ;�E 2 [�E;min; �E;max] = [0:23; 1] : (7)Note that, unlike the case in the two-dimensional version of this game in [6], [7],[19], [20], [21], and [26], the nE;max-constraint is more restrictive, since sharpturns of E are optimal for many initial constellations; see also [10]. The Machlimit at high altitudes and the maximum lift coe�cient for low speeds are ofsecondary importance and are not taken into account. These constraints can,however, be included in a similar way. Also a visibility condition for the missileto keep the aircraft in its radar cone, can be included analogously.5

3 Necessary Conditions for Optimality3.1 Di�erential Game ApproachIn the sequel it is assumed that the game is started with an arbitrary initialstate z0 for which capture is guaranteed against all admissible strategies of E,provided that P plays optimally. The set of all optimal trajectories emanatingfrom these initial states span a subset in the state space, the capture zone.It is separated from the remainder of the state space, the escape zone, by asubmanifold, the barrier. Note that the derivation of necessary conditions inthis section is valid only for initial states in the capture zone.Only admissible strategiesuP (t; z) = � nP (t; z)�P (t; z) � = �P (t; z) ;uE(t; z) = 0@ nE(t; z)�E(t; z)�E(t; z) 1A = �E(t; z) (8)are considered; see, e. g., Ba�sar and Olsder [1]. The players P and E haveperfect information about the actual state, but no information about the actualor future control of the opposite player. A strategy �P � is called optimal for Pif J(�P �;�E ; t; z) � J(�P ;�E ; t; z) (9)holds for all admissible strategies �E , for all t, and for all z within the capturezone. Vice versa, a strategy �E� is called optimal for E ifJ(�P ;�E; t; z) � J(�P ;�E�; t; z) (10)holds for all admissible strategies �P , for all t, and for all z within the capturezone.The optimal value V � of the objective function as a function of the actualtime t and the actual state z is de�ned byV �(t; z) = J(�P �;�E�; t; z) : (11)A trajectory generated by optimal strategies will be called an optimal tra-jectory. Along optimal trajectories, the optimal strategies yield optimal con-trols uP �(t) := uP �(t; z(t)) and uE�(t) := uE�(t; z(t)). Henceforth, we willspeak about optimal controls if we have the values of the optimal strategiesalong optimal trajectories in mind.The minimax principle of Isaacs [16] and [17] yields local necessary conditionswhich must be satis�ed by the optimal controls uP � and uE�, i. e., for all t along6

the optimal trajectories. De�ning auxiliary functions by~uP (t; z; V �t (t; z); V �z (t; z); uE) = arg minuP ddtV �(t; z)= arg minuP �V �t (t; z) + V �z (t; z) f(t; z; uP ; uE)� ;~uE(t; z; V �t (t; z); V �z (t; z); uP ) = arg maxuE ddtV �(t; z)= arg minuE �V �t (t; z) + V �z (t; z) f(t; z; uP ; uE)� ;(12)the optimal controls must satisfyuP �(t; z; V �t (t; z); V �z (t; z))= arg minuP �V �t (t; z) + V �z (t; z) f(t; z; uP ; ~uE(t; z; V �t ; V �z ; uP ))� ;uE�(t; z; V �t (t; z); V �z (t; z))= arg maxuE �V �t (t; z) + V �z (t; z) f(t; z; ~uP (t; z; V �t ; V �z ; uE); uE)� ; (13)for all t.In Eqs. (12) and (13), the minimization and maximization must be per-formed only for controls uP and uE which are admissible in t. Because of theseparability of dV �=dt, the auxiliary functions ~uP and ~uE coincide with the op-timal controls uP � and uE�. Therefore, one need not to distinguish upper andlower values of the game. Since the game has a terminal payo� [see Eq. (4)],the optimal value V � satis�es the nonlinear partial di�erential equation of �rstorder V �t + V �z f (t; z; uP � (t; z; V �t ; V �z ) ; uE� (t; z; V �t ; V �z )) = 0 (14)for all (t; z) in the capture zone. This equation is known as Isaacs equation(see [16] and [17]). Unfortunately, Eq. (14) cannot be used directly for com-puting a pair of optimal strategies, since it is a nonlinear partial di�erentialequation for 13 independent variables here. A state space reduction to theessential variables can decrease the dimension only to 10. Moreover, the func-tion V � is not continuously di�erentiable in the entire capture zone. However,an equivalent system of ordinary di�erential equations along a characteristiccurve can be formulated identifying the gradient (Vt�; VxP �; : : :)> at (t; z) withthe merely time-dependent adjoint functions (��; �xP ; : : :)>. For this purpose,the system (1) is �rstly rewritten in autonomous form using � = t as a new statevariable, ddt � �z � = � 1f(�; z; uP ; uE) � : (15)The new initial condition for � is �(0) = 0. Secondly, the system for the adjoint7

variables is given, according to [17], byddt � ��� � = �� @H@(�; z)�>= � �> @f(�; z; uP �; uE�)@(�; z)+ @H@(uP ; uE) @@(�; z) � uP �uE� �!> ; (16)where the Hamiltonian is de�ned byH(�; z; ��; �; uP ; uE) = �� + �>f(�; z; uP ; uE) : (17)Note that there holdsH(�; z; ��; �; uP �; uE�) = ddtV �(�; z) : (18)Following this procedure, the optimal controls uP � and uE� in Eq. (13) can nowbe identi�ed along optimal trajectories throughuP �(t) = uP ��t; z(t)� = uP ��t; z(t); V �t (t; z(t)); V �z (t; z(t))�= uP ��t; z(t); ��(t); �(t)� ;uE�(t) = uE��t; z(t)� = uE��t; z(t); V �t (t; z(t)); V �z (t; z(t))�= uE��t; z(t); ��(t); �(t)� ; (19)and the optimal strategies �P � and �E� are obtained along optimal trajectoriesfrom the optimal control histories uP � and uE� via�P �(t; z(t)) = uP �(t) ; �E�(t; z(t)) = uE�(t) : (20)These equations provide an open-loop representation of the optimal strategies;for details see [1].3.2 Open-Loop Representations of Optimal Strategies forthe MissileIn this subsection, it is assumed that the nP;max-constraint is not active duringthe entire maneuver. This assumption is valid for all numerically computedoptimal trajectories presented in Section 4. Furthermore, it can be shown thatnP �(t) = nP;min holds only for isolated t; see Grimm [12]. FromH(�; z; ��; �; nP ; �P ; nE ; �E ; �E) = H1(�; z; �)nP 2+H2(z; �)nP cos�P +H3(z; �)nP sin�P+H4(�; z; ��; �; nE ; �E ; �E) ; (21)8

the optimal controls of the missile P are computed using the minimax principle,@@nP H(�; z; ��; �; nP �; �P �; nE ; �E ; �E) = 0 ;@@�P H(�; z; ��; �; nP �; �P �; nE ; �E ; �E) = 0 ;@2@nP 2H(�; z; ��; �; nP �; �P �; nE ; �E ; �E) � 0 ;@2@�P 2H(�; z; ��; �; nP �; �P �; nE ; �E ; �E) � 0 : (22)This yields nP � = mP g UP2DI;P vP �vP ; (23)cos�P � = �� PUP ; (24)sin�P � = � ��PUP cos P (25)with UP =q� P 2 + (��P = cos P )2and the necessary sign condition �vP � 0.3.3 Open-Loop Representations of Optimal Strategies forthe AircraftThe di�erent optimal control laws for E depend on the functionUE =q� E 2 + (��E= cos E)2 : (26)During the numerical computations it turned out that the singular case UE � 0can occur only on dynamic-pressure-constrained subarcs. Hence, we will haveto distinguish between regular and singular constrained subarcs. However, we�rst consider state-unconstrained subarcs.3.3.1 Dynamic Pressure Constraint InactiveThe optimal controls of E can be obtained by maximizing the HamiltonianH(�; z; ��; �; nP ; �P ; nE; �E ; �E) = �H1(z; �)nE2+ �H2(z; �)nE cos�E + �H3(z; �)nE sin�E+ �H4(z; �) �E + �H5(�; z; ��; �; nP ; �P ) (27)9

over the set of admissible controls [nE;min; nE;max]�]� �; �] � [�E;min; �E;max].This yields nE� = 8><>: nE;max ; if �vE � 0,min� mE g UE2DI;E vE �vE ; nE;max� ; if �vE > 0, (28)cos�E� = � EUE ; (29)sin�E� = ��EUE cos E ; (30)�E� =8><>: �E;min ; if �vE < 0,unde�ned ; if �vE = 0,�E;max ; if �vE > 0. (31)3.3.2 Dynamic Pressure Constraint ActiveUnlike the situation in the case of optimal control problems, state variable in-equality constraints or, more precisely, state constraints of order greater thanzero, are not well understood in di�erential games. This obstacle can generallybe remedied by strengthening the state constraint by a mixed control-state con-straint; see, e. g., [2], and [21]. Their theoretical treatment is well understood.Unfortunately, this procedure leads to optimal solutions with a chattering con-trol here due to the nonconvexity of the hodograph. An extended class ofstrategies would have been permitted then which, however, cannot be imple-mented for aircraft. Therefore, the well-known necessary conditions of optimalcontrol theory for state constraints of higher order (see, e. g., Bryson, Denham,and Dreyfus [8], Jacobson, Lele, and Speyer [18], and Maurer [23]) are appliedto the present di�erential game despite the lack of a rigorous proof in the con-text of di�erential games. This approach can be justi�ed according to [21] asfollows. The state-constrained di�erential game problem can be imbedded intoa one-parameter family of more stringent mixed-state-control-constrained prob-lems. By weakening these control-state-constraints, numerical convergence ofthe associated one-parameter family of solutions has been observed in [21] to-wards the solution associated with those necessary conditions. For more details,see [21].If the dynamic pressure constraint holds, the set of admissible controls isrestricted by the condition�(hE) vE�12 @ ln(�(hE))@hE vE2 sin E � g sin E+ 1mE ��E TE;max �D0;E � nE2DI;E�� � 0 : (32)10

This is implied by the �rst derivative of the dynamic pressure constraint (5).The aircraft E again attains its optimal controls by maximizing the Hamilto-nian. Since the set of admissible controls is restricted by Eq. (32), this conditionis, according to [8], adjoined toH by a Lagrange multiplier �qE which necessarilymust be nonpositive. The optimal controls are then given bynE� = " 1DI;E�12 @ ln(�(hE))@hE mE vE2 sin E�mE g sin E + TE;max �D0;E�#1=2 ; (33)cos�E� = � EUE ; (34)sin�E� = ��EUE cos E ; (35)�E� = �E;max ; (36)provided that �vE > 0 and UE > 0. From a discussion of the Hamiltonian, therefollows that �vE < 0 cannot occur on dynamic-pressure-constrained subarcs;compare [21]. Note that the adjoint equations di�er from those on unconstrainedsubarcs, denoted by the superscript \free",_� = _�free � �qE � @@z ddtQ(z)�> : (37)Furthermore, the interior point condition qE(tentry) = qE;max at the junctionpoint tentry induces discontinuities in the adjoint variables, namely�(tentry+) = �(tentry�)� � � @@zQ(z)�> (38)with a necessarily nonpositive parameter �,� = �qE (tentry+) : (39)If UE � 0 holds on a certain subinterval, the Hamiltonian has no uniquemaximum on the set of admissible controls. From ��E � 0 and _��E � 0, it canbe derived �E = const yieldingsin�E� = 0 ; (40)cos�E� 2 f�1; 1g : (41)Note that on singular subarcs, the optimal ight path of E is in a vertical plane.The control that keeps � E � 0 can be obtained by di�erentiating this identity11

twice with respect to time. Then there holdsnE� cos�E� = cos E �1 + 12 g @ ln(�(hE))@hE vE2� : (42)Since the activity of the dynamic pressure constraint must be maintained,Eq. (32) with the equality sign yields�E� = 1TE;max�D0;E +mE g sin E�12 @ ln(�(hE))@hE mE vE2 sin E +DI;E nE�2� : (43)An additional necessary condition on singular subarcs is@@(nE cos�E) �� E � 0 : (44)As long as this condition holds, the singular subarc cannot be left withoutcontradiction to the minimax principle. For, one obtains two local maximaof the Hamiltonian for all nE cos�E di�ering from the respective value (42)on the singular subarc. None of these maxima can be chosen as candidateoptimal, since the other immediately yields a larger value of the Hamiltonian;see [21] for more details. Equation (44) is always satis�ed here on singularsubarcs. Hence, optimal trajectories always terminate on singular subarcs, ifthey include a singular subarc and if no other constraint is violated by thesingular control. If the game terminates on a singular subarc, it can be easilyshown from _� E(tf ) = 0 that the velocity vector of E at the �nal time tf isparallel to the line of sight.Finally, it should be mentioned that singular subarcs are located in a singularsurface of universal type. Entering the universal surface, the adjoint variablesare continuous.3.4 Jump Conditions and Interior Point ConditionsThe following conditions are related to the transition points (i) between the dif-ferent thrust phases of the missile, (ii) between nE;max-constrained subarcs with�E� = �E;min and nE;max-constrained subarcs with �E� = �E;max, (iii) betweennE;max-constrained subarcs and free subarcs, (iv) between free subarcs andqE;max-regular-constrained subarcs, (v) between qE;max-regular-constrainedsubarcs and free subarcs, and (vi) between qE;max-regular-constrained subarcsand qE;max-singular-constrained subarcs.(i) The discontinuities in the right-hand side of the dynamic system (1)induced by the di�erent thrust phases of P , create singular surfaces in the statespace which lead to jumps in the adjoint vector when the optimal trajectory12

penetrates these surfaces. The two transition points t1 = 3 [s], and t2 = 15 [s]between the di�erent thrust phases must be treated as interior point conditions.The continuity of the Hamiltonian implies that �� has discontinuities at ti,i = 1; 2, ��(ti�)� ��(ti+) = �vP (ti)mP (ti) �TP;max(ti+)� TP;max(ti�)� ; (45)see Leitmann [22].(ii) The switching point t�E associated with that junction is determined bythe interior point condition �vE (t�E ) = 0 : (46)At this point, a bang of �E� from the minimal admissible value to the maximaladmissible value takes place.(iii) The optimal load factor nE� maximizing the Hamiltonian H movesfrom the nE;max-boundary into the interior of [nE;min; nE;max]. Therefore, theassociated switching point tnE is determined by [compare (28)]mE g UE2DI;E vE �vE ����t=tnE = nE;max : (47)(iv) It can be shown that the control variables of the aircraft E are continu-ous at the transition point tentry into a dynamic-pressure-constrained subarc.The adjoint vector is discontinuous, the jump is given by Eq. (38). Note thatthe adjoint equations must be modi�ed according to Eq. (37) on those subarcs.The switching point tentry is determined by the interior point condition12�(hE(tentry)) vE(tentry)2 = qE;max : (48)(v) The optimal load factor nE� maximizing the Hamiltonian does no longersatisfy the condition (d=dt)Q = 0. Therefore, the optimal trajectory leavesthe dynamic pressure constraint. The exit point texit is determined by [com-pare (28)] mE g UE2DI;E vE �vE ����t=texit = nE�(texit�) : (49)Note that this condition is equivalent to �qE (texit�) = 0.(vi) The optimal control variables nE� and �E� are discontinuous at thetransition point tsing into a singular subarc. The following conditions can beshown, UE(tsing�) = 0 ; (50)_�E(tsing�) = 0 ) ( sin�E�(tsing�) = 0 ;cos�E�(tsing�) 2 f�1; 1g ; (51)13

_� E (tsing�) = 0 ; (52)_��E (tsing�) = 0 : (53)3.5 Two-Point Boundary ConditionsThe initial conditions are given by Eqs. (2). In order to obtain conditionsat t = tf for the adjoint variables, the terminal manifold described by Eq. (3)is parameterized; see [17]. Partial di�erentiation of the value V �(�(tf ); z(tf ))with respect to the parameters yields a system of linear equations for �(tf ).One obtains, for trajectories not ending on the dynamic pressure limit,�xE (tf ) [yE(tf )� yP (tf )] � �yE (tf ) [xE(tf )� xP (tf )] = 0 ;�xE (tf ) [hE(tf )� hP (tf )]� �hE (tf ) [xE(tf )� xP (tf )] = 0 ;�xP (tf ) + �xE (tf ) = 0 ; �yP (tf ) + �yE (tf ) = 0 ;�hP (tf ) + �hE (tf ) = 0 ; (54)�vP (tf ) = 0 ; �vE (tf ) = 0 ;� P (tf ) = 0 ; � E (tf ) = 0 ;��P (tf ) = 0 ; ��E (tf ) = 0 ;��(tf ) = 1 ;and for those which do,�xE (tf ) [yE(tf )� yP (tf )] � �yE (tf ) [xE(tf )� xP (tf )] = 0 ;�xE (tf ) [hE(tf )� hP (tf )]� �hE (tf ) [xE(tf )� xP (tf )] = 0 ;�xP (tf ) + �xE (tf ) = 0 ; �yP (tf ) + �yE (tf ) = 0 ;�hP (tf ) + �hE (tf )� 12 �vE (tf ) @ ln(�(hE(tf )))@hE(tf ) vE(tf ) = 0 ; (55)�vP (tf ) = 0 ; �vE (tf ) = arbitrarily nonnegative ;� P (tf ) = 0 ; � E (tf ) = 0 ;��P (tf ) = 0 ; ��E (tf ) = 0 ;��(tf ) = 1 :Together with the Isaacs equationH(�; z; ��; �; nP �; �P �; nE�; �E�; �E�)��t=tf = 0 ; (56)the adjoint variables are determined at t = tf . An additional terminal condition,which determines the �nal time tf is given by the capture condition (3).14

3.6 Singular SubarcsOn state-constrained subarcsqE(hE ; vE) � qE;max ; (57)the optimal controls may become singular indicated by UE � 0. This equationis equivalent to � E � 0 ; (58)��E � 0 : (59)These three identities de�ne a universal-type singular surface of codimensionthree.For the 12 state variables, the 13 adjoint variables, and the parameters tentry,tsing, and tf , there are 28 interior and boundary conditions to be satis�ed,namely the 12 initial conditions (2), the capture condition (3), the 13 terminalconditions (55) and (56), the entry condition (48) into the state-constrainedsubarc, and the entry condition (50) into the singular subarc.It should be mentioned, that Eq. (50) itself cannot be chosen for numericalreasons. Instead, one of the conditions� E (tsing) = 0 ; (60)��E (tsing) = 0 : (61)is prescribed. The other one is checked afterwards and can always be found tobe ful�lled with an absolute error of at most 10�6.If additional constraints become active here, e. g., the altitude constraint (6),the switching structure might become even more complicated; see [12] and Sey-wald, Cli�, and Well [28] for similar problems in optimal control where morecomplicated switching structures occur due to prescribed terminal conditions.This section completes now the derivation of well-de�ned multipoint bound-ary-value problems for all switching structures that have been obtained duringthe numerical computations by means of the multiple shooting method.4 Numerical ResultsAlthough superior with respect to accuracy and reliability, the multiple shoot-ing method su�ers from the fact that an initial estimate of all variables mustbe provided. This di�culty is surmounted by using a trajectory from [21] asinitial guess where ight scenarios in a vertical plane have been investigated.For, if the initial positions and the velocity vectors of P and E are placed in acommon vertical plane, the optimal trajectories remain in this plane. The pro-cedure in [21] is based on the following idea: Firstly, an optimal control problem15

is established which is related to a family of di�erential games including the dif-ferential game to be solved. Secondly, the optimal control problem is solved bymeans of a robust direct collocation method due to [30] which is not so exactingwith respect to an appropriate initial guess. This method additionally yields anapproximation for the adjoint variables and a hypothesis of the switching struc-ture. Thereafter, the multiple shooting method is employed in connection withhomotopy techniques to solve the di�erential games. Finally, the trajectoriesfor arbitrary initial conditions in the capture zone are computed by means ofhomotopy techniques, too.xP ;xE = [km]yP ;yE = [km]

hP ;hE = [km]

Figure 1: Optimal ight path of P (gray) and E (black) and projections intothe coordinate planes.To illustrate this, the following set of initial conditions is chosenxP;0 = 0 [m] ; xE;0 = 12500 [m] ;yP;0 = 0 [m] ; yE;0 = 1000 [m] ;hP;0 = 5000 [m] ; hE;0 = 6000 [m] ;vP;0 = 250 [m/s] ; vE;0 = 400 [m/s] ; P;0 = 0 [deg] ; E;0 = 0 [deg] ;�P;0 = 0 [deg] ; �E;0 = 120 [deg] : (62)Figure 1 shows the optimal ight paths of P and E in a (x; y; h)-coordinatesystem. Additionally, the projections of the ight paths into the coordinateplanes are drawn (thin dashed lines). It can be seen that the optimal ight pathsof P and E enter a common vertical plane. The missile P ascends immediately16

after the launch in order to exploit the low drag in the higher regions of theatmosphere. Since the initial angle �E;0 = 120 [deg] is quite unfavourable,the aircraft E is forced to turn right as sharp as possible. Hence, a nE;max-constrained subarc occurs during the �rst 7 seconds of the maneuver; see Fig. 3.

vP ; vE = [m/s]

hP ; hE = [km] ight envelope

Figure 2: History of h and v for P (gray) and E (black).Figure 2 shows the history of the optimal trajectory in an altitude-velocitydiagram. The shaded area marks the set of all pairs (hE ; vE) for which E canmaintain a stationary horizontal ight. The boundary of this set is called the ight envelope of the aircraft E. Note that the lower right part of the ight en-velope is determined by the dynamic pressure constraint. It should be mentionedthat only the mid course guidance of the missile is modelled here appropriately.The �nal approach immediately before the hit must be left to special guidancelaws. The corners in the curve of P mark the two transitions between the dif-ferent thrust phases of the missile. The e�ects of these transition points governalso the history of the optimal load factor nP � in Fig. 3. The corner in theoptimal load factor nE� at t = 27 [s] marks the junction point between inactiveand active dynamic pressure constraint. At t = 29 [s] the entry point of the sin-gular subarc is reached. Note that nE� as well as �E� are discontinuous at tsing;see Fig. 4. Figure 5 shows the history of the optimal throttle �E� which is alsodiscontinuous at tsing. However, the jump in �E� is below drawing accuracy.17

t = [s]

nP �; nE�

Figure 3: Optimal load factors for P (gray) and E (black).

t = [s]

�P �; �E� = [deg]

Figure 4: Optimal bank angles for P (gray) and E (black).18

t = [s]

�E�

Figure 5: Optimal throttle for E.In Fig. 6 eight optimal ight paths are depicted in the (x; y; h)-space. Forthe sake of visibility, only the projections into the (x; y)-plane are drawn. These ight paths belong to the initial conditionsxP;0 = 0 [m] ;yP;0 = 0 [m] ; yE;0 = 0 [m] ;hP;0 = 5000 [m] ; hE;0 = 10500 [m] ;vP;0 = 250 [m/s] ; vE;0 = 300 [m/s] ; P;0 = 0 [deg] ; E;0 = 0 [deg] ;�P;0 = 0 [deg] ; �E;0 = 90 [deg] ; (63)xE;0 = [m] 2 f3000; 4500; 6000; 7500; 9000; 10500; 12000; 13500g :The dynamic pressure constraint becomes active only for the trajectory withthe largest initial distance. The two optimal trajectories related to the smallestand the largest value of xE;0 lie close to the barrier. The di�erences of theoptimal ight paths, depending on the initial distance between P and E, areconsiderable.19

xP ;xE = [km] yP ;yE = [km]

hP ;hE = [km]

Figure 6: Optimal ight paths for di�erent initial constellations. Flight pathsof P are gray and those of E are black. Initial conditions di�er only in xE;0.Finally, Fig. 7 shows seven optimal ight paths related to the following setof initial conditionsxP;0 = 0 [m] ; xE;0 = 15000 [m] ;yP;0 = 0 [m] ; yE;0 = 0 [m] ;hP;0 = 10000 [m] ; hE;0 = 10000 [m] ;vP;0 = 250 [m/s] ; vE;0 = 500 [m/s] ; P;0 = 0 [deg] ; E;0 = 0 [deg] ;�P;0 = 0 [deg] ; (64)�E;0 = [deg] 2 f0; 30; 60; 90; 120; 150; 177g :As in Fig. 6, the turns of the optimal paths into common vertical planes canbe seen. Each trajectory terminates on a singular subarc. Note that the almosthead-on initial constellation for �E;0 = 177 [deg] is close to a dispersal surfacein the state space beyond which a symmetric situation can be found. For thetail-chase initial constellations, the missile P again ascends into the regions oflow drag. This behavior is optimal for P if the duration of the game is not tooshort. 20

xP ;xE = [km] yP ;yE = [km]

hP ;hE = [km]

Figure 7: Optimal ight paths for di�erent initial constellations. Flight pathsof P are gray, those of E are black. Initial conditions di�er only in �E;0.5 ConclusionsOptimal trajectories and their associated open-loop representations of the op-timal strategies can be computed for complicated pursuit-evasion games in theentire capture zone. These open-loop representations provide information bywhich optimal strategies can be approximated globally, e. g., by the subsequentsolution of neighboring boundary-value problems [2], by Taylor series expansionsaround many optimal trajectories ([3], and [4]), or by neural networks [27]. Thelast two of these methods are real-time feasible. This idea is generally applicableand not con�ned to the air-combat problem investigated in this paper.This air-combat problem includes a �rst-order state variable inequality con-straint for the dynamic pressure of the aircraft. The numerical solutions of thisdi�erential game have been obtained by applying the known necessary condi-tions of optimal control theory. In addition, the controls may become singularon state-constrained subarcs. The associated singular surface is of universal typeand has codimension three. All the necessary conditions have been examined fornot contradicting the minimax principle. More detailed and rigorous investiga-tions concerning di�erential games with state constraints of order greater thanzero are still pending. 21

AcknowledgementsThe authors would like to thank Professor R. Bulirsch of Munich Universityof Technology, Department of Mathematics, for his encouragement. The au-thors are also indebted to the Reviewers for their helpful comments. This workhas been partly granted by the German National Science Foundation withinthe Project \Applied Optimization and Control", and by FORTWIHR, theBavarian Consortium on High Performance Scienti�c Computing.REFERENCES[1] Bas�ar, T. , and Olsder, G. J., Dynamic Noncooperative Game Theory ,Academic Press, London, Great Britain, 1982, 2nd ed., 1994.[2] Breitner, M. H., Construction of the Optimal Feedback Controller forConstrained Optimal Control Problems with Unknown Disturbances , in:R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, Inter-national Series of Numerical Mathematics 115, Birkh�auser, Basel, Switzer-land, pp. 147{162, 1994.[3] Breitner, M. H., Real-Time Capable Approximation of OptimalStrategies in Complex Di�erential Games , in: M. Breton, G. Zaccour(Eds.), Proceedings of the Sixth International Symposium on DynamicGames and Applications, St-Jovite, Qu�ebec, GERAD, �Ecole des Hautes�Etudes Commerciales, Montreal, Canada, pp. 370{374, 1994.[4] Breitner, M. H., Robust-optimale R�uckkopplungssteuerungen gegen un-vorhersehbare Ein �usse: Di�erentialspielansatz, numerische Berechnungund Echtzeitapproximation, PhD thesis, Department of Mathematics, Mu-nich University of Technology, Munich, Germany, in preparation.[5] Breitner, M. H., and Pesch H. J., Reentry Trajectory Optimizationunder Atmospheric Uncertainty as a Di�erential Game, in: T. Ba�sar, andA. Haurie (Eds.), Advances in Dynamic Games and Applications, An-nals of the International Society of Dynamic Games 1, Birkh�auser, Basel,Switzerland, pp. 70{88, 1994.[6] Breitner, M. H., Grimm, W., and Pesch, H. J., Barrier Trajectoriesof a Realistic Missile/Target Pursuit-Evasion Game, in: R. P. H�am�al�ainen,and H. K. Ehtamo (Eds.), Di�erential Games | Developments in Model-ling and Computation, Lecture Notes in Control and Information Sciences156, Springer, Berlin, Germany, pp. 48{57, 1991.[7] Breitner, M. H., Pesch, H. J., and Grimm, W., Complex Di�erentialGames of Pursuit-Evasion Type with State Constraints, Part 1: Necessary22

Conditions for Optimal Open-Loop Strategies, Part 2: Numerical Compu-tation of Optimal Open-Loop Strategies , Journal of Optimization Theoryand Applications 78, pp. 419{441, 443{463, 1993.[8] Bryson, A. E., Denham, W. F., and Dreyfus, S. E., Optimal Pro-gramming Problems with Inequality Constraints, I: Necessary Conditionsfor Extremal Solutions , AIAA Journal 1, pp. 2544{2550, 1963.[9] Bulirsch, R., Die Mehrzielmethode zur numerischen L�osung von Rand-wertproblemen und Aufgaben der optimalen Steuerung , Report of the Carl-Cranz-Gesellschaft, DLR, Oberpfa�enhofen, Germany, 1971; Reprint, De-partment of Mathematics, Munich University of Technology, Munich, Ger-many, 1985 and 1993.[10] Fink, G., Numerische Berechnung optimaler Strategien f�ur ein realit�ats-nahes Di�erentialspiel einer Verfolgung im dreidimensionalen Raum, Dip-loma thesis, Department of Mathematics, Munich University of Techno-logy, Munich, Germany, 1994.[11] Greenwood, N. J. C., Applied Di�erential Games in the Aerial Scen-ario, PhD thesis, Department of Mathematics, University of Queensland,Brisbane, Australia, 1994.[12] Grimm, W., Lenkung von Hochleistungs ugzeugen. Vergleich von opti-maler Steuerung und fastoptimaler Echtzeitsteuerung , PhD thesis, Depart-ment of Mathematics, Munich University of Technology, Munich, Germany,1992.[13] Guelman, M., Shinar, J., andGreen, J., Qualitative Study of a PlanarPursuit Evasion Game in the Atmosphere, in: Proceedings of the AIAAGuidance, Navigation and Control Conference, Minneapolis, Minnesota,1988.[14] Gutman, S., and Katz, D., On Guaranteed-Cost, Closed-Form Guidancevia Simple Linear Di�erential Games , in: Proceedings of the 27th IEEEConference on Decision and Control, Austin, Texas, 1988.[15] Hiltmann, P., Chudej, K., and Breitner, M. H., Eine modi�zierteMehrzielmethode zur L�osung von Mehrpunkt-Randwertproblemen | Ben-utzeranleitung , Report No. 14, Deutsche Forschungsgemeinschaft, Sonder-forschungsbereich 255 \Transatmosph�arische Flugsysteme", Departmentof Mathematics, Munich University of Technology, Munich, Germany,1993.[16] Isaacs, R., Games of Pursuit , Paper No. P-257, RAND Corporation,Santa Monica, California, 1951. 23

[17] Isaacs, R., Di�erential Games , Wiley, New York, New York, 1965;Krieger, New York, New York, 1975.[18] Jacobson, D. H., Lele, M. M., and Speyer, J. L., New NecessaryConditions of Optimality for Control Problems with State Variable Inequal-ity Constraints , Journal of Mathematical Analysis and Applications 35,pp. 255{284, 1971.[19] Lachner, R., Realit�atsnahe Modellierung eines Di�erentialspieles mitZustandsbeschr�ankungen und numerische Berechnung optimaler Trajekt-orien, Diploma thesis, Department of Mathematics, Munich University ofTechnology, Munich, Germany, 1992.[20] Lachner, R., Breitner, M. H., and Pesch, H. J.,Optimal Strategies ofa Complex Pursuit-Evasion Game, in: R. Fritsch, and M. Toepell (Eds.),Proceedings of the 2nd Gauss Symposium, Munich, Germany, 1993; toappear in Journal of Computing and Information, 1995.[21] Lachner, R., Breitner, M. H., and Pesch, H. J., E�cient NumericalSolution of Di�erential Games with Application to Air Combat , ReportNo. 466, Deutsche Forschungsgemeinschaft, Schwerpunkt \Anwendungs-bezogene Optimierung und Steuerung", Department of Mathematics, Mu-nich University of Technology, Munich, Germany, 1993.[22] Leitmann, G., Einf�uhrung in die Theorie optimaler Steuerung und derDi�erentialspiele, R. Oldenburg, Munich, Germany, 1974.[23] Maurer, H., Optimale Steuerprozesse mit Zustandsbeschr�ankungen, Ha-bilitationsschrift, University of W�urzburg, W�urzburg, Germany, 1976.[24] Miele, A., Flight Mechanics I, Theory of Flight Paths , Addison Wesley,Reading, Massachusetts, 1962.[25] Pesch, H. J., O�ine and Online Computation of Optimal Trajectories inthe Aerospace Field , in: A. Miele and A. Salvetti (Eds.), Applied Math-ematics in Aerospace Science and Engineering, Plenum, New York, NewYork, pp. 165{219, 1994.[26] Pesch, H. J., Solving Optimal Control and Pursuit-Evasion Game Prob-lems of High Complexity , in: R. Bulirsch, and D. Kraft (Eds.), Compu-tational Optimal Control, International Series of Numerical Mathemat-ics 115, Birkh�auser, Basel, Switzerland, pp. 43{64, 1994.[27] Pesch, H. J., Gabler, I., Miesbach, S., and Breitner, M. H., Syn-thesis of Optimal Strategies for Di�erential Games by Neural Networks , in:G. J. Olsder (Ed.), Advances in Dynamic Games and Applications, Annalsof the International Society of Dynamic Games, this volume, Birkh�auser,Basel, Switzerland, 1995. 24

[28] Seywald, H., Cliff, E. M., andWell, K. H., Range Optimal Traject-ories for an Aircraft Flying in the Vertical Plane, Journal of Guidance,Control, and Dynamics 17, pp. 389-398, 1994.[29] Stoer, J., Bulirsch, R., Introduction to Numerical Analysis , Springer,New York, New York, 2nd ed., 1993.[30] von Stryk, O., Numerische L�osung optimaler Steuerungsprobleme:Diskretisierung, Parameteroptimierung und Berechnung der adjungier-ten Variablen, Fortschritt-Berichte VDI, Reihe 8, Nr. 441, VDI-Verlag,D�usseldorf, Germany, 1995

25