Autonomous optimal control problems

Autonomous optimal control problems


715KB Sizes 1 Downloads 51 Views

Vol. 51 (2003)


No. 2l3

AUTONOMOUS OPTIMAL CONTROL PROBLEMS B. LANGEROCK Department of Mathematical Physics and Astronomy, Ghent University, Krijgslaan 281 S9, B-9000 Ghent, Belgium (e-mail: [email protected] http:/ (Received September 2, 2002) A geometric version of the maximum principle for autonomous optimal control problems is derived and applied to the length-minimizing problem in sub-Riemannian geometry and to Lagrangian mechanics on Lie-algebroids. Keywords

Autonomous geometric control theory.

2QOOMSC: 49Kxx, 53Cxx.

1. Introduction

The development of a differential geometric setting for optimal control theory was carried out, among others, by H. J. Sussmann in [12], where a coordinate-free formulation of the maximum principle was given. In [4] we gave a proof of a coordinate-free version of the maximum principle for (time-dependent) optimal control systems with fixed endpoint conditions, relying on an approach due to L. S. Pontryagin et al. in [5]. As a side result of our approach, we were able to give some necessary and sufficient conditions for the existence of so-called (strictly) abnormaZ extremzls (for an example of a strictly abnormal extremal, we refer to [7]). In this paper, it is our goal to prove the maximum principle for autonomous optimal control problems and apply it to subRiemannian geometry [9, lo] and Lagrangian systems on Lie-algebroids [ 1, 6, 131. The outline of the paper is as follows. In the remainder of this section, we first recall the notion of a geometric control structure as described in [4] in a timedependent setting, as well as the notions of an optimal control problem with fixed and variable endpoints, respectively. We then consider an adapted version of these notions for the autonomous case. In Section 2 we briefly review the approach to the maximum principle presented in [3] and [4] and use it as a starting point to derive a version of the maximum principle for autonomous optimal control problems. Section 3 contains some specific results for linear autonomous optimal control problems and in Section 4 we discuss some applications. We start by recalling the definition of a geometric control structure (see [4] for more details). A geometric control structure is a triple (t, u, p), where (i) t : M + II2 P591



and (ii) u : U + M are fibre bundles, with typical fibres denoted by, respectively, r;! and C, where Q is called the conjguration space and C the control domain. The control bundle U is related to the first jet bundle of t by means of a bundle map (iii) p : U -+ J’t, with ti.0 o p = u (where r1.0 : J’s -+ M denotes the standard projection, see for instance [S]). This is represented schematically by the following commuting diagram:


Let u denote a smooth section of r o u, i.e. u : I = [a, b] + U such that = t (we assume that I is a compact interval in W and that u admits a r(iMt))) smooth extension to an open interval containing I). Then u is a control if pou = j’c with c = uou the base section of u. Given any section c of t, c is called a controlled section if c is the base section of a control. We say that the control u takes the point c(a) to the point c(b), and is represented schematically by c(a) -fk c(b). In order to fix the ideas we first investigate locally the notion of smooth controls. Let (t , x’) denote bundle-adapted coordinates on M and, similarly, let (t , xi, u’) denote bundle-adapted coordinates on U. Then the control u locally satisfies pi (t, ci (t), u”(t))

= 2 (t),

for all t. It is easily seen that these equations correspond to the definition of control in [SJ. However, it turns out that the class of smooth controls should be further extended to sections admitting (a finite number of) discontinu.ities in the form of certain ‘jumps’ in the fibres of u, such that the corresponding base section is piecewise smooth. For instance, assume that ~1 : [a, b] --+ U and u2 : [b, c] + U aTe two smooth controls with respective bases cl and 122,such that cl(b) = c~(b). The composite control 242- u 1 : [a, c] + U of ui and u2 is defined by Ul(f), I42 *

u1(t) = i U20)?



t E

[a,bl, lb,


It is readily seen that, although in general u2 1 u1 is discontinuous, at t = b, the base section u o (~2. ~1) is continuous. This definition can be easily extended to any finite number of smooth controls, yielding what we shall call, in general, a control (a detailed definition can be found in [4]).



Xn the remainder of this section we focus our attention on optimal control problems. Assume that a cost function L on the control space U is given, i.e. L E P(U). With any control u : [a, b] -+ M we are now able to define its cost ,7(u), J(u) = 1” L(u(t))dt



A control u taking x = u(u(a)) to y = u(u(b)) is said to be optimal if, for any other control u’ taking x to y we have J-(u) I JYU’) . The problem of finding the optimal controls taking a given point to another given point is called an optimal control problem with fixed endpoint conditions. On the other hand, assume that two immersed submanifolds i : Si + M and j : Sf + It4 are given. A control u taking a point x E i(Si) to a point y E j (Sf) is said to be (Si, Sf)-optimal if, given any other control u’ taking x’ E i(Si) to y’ E j($), one gets J(u) 5 J(u’). The problem of finding the (Si, Sf)-optimal controls taking a point in Si to a point in & is called an optimal controE problem with variable endpoint conditions (see [3]). An autonomous geometric control structure consists of a pair (c, 5), where i7 : C + Q is a bundle and 5 : C + T Q a bundle map fibred over the identity on Q. We can then consider the following geometric control structure (t , u, p), in the sense defined above, associated with (G, 5): 1. t : M = W x Q + W : (t, q) I-+ t(t, q) = t, 2. u : U = R x C + M : (t, p) I+ u(t, p) = (t, c(p)), and 3. p : U + J’t : (t, p) H (t, $(p)), since J’t S R x TQ . Let ii : [a, b] + C denote a curve in the control domain C. Then fi is called jkdmissible if $(2(t)) = Z(t), where Z(t) = [email protected](t)) is called the base of ii. We now translate concepts defined in the autonomous geometric control structure (G, 6) to known concepts in the associated geometric control structure (t , u, p), For instance, it is an easy exercise to see that every ~-admissible curve ii : [a, b] + C determines a smooth control u : [a, b] + U with u(t) = (t, i(t)). On the other hand, if we assume that u : [a, b] + U is a smooth control, then u can be written as u(t) = (t, ii(t)) since u is a section of t o u. The. curve fi : [a, b] + C is p-admissible since P(u(t)) = (t, 5$(t)) and j,‘c = (t, C(t)) (where c(t) = (t, c(t)) denotes the base of u). This correspondence between ~-admissible curves and smooth controls is easily extended to the more general notion of composite controls. Similar to the control setting, we say that the &admissible curve G takes Z(a) to Z(b). Consider a function L on C. Then we can define a cost function on U by considering L = p$, where pc : U + C denotes the natural projection. We define a functional on the set of &admissible curves by g(ii) = lb @(t))dt a




where we have used the same notation as in the previous section since J’(i;) = J(u), with u being the control associated with i. A $-admissible curve ii : [a, b] + C, taking p to q, is called optimal if, given any &admissible curve k’ : [a, b] + C taking p to q, one has J(i) 5 J(3). Note that ii is optimal with respect to L iff the associated control u(t) = (I, i(t)) is optimal with respect to L. We say that U is strongly optimal if, given any other control ii’ : [a’, b’] 4 C taking p to q, one gets J(6) 5 J(c’). Consider S, = W and i, : S, + R x Q : s I+ (s, p). Similarly we put S, = W with jy : S, + W x Q : s w- (s, q). Using the above notation, we can write Si = S,, and ,Sf = S,. It is now easily seen that a &admissible curve i, taking p to q is strongly optimal iff the associated control u is (S,, &)-optimal with respect to PEE. The problem of finding the optimal $admissible curves taking p to q, is called the autonomous optimal control problem. 2.

The maximum principle

2.1. Non-autonomous optimal contrd problems We now proceed towards the formulation of the maximum principle for nonautonomous optimal control problems, proven in [4], providing necessary conditions for optimal controls. We first define the notion of a multiplier of control u. For that purpose, we construct a l-parameter family of closed two-forms on U xy V*t (where Vt = ker Tt is the vertical subbundle of TM and V*t its dual). Let 6 be the closed two-form on the fibred product U x M T*M, obtained by pulling back the canonical symplectic form on T* M by the projection U x M T* M + T*M. Next, for any real number J. we can define a section crA of the fibration U xy T*M + U x~ V*t in the following way. Take u, E U,, q,,, E V$ and put ~~(&I, 17m)= (u,, (Y,), where a+,, E T,*M is uniquely determined by the conditions (i) (cx~,T(p(u,)))+U(u,) = 0 and (ii) crm projects onto q,,,. As usual, T : J’t + TM represents the total time derivative defined by T(j,‘c) = &~(a,), for jic E J’r arbitrary and with 3, the standard vector field on W. The mapping ai is smooth, as can be easily seen from the following coordinate expression: putting urn = (t, xi, u”) and I],,,= pi dx;‘,, a straightforward computation gives CA(t,

xi,Uatpi) =


Xi, U",


Xi, U')pi -


Xi, Id’), pi)


We can now use OA to pull-back the closed two-form i;, to a closed two-form on U x~ V*t, which will be denoted by WA= or:&. Herewith, we can introduce the following definition of a multiplier. DEFINITION 1. Given a control u : [a, b] + U, a pair (Q, h) consisting of a piecewise smooth section g of V*s along c = v o u and a real number h, is called the multiplier of u if the following conditions are satisfied: 1. i(ir(,),i(r))wk= 0 on every smooth part of the curve (u(t), q(t)),




2. given any to E [a, b], and if we put ok(u(to), q(to)) = (u(t~),c&

then the function u’ I-P (a~, [email protected](d))) + AL(u’) defined on u-‘(c(t0)) attains its global maximum for u’ = u(ta), 3. (q(t), kc)# (0,O) for all t E [a, b]. We then have the following result (cf. [4]). 1. Assume that x -11 y ad (q, A) with A 5 0.

that u is optimal.



Then there exists a

In [3] we have considered optimal control problems with variable endpoints. Assume that Si and Sf denote two immersed submanifolds of M, where either Si or Sf reduce to a point (note that if we assume u, with x -$ y, to be (Si, Sf>optimal, then u is also (Si, S))-optimal, where Si = Si and Si = {y] or Si = Sf and Si = {x}). Let us denote the annihilator of a linear subspace W in a vector space V by W”. THEOREM

2. tit



there exists a multiplier

1. h10, 2. ~A(U(U),q(U)) E aMb),

[email protected]))




E Si



E sf.







(q, A) such that


E uwysf)~“,

if if








2.2. Autonomous optimal control problems

Consider an autonomous optimal control problem as defined in Section 1 and how it is related to a non-autonomous optimal control problem. Let WQ denote the canonical symplectic form on T* Q, and consider the closed two-form &Q on C x Q T*Q, which is the pull-back of aQ under the projection ppQ : C xQ T*Q + T*Q. We can now prove the following version of the maximum principle for autonomous optimal control problems. THEOREM3. Zf a jkdmissible curve il : [a, b] + C is optimul with respect to L then there exist a piecewise smooth one-form G(t) along E(t) = c(ii(t)) and a real number h 5 0 such that: l* i(i(tj,;i(rjjwQ = -dhk(fi(t), fi(t)) on every smooth part of the curve (c, G)(t), with hA E P(C xQ T*Q) defined by hi(i$,, &,> = (JP, F(i$)) + hL(i$) for arbitrary (GL, &,) E C xQ T*Q; 2. given any t E I, the function

ii’H hA(ii’, G(t)) on Ccct) = c-‘(c(t)) attains a global maximum for ii’= G(t); 3. hk(ii(t), G(t)) = const for all t; 4. (e(t), I.) # 0 for all t E 1. Zf I? is strongly optimal then condition (3) is to be replaced by hk(ii(t), G(t)) = 0.



Proof: Since we already know from Section 1 that if ii is optimal, then the associated control u is also optimal in the non-autonomous setting. This correspondence permits us to apply Theorem 1. The remainder of this proof consists of translating the necessary conditions from the time-dependent setting to the autonomous one. First, note that V*t 2 II8 x T*Q and that the section a~ can be written as a*(~‘, <) = -hk(~c(u’), C)dt+< where (u’, <) E U XMV*t. Recalling the definition of a multiplier we know that there exists a piecewise smooth one-form q(t) = (r, G(t)) and h 5 0 such that u denotes the control associated with U, 1. i(ir(l),i/(,))W = 0 on every smooth part of the curve (u, q)(t), 2. given any t E I, the function U’H (ak(~(t), q(t)), T(p(u’))) + hL(u’) defined on u-‘(c(t)) attains a global maximum for u’ = u(t), 3. (r](t), h) # 0 for all t E I. The closed two form wk equals (with .a slight abuse of notations) C&J- dhA A dt. The function u’ * (ok(@)r N)), T(P(~‘))) + Wu’)

equals -h*(ii(t), fi(t)) + hk(pc(u’), G(t)). attains its global maximum for U’= u(t) Given any tangent vector X = X+X’& one has ixw~. = ii&, -dhk(%)dt+X’dhk. X’ = 1 and X = (i(t), G(t)), and Eq. (1)

We conclude that hh(pc(u’), fi(t)) also or for U’= pi = c(t). with X E T(CxpT*Q), E T(llx~V*t) If we assume that X = (h(t), G(t)), then is equivalently rewritten as ic;cij,;icrjj&~=


Since (i;, 6) solves the implicit Hamiltonian system with Hamiltonian hk, the function hA is constant on every smooth part of the curve (L(t), G(t)). Thus it remains to prove that h*@(t), G(t)) is continuous. Consider therefore a discontinuous point (at t = to) of ii(t), and assume that we have fixed an adapted coordinate chart containing it. Then, hi(z’tt), G”(t), e/(t)) 2 hA(E’(t), tJJa,iii(t)) for all wa. If we consider left and from the right for as a function of t. It now hk is zero on (ii(t), Q(t)).

this inequality and take successively the limit from the t -+ to, we obtain the continuity of h*@(t),

G”(t), iii(t))

remains to prove, in the case of strong optimality, that We make use of Theorem 2. From the fact that

~A(u(u), V(U)) = -hA(ii(a)y c(u))dt + fii(u)dx’ E (TSP)’ = V*ty we obtain hA(C(a), c(u)) = 0.


Before proceeding to the next section, we introduce some additional definitions. Assume that a &admissible curve G is given. A pair (6, h), where fi denotes a one form along the base of ii and a real number h, is &led a Zocul multiplier if the conditions (l), (3) and (4) from Theorem 3 are satisfied. If, in addition, condition (2) is satisfied then (fi, Ai>is called a global multiplier. Note that the implicit Hamiltonian system in condition (l), implies that hk attains a local extremum which



justifies the above definitions. It is well known from literature that the +ulmissible curve i is called a gZobu2 (local) extremal if it admits a global (local) multiplier (fi, A) with h I 0. Furthermore if h = 0, then ii is &led an abnormal extremal, and if h < 0, then fi is called a normal extremal. Using these definitions, Theorem 3 says that any optimal @admissible curve is a global extremal. Note that, given any global multiplier (fi, A), for any (II> 0, the pair (kj, aA) is also a multiplier. Therefore, we shall henceforth always assume that the multiplier (fi, A) is ‘normalised’ in the sense that h equals 0, 1 or -1. 3.

Linear autonomous control problems

In the following we shall concentrate on Zinear autonomous geometric optimal control structures, i.e. autonomous geometric optimal control structures satisfying the additional conditions that i? : C + Q is a linear bundle and 5 is a linear bundle map. We first consider the maximality condition derived in Theorem 3. Fix any to E T*Q and let x0 = XQ([O). The function fi )-_)h~(ii, co) for any ii E C,, attains its local extremum at i? = ii0 iff

or equivalently @*(<)= -AFL(u), where FL : C + C* denotes the fibre derivative of L and fi* : T*Q + C* is the dual of the linear bundle map 5. If 6 is an abnormal local extremal (i.e. there exists a local multiplier (fi, A) with h = 0), then j*(q(t)) = 0 for any t or, equivalently, G(t) E (F(C,,,,))O (the annihilator space of the image of Cq(,) under 5). Moreover, in this specific case, the function ii’H h&i’, G(t)) equals 0 for all ii’E Cq(,), which implies that ii is a global abnormal extremal. We conclude that for linear autonomous control problems, the abnormal local extremals are global abnormal extremals. On the other hand, if fi is a normal local extremal, i.e. A = -1, then 5*(c) = FL(u). We say that L is a regular cost if the fibre derivative of L is invertible. We say that a curve G(t) in T*Q generates a curve ii(t) in C, if i;(t) = (FL)-‘(ji*(fi(t))). In this case, if ij is piecewise smooth, then the curve U generated by ij is also continuous. Therefore, a normal local extremal is a piecewise smooth curve in C. Moreover, from the following proposition it follows that it is a smooth curve. THEOREM4. Assume that L denotes a regular cost. Every normal local extremal ii is generated by an integral curve fi(t) of the Hamiltonian vector jeld XG on T*Q associated with the function G(C) = hk(FL-‘(5*(C), {), for C E T*Q and with h = -1. The converse also holds, i.e. every integral curve of XG generates a normal local extremal.



Proof: We assume that 6 is a local extremal and we fix a local multiplier (6, A) with h = - 1. Consider the function L : T” Q -+ C x v T* Q defined by

Note that L is a section of the bundle PT:Q : C XQ T*Q -+ T*Q, with pp~ the projection onto the second factor. Then it is easily seen that L*h* = G and that C*&Q = WQ. Recall the implicit Hamiltonian system: the multiplier has to satisfy i(bjt),i,(t))fQ = -dhi(fi(t), G(t)), and consider the tangent vectors X = (i;(t), ij(t)) = TL(ij(t)) and Y = TC(tu) E T(C xQ T*Q) for w E T(T*Q) arbitrary. Then &Q(X, Y) = oQ(i(r), w). By substituting C*hh = G we obtain that zt”l(r)wQ = -dG(q(t)) for every smooth part of fi, By uniqueness of solutions to differential equations, it follows that ;i is smooth (and therefore we have that ii is smooth). On the other hand, assume that i~(~)mQ= -dG(fi(t)) and consider the smooth curve i;(t) = (FL)-‘(5*(6(t))) in C. Then, by reversing the above arguments, we obtain ~Q(X, Y) = Ah*(Y) with X = (h(t), G(t)) = TL(G(t)) and Y = TL(w) E T(C xQ T*Q) for w E T(T*Q) arbitrary and h = -1. It remains to check that this is also valid for arbitrary Y E T(C XQ T*Q). Since C is a section of PT*Q, any element Y in T(C x Q T* Q) can be written as Y = TL(w) + Z where w = T~T*Q(Y) E T(T*Q) and Z E ker TPT*Q. Since &Q = P;‘*~oQ it is easily seen that izC%Q= 0. From this we conclude that i+Q = -dh*(i(t), G(t)) for arbitrary t. 0 4.


EXAMPLE 1. Consider a sub-Riemanman structure (Q, D, h), where Q is a manifold, D is a regular (i.e. constant rank and smooth) distribution on Q and h is a Riemannian bundle metric on D. Let i : D -+ TQ denote the natural injection of D into TQ (note that V : D + Q can be considered as a linear bundle on Q). We would like to solve the length-minimizing problem in the sub-Riemannian structure (Q, D, h), i.e. we have to solve the autonomous optimal control problem with control structure (Q, c, i) and cost E E P(D), where E(v) = ih(v, u). It is easily seen that E is a regular cost, i.e. FE = bh, with bh defined by h(v, W) = (bh(v), w) for arbitrary u, w E D. Let jth denote the inverse of bh. The function G, introduced in the above theorem, takes the form G(C) = (5, i&(i*({)))) - $h(j&(i*(t)), &(i*({))). If we consider the tensor g E TQ @ TQ defined by dt, 5) = Wh(i*K)), !hG*W)) wir.b t,< E T*Q, then G(q) = kg(J, 0. In [2] we have further investigated the equations of a local extremal using connections over a bundle map and we gave necessary and sufficient conditions for abnormal extremals.

In the next example we consider the case where C is a Lie-algebroid with anchor map 5. We refer to [ 1, 6, 131 where the importance of this specific case for a generalisation of Lagrangian mechanics is thoroughly investigated. It should



be noted that we only derive the Lagrangian equations, using the theory developed above. EXAMPLE2. Assume that C is a Lie-algebroid with anchor map 5 and assume that L is a regular cost. It is a well known fact that the Lie-algebroid structure determines a Poisson structure on C*, where the bracket on Cm(C*) is denoted by {., .}c*, whereas the Poisson bracket on Cm(T*Q) is denoted by {e,e}. It is also well known that both Poisson structures are 5* connected, i.e. if x = ,Z*, then given arbitrary f, g E ?(C*) the following equality holds: {x*f, x*g} = x *{f, g}c*. This implies that any Hamiltonian vector field X, on C* is p*-connected to the Hamiltonian vector field X,*f on T*Q. Consider the function G on T*Q, introduced in the previous section. Define g E Cm(C*) by g(ar) = (OL, FL-‘(a)) L(FL-‘(a)). Then it is easily seen that x*g = G. This guarantees that, given any integral curve q(t) of XG, ?*(r(r)) is an integral curve of X, and, conversely, any integral curve cr(t) of X, through a point in the image of $* is the projection under 5* of an integral curve of XG. From this we conclude that, in the case where C* is a Lie-algebroid, the integral curves of X, through a point in the image of p* are projections of normal local extremals. Acknowledgements This research is supported by a grant from the “Bijzonder onderzoeksfonds” of Ghent University. I am indebted to F. Cantrijn for many discussions and careful reading of this paper. REFERENCES [ll J. F. Caritiena and E. Martfnez: Lie Algebroid Generalization of Geometric Mechanics, in: Lie Algebroids and Related Topics in Di#venrial Geometry, P. U&&ski J. Kubarski and R. Wolak Eds., Banach Center Publications, vol. 54, 2001, pp. 201-215. B. Langerock: A Connection Theoretic Approach to Sub-Riemannian Geometry, J. Geom. Phys. (2003), to appear. [31 B. Langerock: Con& Problems With Variable Endpoints, Discret. Contin. Dyn. Syst., An expanded volume for the Wilmington Meeting, May 24-27, 2002 (2003), to appear. [41 B. Langerock: Geometric aspects of the maximum principle and lifts over a bundle map, Acta. Appl. Math. (2003), to appear. Theory [51 L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamklelidze and E. F. Mishchenko: The Muhemahal of Optimal Processes, Wiley, Interscience, 1962. [61 E. Martfnez: Acta. Appl. Marh. 67 (2001), 295-320. [71 R. Montgomery: Abnormal minimizers, Siam J. Conwol and Oprimization 32 (1994), 1605-1620. @I D. J. Saunders: The geometry of Jet Bundles, Cambridge University Press, Cambrige 1979. 191 R. S. Strichartz: J. Dig Geom. 24 (1986). 221-263. t101 R. S. Strichartz: J. Dig Geom. 30 (1989), 595-596. [ill H. J. Sussmann: Tmns. Amer. Marh. Sot. 180(1973), 171-188. [121 H. J. Sussmann: An Introduction to the Coordinate-Free Maximum Principle, in: Geometry of Feedback and Opknal Conrrol, B. Jakubczyk and W. Respondek Eds., Marcel Dekker, New York 1997, pp. 463557. [I31 A. Weinstein: Lagrangian Mechanics and Groupoids, in: Mech&cs Day, W. F. Shadwick P. S. Krishnaprasad and T. S. Ratiu Eds., Fields Institute Communications, American Mathematical Society, Providence, RI 1996, pp. 207-232.