Non-singular optimal control for fed-batch fermentation processes with a differential-algebraic system model

Non-singular optimal control for fed-batch fermentation processes with a differential-algebraic system model

Non-singular optimal control for fed-batch fermentation processes with a differential-algebraic system model P.-C. Fu” and J. P. Barford Department of...

760KB Sizes 0 Downloads 20 Views

Non-singular optimal control for fed-batch fermentation processes with a differential-algebraic system model P.-C. Fu” and J. P. Barford Department of Chemical Engineering, Sydney University, NSW 2006, Australia Received 2 June 1992; revised 7 July 1993

The problem of optimal control for fed-batch fermentation processes is studied with nonlinear differentialalgebraic system modelling. A non-singular optimal control strategy has been developed as a result of the necessary condition analysis of non-singularity of the Hamiltonian function established for the processes. Proof of the optimality of the proposed feeding policy is given. The difficulty associated with singularity of the fed-batch operation mode can thus be avoided. The ethanol fermentation process from glucose by S. cerevisiaeis taken as an example for the optimization application. It has been found that previous investigations by some authors, with different optimization methods, led to overestimation of the product formation from the process. A constraint is thus put on the specific productivity, which if unconstrained is

responsible for the existing inaccurate predictions. This constraint takes into account the stoichiometry involved in the fermentation. Our simulation study has shown that a realistic result can be achieved with the proposed non-singular optimization scheme. Keywords: fed-batch fermentation; differential-algebraic

The problem of optimal control for fed-batch fermentation processes has drawn great attention over the last two decades. The main reason is that fermentation involves high cost with relatively low yields. Hence it is very profitable to optimize the fermentor’s performance to maximize the biomass growth and the product yield, while minimizing substrate consumption and fermentation time. Fed-batch operation mode is usually chosen because it effectively overcomes such effects as substrate inhibition, catabolite repression, product inhibition, glucose effects and auxotrophic mutation, etc.‘-‘. However, it raises the singular control problem for determination of the optimal substrate addition policies in which the feed rate appears linearly in the system and/or in the performance index to be optimized so that Pontryagin’s maximum principle4 fails to provide a solution. Due to limited progress in singular control theory, most of the fed-batch fermentation processes considered so far have been limited to low-order processes’. Moreover, there is no guarantee that a singular control is minimizing, or that it will enter the true optimal solution even if the possibility of a singular solution does exis?. Some efforts have been made to convert the singular control problems into semi-singular or non-singular ones

‘To whom all correspondence [email protected]

0959-l 524/93/04021 l-08 0 1993 Buttenvorth-Heinemann



be addressed.


system; ethanol formation

to overcome the above difficulty. One approach is to obtain the solution to the singular control problem from that of the semi-singular problem. For example, Modak et al.’ studied in detail the general characteristics of the optimal feed rate profiles deduced for various fed-batch fermentation processes by analysing singular controls and singular arcs. As a result, the optimal feed rate profile determination problem was converted into a problem of determining switching times between bangbang intervals and singular intervals. Hong* has also treated the singular control problem by considering the conjunction point between the singular and bang-bang arcs, and hence he obtained an analytical expression for it in terms of the concentrations of substrate, cell and product, and the liquid volume. The optimal control policy is related to the so-called ‘conjunction condition’. Alternatively, many authors have taken system variables other than the feed rate as the control variable. Among them, Lim et ~1.~have deduced different feeding programs depending on general characteristics of specific rates of biomass growth and product formation and initial conditions for models with four state variables. The authors proposed transformation of the state vector which produced a non-singular control problem. It is achieved by replacing substrate concentration by the new state variable (23, - 5’) x V, making the model equatio^ns nonlinear in volume. Optimal feeding rate is then calcu-

J. Proc. Cont. 1993, Volume 3, Number 4


Non-singular optimal control: P.-C. Fu and J. P. Batford

lated from a profile of the culture volume during fermentation. Guthke and Knorre” have used substrate concentration as a control variable but disregarded constraints imposed on feeding rate and volume. San and Stephanopoulos” have suggested substrate concentration in feed as a control variable and derived an optimization algorithm with constraints included on the state and control variables. Their results predicted significant improvements in productivity due to an increase in controllability, in contrast with previous results’* when feed rate is the only control variable. Modak and Lim’ have used the culture volume instead of the feed rate as the, control variable in their optimization scheme, which leads to obtaining the optimal feed rate by differentiating the continuous function v*(t). Most recently, Kurtanjek” reported a simple method for multidimensional optimization of fed-batch fermentations based on the use of the orthogonal collocation technique. It reformulated the state and control variables to obtain a non-singular form of the optimization problem and used substrate concentration in the fermentor as an auxiliary decision variable. As demonstrated by Chen and Hwangi4,“, the singular control problem of fed-batch fermentation processes can be converted into finite dimensional optimization problems which concern optimal parameter selection and control for differential-algebraic systems (DAS) involving various constraints. The problems considered can then be numerically solved by constrained quadratic programming methods. In this work we use DAS modelling’4~‘5to describe the fermentation processes, and suggest a different approach to formulate the control problem. A non-singular optimal control strategy can be derived with the application of Pontryagin’s maximum principle. The optimality of the proposed optimization scheme will be proven in the Appendix. For illustration and comparison, the ethanol fermentation process by S. cerevisiae will be examined again to discuss the reason for the unrealistic simulation results which were produced by the previous investigations, and to demonstrate the necessity of a strictly constrained optimization problem for successful application of the advanced control theory in the biotechnological field.

are, respectively, the state and control vectors. The DAEs in Equation (1) can be grouped as: dY Edt = AYW)

The dynamics of fed-batch fermentors have been modelled by the following nonlinear differential-algebraic equations’4”5:


where E = diag[Z,,,,O,,_,J,f = V;‘V;‘]‘. Z, is the m X m identity matrix and O,_, is the (n - m) x (n - m) null matrix. In Equation (l), the superscript T denotes the transpose. For a given DAS (l), it is desired to find the optimal control u(t) which satisfies the lower and upper bounds: &,min








that will drive the system from an admissible initial state Y(to) = y. E So to an admissible final state Y(tr) = yf E Sl while the cost functional: J(u) = Go(Y(tr),t,) + !;Fo(Y,~)dt


is minimized, subject to the following type of constraints imposed on the state and control vectors: h( Y,u) 2 0


The necessary condition of the Pontryagin’s maximum principle is expressed in terms of the Hamiltonian function defined by the state vector Y, the control u and the adjoint, or the costate, vector I: H = F,( Y,u,t) + nTflY,u,t) = Fo( Y,U,?) + A>‘( Y,u,t) + @z( Y,W)


The costate variables are defined by the adjoint equations: d3. _=

-_ aH



with the specific terminal conditions given by:


Problem statement





Y(tr)Jr>) a wf)

The necessary condition for optimality requires that the control u(t) satisfy:

q And there exists at least one element of Y: letting

0 d fi( Yw)



# constant

with the initial conditions Y(0) = y(t,,), where: y =






* * .

YmlYm+,, * f *Yn3’ E R”

J. Proc. Cont. 1993, Volume 3, Number 4

For the usual ordinary differential equation (ODE) model, there exists only the first items on the right-hand

Non-singular optimal control: P. -C. Fu and J. P. Barford

sides in Equations (6) and (9). This results in the control variable u(t) appearing linearly in the Hamiltonian function and further discussion will be needed for the decision of the control’6 u(t). On the other hand, the Hamiltonian function can be minimized with respect to the choice of u(t) and a non-singular solution can be attained for our scheme if relation (10) exists. Therefore, we denote relation (10) as the non-singular condition. The analytical expression of optimal control U* can be obtained by the solution of Equation (9).

Y3 =



Y4 =



Y5 =



The original DASs can be converted as: dy’ _ -dt y4


dyz _ dt - Y, expk

Non-singular optimization of a fed-batch fermentation process

- y2)



Here we consider a free terminal-time fed-batch fermenin which ethanol is produced by S. tation process8~‘4V’5 cerevisiae and the production rate of ethanol is inhibited by itself. Its dynamic behaviour has been described with DAS modelling’4.‘5 as follows: dx ;iT=X/l-U$

(11) (12)




0 = 4ltYl~Y2,Y3WY4,4


0 = B2t_hY2,Y3(~),Y5,U


where (x(o)/Y~,+s(o)-s,)~+s~-e~


41 =y4-



dp ;i7=n”-U$


dV -_=a dt





V(O)+ s,,-=Wl_vJ exP6’2) k:+ (x(O)/r, + s(O)- so)y,

’ + (44



The Hamiltonian function for this system is:



k,+s P


exp64 (28)

The Monod equations for p and E were derived by Aiba et al.“: p=



= 4y4 + jlzy5ewC_h - y2) + Su +














and the adjoint equations are:


where X, S and P are the concentrations of cell mass, substrate and product, respectively, V is the broth volume in the fermentor, p is the specific growth rate, E is the specific productivity, YXsis yield coefficient, S,, is the substrate concentration of the feed, and u is the feeding rate. Note that Equations (11) and (12) can be combined to give’: s = (X(0)/Y,, + S(0) - so,?


+ s, - f



dh --



= hhexph



d& _ --






- y2) + A4g+A5g

dil,_&+& dt - 4ay3 dt

Here X(O), S(0) and V(0) represent the initial conditions of cell mass, substrate and broth volume, respectively. By using Kelly transformations’8, we have:


say, A4

da5 _ --a2 exrb -y2) - a5 dt

yl = 1nGW


The cost functional is chosen as:

YZ= ln(PV


J(u) =


J. Proc. Cont. 1993, Volume 3, Number 4








Non-singular optimal control: P.-C. Fu and J. P. Barford Table 1 Comvarison of simulation results by different feeding policies End time tXh)

Cell cont. Wr)(g 1-1)

Ethanol cont. JYtr)(g l-1)

Objective function



14.8 14.5 15 15 15

105 100.37 99.98 105.7 75.68

-21000 - 20 073.7 - 19 996.2 -21 140.2 - 15 132

Singular Sub-optimal Optimal on-off Non-singular Non-singular

Hong8 Chen and Hwang’” Chen and Hwangu This work (case 1) This work (case 2)

59.05 54 50 56 50.67

Then the value of the costate vector can be derived at the terminal time: aTof) = WI &lT = Mf)

&Of) Ud



= [OeQ(‘f)0 0 OIT


The non-singular condition, Equation (lo), can be satisfied, that is: $!i



_ A a41 dY3


;3.aq2 dY3





dy3 -_= du



Here i = 1,2. From Equations (9), (33) and (39), we thus obtain:

Setting Equation (40) to be zero, it can be deduced that: du -=24




Applying the integrals on both sides of Equation (41) we derive the optimal solution: u*(t) =

c,a&) +



Here c, and co are the coefficients. Using the terminal condition (37), we obtain: A,(&) = 0, co = u(tf), and letting u(to) = 0, then:

u(t3 c’



tation has been presented. However, it is difficult to derive the analytical expression of the non-singular optimal control in a direct manner because of the nonlinear nature of the control problem. Computational techniques are therefore needed for solving nonlinear twopoint boundary value problems. Let u’(t) be the ith approximation (iteration) on the optimal control u*(t). The corresponding gradient aH/ hi is evaluated by integrating the state equations (from Equation (23) to Equation (25)) forward with u(t) = ~~(2) and the given initial condition, integrating the adjoint equations (from Equation (31) to Equation (35)) backward with the known terminal condition, and then computing the gradient from Equation (40) for the new modification of the control variable. The steepest descent algorithm is:

d+‘(t) = u’(t) +



Here LJis the iteration step. The computation mentioned above should be repeated until AU, does not change appreciably from iteration to iteration. To solve the given problem by the steepest descent algorithm, we chose the kinetic constants for S. cerevisiae growing on glucose’ as follows: b = 0.408 h-‘; kp = 16.0 g 1-l; k, = 0.22 g 1-l; q. = 1.0 h-‘; kp, = 71.5 g 1-l; k,. = 44 g 1-l; Y_, = 0.1. It is assumed that the fermentor is initially filled with 10 1 of broth in which the concentrations of cell mass and substrate are 1 g 1-l and 150 g l-‘, respectively, and there is no ethanol. Also it is assumed that the maximum capacity of the fermentor is 200 1 and the maximum feeding rate is restrictedI up to 12 1 h-‘. Under these conditions, the problem considered here is to find the optimal feeding rate u(t) and the fermentation time tf for the DAS in Equations (23) to (27) such that the cost functional is minimized while subjected to the following constraints:


0 5 u(t) 5 12 We have arrived at the two-point boundary value problem whose solution with the associated boundary conditions will establish the optimal state trajectory and optimal control.

Simulation and discussion In the previous section, the development of the nonsingular optimal feeding strategy for fed-batch fermen-


J. Proc. Cont. 1993, Volume 3, Number 4

S20 v(tJ = 200 In the literature, some investigators have developed the feeding policies to optimize the fermentation process and obtained the simulation results as shown in the first three rows in Table 1. The stoichiometry of the ethanol formation from glucose for S. cerevisiae is:

Non-singular optimal control: P. -C. Fu and J. P. Batford


time (hour)

Fiiure 1 Optimal substrate feeding rates during the fermentation period (case 1: dashed line; case 2: solid line)

C6H,206 + 2C2Hs02 + 2C02

Figure 2 Fermentor volumes during the fermentation period (case 1: dashed line; case 2: solid line)


In this case, 3 kg of glucose is supplied, and a maximum of 1.533 kg of ethanol produced. Therefore the maximum possible ethanol concentration in the fermentor is 76.65 g 1-l. Hence, the predicted values of P(tr) by the previous investigations are all unrealistic, since they exceed the theoretical calculation under the given condition. The reason is that the above equations are not accurate enough to be used for optimization purposes*. In particular, Equation (29) is responsible for the overestimation in the literatures”4”s since it is totally unconstrained. To solve the problem, a constraint on the specific productivity E was introduced. For the fermentation process, the following equation can be derived: dP -_= dt

Y dS PSdt

(46) Figure 3 Specific growth rates and productivities time (case 1: dashed line; case 2: solid line)



In the steady state: 1 dP 6=livt


Combining Equations (46) and (47), we get: Yp, dS . YpsAS 61Xdt=XE


Since Yps = 0.51 is the theoretical yield of ethanol from glucose, the value of E from Equation (16) should be always less than or equal to the value of the righthand side of Equation (48) to meet the stoichiometry (Equation (45)). A computer study has been carried out with the nonsingular feeding strategy with the simulation results in Figures I to 5. For comparison purposes, both the specific productivity unconstrained case (case 1, dashed lines) and the specific productivity constrained case (case 2, solid lines) were considered. The profiles for case 1 are similar to those derived by Chen and Hwang’s meth-


,),,,,,,),,) I”


I ,)(,,,(,, a0







kmc~~trlio~~ %nc (IL)

4 Cell levels during the fermentation period (case 1: dashed line; case 2: solid line) Figm

J. Proc. Cont. 1993, Volume 3, Number 4


Non-singular optimal control: P.-C. Fu and J. P. Batford

Figure 5 Substrate consumption and product formation during the fermentation period. In case 1 (dashed line), the predicted value of P(t) exceeds the ceiling of maximum possible product concentration. In case 2 (solid line), P(t) is a realistic response to the optimal profile

ods14, but the predicted value of the product concentration at the terminal time is the same as Hong’s result’. A maximum possible product concentration was fixed in Figure 5. It can be seen that when the specific productivity is restricted by Equation (46), the calculated ethanol formation is under the maximum theoretical value. If unconstrained, it will go beyond this limit. The simulation for case 2 leads to a shorter fermentation time, and smoother changes in both substrate consumption (Figure 5) and cell growth (Figure 4) during the cultivation. Monod equations are frequently used in the kinetic descriptions for growth of micro-organisms and the formation of metabolic products. However, since they reflect the empirical relationships only, care must be taken when they are employed as part of the models for biochemical processes on which optimization theory will be applied. This is because the application of optimization theory should be based on a precise mathematical model of the process. Understanding such requirements in the optimization of complex biochemical processes will be helpful to bridge the gap between control engineers and biotechnologists. In addition, it is also important for control engineers to understand fully the biological/process implications of proposed optimization schemes before they are developed. In this paper, we have shown that lack of attention to the maximum theoretical yield leads to impractical feeding schemes8’143’5, which result in unattainable product concentrations and therefore such schemes should be subject to consideration of whether their proposed feeding strategies are of practical use for the process of ethanol fermentation. The control problem for fed-batch operation is usually a singular control problem5. Two ways so far to treat the problem are: finding a singular solution for a specified fermentation process437*s,or choosing other variables instead of the feed rate as the control variable”‘. There is no guarantee using the first method, however, that a singular control is minimizing, or that it will enter the true optimal solution even if the possibility of a singular solution does exist6. The second method cannot explicitly


J. Proc. Cont. 1993, Volume 3, Number 4

take into account the physical constraints imposed upon the flow rate and the volume capacity5. We suggest that non-singular optimal control can be attained even if the feed rate is used as the control variable. Singularity may be avoided when it is possible to deal with the control problem as a two-point boundary value problem to derive the optimal control profile theoretically. It can be found that fed-batch fermentation processes are governed by the specific growth rate and specific product formation rate throughout the fermentation period, while the two specific rates are functions of the substrate concentration in the bioreactor. The proposed optimal feeding strategy takes into account the change rates of the two specific rates with respect to the control variable; it is thus easier to put constraints on the DASs to form a strictly constrained optimization problem. The optimality of the non-singular control strategy, Equation (42), can also be proven on the basis of the singular optimal control point of view as shown in the Appendix, in which a proof that the linear relationship between the control variable u(t) and the adjoint variable &(t) fulfils the generalized Legendra-Clebsch condition20’2’is given. Therefore, the conclusion can be made that: 1) Equation (42), or Equation (a22) in the Appendix, is a unique optimal solution of the control problem; 2) The proposed control strategy is non-singular since it can be developed by both the application of the Pontryagin’s maximum principle and the generalized Legendra control problem”.

Conclusion We have demonstrated that the application of Pontryagin’s maximum principle for optimizing fed-batch fermentation processes results in a non-singular optimal control solution even if the feed rate is taken as the control variable. This approach is able to avoid both the singular control problem in which limited progress has been made, and the choice of other system variables as the control variable which may suffer from the problem of not explicitly taking into account the physical constraints imposed on the flow rate and the volume. Application of the optimization scheme on the ethanol fermentation process by S. cerevisiae has been studied with the stoichiometric constraints. A constraint has been put on the maximum ethanol yield. The absence of such a constraint has been responsible for the failure of previous optimization schemes to achieve realistic results. Our simulation has proven that the reformulation of the optimal control problem increased the accuracy of the system response to the feeding profile.

References 1 Whitaker, A. Proc. Biochem. 1980,15, 10 2 Yamane, T. and Shim& S. Adv. Biochem. Eng. Biotechnol, 1984, 30,147

Non-singular optimal control: P. -C. Fu and J. P. Batford 3 Parulekar.

S. J. and

Lim. H. C. AC/V. Biochem. Eng. Biotechnol.

198532.207 4 Pontryagin,

5 6 7 8 9 IO II 12 I3 I4 I5 I6 I7 I8 19

20 21 22

L. S.. Boltyanskii, V. G., Gamkrelidze, R. V. and Mishchenko. E. F. ‘The Mathematical Theory of Optima1 Processes’. Interscience. New York, 1962 Modak. J. M. and Lim. Y. C. Bio/echno/. Bioeng. 1989, 33, I I Sage% A. P. and White. C. C. ‘Optimum SystemsControl’, 2nd ed., Prentice-Hall. Enrlewood Cliffs. N.J.. 1985 Modak. J. M.. LTm. H. C. and Tayeb. Y. J. Biotechnol. Bioeng. 1986. 28. 1396 Hong. J. Biotechml. Bioeng. 1986, 28. 1421 Lim. H. C.. Tayeb. Y. J.. Modak. J. M. and Bonte, P. Biotrchnol. Bioer~~. 1986. 28. I408 Guthke. R. and Knorre. W. A. Bio/&mo/. Bioeng. I98 I, 23, 2771 San. K.-Y. and Stephanopoulos. G. Biotechnol. Bioeng. 1989.3472 Bajpai. R. K. and keuss.. R. Biotehol. Bioetg. 1981. 23, 717 Kurtanjek. 2. Biotedvd. Bioettg. 1991, 37, 8 I4 Chen. C.-T. and Hwang. C. C’l~en?.Elg. Conmfil. 1990, 97, 9 Chen. C.-T. and Hwang. C. I/U/. Etg. Client. Re.7. 1990, 29. 1869 Kelly. J. H.. Kopp. R. E. and Moger. H. G. in ‘Topics in Optimization’. Vol. 63 (Ed. G. Leitmann), Academic Press, New York, 1967 Aiba. S.. Humphrey. A. E. and Millis. N. F. ‘Biomechanicai Engineering’. University of Tokyo Press, 1973. I48 Kelly. J. H. SIAM J. Cotrtrol. 1965, 2. 234 Bell. D. J. and Jacobson, D. H. ‘Mathematics in Science and Engineering’, Vol. I I7 (Ed. R. Bellman), Academic Press, New York. 1975 Robbins. H. M. IBM J. Ras. Du. 1967, 3. 361 Goh. B. S. SIAM J. Cot1tr.d 1966, 4, 716 Speyer. J. L. J. Aimwfi. 1973. IO. 763

The first derivative of the Hamiltonian with respect to u becomes:

funtion (30)

H,, = A,


Taking the first derivative respect to time: I?,



(a3) with



is obvious from Equation (33) that:



of Equation



The partial derivative of Equation (a4) with respect to is: (a6)

while the second derivative of Equation (a3) with respect to time is:

Nomenclature Changing the sequence of the derivation:

(‘I,. (‘I

coefficients of the feeding strategy E identity matrix H Hamiltonian J objective function li,,.k,,,./i..X..,Monod constants null matrix 0 P product concentration (g I ‘) substrate concentration (g I ‘) substrate concentration of the feed (g I ‘) fermentation time (h) initial time of fermentation 11, terminal time u control variables V liquid volume (I) cell concentration (g I ‘) transformed variables state variables yield of product from substrate yield coefficient

8 dq, ndaYl=a 4dt ay, 4ay, dt It can be deduced from Equations (23) to (25) that:

a da _ --__ a3 dt

84, --+_---f_~ ay,

a 84,dy, ay,ay,dt +

34, dy,

%y5e~,-1’. + !&



a%, __

a?h =J&J”+

dy, dt

ay, dt







Similarly: Greek E



specific productivity (h ‘) maximum productivity adjoint variables specific growth rate (h ‘) maximum growth rate iteration step

? fi h 0





ah dt Substituting Equations (a7), we have:

(a9) and (alO) into Equation

Appendix: Proof of optimality of the proposed control strategy as the singular optimal control problem Considering our case as a singular problem, modify Equations (26) and (27) to:

we can

(al 1) We can obtain:

0 = d_vI,Y2,Y3,Y4)






= [A4&] [$J !$-’ = &(%) 3



J. Proc. Cont. 1993, Volume 3, Number 4


Non-singular optimal control: P.-C. Fu and J. P. Barford

Considering the generalized Legendra-Clebsch condition20’2’ which is taken as a necessary condition for optimality”: ;$Hu

Substituting Equation (a17) into Equation (a16), we have:


= 0


for all t E [to&] where p is odd, and

Since the system variable y3, or the volume, is a monotonic increasing function of time along the singular arc, and u may possess the same property, it implies that:

~3 d2q (- l)qGPH,‘O


for all t e [t&l

necessary For p = 1, Equation (a13) is a powerful ^^ condition in the case of vector controlsLL, and it is trivially satisfied when the control is a scalar”. For this case, the following constraints can be obtained.

Then we get: d, = - ik,] I 0


and c, = -(k21 I 0

(al61 It can be seen from Equation (a15) that there should be a linear relationship between i&(t) and u(t) as: A:(t) = d,u*(t)+&


or u*(t) = c&(t)


+ co

J. Proc. Cont. 1993, Volume 3, Number 4



where k, and k2 are positive coefficients. Then Equation (a18) can be rewritten as: u(t) = -





The proposed control strategy in Equation (42) can thus be derived by application of the generalized Legendra-Clebsch condition.