Upload
others
View
154
Download
12
Embed Size (px)
Citation preview
MATLABOptimizationToolbox(optimtool)
Dr. RajeshKumar
PhD,PDF(NUS,Singapore)SMIEEE(USA),FIET(UK)FIETE,FIE(I),LMCSI,LMISTE
Professor, DepartmentofElectricalEngineeringMalaviya NationalInstituteofTechnology, Jaipur, India,
Mobile: (91)[email protected],in, [email protected]:http://drrajeshkumar.wordpress.com/http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Contents• Minimizationalgorithm
– fgoalattain– fmincon– fminimax– fminunc
• Equationsolving– fsolve– fseminf
• Linearprogramming– linprog– intlinprog
• Leastsquareproblems– lsqlin– lsqnonlin– lsqcurvefit (curvefitting)
• Quadraticprogramming– quadprog
• GlobalOptimizationToolbox– ga (geneticalgorithm)– particaleswarm (Particleswarmoptimization)– simulannealbnd (simulatedannealingalgorithm)– gamultiobj (multi-objectivega)– patternsearch
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Solvemultiobjective goalattainmentproblems
,
( ) weight. goal(x) 0( ) 0
minimize ..
x y
F x ycceq xA x b
Aeq x beqlb x ub
− ≤⎧⎪ ≤⎪⎪ =⎨
≤⎪⎪ =⎪
≤ ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Solvemultiobjectivegoalattainmentproblems
• Where– weight, goal, b,and beq arevectors– A and Aeq arematrices– c(x), ceq(x),and F(x)arefunctionsthatreturnvectors– F(x), c(x),and ceq(x)canbenonlinearfunctions
,
( ) weight. goal(x) 0( ) 0
minimize ..
x y
F x ycceq xA x b
Aeq x beqlb x ub
− ≤⎧⎪ ≤⎪⎪ =⎨
≤⎪⎪ =⎪
≤ ≤⎩
Linearinequalityconstraints
RangeofxLinearequalityconstraints
Nonlinearinequalityconstraints
Nonlinearequalityconstraints
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Example Anoutputfeedbackcontroller, Kisdesigned
producingaclosedloopsystem
Withdesignconsideration,closelooppoles[-5,-3,-1]andgain-4<K<4
( )x A BKC x Buy Cx= + +
=
&
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Example Anoutputfeedbackcontroller, Kisdesigned
producingaclosedloopsystem
Withdesignconsideration,closelooppoles[-5,-3,-1]andgain-4<K<4
( )x A BKC x Buy Cx= + +
=
&
[ ]
0.5 0 0 1 01 0 0
0 2 10 ; 2 2 ;0 0 1
0 1 2 0 1
goal= 5 3 1 ;weight (goal);
1 1K0 ;
1 1
4 4 4 4; ;
4 4 4 4
A B C
abs
lb ub
−⎡ ⎤ ⎡ ⎤⎡ ⎤⎢ ⎥ ⎢ ⎥= − = − = ⎢ ⎥⎢ ⎥ ⎢ ⎥ ⎣ ⎦⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦
− − − =
− −⎡ ⎤= ⎢ ⎥− −⎣ ⎦
− −⎡ ⎤ ⎡ ⎤= =⎢ ⎥ ⎢ ⎥− −⎣ ⎦ ⎣ ⎦
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Createfunctionfile, eigfun.m.
function F = eigfun(K,A,B,C) F = sort(eig(A+B*K*C)); % Evaluate objectives
• NextenterthesystemmatrixandinvokeanoptimizationroutineA= [-0.5 0 0; 0 -2 10; 0 1 -2]; B = [1 0; -2 2; 0 1]; C = [1 0 0; 0 0 1]; K0 = [-1 -1; -1 -1]; % Initialize controller matrix goal = [-5 -3 -1]; % Set goal values for the eigenvaluesweight = abs(goal); % Set weight for same percentage lb = -4*ones(size(K0)); % Set lower bounds ub = 4*ones(size(K0)); % Set upper bounds
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Createfunctionfile, eigfun.m.
function F = eigfun(K,A,B,C) F = sort(eig(A+B*K*C)); % Evaluate objectives
• NextenterthesystemmatrixandinvokeanoptimizationroutineA= [-0.5 0 0; 0 -2 10; 0 1 -2]; B = [1 0; -2 2; 0 1]; C = [1 0 0; 0 0 1]; K0 = [-1 -1; -1 -1]; % Initialize controller matrix goal = [-5 -3 -1]; % Set goal values for the eigenvaluesweight = abs(goal); % Set weight for same percentage lb = -4*ones(size(K0)); % Set lower boundsub = 4*ones(size(K0)); % Set upper bounds
options = optimoptions('fgoalattain','Display','iter');
[K,fval,attainfactor]=fgoalattain(@(K)eigfun(K,A,B,C),... K0,goal,weight,[],[],[],[],lb,ub,[],options)ht
tps://
drra
jesh
kum
ar.w
ordp
ress
.com
fgoalattain• Result
K = -4.0000 -0.2564 -4.0000 -4.0000
fval =-6.9313-4.1588-1.4099
attainfactor = -0.3863
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fmincon• Findminimumofconstrainednonlinearmultivariablefunction
• Syntex[x,fval]= fmincon(fun,x0,A,b,Aeq,beq,lb,ub)
(x) 0( ) 0
minimize ( ) ..
x
cceq x
f x A x bAeq x beqlb x ub
≤⎧⎪ =⎪⎪
≤⎨⎪ =⎪
≤ ≤⎪⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fmincon• Example FindtheminimumvalueofRosenbrock's function
withfollowingconstraints
atstartingpoint(-1,2)
( ) ( )2 22
2 1 1100 1x x x− + −
1 22 1x x+ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fmincon• Example FindtheminimumvalueofRosenbrock's function
withfollowingconstraints
atstartingpoint(-1,2)
• MatlabCodex0 = [-1,2]; A = [1,2]; b = 1;
x = fmincon(fun,x0,A,b)
( ) ( )2 22
2 1 1100 1x x x− + −
1 22 1x x+ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fmincon• Example FindtheminimumvalueofRosenbrock's function
withfollowingconstraints
atstartingpoint(-1,2)
• MatlabCodex0 = [-1,2]; A = [1,2]; b = 1;
x = fmincon(fun,x0,A,b)• Solution
x = 0.5022 0.2489
( ) ( )2 22
2 1 1100 1x x x− + −
1 22 1x x+ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminimax• Findstheminimumofaproblemspecifiedby
• Syntex[x,fval]= fminimax(fun,x0,A,b,Aeq,beq,lb,ub)
(x) 0( ) 0
min max ( ) ..
ix i
cceq x
f x A x bAeq x beqlb x ub
≤⎧⎪ =⎪⎪
≤⎨⎪ =⎪
≤ ≤⎪⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminimax• Example Findvaluesof x thatminimizethemaximumvalueof
where
atstartingpoint(0.1,0.1)
[ ]1 2 3 4 5( ), ( ), ( ), ( ), ( )f x f x f x f x f x2 2
1 1 2 1 22 2
2 1 2
3 1 2
4 1 2
5 1 2
( ) 2 48 40 304
( ) 3
( ) 3 18
( )
( ) 8
f x x x x xf x x xf x x x
f x x xf x x x
= + − − +
= − +
= + −
= − −
= + −
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminimax• First,writeafilethatcomputesthefivefunctionsat x
function f = myfun(x) f(1)= 2*x(1)^2+x(2)^2-48*x(1)-40*x(2)+304; f(2)= -x(1)^2 - 3*x(2)^2; f(3)= x(1) + 3*x(2) -18; f(4)= -x(1)- x(2); f(5)= x(1) + x(2) - 8;
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminimax• First,writeafilethatcomputesthefivefunctionsat x
function f = myfun(x) f(1)= 2*x(1)^2+x(2)^2-48*x(1)-40*x(2)+304; f(2)= -x(1)^2 - 3*x(2)^2; f(3)= x(1) + 3*x(2) -18; f(4)= -x(1)- x(2); f(5)= x(1) + x(2) - 8;
• Next,invokeanoptimizationroutinex0 =[0.1; 0.1]; % Make a starting guess solution [x,fval] = fminimax(@myfun,x0);
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminimax• First,writeafilethatcomputesthefivefunctionsat x
function f = myfun(x) f(1)= 2*x(1)^2+x(2)^2-48*x(1)-40*x(2)+304; f(2)= -x(1)^2 - 3*x(2)^2; f(3)= x(1) + 3*x(2) -18; f(4)= -x(1)- x(2); f(5)= x(1) + x(2) - 8;
• Next,invokeanoptimizationroutinex0 =[0.1; 0.1]; % Make a starting guess solution [x,fval] = fminimax(@myfun,x0);
• Afterseveniterations,thesolutionisx =
4.0000 4.0000
fval = 0.0000 -64.0000 -2.0000 -8.0000 -0.0000
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminunc• Findstheminimumofaproblemspecifiedby
where f(x)isafunctionthatreturnsascalarx isavectororamatrix
• Syntex[x,fval]= fminunc(fun,x0)
min ( )xf x
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminunc• Example Minimizethefunction
atstartingpoint(1,1)2 21 1 2 2 1 2( ) 3 2 4 5f x x x x x x x= + + − +
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminunc• Example Minimizethefunction
atstartingpoint(1,1)• Matlab Code
fun = @(x)3*x(1)^2 + 2*x(1)*x(2) + x(2)^2 - 4*x(1) + 5*x(2);
x0 = [1,1]; [x,fval] = fminunc(fun,x0);
2 21 1 2 2 1 2( ) 3 2 4 5f x x x x x x x= + + − +
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fminunc• Example Minimizethefunction
atstartingpoint(1,1)• Matlab Code
fun = @(x)3*x(1)^2 + 2*x(1)*x(2) + x(2)^2 - 4*x(1) + 5*x(2);
x0 = [1,1]; [x,fval] = fminunc(fun,x0);
• Afterafewiterations, fminunc returnsthesolutionx =
2.2500 -4.7500fval =
-16.3750
2 21 1 2 2 1 2( ) 3 2 4 5f x x x x x x x= + + − +
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fseminf• Findstheminimumofaproblemspecifiedby
Ki(x,wi) ≤ 0 arecontinuousfunctionsofboth x andanadditionalsetofvariablesw1,w2,...,wn
• Syntex[x,fval]= fseminf(fun,x0,ntheta,seminfcon,…
A,b,Aeq,beq,lb,ub)– ntheta -Numberofsemi-infinite constraints– Seminfcon - Semi-infiniteconstraintfunction
( , ) 0,1(x) 0( ) 0
minimize ( ) ..
i i
x
K x w i ncceq x
f xA x b
Aeq x beqlb x ub
≤ ≤ ≤⎧⎪ ≤⎪⎪ =⎨
≤⎪⎪ =⎪
≤ ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fseminf• Example Minimizesthefunction
subjecttotheconstraints( )21x−
( ) ( )2
0 2
1 1( , ) 0 for all 0 12 2
x
g x t x t t
≤ ≤
= − − − ≤ ≤ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fseminf• Example Minimizesthefunction
subjecttotheconstraints
• Theunconstrainedobjectivefunctionisminimizedat x = 1.However,theconstraint,
implies
( )21x−
( ) ( )2
0 2
1 1( , ) 0 for all 0 12 2
x
g x t x t t
≤ ≤
= − − − ≤ ≤ ≤
2( , ) 0 for all 0 1g x t t≤ ≤ ≤12x ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fseminf• Writetheobjectivefunctionasananonymousfunction
objfun = @(x)(x-1)^2;
• Writethesemi-infiniteconstraintfunction,– Includesthenonlinearconstraints([ ]inthiscase)– Initialsamplingintervalfor t (0to1instepsof0.01inthiscase)– Thesemi-infiniteconstraintfunctiong(x, t):function [c, ceq, K1, s] = seminfcon(x,s)% No finite nonlinear inequality and equality constraints c = []; ceq = []; % Sample setif isnan(s) % Initial sampling interval
s = [0.01 0]; end t = 0:s(1):1; % Evaluate the semi-infinite constraint K1 = (x - 0.5) - (t - 0.5).^2;
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fseminf• Call fseminfwithinitialpoint0.2,andviewtheresult:
x = fseminf(objfun,0.2,1,@seminfcon)
x =0.5000
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Solvesanonlinearproblemspecifiedby
whereF(x)isafunctionthatreturnsavectorvalue
• Syntex[x,fval]= fsolve(fun,x0)
( ) 0F x =
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Example Solvetwononlinearequationsintwovariables.The
equationsare:-( ) ( )
( ) ( )
1 2 22 1
1 2 2 1
1
1cos sin2
x xee x x
x x x x
− +− = +
+ =
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Example Solvetwononlinearequationsintwovariables.The
equationsare:-
• ConvertinginF(x)=0
( ) ( )
( ) ( )
1 2 22 1
1 2 2 1
1
1cos sin2
x xee x x
x x x x
− +− = +
+ =
( ) ( )
( ) ( )
1 2 22 1
1 2 2 1
1 0
1cos sin 02
x xee x x
x x x x
− +− − + =
+ − =
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Matlab Codeforfunctionfile
function F = root2d(x) F(1) = exp(-exp(-(x(1)+x(2)))) - x(2)*(1+x(1)^2); F(2) = x(1)*cos(x(2)) + x(2)*sin(x(1)) - 0.5;
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Matlab Codeforfunctionfile
function F = root2d(x) F(1) = exp(-exp(-(x(1)+x(2)))) - x(2)*(1+x(1)^2); F(2) = x(1)*cos(x(2)) + x(2)*sin(x(1)) - 0.5;
• Solvethesystemofequationsstartingatthepoint [0,0].fun = @root2d; x0 = [0,0]; x = fsolve(fun,x0)
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
fsolve• Matlab Codeforfunctionfile
function F = root2d(x) F(1) = exp(-exp(-(x(1)+x(2)))) - x(2)*(1+x(1)^2); F(2) = x(1)*cos(x(2)) + x(2)*sin(x(1)) - 0.5;
• Solvethesystemofequationsstartingatthepoint [0,0].fun = @root2d; x0 = [0,0]; x = fsolve(fun,x0)
• Resultsx =
0.3532 0.6061http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
linprog• Solvelinearprogrammingproblems
• Syntaxx = linprog(fun,A,b,Aeq,beq,lb,ub)
.min .T
x
A x bf x Aeq x beq
lb x ub
≤⎧⎪
=⎨⎪ ≤ ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
linprog• Example Linearprogramming
1 2
21
1 21 2
12
1 2
1 2
2
1421min( ) subject to
3 141
2
x
x xxx
x xx x
x x
x xx x
+ ≤⎧⎪⎪ + ≤⎪
− ≤⎪− − ⎨
⎪− − ≤⎪− − ≤ −⎪⎪− + ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
linprog• Example Linearprogramming
1 2
21
1 21 2
12
1 2
1 2
2
1421min( ) subject to
3 141
2
x
x xxx
x xx x
x x
x xx x
+ ≤⎧⎪⎪ + ≤⎪
− ≤⎪− − ⎨
⎪− − ≤⎪− − ≤ −⎪⎪− + ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
linprog• Example Linearprogramming
• Matlab Code% Objective Function f = [-1 -1/3];A = [1 1; 1 1/4; 1 -1;-1/4 -1; -1 -1; -1 1]; B = [2 1 2 1 -1 2];% calling linprogx = linprog(f,A,b,[],[],[],[])
1 2
21
1 21 2
12
1 2
1 2
2
1421min( ) subject to
3 141
2
x
x xxx
x xx x
x x
x xx x
+ ≤⎧⎪⎪ + ≤⎪
− ≤⎪− − ⎨
⎪− − ≤⎪− − ≤ −⎪⎪− + ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
linprog• Results
Optimal solution found.x =
0.6667 1.3333
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
intlinprog• Mixed-integerlinearprogramming(MILP)
• Syntaxx = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub)– intcon –integer variables
(intcon)are integers.
min.
T
x
xA x b
f xAeq x beqlb x ub
⎧⎪ ≤⎪⎨
=⎪⎪ ≤ ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
intlinprog• Example Mixed-integerlinearprogramming(MILP)
2
1 21 2
1 2
1 2
is an integer2 14
min(8 ) subject to4 332 20
x
xx x
x xx xx x
⎧⎪ + ≥ −⎪
+ ⎨− − ≤ −⎪⎪ + ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
intlinprog• Example Mixed-integerlinearprogramming(MILP)
• Matlab Code% Objective Function and integer variablef = [8;1]; intcon = 2;A = [-1,-2; -4,-1; 2,1]; B = [14;-33;20];% calling intlinprogx = intlinprog(f,intcon,A,b)
2
1 21 2
1 2
1 2
is an integer2 14
min(8 ) subject to4 332 20
x
xx x
x xx xx x
⎧⎪ + ≥ −⎪
+ ⎨− − ≤ −⎪⎪ + ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
intlinprog• Results
LP: Optimal objective value is 59.000000x =
6.5000 7.0000
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqcurvefit• Solvenonlinearcurve-fitting(data-fitting)problemsinleast-
squaressense
Nonlinearleast-squaressolverfindscoefficientsx thatsolvetheproblem.Whereuserdefineddatashouldbefeedinvectorform
• Syntaxx = lsqcurvefit(fun,x0,xdata,ydata,lb,ub)
( )22
2min ( , ) min ( , )x x iF x xdata ydata F x xdata ydata− = −∑
( , (1))( , (2))
( , )
( , ( ))
F x xdataF x xdata
F x xdata
F x xdata k
⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥⎢ ⎥⎣ ⎦
Mht
tps://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqcurvefit• Example Simpleexponentialfit
xdata =[0.91.513.819.824.128.235.260.374.681.3];ydata =[455.2428.6124.167.343.228.113.1-0.4-1.3-1.5];
2xdata1ydata xx e=
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqcurvefit• Example Simpleexponentialfit
xdata =[0.91.513.819.824.128.235.260.374.681.3];ydata =[455.2428.6124.167.343.228.113.1-0.4-1.3-1.5];
• Matlab codexdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 … 74.6 81.3]; ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 … -0.4 -1.3 -1.5];% Create a simple exponential decay modelfun = @(x,xdata)x(1)*exp(x(2)*xdata); % Fit the model using the starting pointx0 = [100,-1]
x = lsqcurvefit(fun,x0,xdata,ydata,[],[])
2xdata1ydata xx e=
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqcurvefit• Results
Local minimum possible.x =
498.8309 -0.1013
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqlin• Solveconstrainedlinearleast-squaresproblems
solvesthelinearsystem C*x = d
• Syntaxx = lsqlin(C,d,A,b,Aeq,beq,lb,ub)
2
2
.1min . .2x
A x bC x d Aeq x beq
lb x ub
≤⎧⎪
− =⎨⎪ ≤ ≤⎩
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqlin• Example FindthexthatminimizesthenormofC*x- dforan
overdeterminedproblemwithlinearequalityandinequalityconstraintsandbounds.
• Matlab codeC = [0.9501 0.7620 0.6153 0.4057;
0.2311 0.4564 0.7919 0.9354; 0.6068 0.0185 0.9218 0.9169;0.4859 0.8214 0.7382 0.4102; 0.8912 0.4447 0.1762 0.8936];
d = [0.0578; 0.3528; 0.8131; 0.0098; 0.1388];A =[0.2027 0.2721 0.7467 0.4659;
0.1987 0.1988 0.4450 0.4186; 0.6037 0.0152 0.9318 0.8462];
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqlin• Matlab codeconti.
b =[0.5251; 0.2026; 0.6721]; Aeq = [3 5 7 9]; beq = 4; lb = -0.1*ones(4,1); ub = 2*ones(4,1);
% call lsqlin to solve the problemx = lsqlin(C,d,A,b,Aeq,beq,lb,ub)
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
lsqlin• Matlab codeconti.
b =[0.5251; 0.2026; 0.6721]; Aeq = [3 5 7 9]; beq = 4; lb = -0.1*ones(4,1); ub = 2*ones(4,1);
% call lsqlin to solve the problemx = lsqlin(C,d,A,b,Aeq,beq,lb,ub)
• Resultsx = -0.1000 -0.1000 0.1599 0.4090 ht
tps://
drra
jesh
kum
ar.w
ordp
ress
.com
• Tosolvenonlinearleast-squarescurvefittingproblems
• TrustregionreflectivealgorithmorLevenburgMarquardt(setinOptions)
• lsqnonlin stopsbecausethefinalchangeinthesumofsquaresrelativetoitsinitialvalueislessthanthedefaultvalueofthefunctiontolerance.
• Syntaxx = lsqnonlin(fun,x0,lb,ub)
2 22
1min ( ) min ( )
n
ix x if x f x
=
= ∑
lsqnonlin
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• ExampleFitasimpleexponentialdecaycurvetodata.Themodelconsideredis:
where0≤t≤ 3,andε is normallydistributednoise,mean0andstandarddeviation0.05.
1.3ty e ε−= +
lsqnonlin
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• ExampleFitasimpleexponentialdecaycurvetodata.Themodelconsideredis:
where0≤t≤ 3,andε is normallydistributednoise,mean0andstandarddeviation0.05.
• Matlab coderng default; % for reproducibility d = linspace(0,3);y = exp(-1.3*d) + 0.05*randn(size(d)); fun = @(r)exp(-d*r)-y;x0 = 4;
x = lsqnonlin(fun,x0,[],[])
1.3ty e ε−= +
lsqnonlin
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Resultsplot(d,y,'ko',d,exp(-x*d),'b-')legend('Data','Best fit')xlabel('t')ylabel('exp(-tx)')
lsqnonlin
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Findsaminimumforaquadraticprogrammingproblemthatisspecifiedby:
• Syntaxx = quadprog(H,f,A,b,Aeq,beq,lb,ub,options)
- options – optimtool options forspecificfunction
1min .2
T T
x
Ax bx Hx f x Aeq x beq
lb x ub
≤⎧⎪
+ ≤⎨⎪ ≤ ≤⎩
quadprog
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Example Solvethegiventhequadraticprogrammingproblem:
subjectto
2 21 2 1 2 1 2
1( ) 2 62
f x x x x x x x= + − − −
1 2
1 2
1 2
1 2
22 2
2 30 ;0
x xx xx xx x
+ ≤
− + ≤
+ ≤
≤ ≤
quadprog
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Example Solvethegiventhequadraticprogrammingproblem:
subjectto
• Matlab CodeH = [1 -1; -1 2]; f = [-2; -6]; A = [1 1; -1 2; 2 1]; b = [2; 2; 3]; lb = zeros(2,1);options = optimoptions('quadprog', 'Algorithm', 'interior-point-convex','Display','off'); [x,fval]= quadprog(H,f,A,b,[],[],lb,[],[],options);
2 21 2 1 2 1 2
1( ) 2 62
f x x x x x x x= + − − −
1 2
1 2
1 2
1 2
22 2
2 30 ;0
x xx xx xx x
+ ≤
− + ≤
+ ≤
≤ ≤
quadprog
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Resultx =
0.66671.3333
fval =
-8.2222
quadprog
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Genetic Algorithm (GA)• Geneticalgorithmsolverformixed-integerorcontinuous-
variableoptimization,constrainedorunconstrained• Syntax
x = ga(fun,nvars,A,b,Aeq,beq,lb,ub)
– nvars isthedimension(numberofdesignvariables)offun
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Genetic Algorithm (GA)• Example Usethegeneticalgorithmtominimizethe
ps_example function{inbuiltinMatlab},withfollowingconstraints:-
1 2
2 1
1
2
15
1 63 8
x xx x
xx
+ ≥
= +
≤ ≤
− ≤ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Genetic Algorithm (GA)• Rearrangingconstraints
• Matlab CodeA = [-1 -1]; b = -1;Aeq = [-1 1]; beq = 5lb = [1 -3]; ub = [6 8];%Set bounds lb and ubfun = @ps_example;
x = ga(fun,2,A,b,Aeq,beq)
1 2
1 2
1
2
15
1 63 8
x xx x
xx
+ ≥
− + =
≤ ≤
− ≤ ≤
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Genetic Algorithm (GA)• Results
x =-2.0000 2.9990
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• BoundconstrainedoptimizationusingParticleSwarmOptimization(PSO)
• Minimizesobjectivefunctionsubjecttoconstraints• MATLABbuiltinfunctionparticleswarm
– Maydefineasanunconstrainedproblemorfindsolutioninarange
– Allowstospecifyvariousoptionsusingoptimoptions
• Syntaxx = particleswarm(fun,nvars,lb,ub,options)
particleswarm
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
particleswarm• Exampletheobjectivefunction.
Setboundsonthevariables
2@( ) (1)*exp( ( ) )fun x x norm x= −
[ 10, 15]and [15,20]lb ub= − − =
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
particleswarm• Exampletheobjectivefunction.
Setboundsonthevariables
• Matlab Code%Define the objective function.fun = @(x)x(1)*exp(-norm(x)^2);
%Call particleswarm to minimize the function.rng default % For reproducibility nvars = 2;x = particleswarm(fun,nvars)
2@( ) (1)*exp( ( ) )fun x x norm x= −
[ 10, 15]and [15,20]lb ub= − − =
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
particleswarm• Result
x = 629.4474 311.4814
%This solution is far from the true minimum, as %you see in a function plot.fsurf(@(x,y)x.*exp(-(x.^2+y.^2)))
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• FindstheminimumofagivenfunctionusingtheSimulatedAnnealingAlgorithm.
• Maydefineupperandlowerbounds• Usefulinminimizingfunctionsthatmayhavemanylocal
minima
• Syntax[x,fval] = simulannealbnd(fun,x0,lb,ub,options)
simulannealbnd
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Example MinimizeDeJong’s fifthfunctionà 2Dfunctionwithmanylocalminima.
• Startingpointà [0,0]• UBis64andLBis-64
DeJong’s FifthFunctionPlot– ManyLocalMinima
simulannealbnd
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Plot De Jong’s function and assign fcn = @dejong5fcn
In main program, simulannealbnd with
starting [0,0]
Set LB and UB. Call simlannealbnd
with LB and UB
No Lower and Upper Bounds
fun = @dejong5fcn;x0 = [0 0];x = simulannealbnd(fun,x0)
Outputx =
-31.9785 -31.9797
With LB and UBfun = @dejong5fcn;x0 = [0 0];lb = [-64 -64];ub = [64 64];x = simulannealbnd(fun,x0,lb,ub)
Outputx =
0.0009 -31.9644
Note: The simulannealbnd algorithm uses the MATLAB® random number stream, so different results may be obtained after each run
simulannealbnd (example)
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Multi-objective GA• FindParetofrontofmultiplefitnessfunctionsusinggenetic
algorithm• Syntax
x = gamultiobj(fitnessfcn,nvars,A,b,Aeq,beq,lb,ub)
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Multi-objective GA• Example ComputetheParetofrontforasimplemulti-
objectiveproblem.Therearetwoobjectivesandtwodecisionvariables x 2
1
2
2
( )
2( ) 0.5 2
1
f x x
f x x
=
⎡ ⎤= − +⎢ ⎥
⎣ ⎦
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Multi-objective GA• Example ComputetheParetofrontforasimplemulti-
objectiveproblem.Therearetwoobjectivesandtwodecisionvariables x
• Matlab codefitnessfcn = @(x)[norm(x)^2,0.5*norm(x(:)-[2;-1])^2+2];
rng default % For reproducibility x = gamultiobj(fitnessfcn,2,[],[],[],[],[],[])
21
2
2
( )
2( ) 0.5 2
1
f x x
f x x
=
⎡ ⎤= − +⎢ ⎥
⎣ ⎦
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Multi-objective GA• Results
Optimization terminated: average change in the spread of Pareto solutions less than options.FunctionTolerance. x =
-0.0072 0.0003 0.0947 -0.0811 1.0217 -0.6946 1.1254 -0.0857
-0.0072 0.0003 0.4489 -0.2101 1.8039 -0.8394 0.5115 -0.6314 1.5164 -0.7277 1.7082 -0.7006 1.8330 -0.9337 0.7657 -0.6695 0.7671 -0.4882 1.2080 -0.5407 1.0075 -0.5348 0.6281 -0.1454 2.0040 -1.00641.5314 -0.9184
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
Multi-objective GA• Plotthesolutionpoints
plot(x(:,1),x(:,2),'ko')
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Findalocalminimumagivenfunctionusingpatternsearchsubjectto– linearequalities/inequalities,– lower/upperbounds,– non-linearinequalities/equalities
• Specificoptionsmaybegivenusingoptimoptions
• Syntaxx = patternsearch(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon)
patternsearch
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Example1 Minimizeanunconstrainedproblemusingauserfunctionandpatternsearch solver Define a user
function (obj1) that describes the
objective
In main program, call the function obj1 and define starting point
Use patternsearch to find the minima
Main Codefun = @obj1;x0 = [0,0];x = patternsearch(fun,x0)
Function Codefunction[y] = obj1(x)y = exp(-x(1)^2-x(2)^2)*(1+5*x(1) + 6*x(2) + 12*x(1)*cos(x(2)));
Outputx =
-0.7037 -0.1860
y = exp(-x(1)2-x(2)2)*(1+5x(1) + 6*x(2) + 12x(1)cos(x(2)));
patternsearch
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com
• Example2 Solvingthepreviousproblemwithlowerbound(LB)andupperbound(UB)specifiedandstartingat[1,-5]:
Main Codefun = @obj1;lb = [0,-Inf];ub = [Inf,-3];A = [];b = [];Aeq = [];beq = [];x0 = [1,-5];x = patternsearch(fun,x0,A,b,Aeq,beq,lb,ub)
Outputx =
0.1880 -3.0000
1
2
03
xx
≤ ≤ ∞
−∞ ≤ ≤ −
patternsearch
http
s://
drra
jesh
kum
ar.w
ordp
ress
.com