4. Thai Menu Problem [TMP] Formulation · The combinatorial question is how many relief packages of each type to . 4 Thai Menu Problem [TMP] Formulation 119 load into the cargo bay

118

4. Thai Menu Problem [TMP] Formulation

4.1 Parameterising a Combinatorial Problem

4.1.1 Combinatorial Optimisation Revisited

Combinatorial vs. Parametric Optimisation

In terms of problem formulation, the distinction between parametric and combinatorial optimisation has

tended to be defined in terms of the discreteness and/or finiteness of the solution sets. Consider Linear

Programming (LP) [Dantzig 1966; Solow 1984]:

max ,

, ,

[ , )

c x c

Ax b A b

x

t K

M K M

K

∈ℜ

≤ ∈ℜ ∈ℜ

∈ ∞

×subject to

0

(1)

and the Knapsack Problem (KnP) [Martello & Toth 1990]:

max ,

, ,

c n c

a n b a b

n

t K

t K

K

∈ℜ

≤ ∈ℜ ∈ℜ

∈

subject to

N

(2)

The former is an example of parametric optimisation, the latter combinatorial optimisation. However, save

for the variable space description, i.e. LP’s x K∈ ∞[ , )0 vs. KnP’s n K∈N (and the fact that the KnP

generally has but one constraint relation) the two problem statements/formulations are virtually

indistinguishable from one another. We can almost see a combinatorial problem as a parametric problem

with additional constraints enforcing discreteness the solution set, i.e. alluding to the set/sub-set

relationship N K K⊂ ∞[ , )0 .

Consider a pedagogical example KnP: a Red-Cross cargo plane, on an emergency relief mission, is to

carry, nk =1 ‘foodstuff bundles’, nk =2 ‘medical kits’, and nk =3 ‘shelter materials’. Each ‘relief package’

contributes additively to the overall objective of the mission and utilises a certain amount of weight (or

space) allowance, and these objective (contribution) and constraint (resource utilisation) coefficients

depend on the type of relief package. But the coefficient values are constant and are the same for every

relief package of a given type. The combinatorial question is how many relief packages of each type to

4 Thai Menu Problem [TMP] Formulation

119

load into the cargo bay. The combinatorial optimisation question is how to do so such that the objective

function is maximised, while the weight/space constraint is not exceeded.

In practical terms, parametric and combinatorial optimisation are also distinguished by the continuity vs.

discreteness of the trajectorial path taken by an optimisation algorithm. From iteration to iteration, a

combinatorial algorithm generally ‘jumps’ from one combinatorial solution to the next, while a parametric

algorithm generally takes continuous1 steps in the direction of the steepest gradient.

We propose to look past such an algorithmic basis for distinction (including the existence of polynomial

time algorithms, i.e. [Karmarkar 1984], for an LP but not for a KnP), and instead to think of the

compositional aspect of the problem solution itself. This is in keeping with our ideal of not letting

considerations with regards to the optimisation algorithm (exploration mechanism and optimisation

directives) impinge on our formulating an appropriate handle on the optimisation problem (solution

representation and performance evaluation).

We propose the following viewpoint: in a parametric analysis, we deal with singular solutions; whereas,

in a combinatorial analysis, we deal with compositional structures of solution-components, whence

compositional solutions. In parametric optimisation, the solution space is defined in terms of the

parametric space pertaining to a single solution entity, i.e. a single portfolio, a single regression equation,

and so on. On the other hand, in combinatorial optimisation, the solution space is defined in terms of the

combinatorial space pertaining to a multiplicity of components, i.e. a collection of KnP Red-Cross aid

packages, a set of TSP inter-city routes, and so on. A purely parametric space pertains to the definitive set

of attributes of the solution; a purely combinatorial space pertains to a compositional structure, but does

not deal with each individual component’s definitive set of attributes.

There is nothing parametric about the KnP solution space, as every parametric attribute associated with

each solution-component is a (non-optimised) problem-defined constant. For instance, with

( )ct

= 81 6 2 7. . we always add 8.1 ‘units of objective utility’ for each foodstuff bundle, 6.2 for each

medical kit, and so on. Similarly, with ( )at

= 57 0 9 13. . . , each foodstuff bundle uses 5.7 units of

resource, while each medical kit uses up 0.9, and so on.

We foreshadow four significant developments. First, the parameters, or parametric attributes, associated

with the various solution-components within a combinatorial structure will be ‘individualised’. That is,

the foodstuff bundles need not all add the same amount to the objective function, nor necessarily subtract

the same amount from the resource pool. Second, these individualised attributes will be variable, and

subject to optimisation, whence leading up to our combinatorial-parametric solution space. Third, to

enable an optimisation search over this complex combinatorial and parametric space, the parametric

exploration mechanism will have to be distributed. That is, the underlying solution-components will be

1 An LP simplex algorithm happens to take discrete, enumerable paths, but this is because the simplex algorithm exploits the finiteness of the number of ‘corners’ of a hypercube; the underlying solution space itself remains continuous.

Evolutionary Optimisation and Financial Model-Trading

120

responsible for some form of parametric self-adaptation. No global, all knowing procedure will be

responsible for all the parametric variations that will take place throughout the course of the algorithmic

optimisation process. Finally, this distributed property means that, ultimately, the optimisation algorithm

at large will not need to ‘know’ anything about the nature of the embedded variable space associated with

each individual solution-component. In other word, the distributed property enables encapsulation of the

solution-components’ parametric attributes.

This (problem formulation) chapter addresses the first two developments. That is, we undertake to

“parameterise a combinatorial optimisation problem/solution space”, using KnP as our purely

combinatorial starting point. The latter two developments will be taken up in the next (optimisation

methodology) chapter.

OO View of Combinatorial Solution Structures

Our new ‘handle’ on a combinatorial problem solution can in fact be described quite naturally from an OO

modelling and implementation viewpoint, i.e. one in which system entities are captured as class objects.

We propose the following OO viewpoint: a combinatorial, compositional solution structure is defined as a

collection of objects from a collection of classes. For instance, consider a hypothetical KnP solution

( )nt

= 3 9 2 . This entails an inclusion of 3 objects of the first class, i.e. 3 foodstuff bundles, 9 objects

of the second classes, i.e. 9 medical kits, and so on, where we have K = 3 classes of objects to select

from. In general, we say that there are K >1 classes of solution-component objects, or simply component

objects.

In a KnP, note how for each class k k K, , ,= 1K , every object is ‘generic’ in the sense that all the

objects instantiated from a particular class are commonly represented by the same constant values of the

objective coefficient ck and the constraint coefficient ak . That is to say, there is class identity without

object identity. Such a class is said to be ‘comprehensively generic’ in this respect. This ‘class identity

sans object identity’ property is critical for KnP, for it enables: one set of values (i.e. the objective

coefficient ck and the constraint coefficient ak ) to parameterise all objects of a given class

k K∈{ , , }1K , and (ii) the presence of a multiplicity of objects to be represented by the class designator

k K∈{ , , }1K and a single number, the multiplier variable n k Kk , , ,= 1K (i.e. used to multiply to the

class-specific ck and ak ). There is no need, as it were, to get more intimate with the individual object and

its object attributes. If all foodstuff bundles are absolutely identical, a collection of, say, n1 50= foodstuff

bundles is more economically represented by a generic and universal model of a foodstuff bundle (only c1

and a1 are relevant to the analysis at hand) plus a numerical factor of 50 to indicate the lot size. If every

medical kit adds the same amount to the ‘utility function’ and uses up the same amount of air-lift resource,

then 50 medical kits altogether give and take exactly 50 times as much as a single unit, taken and procured

in isolation. Moreover, the values of ck ’s and ak ’s are constant, the KnP solution-component classes are


121

said to be ‘comprehensively generic and constant’. The class-specific and constant definition of a KnP’s

ck ’s and ak ’s corresponds to the OO notion of constant class attributes. If we adopted this OO

viewpoint, then “parameterising a purely combinatorial solution space” can be restated equivalently in

terms of “turning constant class attributes into non-constant object attributes”.

We first describe our way of viewing a purely combinatorial KnP solution as an ensemble of solution-

component class objects (4.1.2). After briefly reviewing an existing parametric KnP formulation (4.1.3),

we then consider the parametric attributes to be non-constant optimisable variables associated with the

various classes of solution-components (4.1.4) and ultimately with the individual solution-component

objects themselves (4.1.5). Our direct combinatorial-parametric optimisation problem is then christened

the Thai Menu Problem [TMP]this to reflect the nature of the optimisation problem, its solution space,

and implications in terms of engineering an appropriate optimisation methodology.

4.1.2 Objectified Knapsack Problem

Taking a KnP formulation, stated in terms of vector elements:

max

, , ,

c n

a n b

n k K

k kk

K

k kk

K

k

=

=

∑

∑ ≤

∈ =

1

11

subject to

N K

(3)

the first chore is to rewrite the KnP formulation using our OO notation superscript:

( )

max c n

q n Q

n n n n

k k

k

K

k k

k

K

k K t K

=

=

∑

∑ ≤

≡ ∈

1

1

1

subject to

L L N

(4)

From now, we will focus on the objective function aspect of the problem, worrying about the constraint

definition as needed.2 The second chore is to replace n k ∈N with 11j

n k

=∑ , but with the stipulation that

Expression jj=∑ =1

00 . In other words, a non-negative integer (counting number) n k is replaced by a

summation of the value ‘1’ n k times if n k is greater than zero, and zero, if n k is exactly zero, a

quirky, but harmless, substitution. Substituting this into the objective function and moving the constant

terms (which actually does not contain any j-indexed quantity or expression) inside the j-indexed

summation, we obtain the following restatement of the objective function:

2 In fact, as we are ultimately heading toward a genetic-evolutionary optimisation methodology, the constraint set/feasibility description will be taken care of automatically by an appropriately designed genetic encoding scheme and corresponding genotype-phenotype map.


122

c n c ck k

k

Kk

j

n

k

Kk

j

n

k

Kk k

= == ==∑ ∑∑ ∑∑= ⋅

=

1 11 11

1 (5)

From our mathematically trivial substitution, this particular form will pave the way for us to start thinking

in terms of objects. That is, in a KnP the system objective is an additive function, adding the objective

coefficients from all the nt 1 solution-component objects:

o j k kj n k K, , , , , ,= =1 1K K (6)

The class information comes in to play only because in a KnP each solution-component’s objective

coefficient is solely dependent on its class designation k K∈{ , , }1K (i.e. class identity without object

identity).

Let us summarily observe that: (i) the KnP objective function is additive; (ii) same-classed solution-

component objects contribute exactly the same amount to the objective function value, (iii) which is a

problem-defined constant. We refer to these features, respectively, as (i) the ‘additive property’, (ii) the

‘generic property’, and (iii) the ‘constant property’ of a KnP objective function. One by one, these will be

generalised away. The ‘constant property’ is the first to go, meaning that what had hitherto been a

problem-defined constant is now free to vary, hence constituting part of the solution variable, whence

subject to optimisation. Next, the ‘generic property’ will be generalised away, as it distinguishes between

optimisable class-specific parameterisation (simple optimisation search over a ‘statically allocated’

search space) and optimisable object-specific parameterisation (complex optimisation search over a

‘dynamically allocated’ search space). In the latter case, not only are the various parameter values

variable and subject to optimisation, the dimensionality of the search space itself will be variable and

subject to optimisation as well! Finally, our two-stage combinatorial-parametric optimisation problem can

be formulated without concerning ourselves with the ‘additive property’, which is a property of how we

use the collection of solution-components together as an ensemble, and therefore the most ‘problem

dependent’ aspect under consideration.

Component Object Behaviour

Instead of thinking about the scalar objective coefficients, let us think in terms of a utility contribution

function defined on these component objects:

( )c c c j n k Kj k k k ko k= = = =. , , , , , ,1 1K K (7)

where k k c k K. , , ,= 1K , are class attributes with constant values c k Kk , , ,= 1K .

In terms of the C++ programming language, we depict the following example, where a ‘FoodStuff’ class

defines the utility contribution class method ObjectiveCoefficient(), which always returns the constant

class attribute m_s_ClassSpecificObjectiveCoefficient:


123

// header (*.h) file class FoodStuff { Protected: static const double m_s_ClassSpecificObjectiveCoefficient; public: static double ObjectiveCoefficient() {return m_s_ClassSpecificObjectiveCoefficient;} /* etc. */ };

Note how m_s_ClassSpecificObjectiveCoefficient is modified with the C++ reserved word ‘static’,

meaning that every object of this class shares this class attribute, a consequence of the ‘generic property’

or lack of object identity. This is the definitive difference between class-specific and object-specific

parameterisation which will be of consequence. Similarly, ObjectiveCoefficient() is also ‘static’, meaning

that it is a class method, to be called with the class name reference ‘FoodStuff::’, hence:

// implementation (*.C) file const double FoodStuff::m_s_ClassSpecificObjectiveCoefficient = 8.0; main() { cout << “Each foodstuff object contributes the SAME value ” << FoodStuff::ObjectiveCoefficient() << “ to the objective function, and this value may NEVER change.”; }

Object-Ensemble Behaviour

Just as we started to think of each individual component’s objective coefficient as a utility contribution

function, the overall KnP objective function itself is defined on the object-ensemble itself:

f n

k

K

j

nj k

k

( ) =

= =⊕⊕Φ

1 1o

(8)

Having used the generalised object and ensemble notation, we have yet to generalise away the ‘additive

property’, ‘generic property’, and ‘constant property’. So at this point, we can still ‘retrace’ the objective

function completely back to the KnP form:

( )

( )

Φk

K

j

nj k

additive property

j k

j

n

k

K

j k

j

n

k

K

generic property

k

j

n

k

K

k

j

n

k

K

constant property

k k k

k

K

j

n

k

K

k k

k k

k k

c

c c

c c c n f n

= = ==

== ==

== ===

⊕⊕ ∑∑

∑∑ ∑∑

∑∑ ∑∑∑

=

=

= = =

1 1 11

11 11

11 111

o o

o k

k

.

. ( )

(9)

In C++, we depict the following example of a KnP solution class ‘RedCrossAidCargo’. Because each KnP

relief package class, be it ‘FoodStuff’, ‘MedicalKit’, or ‘ShelterUnit’, is ‘comprehensively generic’, there is

no need to maintain any actual aid package object as part of the RedCrossAidCargo class definition. In

other word, we only need to work with the multiplier factors n FoodStuff (‘m_NumFoodStuffs’),

n MedicalKit (‘m_NumMedicalKits’), and n ShelterUnit (‘m_NumShelterUnits’), which together specify the


124

complete solution to the problem as it now stands:

class RedCrossAidCargo { protected: unsigned m_NumFoodStuffs; unsigned m_NumMedicalKits; unsigned m_NumShelterUnits; public: double ObjectiveFunctionValue() const { return (m_NumFoodStuffs * FoodStuff::ObjectiveCoefficient() + m_NumMedicalKits * CMedicalKit::ObjectiveCoefficient() + m_NumShelterUnits * ShelterUnit::ObjectiveCoefficient()); } };

4.1.3 Parametric Knapsack Problem

Following [Carstensen 1983], the KnP objective function can be parameterised thus:

( )f n a b nk k k

k

K[ ] ( )τ τ= +

=∑

1

(10)

where the various a k ’s and b k ’s are still problem-defined constants, but there is a free parameter τ

floating about. This is all well and nice, but offers us very little to go on from in the sense that this

parameter definition, and its use as a multiplier in a linear expression, is common to all classes of objects.

There is no class identity, never mind object identity. In fact, the entire parameter space of this

parameterised combinatorial problem is simply given by ℜ , as τ ∈ℜ , and fixing the value of τ reduces

a bk kτ + to a mere constant. Moreover, τ is not optimised. For us, introduced problem parameters will

be subject to optimisation.

4.1.4 Class-Specific Parameterisation


Among the three features of a KnP objective function, the first to go is the ‘constant property’. For each

component class, the class-specific objective coefficient is now a utility contribution function defined on a

set of class-specific function-parameters θ k k∈Θ , which are henceforth optimisable variables:

( )k k k k kc c j n k K. , , , , , ,= = =θ 1 1K K (11)

Note that there is still no reference to the component object index j in ( )c k Kk kθ , , ,= 1K . This is

because the ‘generic property’ has not been generalised away at this point, and all class k k objects still

share the common parametric class attribute θ k k∈Θ .

The reason we want to start thinking about a utility contribution function ( )c k kθ rather than a


125

utility contribution value c k is because we are moving toward the case we are interested in, namely a

solution ensemble (signal-ensemble trading model) whose components are in fact parameterised functions

(signal models).

Variable class-specific parameterisation is depicted in the following example:

( )( )

c a b

c a b

1 1 1 1 1 1

2 2 2 2 2 2

θ θ θ

θ θ θ

= + ∈ℜ

= + ∈ℜ

,

,

(12)

However, this example unimaginatively ‘under-realises’ the generalisation potential of class-specific

parameterisation. Instead, consider another example with K = 3 object classes:

( ) ( )( ) ( ) ( )( )

c

c

c

1 111

21 2

11

21 1 1 2

2 21

222 4

32 2 2

3 3 10 3 3

2 4 05

2 0

100 3 1 2 4 5 5013

23

θ θ θ θ θ θ

θ θ θ θ θ

θ θθθ

= + + ⋅ − ⋅ ⋅ ∈ = ℜ

= + + + ∈ = ℜ×ℜ × ∞

= − + ∈ = ×

. ,

ln , [ , )

, { , , } [ , ]

Θ

Θ

Θ

(13)

where we note that each class may differ in their respective utility contribution functions, and in fact

require different sets of function-parameters, i.e. Θ Θ Θ1 2 3≠ ≠ , thus forming a variety of attribute

spaces. In general, let us denote the number of attribute fields with D K . For instance, for the above

examples:

Θ

Θ

Θ

1 2 1

2 2

3 3

2

0 3

1 2 4 5 50 2

= ℜ ⇒ =

= ℜ×ℜ× ∞ ⇒ =

= × ⇒ =

D

D

D

[ , )

{ , , } [ , ]

(14)

In C++, we redefine the FoodStuff class such that each FoodStuff::ObjectiveCoefficient() class method

now calculates the utility contribution as a function of two parameters, ‘m_s_Theta1’ and ‘m_s_Theta2’:

class FoodStuff { protected: static double m_s_Theta1; static double m_s_Theta2; public: static double ObjectiveCoefficient() const { return (2.0 + m_s_Theta1 + 4.0 * pow(m_s_Theta2,2.0) - 0.5 * m_s_Theta1 * m_s_Theta2); } /* etc. */ };

where ObjectiveCoefficient() is still called with the ‘FoodStuff::’ class reference:

// implementation (*.C) file main() { cout << “Each foodstuff object contributes the SAME value ” << FoodStuff::ObjectiveCoefficient() << “ to the objective function, and this value MAY change.”; }


126

Note that we are still stuck with the ‘static’ qualifier, even though we are rid of the ‘const’ qualifier.


With the ‘additive property’ still around, we can retrace our objective function back to almost the original

KnP form:

( ) ( )Φ

k

K

j

nj k k

j

n

k

Kk k

j

n

k

Kk k k

k

Kk k k

c c n c= = == == =⊕⊕ ∑∑ ∑∑ ∑

= = =

1 1 11 11 1o k . θ θ

(15)

which stops short of simplifying the various functions ( )c k kθ ’s to problem-defined constants c k ’s.

The original KnP objective function can now be viewed as a specialisation of our generalised form:

{ ( )f n c n n ck k

k

Kk k k

k

K

constantk

( ) = == =∑ ∑

parametricconstant

functionevaluation

1 1θ

θ

6 74 84

(16)

In terms of C++ programming, as befitting an OO framework, the implementation of RedCrossAidCargo

remains untouched. In other word, while the behaviour of the component objects in terms of the utility

contribution functions have changed, the behaviour of the object-ensemble in terms of combining the

individual utility contributions remains unchanged.

Statically-Allocated Combinatorial-Parametric Solution Space

Let us consider the solution space associated with this optimisation problem. Because there is only one

single parameter space associated with each and every component class, we can enumerate the entire

parametric space thus:

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

=

=

∈ ≡ × × × ×=∏

1

11

1

1

1

1

1

1

M

M

M

M

M

M

M

L Lk

K

D

k

D

k

K

D

K

k

k

Kk K

k

K

Θ Θ Θ Θ

(17)

From this, we can also enumerate the solution space associated with the problem:


127

x ≡

=

=

=n

n

n

n

n

n

n

n

n

nk

K

k

K

k

K

k

K

k

K

θ θ

θ

θ

θ

θ

θ

θ

1

1

1

1

1

11

M

M

M

M

M

M

M

M

M

M

M

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

D

k

D

k

K

D

K

k

K

D

k

D

k

K

D

K

k

K

k

K

n

n

n

1 1

1

1

1

1

11

1

1

1

=

M

M

M

M

M

M

M

M

M

M

M

∈ ×=∏N K k

k

K

Θ1

(18)

where we enumerate both the combinatorial variables xkkn k K≡ =, , ,1K , and their variable space

N K , as well as their parametric variables xK kk k K+ ≡ =θ , , ,1K , and the variable space Θ k

k

K

=∏ 1.

In terms of a computer program, this means that we can statically allocate all the memories needed to

specify a solution variable to such an optimisation problem. In terms of a genetic-evolutionary

optimisation methodology, this means that there is to be one fixed genotype data structure for every single

solution prototypes within a simulated evolutionary population.

But proceeding further, let us ‘reshuffle’ the variables around such that both the combinatorial and the

parametric variables associated with a given component class are given adjacent address, thus:

x ≡

=

n

n

n

n

n

n

k

k

K

K

D

k

k

D

k

K

K

D

K

k

K

1

1

1

11

1

1

1

1

θ

θ

θ

θ

θ

θ

θ

θ

θ

M

M

M

M

M

M

M

=

n

n

n

D

k

k

D

k

K

K

D

K

k

K

1

11

1

1

1

1

θ

θ

θ

θ

θ

θ

M

M

M

M

M

=

∈ × × × ×

n

n

n

D

k

k

D

k

K

K

D

K

K

k

K

1

11

1

1

1

1

1

θ

θ

θ

θ

θ

θ

M

M

M

M

M

LN NΘ Θ

(19)

Class-Specific Combinatorial-Parametric Optimisation Problem


128

We now have, in general terms, an optimisation problem defined over a (combinatorial as well as

parametric) solution space:

max

x N No

∈ × × × × = =⊕⊕

Θ ΘΦ

11 1L K

k

k

K

j

nj k

(20)

Now we ‘separate the combinatorial from the parametric’, treating this as a two-stage optimisation

problem. There are two possible formulations, pre-staging the combinatorial with the parametric, or the

other way around. It does not seem to matter which of the following form is chosen:

max

max max

max max

x N N

N

N

o

o

o

∈ × × × × = =

∈ × × ∈ = =

∈ ∈ × × = =

⊕⊕

⊕⊕

⊕⊕

⇔

⇔

Θ Θ

Θ Θ

Θ Θ

Φ

Φ

Φ

1

1

1

1 1

1 1

1 1

L

L

L

K

k

K K

k

K K

k

k

K

j

nj k

n k

K

j

nj k

n k

K

j

nj k

θ

θ

(21)

However, the above solution space enumeration is somewhat misleading. Suppose that

∃ ∈ ∋ =l K n l{ , , }1 0K , then the Θ l parameter space is completely irrelevant to the problem at large. In

other words, in this seemingly one-stage optimisation problem, there lurks a well-defined multi-stage

problem precedence, with a combinatorial problem ‘pre-staging’ the parametric one, i.e. whereby the

combinatorial variable n K∈N determines the relevance of the parametric space Θ kk

K

=∏ 1.

4.1.5 Object-Specific Parameterisation


Having generalised away the ‘constant property’ of a KnP, the second to go is the ‘generic property’ of a

KnP objective function:

( ) ( )c c c j n k Kk j k j k k j k ko o= ≡ = =. , , , , , ,θ 1 1K K (22)

Here the component object index j carries on through. Every class object has its own individual

parameter, i.e. parametric object attribute, θ j k k∈Θ not shared by any other. For example:

( ) ( )( ) ( )( )( )( )

c

c

c

c

c

a a a a a a

b b b b b b

u u

v v

w

u

u

v

v

w

w

1 11

12

1 2

11

21 1 1

1 11

12

1 2

11

21 1 1

2 2 10 2 2

2 2 10 2 2

2 2 10

2 4 05

2 4 05

100 3

100 3

100 3

12

22

12

22

12

22

θ θ θ θ θ θ

θ θ θ θ θ θ

θ θ

θ θ

θ

θθ

θθ

θθ

= + + ⋅ − ⋅ ⋅ ∈

= + + ⋅ − ⋅ ⋅ ∈

= − + ∈

= − + ∈

= − +

. ,

. ,

,

,

Θ

Θ

Θ

Θ

, θ w 2 2∈Θ

(23)

Note that even though the parametric attribute fields are individualised, the number of fields are not, i.e.


129

we still have the same values of D kk , , ,= 1 2 3.

In C++, the FoodStuff class is now redefined in terms of non-‘static’ object attributes m_Theta1 and

m_Theta2:

// header (*.h) file class FoodStuff { protected: double m_Theta1; double m_Theta2; public: /////////////////////////////////////////////////////////////////////////////////////////////////// double GetTheta1() const {return m_Theta1;} double GetTheta2() const {return m_Theta2;} /////////////////////////////////////////////////////////////////////////////////////////////////// void SetTheta1(double T1) {m_Theta1 = T1;} void SetTheta1(double T2) {m_Theta1 = T2;} /////////////////////////////////////////////////////////////////////////////////////////////////// double ObjectiveCoefficient() const { return (2.0 + m_Theta1 + 4.0 * pow(m_Theta2,2.0) - 0.5 * m_Theta1 * m_Theta2); } };

where ObjectiveCoefficient() is an object method, to be called with an object name reference, thus:

// implementation (*.C) file main() { FoodStuff FirstFoodStuff; FoodStuff SecondFoodStuff; FirstFoodStuff.SetTheta1(1.2); FirstFoodStuff.SetTheta2(3.4); SecondFoodStuff.SetTheta1(5.6); SecondFoodStuff.SetTheta2(7.8); cout << “The first foodstuff package adds ” << FirstFoodStuff.ObjectiveCoefficient() << “ to the objective function.” << endl << “The second foodstuff package adds ” << SecondFoodStuff.ObjectiveCoefficient() << “ to the objective function.” << endl; }

freeing each FoodStuff object to have its own value of m_Theta1 and m_Theta2. In other word, class

FoodStuff is no longer ‘comprehensively generic’. There is now object identity.


With the ‘additive property’ still around, we can retrace our objective function back to:

( )Φk

K

j

nj k j k

j

n

k

Kk j k

j

n

k

Kk k k

c c= = == ==⊕⊕ ∑∑ ∑∑

= =

1 1 11 11o o . θ

(24)

which stops short of simplifying the summation ( )c k j kj

n k

θ=∑ 1

to the multiplication ( )n ck k kθ .


130

The original KnP objective function can now be viewed as a specialisation of our generalised form:

( ) ( )f n c n n c ck k

k

Kk k k

k

K

constant

k j k

j

n

k

K

constantk

k

j k k

( ) = = == = == =

∑ ∑ ∑∑1 1 11

θ θθ θ θ

(25)

In terms of C++ programming, the implementation of the RedCrossAidCargo object-ensemble class now

has to be redefined. It no longer suffices to maintain m_NumFoodStuffs, m_NumMedicalKits, and

m_NumShelterUnits. A RedCrossAidCargo solution really needs to maintain actual FoodStuff, MedicalKit,

and/or ShelterUnit component class objects:

class RedCrossAidCargo { protected: ArrayOf<FoodStuff> m_FoodStuffPackage; ArrayOf<CMedicalKit> m_MedicalKitPackage; ArrayOf<ShelterUnit> m_ShelterUnitPackage; public: double ObjectiveFunctionValue() const { int i; double SumCoefficients = 0.0; for(i=0;i<m_FoodStuffPackage.GetSize();i++) SumCoefficients += m_FoodStuffPackage[i].ObjectiveCoefficient(); for(i=0;i<m_MedicalKitPackage.GetSize();i++) SumCoefficients += m_MedicalKitPackage[i].ObjectiveCoefficient(); for(i=0;i<m_ShelterUnitPackage.GetSize();i++) SumCoefficients += m_ShelterUnitPackage[i].ObjectiveCoefficient(); return SumCoefficients; } };

where:

n FoodStuff = m_FoodStuffPackage.GetSize()

n MedicalKit = m_MedicalKitPackage.GetSize()

n ShelterUnit = m_ShelterUnitPackage.GetSize()

Dynamically-Allocated Combinatorial-Parametric Solution Space

Let us consider the solution space associated with this combinatorial-parametric optimisation problem.

Here the actual parametric space associated with a particular solution is dictated by the combinatorial

configuration of that solution, and is given by:

[ ] [ ] [ ] [ ] [ ]Θ Θ Θ Θ Θn k n

k

Kn k n K nk k K

≡ = × × × ×=∏

1

11

L L (26)

For example:

( ) [ ] [ ] [ ] [ ]( ) [ ] [ ] [ ]

n

n

t n

t n

= ⇒ = × ×

= ⇒ = ×

3 9 2

1 0 6

1 3 2 9 3 2

1 1 3 6

Θ Θ Θ Θ

Θ Θ Θ

(27)

As such, the combinatorial aspect of a particular solution can be inferred from the parametric space, once

enumerated. In fact, a particular solution is not a concatenation between the combinatorial solution and

the parametric solution, as was the case with class-specific parameterisation. Here the enumerated

parametric content is the complete solution, thus:


131

[ ]x ≡ =

=

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

n

n

k

n k

K

n K

n

k

n k

K

n K

k

K

k

K

1 1

1

1

1

1 1

1

1

1

11M

M

M

M

M

M

M

M

M

M

=

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

11 1

1 1

11

1

11

1

1

11

1

1

1

1

1

1

M

M

M

M

M

M

M

M

M

M

M

D

n

D

n

k

D

k

n k

D

n k

K

D

K

n K

D

n K

k

k

k

k

K

K

K

K

=

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

θ

11 1

1 1

11

1

11

1

1

11

1

1

1

1

1

1

M

M

M

M

M

M

M

M

M

M

M

D

n

D

n

k

D

k

n k

D

n k

K

D

K

n K

D

n K

k

k

k

k

K

K

K

K

[ ]

∈Θ n

(28)

Object-Specific Combinatorial-Parametric Optimisation Problem

As is clear from our enumeration of the parametric space, there is a well defined order of problem

precedence. Specifically, there is one complete parametric optimisation problem posed per every

combinatorial optimisation solution proposed. Formally, this Thai Menu Problem [TMP] is given by:

[ ] [ ]max maxn k

K

j

nj k

K n n

k

∈ ≡ ∈ = =⊕⊕

N x

oθ Θ

Φ1 1

(29)

In Dynamic Programming (DP) [Bellman 1957] parlance, the parametric optimisation is the stage-one

problem, while the combinatorial optimisation is the stage-two problem, both defined in relations to the

combinatorial-parametric solution space. Our KnP is now fully ‘objectified’ and parameterised.

Unlike optimisable class-specific combinatorial-parametric solution space, here it is impossible to

interchange the multi-stage problem precedence. Moreover, note that by specifying [ ] [ ]x ≡ ∈θ n nΘ as the

solution variable to the problem, n ceases to be a ‘true’ (combinatorial) variable in the sense that it can

be inferred from the ‘aggregated’ (parametric) solution space:


132

[ ] [ ] [ ] [ ] [ ]n

o

o

o

k

K

n k o

k

Ko k o K ok k K

≡

∋ = = × × × ×=∏

1

1

1 1

M

M

L LΘ Θ Θ Θ Θ

(30)

In terms of C++ programming, we first implement the RedCrossReliefPackage class of multi-class object

wrappers, declared thus:

typedef enum {FOOD, MEDICINE, SHELTER } RELIEF_PACKAGE_TYPE; class ReliefPackage { protected: RELIEF_PACKAGE_TYPE m_WhatKind; FoodStuff m_FoodStuff; CMedicalKit m_MedicalKit; ShelterUnit m_ShelterUnit; public: RELIEF_PACKAGE_TYPE ClassDesignator() const {return m_WhatKind;} double ObjectiveCoefficient() const { if(m_WhatKindOfReliefPackage == FOOD) return m_FoodStuff.ObjectiveCoefficient(); else if(m_WhatKindOfReliefPackage == MEDICINE) return m_MedicalKit.ObjectiveCoefficient(); else return m_ShelterUnit.ObjectiveCoefficient(); } };

then we redefine the RedCrossAidCargo yet again, this time by aggregating a single array structure thus:

class RedCrossAidCargo { protected: ArrayOf<ReliefPackage> m_ReliefPackage; public: double ObjectiveFunctionValue() const { double SumCoefficients = 0.0; for(int i=0;i<m_ReliefPackage.GetSize();i++) SumCoefficients += m_ReliefPackage[i].ObjectiveCoefficient(); return SumCoefficients; } unsigned NumReliefPackagesOfType(RELIEF_PACKAGE_TYPE Flag) const { unsigned SumPackageCount = 0; for(int i=0;i<m_ReliefPackage.GetSize();i++) SumPackageCount += (m_ReliefPackage[i].ClassDesignator() == Flag); return SumCoefficients; } };

where:

n FoodStuff = NumReliefPackagesOfType(FOOD)

n MedicalKit = NumReliefPackagesOfType(MEDICINE)

n ShelterUnit = NumReliefPackagesOfType(SHELTER)

4.2 OO TMP Formulation and Generalisation


133

Here we formalise the general instance of the TMP, whereby its solution space is in fact defined in terms

of OO programming constructs.

Up to now, our introduction and use of the OO framework has been largely motivated by the ease and

elegance with which it helps us model and implement an optimisation problem and its solution (as per

solution representation and performance evaluation), in particular the two-staged TMP and its

combinatorial-parametric solution space. Later on, with our phenotype-object optimisation methodology,

OO framework will also help us model and implement the optimisation algorithm (as per exploration

mechanism and optimisation directives). Pesently, however, we lay out the conditions and arguments as to

when and why OO modelling and implementation framework becomes an integral part of how we define

the optimisation problem class and the solution space itself. First we cite the ‘teamwork’ motivation

behind our ensemble construction.

4.2.1 Teamwork Ensemble and TMP Formulation

Here we get rid of the last vestige of a KnP formulation, namely the ‘additive property’ of the objective

function.

Let us consider a hypothetical ensemble-based regression model which averages over N ≥ 1 component

regression sub-models from K ≥ 1 classes. The hypothetical data-fitting error minimisation problem is

given by:

( ) [ ]

( )

min | ( )

, , , , , , ,

f n y x

n n n n

n N

j j n k K

d dj

n

k

K

d

D

k K t K

t

k j k k

j kk

θ

θ

θ= −

= ∈

=

∀ = > ∈ =

===∑∑∑ f

N

1

11

2

1

1

1 0 1

subject to L L

K KΘ

(31)

Using a Heaviside (threshold) function:

[ ]H τ τ( )

,x

x=

≥

1

0

if

otherwise

(32)

we cite the following example:


134

( ) ( ) ( )

( )[ ] ( ) ( )[ ]( ) ( )( )

( ) ( )

min ; | ,

,

sin

. .

f n n y x x

x

x x

x x

x x

s t nn

n

du A

du

nv B

dv

n

d

D

u Ad

u A

du A

du A

du A

du A u A

v Bd

v B v Bd

v B

A

B

A B

u A

u A u A

θ

θ

θ θ

θ θ θ

θ θ θ

θ

θ θ

= − +

=

+ ⋅ −

− ⋅ − +

= −

=

∈

= ==

+

∑ ∑∑ f f

f

H

H

f

N

1 1

2

1

1

2 3

2 3 4

1 2 3

3

3 4

2

n n N

u n

v n

A B

u A

u A

u A

u A

u A

A A

v B

v B

v B

v B

B B

+ =

=

∈ = ℜ × ∞ =

=

∈ = ℜ × ∞ × ℜ =

θ

θθθ

θ

θθθ

θ

1

2

3

4

3

1

2

3

0 1

0 1

Θ

Θ

( , ) , , ,

( , ) , , ,

K

K

(33)

where we construct a regression model as an ensemble comprising K = 2 classes of component regression

functions: (i) a piecewise-linear ‘phase-transitional’ sub-model ( )f u Adx = θ1

u A +

[ ]( ) ( )H θ θ θ32 3

u A

x xdu A

du A⋅ − - [ ]( ) ( )( )H θ θ θ θ θ3 4

2 3 4

u A u A

x xdu A

du A u A+ ⋅ − + , and (ii) a sinusoidal

‘cyclical’ sub-model ( )f v Bdx = ( )θ θ θ1 2 3

v B v Bd

v Bxsin − . In this two-stage combinatorial-

parametric optimisation formulation, we are not required to commit ourselves a priori to any particular

combinatorial configuration, i.e. to a fixed number 0 ≤ ≤n NA of ‘phase-transitional’ components nor

to a fixed number n N nB A= − of ‘cyclical’ components. The combinatorial specification, together

with the actual object-specific parameterisation, will be subject to variation, and hence optimisation.

It is clear that the objective function is not an additive sum of ‘utility contributions’ from amongst the

component objects. Here, the objective function is the data-fitting error measured vis-à-vis the overall

ensemble-based regression model. It is equally clear that the data-fitting performance depends on how the

component sub-models work together ‘as a team’, i.e. how well the various component sub-models

‘complement’ one another to create an optimal data-modelling engine overall.

But why consider an ensemble-based solution to such a data-fitting/modelling exercise? By way of an

analogy, the motivation is given as follows. It is an elementary lesson in harmonics analysis that

superimposing two sinusoidal waves with different wavelengths, λ( )1 and λ( )2 , yields a periodic

behaviour of a longer wavelength λ( )3 , with higher frequency sub-structure. Suppose that a particular

time-series exhibits this spectral behaviour, but coupled with some white noise. Had we been forced to

replicate this behaviour with a single cyclical model, we may well end up with a cyclical model with a

cyclical behaviour of wavelength λ( )3 . On the other hand, had we allowed ourselves two object-


135

specifically parameterised cyclical sub-models to work together, then we could obtain a better result, i.e.

by letting one cyclical sub-model fit the λ( )1 component, while the other fits the λ( )2 component of the

time-series. As a ‘stand-alone’ regression function, either of these two cyclical sub-models would

obviously under-perform relative to the singular cyclical model (the one fitting the superimposed λ( )3

behaviour), but together, as a team, they would complement one another, and clearly achieve the optimal

performance overall.

Perhaps more important than the notion that two cyclical sub-models complement one another is the

notion that cyclical and phase-transitional sub-models complement one another. While cyclical time-series

sub-models may capture business and economic cycles and seasonal factors well, they do not properly

capture regime shifts, such as a change emanating from an interest rate ‘bump’. Modelling a time-series

which exhibits both kind of behaviours would obviously benefit from the use of both kinds of information.

The problem is that, in general, we could not a priori anticipate how many cyclical and how many phase-

transitional component sub-models would prove to be the optimal combination. This is the combinatorial

aspect of the solution. Of course, each cyclical and/or phase-transitional component sub-models would

still have to be parametrically tuned. This is the parametric aspect of the solution. Clearly, our generalised

combinatorial-parametric formulation caters to the need of both. This simple ‘thought experiment’

suggestively hints at how ‘parsimoniously ambitious’ some of the traditional time-series models are,

basing themselves, as it were, on one parametrically optimised regression model function from one single

class of functions.

Formally, a functional optimisation problem where the solution is a functional ensemble is given by:

{ }max ( )Φ Λk

K

j

nj k j k

uj

n

k

K

u u

uk k

Begin

End

g x= = =

==⊕⊕ ∑

=

1 1 1

1

o f

(34)

where { }Λ f j ku

j

n

k

K

Begin Endx u u uk

( ) , , ,=

=

=

11

K represents the mapping behaviour of a functional

ensemble, and g represents some incremental measure of performance. This is similar to the situation we

already encountered in optimising our class-combinatorial signal-ensemble trading model

[ ]fEnsemble SignalM

Ensemble SignalMθ against the annualised cumulative gain percentage figure.

A Thai Menu as a Combinatorial-Parametric Space

Here we follow the OR tradition of nicknaming a general class of optimisation problems after a

hypothetical, anecdotal situation, one which provides a useful analogy, but is not likely to have been the

original application. It is very doubtful that the famous Travelling Salesman Problem was formulated to

cater to the need of a roving salesman. Nonetheless, the imaginary salesman’s similarly imaginary

decision problem very much captures the essence of this very general class of combinatorial optimisation


136

problems. Though undeniably difficult, NP-hard in fact, at least a TSP solver does not have to optimise

any parametric aspect concerning the various inter-city routes.

Here too, we develop a useful analogy. Our framework of optimisation analysis is developed to tackle a

model construction and optimisation problem in financial model-trading, not to help us decide what dishes

to order and how to have them customised to our taste.

Combinatorial-parametric search space is very much a Thai menu phenomenon. It is a given that there are

countless varieties of Thai dishes. In a ‘proper’ Thai restaurant, we do not get a two-fold paper laminate as

a menu, we get a book, which might well contain a hundred pages. It is indexed with page numberings and

‘subject’ headings. Needless to say, it is a very immense search space. To make matters worse, we do not

order a single dish, but a combinatorial selection of dishes, which are organised into various types. A

prepared dish is thus an object instantiation of its type, i.e. the object class.

Ordering a Thai dish can be a very personal matter. A given dish may be ordered by name, in which case,

it is assumed that anybody ordering that dish would expect the same constant, class-specific set of

attributes. What usually happens is that one makes further specification on how such and such dish should

be prepared. In a family gathering, it is not uncommon to order two plates of the exact same dish, with one

‘hotter’ for adults, and one ‘milder’ for children. That takes us to the subject of variable, object-specific

set of attributes. A set of attributes which prescribe one type of dishes is generally different from that

which ‘parameterises’ another (i.e. ‘smokiness’ attribute does not apply to most classes of ‘soup’ dishes).

The overall objective, the satisfaction to be derived from a meal is a function of the whole ensemble of

dishes presented altogether. How one prescribes the attributes of a particular dish thus often depends on

the presence of other types of dishes, as well as on how the attributes concerning those other dishes may

have been prescribed. One particular dish from a particular type of dishes and with a particular set of

object attributes may complement another particular dish from another particular type of dishes, itself with

another particular set of object attributes. On the other hand, one dish may be particularly popular ‘on a

stand-alone basis’ and yet prove disagreeable in the presence of another dish, also popular by itself, and so

on. So the various dishes within such an ensemble have to ‘work together’, i.e. ‘as a team’.

Now, ordering enough dishes for a gathering of ten from a list of near one thousand dishes, taking into

account the specific object attributes of the individual dishes, can be a very onerous process, implying a

difficult search over a very immense space. The generally preferred heuristic seems to be a two-stage

process. We usually start by tentatively agreeing on the combination of dishes before settling down onto

the various attributes specific to each dish. This being heuristically ‘optimised’, one person may then

suggest another type of dish to be thrown in, altering the combinatorial decision anew, whence the

parametric decision is once again open to debate. Various attributes concerning the previously selected

dishes may now have to be readjusted in view of the newly introduced dish. The two-stage process can

then repeats itself, and so on.


137

4.2.2 TMP Generalisation Based on OO Inheritance

This simply means that some solution-component object classes are derived from other(s), so that the

various classes exist within a well-defined OO inheritance hierarchy. In terms of solution representation, a

derived solution-component object class generally inherits some ‘protected’ object attribute(s) from the

base solution-component object class(es) and defines additional object attribute(s) and/or modify the

solution-component mapping functionality. In terms of exploration mechanism, the object attribute space

associated with the derived class embeds the object attribute space associated with its base class(es). As

such, an optimisation search over the derived solution-component object’s object attribute space entails a

search over the inherited part along with the additionally defined part as well.

For combinatorial-parametric optimisation, we depict OO inheritance relationships amongst component

classes within the TMP formulation thus:

[ ] [ ]

{ }

max max

. . , , , , :

n k

K

j

nj k

d b

Kn

K

n K

k

w p d b K d b

∈∈ × × = =

⊕⊕

∃ ∈ ≠ ∋

No

k k

θ Θ Θ

Φ1

11 1

1

L

K

(35)

Here ‘w p. . ’ stands for ‘where possibly’ (as opposed to ‘subject to’). In other word, we are able to

explicitly capture OO inheritance relationships within our problem formulation, and ‘w p. . ’ is taken as a

form of suggestion of this possibility. In particular, such an inheritance relationship comes in handy when

the derived-class component object encapsulates the base-class component object’s object attribute space,

whence:

[ ] [ ]

{ }

max max

. . , , , , : ,

n k

K

j

nj k

d bb a

Kn

Kn K

k

w p d b K d b

∈∈ × × = =

⊕⊕

∃ ∈ ≠ ∋ ⊆

No

k k

θ Θ Θ

Φ

Θ Θ

1

11 1

1

L

K

(36)

By way of an analogy, newly introduced Thai dishes are often derived from existing ones. As such, they

generally contain additional ingredients and preparation steps in addition to those which have been

‘inherited’ from the existing dishes on which they are based. More ingredients and preparation steps

means additional dimensions in the object attribute space. The additional model parameters have to be

optimised alongside the inherited sub-object attributes. In this manner, a newly derived type of Thai

dishes is a derived solution-component object class.

For instance, suppose that starting with a linear ‘regression function’:

[ ]f β β β β0 10 1

, ( )x x= + (37)

modelled and implemented with:


138

class LinearFunc { protected: double m_Beta0; double m_Beta1; public: double operator()(double x) const { return (m_Beta0 + m_Beta1*x); } /* etc. */ };

we now want to model and implement a quadratic ‘regression function’:

[ ]f β β β β β β0 1 20 1 2

2, , ( )x x x= + + (38)

With k LinearFunc defined, we would simply derive k kQuadraticFunc LinearFunc: thus:

class QuadraticFunc: public LinearFunc { protected: double m_Beta2; public: double operator()(double x) const { return (CLinearFunc::operator()(x) + m_Beta2 * pow(x,2)); } };

with an additional object attribute ‘m_Beta3’ which will ultimately have to be optimised along side the

inherited ‘LinearFunc::m_Beta1’ and ‘LinearFunc::m_Beta2’. Moreover, note how

‘QuadraticFunc::operator()(double)’ essentially builds its mapping functionality from

‘LinearFunc::operator()(double)’.

4.2.3 TMP Generalisation Based on OO Encapsulation

Multi-Stage Combinatorial-Parametric Solution Space Recursion

In a two-stage combinatorial-parametric optimisation problem, our solution is a combinatorial assembly of

parametric components. Now suppose that one such component is, in fact, a combinatorial assembly of

components, some of which, in turn, may themselves be constructed from combinatorial assemblies of

components, and so on.

This kind of combinatorial-parametric recursion is very difficult to handle, as it amounts to a multi-stage

DP formulation. In the most general case, we may not even have an a priori handle on how many

embedded stages could be involved. It would be helpful if we were somehow able to ‘ignore’ this

recursion in the solution space when we need to, while still being able to faithfully capture the complex,

recursively embedded structure of the overall solution. How could we deal with such a situation?

With OO encapsulation, we can employ an array of parametric objects without having a direct handle on

the individually encapsulated object attributes. So in fact, the ‘parametric’ objects need not be ‘truly


139

parametric’ in nature at all, and may in fact be ‘combinatorial’ or ‘combinatorial-parametric’ in nature, i.e.

encapsulating a combinatorial or combinatorial-parametric attribute spaces of their own. As such, this

multi-stage extension can be handled as though we were still dealing with a two-stage solution space. It is

an OO equivalent of the mathematical notion of counting: “once we have two, we have infinity.”

The crucial point is that the very same object-array construct, used to model and implement a

compositional solution, is also available to be used to model and implement a compositional solution-

components. Moreover, if we were somehow able to engineer a mechanism by which a solution object-

array is ‘responsible for its own’ exploration mechanism, i.e. independent of how the optimisation

algorithm at large actualises the optimisation directives, then this same mechanism would be available for

the object-arrays used to model and implement compositional solution-components as well, thereby

carrying out combinatorial and parametric search at all the embedded levels.

For example, consider an ‘ensemble’ regression model comprising component functions (regression sub-

submodels) from various classes. Suppose we had defined k FuncA and k FuncB as our component

function classes and k EnsembleFuncAB as our multiple-function ensemble regression function class where:

( )f

f fEnsembleFuncAB

FuncA FuncA

FuncA FuncB

FuncA FuncA

( )

( ) ( )

,x

x x

n nx

a

a

nb

b

n

=

+

+∈ℜ

= =∑ ∑

1 1

(39)

Having done all that, we now add another class of regression sub-model k FuncZ to the mix, and define

k EnsembleFuncABZ as our multiple-function ensemble regression function class where:

( )f

f fEnsemleFuncABZ

EnsembleFuncAB FuncZ



( )

( ) ( )

,x

x x

n nx

a

a

nb

b

n

=

+

+∈ℜ

= =∑ ∑

1 1

(40)

Now, note how f FuncEnsembleABZ is able to call each f a a nEnsembleFuncAB EnsembleFuncAB, , ,= 1K , with

no ‘appreciation’ or ‘concern’ whatsoever that each is itself a ensemble function, encapsulating a

combinatorial assembly of k FuncA and k FuncB class objects. As far as f FuncEnsembleABZ is concerned,

the fact that each f a a nEnsembleFuncAB EnsembleFuncAB, , ,= 1K , would have to go and take the ‘ensemble

average’ amongst the output of the k FuncA and k FuncB component functions is of no concerned at this

level. So it is a matter of viewpoint: with respect to k FuncA and k FuncB objects, f FuncEnsembleAB is an

ensemble function; with respect to f FuncEnsembleABZ, f FuncEnsembleAB is but a component function.

So what is to stop someone from using f FuncEnsembleABZ as a ‘mere’ component function? At the

opposite end, how do we really know that f FuncA , or f FuncB , is a ‘simple’ parameterised function? It

may well be that f FuncA in fact comprises an ensemble of its own functional components. The only


140

thing we know is that f FuncA cannot use f FuncEnsembleAB or f FuncEnsembleABZ as its component

function (though it may well use f FuncB ).

So far, with OO encapsulation, we are able to ignore the ‘recursive nature’ of the recursively multi-stage

combinatorial-parametric solution space. That is, until we realise that in terms of performing search and

optimisation, the optimisation algorithm would need to be able to get a handle on all the necessary model

parameters, no matter how deeply ‘nested’ they are within this combinatorial-parametric recursion!

Otherwise, how would the these parameters undergo changes or variation. With no variation, no

optimisation!

What we need now is for the variously nested model parameters to be able to somehow undergo ‘self-

adaptation’ (i.e. speaking in terms of exploration mechanism), but how would they know where to go (i.e.

speaking in terms of optimisation directives)? Who would guide them? For if the objective function (i.e.

speaking in terms of performance evaluation) is no longer a simple, additive sum of individual

components’ utility contributions, no individual component object would be able self-adapt its

encapsulated model parameters with the full understanding of how such and such movement would effect

the entire solution as a whole.

Here is the crux of our entire situation. Having generalised a combinatorial solution into a combinatorial

ensemble of parametric objects, and into an ensemble structure of component class objects, some of which

may be further encapsulating ensemble structures, and so on, recursively, we have now ended up with a

very complex, nested search space. On the one hand, we want to enumerate all the parametric content of

the entire solution set. This ‘parametric transparency’ is for the benefit of the optimisation algorithm. On

the other, we want to leave them fully embedded within the various component objects. This ‘attribute

hiding’ is for the benefit of generalising the optimisation problem/solution space. Somehow, the OO

distributive principle, where the collective set of solution model parameters are distributed into object-

encapsulated chunks (i.e. speaking in terms of solution representation), would have to apply to the

optimisation/search process as well.

To summarise: in terms of (I) solution representation, the OO encapsulation principle has been a

Godsend, enabling us to capture a very complex, recursively combinatorial-parametric space as a simple

combinatorial space of component objects; in terms of (II) performance evaluation, the OO language

mechanism for overloading function names has been invaluable, enabling us to call on the mapping

functionality for each of the recursively embedded component objects. But in terms of (III) exploration

mechanism and (IV) optimisation directives, we will need to finally let go of our outmoded concept of an

optimisation algorithm as a global program accessing each solution object’s entire parametric contents

and manipulating their values directly. This eagerly awaits our methodology chapter.

A Thai Menu as a Multi-Stage Object Space

Multi-stage, combinatorial-parametric solution space is very much a Thai menu phenomenon. That is,


141

depending on the level of detail one looks at a dish, each dish may be considered a component, such as

when an appetiser dish is considered part of an appetiser set, or it may be considered to be a collection of

components, such as when an appetiser is itself an ensemble of ingredients. A seemingly singular object

such as a particular kind of curry dish, is often itself describable as a combinatorial ensemble. It is not

unusual for a customer to order a particular curry dish, assuming the ‘default’ combinatorial specification

of ingredients, and then impose a personal choice that such and such ingredients be left out, or substituted

with such and such ingredients (combinatorial perturbation), or even that the same set of ingredients be

used in different proportions and/or prepared differently (parametric perturbation), and so on, and yet the

entire dish is still but a component vis-à-vis the dinner ensemble.

Proprietary Component Object Class and Commercial Object Vendor

We finally take away all parametric transparency. Again, consider an ‘ensemble’ regression model

comprising component-functions/sub-models from various classes. Now suppose that some of these

component sub-models were proprietary, i.e. proprietary models which whose dynamically linked library

(DLL), but not source code, has been bought from a software/object vendor. We know how to use it, for

example, as each sub-model class, say k ProprietaryRegressionFunc, implements an object method

f ProprietaryRegressionFunc:ℜ→ ℜ . But aside from their usage, the type of variable it maps from and maps

to, we have no idea what has actually been implemented inside the ‘black box’ for which

f ProprietaryRegressionFunc provides but the user interface. We do not know anything about the actual

function f ProprietaryRegressionFunc, never mind o ProprietaryRegressionFunc’s object attribute space. That is, if

such a component object contains function-parameters, we do not know the variable spaces to which they

belong. These details are proprietary after all. Just as we have data encapsulation from an object-oriented

perspective, this is ‘model’ encapsulation from a business-oriented perspective.

Since there is no direct, global access on proviso, an optimisation algorithm has to ‘operate blind’. It has

to somehow ‘ask’ the component objects to ‘self-adapt’ whatever optimisable attributes are contained

therein. Perhaps there is a ‘feedback’ interface, i.e.

‘ o ProprietaryRegressionFuncContinuePerturbation bool InThePreviousDirection. ( ) ’, by which the optimisation

algorithm can ‘tell’ the o ProprietaryRegressionFunc component object if the most recent adaptation of the

mapping behaviour was ‘helpful’ or ‘hurtful’ vis-à-vis the overall performance of the ensemble regression

model solution (of which it plays a component part). Perhaps even this is not possible, and the

optimisation algorithm can only destroy the poorly performing ensemble solutions which presumably

contain the detrimental component objects and construct new ones.

A Thai Menu as a Proprietary Object Space

Existence and use of proprietary component objects is also very much a Thai menu phenomenon. Often, a

Thai restaurant would offer a number of ‘unique’ dishes, the proprietary component objects, of which we


142

know very little about. We do not know the basic ingredients nor the way they are prepared, but can

evaluate their suitability and compatibility amongst other dishes nonetheless. This can happen with newly

invented dishes as well as modifications from existing ones, as per OO inheritance. Perhaps there are

quantitative way in which we can describe the dishes, and convey our preferences accordingly. A well-

acquainted patron of the restaurant can order the same dish, with an instruction that goes something like

“just like the last time, only more tangy, in a North-Eastern sort of way”. This is an indirect call to

parametric perturbation, as per OO message-passing. Of course, we describe what we want qualitatively,

intending for parametric adjustment, i.e. relative to the dish we had the last time we came in. Little do we

know that perhaps, to make this particular dish taste “North-Eastern”, some ingredients have to be added

and/or removed. In other word, unbeknownst to us, the dish is in fact combinatorial, and adding more, or

less, of the various ingredients entail a combinatorial adjustment process. Maybe it involves a parametric

adjustment process, as making the dish more “tangy” requires roasting some of chilli longer than usual,

the degree to which the chilli is being roasted being a parametric specification. A good Thai chef would

also know that roasting the chilli longer would effect the aromatic balance of that dish, which also depends

on the various other ingredients. So the parametric depends on the combinatorial, but not necessarily the

other way around (well defined problem staging), and so on. The good news is that we do not have to keep

track of all this embedded information. (In fact, we are not entitled to know the ‘secrets’ of dish

preparation anyway.) In terms of optimisation, this means that we cannot directly mandate the

combinatorial and/or parametric contents of the proprietary dishes. We only infer that there is an object

attribute space associated with the particular dish, but we can only ask the chef to alter them on our behalf.

We do this in view of other dishes within the ensemble solution structure we summon as a Thai dinner

ensemble.

Altogether, we now retrace the overall development of our generalised TMP formulation. Starting with a

purely combinatorial KnP formulation, we first incorporated class-specific parameterisation:

max maxn k

K

j

nj k

K K

k

∈ ∈ × × = =⊕⊕

N

oθ Θ Θ

Φ1

1 1L

(41)

Then we incorporated object-specific parameterisation:

[ ] [ ]max maxn k

K

j

nj k

Kn K n K

k

∈∈ × × = =

⊕⊕

N

oθ Θ Θ

Φ1

11 1

L

(42)

which begets us the (two-stage) combinatorial-parametric TMP formulation.

Then we incorporated possible OO inheritance relationship(s) amongst component object classes:

[ ] [ ]{ }

max max

. . , , , , : ,

n k

K

j

nj k

d b b d

Kn K n K

k

w p d b K d b

∈∈ × × = =

⊕⊕

∃ ∈ ≠ ∋ ⊆

No

k k

θ Θ Θ

Φ

Θ Θ

11

1 1

1

L

K

(43)

Finally, with full-fledged OO encapsulation, we (the application user as well as the optimisation


143

algorithm) are no longer at a ‘privilege’ to access and/or manipulate the various model parameters/object-

attributes encapsulated within the various component objects. As such, we are only able to formulate the

global statement:

{ }

max

. .

, , ,

. . , , , , :

Φk

K

j

nj k

t

k

d b

k

s t n N

n k K

w p d b K d b

= =⊕⊕

= ≥

≥ =

∃ ∈ ≠ ∋

1 1

1

0 1

1

o

1

k k

K

K

(44)

which begets us the (possibly multi-stage) TMP formulation, which we can also write in terms of the

mutli-class object emulators/wrappers:

{ }

max

. .

. . , , , , :

Φ

∆

i

Ni

k

Kk

d b

s t N

w p d b K d b

=

◊

◊

=

⊕

≥

=

∃ ∈ ≠ ∋

1

1

1

1

o

k k

k k

where

K

(45)

Note the remarkable aspect of this full-fledged TMP formulation, namely that it is a single-stage problem

over a possibly multi-stage solution space. We will see that Nature, with her Natural Selection principle

and exploration mechanism and multi-species coevolution (combinatorial assembly of individual

organism, each encapsulating an individual genetic parameter space), also optimises over a multi-stage

solution space within a single-stage setting. As an extension of existing EO paradigms, our own

phenotype-object optimisation methodology is a further attempt to harness and simulate this power and

generality of the natural phenomena within a computational optimisation algorithm.

Next we consider situations whereby different types of ‘dishes’ are segregated by ‘functionality’. Some

‘group’ classifies dishes which fulfill the ‘modularised task’ of whetting the appetite; other ‘group’ of

dishes complements cold drinks in a hot climate; yet another ‘group’ consist of dishes which are colourful

and ornamental, serving the visual appeal and operate at the aesthetic rather than gastronomic level. Just

as a certain dish is an object class and a particular serving is a class object, here the ‘group’ of dish

corresponds to our notion of functionality or class template group.

4.2.4 TMP Generalisation with Modular Solution Structures

So far, we have been considering TMP as an optimisation problem whose solution-function comprises an

ensemble of components, an ensemble structure being essentially an indexed, but inherently unordered,

assembly of objects. Now we extend the TMP formulation to the case where the solution is a modular

structure comprising an ordered collection of components.

Why Modular Solution Structure?

But why and when would a solution-function have such a modular structure? From an engineering


144

viewpoint, we consider the case of representing a solution-function as a modular (mapping) structure,

that is, when the solution is modularised into a number M M∈ >N , 1, of modular components, or

simply modules, each component module performing a specialised task. The basic premise is that such a

modular structure realises the ‘division-of-labour’ principle. In essence, each individual component is

necessarily incomplete in terms of representing a complete solution, and has to be used as part of a

modular structure. Performance evaluation, and optimisation analysis, is necessarily performed only on

basis of the solution modular structure as a whole.

On this, the size of the modular structure, the number of modules contained therein, depends to an extent

on how complicate the overall problem solution is, the extent to which we should partition the solution

modelling and implementation task into separable and manageable ‘chunks’. This number M is thus a

problem-defined constant. Let { }m M∈ 1, ,K be the index denoting the mth component module

responsible for the mth modularised functionality.

A traditional, financial time-series forecasting-based trading model formulation is one such example of a

modular solution structure. Here the complete solution-function is a financial time-series trading model.

Its first module is the forecasting engine, the second module being the trading rule which takes in the

forecast from the first module and outputs the market transactions/trading decisions.

The point of having a multiplicity of modules is that we can dispense our effort addressing the various

model components in successive stages. For example, we can concentrate on the time-series forecasting

engine first, then on the trading rule later. Keep in mind, however, that in terms of optimisation, the

performance of the complete solution depends on the beneficial interactions amongst the modular

components. For example, a highly accurate, single-period forecasting engine may work well with one

type of trading rule, while a less accurate, but robust, multi-period forecasting engine may work well with

an altogether different type of trading rule. Still, we could not be sure of an optimal result, or even

compatibility, if we tried to optimise the individual functional modules separately, hence the need for a

combinatorial framework for putting together different classes of modules together to form a cohesive

whole.

Compared to an ensemble solution structure, a modular structure is intrinsically an ordered collection. To

wit, we could not feed the time-series data as input to a trading rule, anymore than we can feed the output

of the trading rule as input to the time-series forecasting function. Even if, in some applications, there

arises a need for some ‘feedback’ loops, the arrangement is never truly symmetric, and one cannot

sensibly interchange two modules if they serve different modularised functionality.

OO Model/Implementation of a Modular Structure

Each component module is modelled and implemented an object, to be instantiated from any class from

within the functionality or (class) template group K mm M, , ,= 1K . In other word, the ‘class identity’


145

of each modular component object m m M, , ,= 1K , is generically referred to by the m-indexed class

template k Km m∈ . Note that these functionality groups are generally, but not necessarily, unique for

each modularised functionality m m M, , ,= 1K , so that is possible that

{ }∃ ∈ ≠ ∋ =r s M r sr s

, , , ,1K K K . The modular structure of these M objects altogether is denoted

by:

m

Mm m m

=⊕ ∋ ∈

1

o k K (46)

In general, our interest will be to capture the mapping behaviour of a modular mapping structure, i.e. a

modular structure of component functions.

For instance, letting the overall mapping functionality of some modular mapping structure f D R: → be

modularised into component functions f D Rm

m m m M: , , ,( ) ( )→ = 1K , suppose that

f D D D Dmm m m M: , , , ,( ) ( ) ( )→ = − ≡+1 11 1K , f D R

MM: ( ) → , and that f D R: → is

simply calculated as a functional composition, then:

y x x

x x y

M m

M m

= =

=

∈ ∈

f f f f

f f f D R

( ) ( )

( ) , ,

oLo oLo

K K

1

1

( 47 )

Essentially this kind of composition mapping can be thought of as a serial processing architecture, with

one object inputting the output of another object, processing it, and outputting the result which may be

used as input to yet another object, and so on. This contrasts with an ensemble mapping structure, i.e. an

ensemble function such as our signal-ensemble trading model, which embodies a parallel processing

architecture.

OO TMP Formulation with Modular Solution Structure

If such a modular (mapping) structure represents a solution to an optimisation problem, then the objective

function will have to be defined on this modular solution structure thus:

max

, , ,

Φm

Mm

m m m M

=⊕

∈ =1

1

o

k Ksubject to K

(48)

which gives us a modular-type TMP, as distinguished from the ensemble-type TMP we have been

working with all along.

In comparison, an ensemble-type TMP solution structure kK

jn j k

k

= =⊕ ⊕1 1o specifies n Nkk

K

=∑ = ≥1

1

object(s) from K ≥ 1 class(es). In essence, there is only one functionality/template group for all object(s)

within the ensemble structure. On the other hand, for a modular-type TMP solution structure


146

mM m

=⊕ 1o , there is a functionality/template group associated with each modularised functionality

m m M, , ,= 1K , where the mth component module o m is an object instantiated from any one of the

Km

≥ 1 object class(es) within the functionality/template group K m.

As with the ensemble-type TMP, here we consider the combinatorial complexity of the ‘stage-two’ space

(the number of of ‘stage-one’ parametric optimisation problems).

We first consider each functionality group K mm M, , ,= 1K as an indexed set. Then we let the

modular component class index { }κ m mK∈ 1, ,K denote the particular class to which the mth object

instantiates from. Moreover, let us define the class configuration vector as an M -dimensional N -vector:

κ

κ

κ

κ

≡

∈

1

M

M

m

M

MN

(49)

Recall that K m Mm

≥ =1 1, , ,K , gives us the number of class(es) within the template group K m,

i.e. Km m

≡ K . To put it in another way, Km

is the number of class(es) whose objects may be

used to fulfil the mth modularised functionality m M= 1, ,K . With that, the number of possible class

configurations, i.e. the number of unique values of κ , is given by:

{ }ℵ

≡

= =∏K K

m

m

Mm

m

M

1 1

(50)

giving us the number of ‘stage-two’ combinatorial solution, each of which poses a complete ‘stage-one’

parametric optimisation problem. This number { }ℵ

=K

m

m

M

1, while polynomial in

K m Mm

≥ =1 1, , ,K , is exponential in M .

Traveling Salesman Problem (TSP) Solution as a Special Case

As an observational note, we remark that the Traveling Salesman Problem (TSP) [Lawler et al. 1985]

can be thought of as a purely combinatorial specialisation of our modular-type TMP, i.e. a ‘stage-two-

only’ TMP with constant, class-specific parameterisation.

To wit, we cite an example 4-city TSP (i.e. one which is equivalent to an undirected fully-connected graph

with 4 nodes). First we note that there are altogether K = − =4 4 1 2 6( ) / component object classes:


147

k k k

k k

k

StLouisChicago StLouisDetroit StLouisCleveland

ChicagoDetroit ChicagoCleveland

DetroitCleveland

, , ,

, ,

( 51 )

from which we note M = 4 class template groups corresponding to 4 modularise functionality (enabling

the salesman to get to each of the 4 cities):

{ }{ }{ }{ }

K k k k

K k k k

K k k k

K k k k

ToStLouis StLouisChicago StLouisDetroit StLouisCleveland

ToChicago StLouisChicago ChicagoDetroit ChicagoCleveland

ToDetroit StLouisDetroit ChicagoDetroit DetroitCleveland

ToCleveland StLouisCleveland ChicagoCleveland DetroitCleveland

=

=

=

=

, ,

, ,

, ,

, ,

( 52 )

Let such a TSP solution be represent by a ‘4-by-4’ matrix { } ( ) ( )A

origins destinations∈ ⊆

× ×014 4 4 4, N of zeros and

ones, i.e. { }a i ji j, { , } , , , ,∈ ∈01 1 4K = { StLouis,…, Cleveland}, where ai j, = 1 indicates that the route from

the ith city, i ∈{ StLouis,…,Cleveland}, to the jth city, j ∈{ StLouis,…,Cleveland}, is included in the trip plan. A

modular structure representation for such a TSP solution would be given by:

{ }m s

m m

m s= =⊕ =

=

ToStLoui

ToCleveland

ToStLoui

ToCleveland

ToStLouis ToChicago ToDetroit ToCleveland

o o

o o o o; ; ;

( 53 )

Consider an example TSP solution is shown below:

{ } ( ) ( )Aorigins destinations

=

•

•

•

•

∈ ⊆× ×

1 0 0

0 0 1

1 0 0

0 0 1

014 4 4 4, N

( 54 )

This particular TSP solution can be represented as a modular solution structure:

{ }m s

m

=⊕ =

∈ ∈

∈ ∈

ToStLoui

ToClevelandStLouisDetroit StLouisChicago DetroitCleveland ChicagoCleveland

StLouisDetroit ToStLouis StLouisChicago ToChicago

DetroitCleveland ToDetroit ChicagoCleveland ToCleveland

o o o o o

k K k K

k K k K

; ; ; ,

, ,

,

( 55 )

which is interpreted as follows: of the KToStLouis

= 3 classes of routes which serves the modularised

functionality “get to St. Louis”, (the) one object from the second modular component class k StLouisDetroit

is chosen. We signify this choice with the modular component class index κ ToStLouis= 2 (i.e. 2nd); of

the KToChicago

= 3 classes of routes which serves the modularised functionality “get to Chicago”, (the)

one object from the first modular component class k StLouisChicago is chosen. We signify this choice with

the modular component class index κ ToChicago= 1 (i.e. 1st). Similarly, κ ToDetroit = 3 and

κ ToCleveland= 2 . Altogether, the class configuration vector description of this TSP solution set in


148

particular is given by:

κ

κκκκ

≡

=

∈

ToStLouis

ToChicago

ToDetroit

ToCleveland

2

1

3

2

3

1

2

3

2

1

3

2

3

2

1

2

1

3

1

3

2

3

2

1

1

2

3

1

, , , , , { }

⊆ 1 2 34

, ,

( 56 )

where we do no get the full Cartesian expansion of { } { } { } { }1 2 3 1 2 3 1 2 3 1 2 3, , , , , , , ,× × × because in a valid

TSP solution, one enters and exits each city exactly once.3 At any rate, we see that the optimisation

problem is of the form4:

max

. . , , ,

Φm

m

m ms t m

=⊕

∈ =ToStLouis

ToCleveland

ToStLouis ToCleveland

o

k K K

( 57 )

Note that our example above considers the case of a TSP for which there is one route object between two

cities. As such, k StLouisChicago , for instance, belongs to both K ToStLouis as well as K ToChicago

class

template groups. For a directed graph TSP, i.e. whereby an inter-city travel cost depends on the direction

of travel, we would have instead defined, say, k FromStLouisToChicago which clearly belongs to K ToChicago

but not to K ToStLouis, demonstrating an even more specialised notion of modularised functionality.

3 This example, of course, is rather trivial: irrespective of the starting city, there are three distinct solutions (not counting the two possible directions of travels).

4 The objective function would have to tally up the route costs and penalise the invalid TSP tours.


149

4. Thai Menu Problem [TMP] Formulation ______________________118 4.1 Parameterising a Combinatorial Problem _____________________________ 118

4.1.1 Combinatorial Optimisation Revisited __________________________________ 118 4.1.2 Objectified Knapsack Problem ________________________________________ 121 4.1.3 Parametric Knapsack Problem ________________________________________ 124 4.1.4 Class-Specific Parameterisation _______________________________________ 124 4.1.5 Object-Specific Parameterisation ______________________________________ 128

4.2 OO TMP Formulation and Generalisation_____________________________ 132 4.2.1 Teamwork Ensemble and TMP Formulation _____________________________ 133 4.2.2 TMP Generalisation Based on OO Inheritance____________________________ 137 4.2.3 TMP Generalisation Based on OO Encapsulation _________________________ 138 4.2.4 TMP Generalisation with Modular Solution Structures _____________________ 143

Bellman, R. (1957). Dynamic Programming. Princeton, NJ, Princeton University Press. Carstensen (1983). “Complexity of Some Parametric Integer and Network Programming Problems.” Mathematical Programming 26: 64-75. Dantzig, G. B. (1966). Lineare Programmierung und Erweiterungen (Linear Programming and Extension). Berlin, Springer. Karmarkar, N. K. (1984). “A new polynomial time algorithm for linear programming.” Combinatorica 4: 373-395. Martello, S. and P. Toth (1990). Knapsack Problems - Algorithms and Computer Implementations. Chichester, John Wiley & Sons. Solow, D. (1984). Linear Programming: An Introduction to Finite Improvement Algorithms, North-Holland.

Documents

4. Thai Menu Problem [TMP] Formulation · The combinatorial question is how many relief packages of each type to . 4 Thai Menu Problem [TMP] Formulation 119 load into the cargo bay