Intelligent Agent Based Model

for an Industrial Order Picking Problem* Byung-In Kim, Robert J. Graves and Sunderesh S. Heragu

Rensselaer Polytechnic Institute

Art St. Onge

St. Onge Company

May 20, 2001

* Supported by NSF Grant #DMI 9900039IIE Annual Conference 2001

Presentation Outline

1. Manufacturing Control Frameworks

2. Industrial Order Picking Problem

3. Intelligent Pick-zone Assignment

4. Intelligent Conveyor Speed Adjustment

5. Conclusions & Discussion

2 / 14

1. Manufacturing Control Frameworks

Control components

Manufacturing entities

Hierarchical

Heterarchical

• Hierarchy: master/slave relationship• Structural rigidity• Difficulty of control system design• Lack of flexibility (Assume Deterministic)

• Interaction of autonomous components• Lack of global information• Difficulty in predicting system performance• Sensitivity to market rules

3 / 14

1. Manufacturing Control Frameworks

Hybrid

• Features of Hierarchical and Heterarchical• Hierarchy and Semi- Autonomous components• Globally optimized solution• Robustness against disturbances

4 / 14

2. Industrial Order Picking ProblemCosmetics Warehouse

Gantry Picking Complex (GPC): 16 pick zones

GPC characteristics: 65142 orders (116589 line items)/day

OAPS (Order Analysis and Planning System)

• Order Processing

• Create Next day’s order sequencing, pick plan,

and replenishment plan

FSS (Finite Scheduling System)

• Make detailed scheduling plan for gantry robots

• Execution of picking and replenishment

5 / 14

2. Industrial Order Picking Problem: GPC

Gantry Robot Pick tote Compartment

Conveyor Drop buffer(b) Picking Zone Layout

(a) GPC Layout

15 11 7 3

14 10 6 2

Sub-zone C Sub-zone B

1612 8 4

13 9 5 1

Sub-zone A

Sub-zone D

43

Pick Zone 12

87

56

1211

910

1615

1314

6 / 14

3. Intelligent Pick-zone Assignment

• Decision Maker: OAPS

• Method: Hierarchical & Static

• Decision Timing : One day

before picking

• Benefit: Ease of Calculation

• Problem: Separates Planning

from Execution

-> Lacks ability to handle

dynamic situation

Old Model New Model

• Decision Maker: FSS

• Method: Intelligent Agent Based

Hybrid & Dynamic

• Decision Timing: The moment

when the line-item enters

the system

• Benefit: Synchronized Planning

& Execution

-> Fault Tolerant -> Last minute changes to picking can be accommodated• Problem: More sophisticated calculation

7 / 14

3. Intelligent Pick-zone Assignment

Negotiation Protocol

Order AgentOrder Agent Pick-zone AgentPick-zone Agent

Task Announcement

Monitoring Bid-board

Select Best Bid

Delete Task, Bid

Confirm Task

Monitoring Task-board

Make a Bid

Bid Submission

Confirm Task

Task-Board

Bid-Board

)_,( timeLrobotgantrypicksduncompleteandcommittedfBidL 8 / 14

3. Intelligent Pick-zone Assignment

Simulation Results with various conveyor speeds

• Errors of the new model are always less than

the original hierarchical model

• Average utilization levels are almost the same

• Standard deviation of utilization levels are almost the same

H ier a r ch ica lM od e l

H ybr idM od e l

H ie r a r ch ica lM od e l

H ybr idM od e l

H ie r a r ch ica lM od e l

H ybr idM od e l

H ie r a r ch ica lM od e l

H ybr idM od e l

C on ve yor fe ed in ter va l 0 .8 0 .8 0 .7 5 0 .7 5 0 .7 0 .7 0 .6 5 0 .6 5N u m ber o f p ick e rr or s 0 0 4 0 5 1 2 0 5 2 3 3 6 7U ti l iza t ion m ean 7 3 .9 7 7 4 .7 8 7 8 .9 4 7 8 .9 2 8 4 .4 4 8 4 .5 1 9 0 .7 1 9 0 .8 1U ti l iza t ion std ev 0 .7 1 0 .7 0 0 .8 5 0 .6 2 0 .8 7 0 .7 1 0 .8 1 0 .6 2

C o n v e y o r F e e d In te rv a l v s . P ic k E r ro rs

0

2 0 0

4 0 0

6 0 0

0 . 8 0 0 . 7 5 0 . 7 0 0 . 6 5

C o n v e y o r I n t e r v a l

Erro

r

H i e r a r c h i c a l

H y b r i d

9 / 14

3. Intelligent Pick-zone Assignment

Utilization changing with breakdown

0.0

20.0

40.0

60.0

80.0

100.0

120.0

0 10000 20000 30000 40000 50000

Time (sec)

Uti

l. le

ve

l

G16

G15

G14

G13

G12

G11

G10

G9

G8

G7

G6

G5

G4

G3

G2

G1

• Flexibility and reconfigurability - machine breakdown scenario: G1-G4 down for 10,000 ~ 20,000 sec - the remaining 12 gantry robots are able to absorb the tasks of the down gantries if they have the needed SKUs

10 / 14

4. Intelligent Conveyor Speed Adjustment

Gantry Complex Agent

Gantry Agent

Gantry Agent

Gantry Agent

Gantry Agent

(1) Can conveyor speed up(down) ?

(2) Everybody is OK with new speed ?

(3) No problem(3) No I can’t

AP-Plex Agent

A-Plex Agent

Manual Agent

(4) I want to reset my conveyor speed. Is it OK to you ?

Negotiation Protocol

11 / 14

4. Intelligent Conveyor Speed Adjustment

(c) 0.75sec - 5min- 0.05sec

0.6

0.7

0.8

0.9

1

0 10000 20000 30000 40000 50000 60000

Time (sec)

(d) 0.75sec - 10min - 0.05sec

0.6

0.7

0.8

0.9

1

0 10000 20000 30000 40000 50000 60000

Time (sec)

(a) 0.75sec - 5min

0.6

0.7

0.8

0.9

1

0 10000 20000 30000 40000 50000 60000

Time (sec)

(b) 0.75sec - 10 min

0.6

0.7

0.8

0.9

1

0 10000 20000 30000 40000 50000 60000

Time (sec)

• More frequent oscillations in (a) than in (b)• Simple logic at the higher level agent to filter requests - Threshold 0.05 sec => Hybrid

* 0.75 sec : starting conveyor feed interval 5 min , 10 min : specified checking interval 0.05 sec : threshold for filtering logic of higher level agent (Gantry Complex agent)

12 / 14

4. Intelligent Conveyor Speed Adjustment

Fault Tolerance- Machine breakdown scenario: G1-G4 down for 10,000 ~ 20,000 sec- With dynamic speed adjustment, number of errors can be reduced

Hierarchical Model Hybrid with Static Speed Hybrid with Dynamic Speed# of picked 110374 113709 113925# of missed picks 6215 2880 2664# of errors after pick 954 81 6Total error 7169 2961 2670# of no-bids (irrecoverable) 2676 2516Gantry pick time 13:35 13:35 14:45Mean util. % 74.80 76.12 71.74STD util. 6.64 8.32 6.91

Using Naive Conservative Logic- Mean Utilization with Dynamic Speed < Mean utilization with Static Speed => conservative

13 / 14

5. Conclusions & Discussions

• Intelligent agent based hybrid model for actual

industrial problem

• Resource assignment problem and dynamic conveyor

speed adjustment

• Hybrid model outperforms pure hierarchical and

heterarchical models

Conclusions

• Hybrid Scheduling and Control System Architecture for

Robustness and Global Optimization

• Guidelines for designing intelligent agent based production/

warehousing planning, scheduling, and control systems

• Chaos concerns in manufacturing

In Progress & Future Works

14 / 14

Hybrid Scheduling and Control System Architecture

Machine/ MHD Agent

PartAgent

Bulletin Board

Higher Level Global Optimizer Agent

Middle Level Guide Agent

Work Load Balancing between Order Trains

Order-Stream is made by OAPSWork Load between trains is unbalanced

Histogram

0204060

80100120140

Bin

Freq

uenc

y

Min 83Max 235Average 143.13Std Dev 17.44Total train # 815

Lack of OAPS => Developed a PreprocessorPair wise exchange between non zip-qualified orders

Min 140Max 146Average 143.11Std Dev 1.84Total train # 815

Histogram

0

50

100

150

200

140.0 141.1 142.1 143.0 144.1 145.1 More

BinFr

eque

ncy

Train Line-Item Number

0

50

100

150

200

250

1 48

95

142

189

236

283

330

377

424

471

518

565

612

659

706

753

800

Train number

Lin

e-I

tem

Nu

mb

er

Train-Line #

0

50

100

150

200

250

1 41 81 121

161

201

241

281

321

361

401

441

481

521

561

601

641

681

721

761

801

Train #

Lin

e It

em

#

Work Load Balancing between Order TrainsSimulation Results with various conveyor speeds

Conveyor Original Bidding Balanced BalancedInterval Error Error Original Bidding

Error Error0.80 0 0 0 00.75 4 0 0 00.70 51 20 9 00.65 523 367 191 39

Conveyor Speed Vs. Error

0

100

200

300

400

500

600

0.80 0.75 0.70 0.65

Conveyor Interval(sec)

Err

or

#

Original

Bidding

Balanced-Original

Balanced-Bidding

• Throughput improvement in the system by balancing workload and using bidding, 12.5 %(0.10/0.80) easily seen