Queue Scheduling and Advance Reservations with COSY

[email protected] IPDPS 2004

Queue Scheduling and Advance Reservations with COSY

Junwei CaoFalk Zimmermann

C&C Research LaboratoriesNEC Europe Ltd.

http://www.ccrl-nece.de/


Outline

Related Work COSY Structure COSY Job Scheduling Scheduling with Advance

Reservations Performance Evaluation Experimental Results Ongoing Work


Related Work Job Scheduling Systems

Catalina, Condor, EASY, LoadLeveler, LSF, Maui, PBS, SGE, STORM, Titan …

Scheduling with Advance Reservations W. Smith, I. Foster, and V. Taylor, “Scheduling

with Advanced Reservations”, IPDPS 2000. Q. Snell, M. Clement, D. Jackson, and C.

Gregory, “The Performance Impact of Advance Reservation Metascheduling”, JSSPP 2000.


COSY Structure

Operating Systems, MPI Environments (NEC, SCore …), …

Users / Applications

Resources

Generic User Interfaces (submit, release, queue, confirm,…)

JobScheduling

Log Management

Resource Monitoring Database Management

Configurations

Job Execution

Plugin Scripts

Specific User Interfaces (mpirun …)


COSY Job Scheduling Queue Scheduling:

First-come-first-served algorithm Aggressive backfilling Multiple queue support Queue and user priority Resource limitation Access control list Batch and interactive support

Advance Reservations: Exact start time Latest end time Two phase commitment


Mean queue wait time increases Resource utilization decreases Schedulable with limit percentage of advance

reservations

Scheduling with Advance Reservations


Challenges …

How to prevent advance reservations from taking start time advantages over queued jobs?

How to satisfy advance reservations as far as possible without sacrificing queue efficiency?

Additional scheduling policies have to be defined (in academic/research environments) …


COSY Strategies

Defining mandatory shortest notice time for advance reservations Using current mean wait time of queued jobs Using times of current mean wait time of queued jobs


Performance Evaluation Performance Metrics

Queue wait time Resource utilization Rejection rate

Workload Modeling Cirne comprehensive model (ANL arrival pattern) 100 requests -> 32 node cluster Time reduced by 180

Experimental Design COSY running with job execution disabled Using different percentages of advance reservations Using different shortest notice time of advance

reservations


Experimental Results

p

0

0. 5

1

1. 5

2

2. 5

0 1 2 3 4 5 6

n

q

5% 10% 15% 20% p

0

20

40

60

80

0 1 2 3 4 5 6

n

u(%)

5% 10% 15% 20%

Queue wait time Resource utilization


Solutions … Policies applied in COSY

COSY takes the current mean wait time of queued jobs as the mandatory AR shortest notice time so that ARs cannot take start time advantages.

The mandatory AR shortest notice time is applied in COSY as 1 time of current mean wait time of queued jobs and increases to 4 linearly as the AR percentage increases from 0% (not inclusive) to 15% so that ARs can be satisfied as far as possible without increasing the queue wait time.

When the AR percentage is more than 15%, COSY will stop accepting ARs temporarily in order to guarantee a proper queue scheduling.

Summary Advantage: adaptive to queue workload. Disadvantage: additional calculation of statistics.


Ongoing Work

COSY performance evaluation in a larger scale and a longer time period

Integration of COSY in GEMSS grid infrastructure to support QoS negotiation

Extension of COSY to support standard languages and protocols (e.g. GRAAP and JSDL)

COSY-G: job scheduling and workload balancing in grid environments

Documents

Queue Scheduling and Advance Reservations with COSY