INRETS, Villeneuve d’Ascq, December 15 th -16 th 2005 ETISEO Project Evaluation process

INRETS, Villeneuve d’Ascq, December 15th-16th 2005

ETISEO ProjectETISEO Project

Evaluation processEvaluation process



Results submissionResults submission

File format transfer toolFile format transfer tool

Evaluation feed-backEvaluation feed-back



Result Submission Format :

• Based on PETS-metrics XML schema, with the addition of events.

• Some attributes will not be processed (segmentation = shape).

• Documentation on this format is available on the web site also with examples.



Result Submission Format :

Extended PETS metrics

developper details

algorithm details

run date

camera used

results

frame

tracked_target

info_2D

segmentation

row_seg

object

info_3D

parent target

event_target

physical_object

contextual_object

speed

Must occur once and only once in the parent element.

Must occur at least once.

Optional but can occur once and only once in the parent element.

Optional to occur but can occur numerously.



Submission content :

- For each sequence you processed, its xml results file using the naming convention:

sequence-name.xml

- The questionnaire describing how your system proceeded on all sequences.



Submission method :

- by email (<10Mo) to: [email protected]

- on ftp www.silogic.fr/etiseo. Connection parameters will be provide to all participants. Send also a notification to [email protected]. Warning : other participants can have access to your results.- by mail to:

Silogic, ETISEO project.Attention of Luc Barthelemy6, rue Camboulives BP 113331036 Toulouse Cedex 1France



Submission method :

In any case, you will receive an acknowledgement to your submission.








Silogic made some tools:

• to convert different video results structures. From a frame basis format (like PETS) to a sequence basis format (like Viper).

• to address specific format incapability.



Silogic will use these tools to convert your xml result file into the ETISEO xml result structure.

Then metrics will be computed on your results.

We recommend that you use a result format as close as possible to the ETISEO result Xml

schema.







Evaluation feedbackEvaluation feedback

Evaluation feedback :

• Metrics results are sent back to participants.

• Multiple submission is allowed.

• Statistics on metrics for each sequence will be available on the web-site.

• Global feedback to all participants.


ScheduleSchedule

• First video corpus diffusion : starting January• Results collection : until end of March• Global metrics results diffusion : July• Second evaluation: May.

First evaluation

Seminar

AprilJan July Oct DecMay

Evaluation review

Seminar

CYCLE 1

Evaluation review

CYCLE 2

Etiseo evaluation

Documents

INRETS, Villeneuve d’Ascq, December 15 th -16 th 2005 ETISEO Project Evaluation process