WMF Holding Point Application Upgrade

Natalia Emelianenko

7/7/2005

Why We Need to Upgrade

1. Imposed migration to J2EE Public Service File upload on the server will not be possible

2. Problems revealed during the code analysis:1) Measurement identification out of control

2) Database design fits to the workflow as it was imagined in the beginning of the project

3) Lack of organizing routings for the engineers

4) Poor security on the Web

Solutions 1. Upload

Upload files directly into the database

Benefits:1. Detailed upload logging and traceability.

2. File replacement can be blocked after the files are downloaded for the analysis.

3. Upload information is accessible via the database, and the files can be managed with SQL.

WMF Workflow

1. Upload – send mail to MAS-MA engineer2. Download 3. Analysis in Excel4. Loading of the information extracted from the

files into the database tables5. Similarity check6. Transfer files to the DB – delete files from the

server7. Mail to PE8. Mail to firm

Workflow with New Version

1. Upload – send mail to MAS-MA engineer2. Download – mark as “analysis started”3. Analysis in Excel4. Loading of the information extracted from the

files into the database tables5. Similarity check6. Mark measurement as “analyzed”7. Mail to PE8. Mail to firm

Solutions 2.1. Identification

Measurement identification:Current scheme: attempts to find the unique combination

date + magnet + time + time of ap2 + …Proposed:let the firm control what is a measurement, i.e. decidewhether the data files being uploaded are

• corrected version of previously uploaded files• results of a new measurement of the same magnet

performed on the same day

Identification Scheme Explained

• A couple of data files for the magnet XXXX tested on DD-MON-YYYY is being loaded for the first time with the measurement ID

DD-MON-YYYY_XXXX_1• Next uploads will be at firm’s discretion

– replacement, ID not changed, or– new measurement, and its ID will be

DD-MON-YYYY_XXXX_2

• Measurement ID is assigned on upload and never more guessed from the data.

Solutions 2.2. Database Design

New database design fits the real data model.

New objects:

• Table for uploads (replaces WMF_DATA_SOURCE): Store both source and info files in one record Log & manage upload, download and analysis

• Table with the list of magnets to manage: Final measurement for each magnet/stage. Renaming of magnets (dead measurements).

Solutions 2.3. Validation

• Analyzer should have the possibility to choose the final measurement for a given magnet at any moment (or until the mail to firm was sent) – easy and safe with the new database design.

• Analysis and similarity check can be performed inside the database with PL/SQL or Java.

Solutions 2.3. PDA Organizer

New functionality:

• List of not yet analyzed downloaded files.• List of measurements to send a message to PE on.• List of measurements to send a message to firm.• List of analyzed measurements for which

messages were not sent for some reason.• Mailing lists stored in the database

Solutions 2.4. Security

• Make the session timeout really working (available for test on the J2EE prototype Report Maker).

• Choose another type of authentication instead of unsafe basic and /or enforce the use the SSL

• Create own password for each firm.

New Features Examples

Next 6 slides show all changes to the PDA holding point Web applications for the users in MAS.

Download 1

Download 2

Mail to PE 1

Mail to PE 2

Mail to firm 1

Mail to firm 2

Upload Illustrations…

Upload 1

Upload 2 – at attempt to load the second measurement for the day

Upload 3 – uploaded as a new measurement

Upload 4 – an attempt to load 3rd measurement for the day

Upload 5 – check all measurements uploaded for the magnet and day

Upload 6 – the 1st measurement data files have been replaced

Download 1

Download 2 – downloaded files disappear from the “waiting list”

Upload 7 – an attempt to upload files after previous ones have been downloaded

Upload 8 – replacement of downloaded files is blocked

Summary of What Was Done

New features:

• Safe measurement identification• File upload managed by the database• Mailing lists managed by the database• Organizing features:

– List of measurements for the analysis

– List of mails to be sent

– Less manual input

Summary for the Discussion

Analysis in Excel• List of new files for the analysis• Skip download – get files directly from the database• Similarity check inside the database • Data loading without conformity check

Workflow management• Do we need to mark a measurement as analyzed without mail to PE ?• Should there be a possibility to replace analyzed measurements ?

Access rights managed in the databaseAre we ready to migrate to the new database design ?

Database Loading Guidelines

The following is proposed by Elena:• Data from any file should be always loaded.• The analyzer should decide the nature of the data

“invalidity” (i.e. should the magnet be re-measured).

• New valid measurements may automatically invalidate the previous ones.

• Tolerances should be stored in the database (to be taken from there for the analysis).

Migration Plan

1. While the safe database migration is being prepared:

1) Fill in mailing lists in the database, define responsibilities.2) Install the new version of the upload (now available for tests

on PDA). Excel macro will not take the measurement ID from the file name but construct it as it is done now.

3) Test the new functionality with the current database design on the prototype and then on PDA.

2. Migration to the new database structures:1) Choose a day when there will be no hanging measurements.2) Transfer the data to the new tables.3) Install the new web applications on PDA and on the prototype.