Workflow Design Description

Prepared by:Peter Carmichael and Joyce Ng

Version (Date) Changes8/22/01 Creation of Design Description8/24/01 Initial Design Review9/5/01 After Clean up and Adding more Figures9/6/01 After Mick’s comments

1. Scope1.1. Identification

This is the Workflow Design Description for the DSpace project.1.2. System Overview

DSpace is a Digital Archive Infrastructure for MIT Libraries. It is intended to be an archive that is maintained for over 50 years. It will initially have a Web interface. During the 50 years it is expected that formats and storage will change, but the archive will live on.Workflow will be the primary method for submitting a set of documents and their metadata to the archive. When the documents and metadata are submitted to a collection within the archive, they will be packaged, and workflow will pass them appropriately to reviewers, editors, and approvers, who will permit them to be ultimately archived.Workflow will also be used for batch submission of documents.

1.3. Document OverviewThis document will be mastered as a Microsoft Word document to maintain the Figures, and disseminated in other formats as needed.

2. Referenced Documents2.1. DSpace Workflow RequirementsWorkflowRequirements.doc

3. Software Item-Wide Design Decisions3.1. Web Based User Interface

It will simplify distribution issues if users bring up the DSpace applications by pointing their browser at a URL.

3.2. Java Class ImplementationIt will blur the boundary between server and client side code if classes are implemented in java. We can in theory run them on any platform (given the right VM). This will also assume that the server side code is run on a single VM on the server.

3.3. Token PassingTokens are containers of key/value pairs and other Tokens; where keys are strings, values are objects. Workflows pass Tokens between Workflows.

3.4. PipelineWorkflows may act as pipelines.Tokens can be asynchronously fed into their Inputs, where they are validated.The Tokens will be passed from workflow to workflow based on the configuration of the workflow’s internal arrangement.This will be important for Workflows like Batch Submit.

3.5. Workflows can be nested Workflows can contain other workflows. When the outer workflow is ended, so are all the inner workflows.

3.6. NotificationWhen there is a need to notify an EPerson, they should be notified by their preferred method.3.6.1. TaskList Notification

The normal method of notification will be via a URL that will show their TaskList. When they view it for the first time it will ask for a username and password or certificate. The browser will be able to remember the username and password the next time they access the page.

3.6.2. Email NotificationA formatted email message will be sent to the user if their preference it to receive an email.

3.7. Static WorkflowsOnce a workflow instance is created it will be static. The state of the workflow will change as tasks are performed, and admin tools will be the only other way of changing them.

4. Software Item Architectural Design

Figure 14.1. WfManager (future)

It is a factory for generating Workflow instances from Workflow Templates and Data.4.1.1. WfTemplatePersistence

The WorkflowTemplates are kept here, so they can be used by WfManager to create workflow instances.

4.2. WfEngineThis will be implemented as a set of cooperating objects, that move tokens between workflows.

4.2.1. WfPersistenceThe workflow instances are kept here, so they can be recovered in the case of a crash. Each workflow will be updated at every state change.

4.3. TaskList ManagementA TaskList will be managed for each active EPerson that actively participates in workflows.

4.3.1. TaskListPersistence

This will serve 3 purposes. It will keep track of each state change, act as a log of activity, and be able to recover the state in the event of a crash.

4.4. UI Admin4.5. Tree Management

See: TreeManagementRequirements.doc and TreeMangementDesign.docTree Management is for managing a Hierarchy of Groups and a collection of EPersons.

4.6. Concept of Execution4.6.1. Create Workflow Template

DSpace Personnel create the workflow templates based on the feedback from the Collection Administrators.

4.6.2. Start WorkflowExample: User brings up a Web Page for the collection, fill it out, and hits one of the possible command buttons (like <Submit>)A command may simply execute a task, or start a more complicated workflow which will cause the WfManager to create a Workflow from the Collection’s WorkflowTemplate.

4.6.3. Start TaskWhen a user goes to their Workspace they will see their tasklist.They may decide to start any TODO task on their list.

4.6.4. End TaskWhen the task is complete, a token is moved to the output of the workflow.

4.6.5. End WorkflowWhen the workflow is over, the EndWorkflow token is passed, and the workflow is ended.

4.7. User Interface DesignThis is to give a flavor of what is to come:4.7.1. TaskList:

Auto Refresh (future)SortedSome columns may be hiddenSome statuses may be hidden

Priority Task Name Posting Date Estimated Time Due Date Status Who Category Reminder

One or all categories may be visibleCategory V

<New Task><Preferences><Help>4.7.2. Preferences:

Column to show:4 Priority4 Task Name% Posting Date% Estimated Time% Due Date4 Status% Who4 Category% Reminder

Statuses to show:4 TODO4 DOING% DONE% CANCELED% EXPIRED

Category to Show: ( )VSort By (Posting Date)VPermit others to view TaskListReminders:

4 RemindersDefault Reminder for new tasks

[ ] hours V after posting date[ ] hours V before due date

4.7.3. HelpURL

GuidanceAdmin Contact

5. Software Item Detailed Design5.1. WfEngine Classes

5.1.1. TokenTokens may hold Key/Value pairs, and other tokens. In Figure 2 and Figure 3, A token has a box around it, and Key = Value.

Figure 2 Token Example (ID Marc)Figure 2 is a Token with 2 Key/Value pairs {ID=1, Marc=123456}

Figure 3 Token Example (ID Docs)Figure 3 is a Token with 1 Key/Value pair {ID=1}, and holds a token with 0 Key/Value pairs, and 3 other Tokens. The first token has 1 KeyValue pair {Doc=123457}, the second token has 1 Key/Value pair {Doc=123458}, the third token has 1 Key/Value pair {Doc=123459}.

5.1.2. Job

Figure 4 JobA Job is any applications or job being done by a certain entity (person or agent).The job has one or more inputs that accept Tokens.The job has one or more outputs that pass Tokens.The job processes each token that arrives at one of its inputs.The job may pass zero or more tokens to any of its outputs.

5.1.3. Workflow

Figure 5 WorkflowWorkflows contain either a simple task or a more complex arrangement of other subWorkflows and Workflow components.Anything added to a workflow are given certain characteristics:RecoveryIf there is a crash, the state of the workflow will be restored.State Change TrackingAs tokens enter or leave a workflow, the state change is recorded.Pause / Resume / AbortThere will be administration capabilities to Pause / Resume / Abort a workflow.Pause / Resume / Abort will work on all the subWorkflows too. The symbol represents the place where the flow of tokens can be shut off (Pause/Resume).Timers

Reminder NotificationsMultiple Reminder Notifications may be added to the Workflow definition that will remind the Workflow Owner with a predefined message.Deadline BehaviorIf the task has not been completed by the deadline, a default token will be sent, or the task could be postponed. Postpone would let someone else start the task instead.

5.1.4. TokenListener / SubscriptionA TokenListener has an input and an output for tokens. When a token arrives all InputListeners are notified that the token has arrived. When a token is consumed, all OutputListensers are notified that the token has left. Input and Output listeners may be added or removed.

Figure 6 TokenListener / Subscription5.1.5. TokenSpace

A TokenSpace is a TokenListener, and it holds the tokens that can be externally consumed. It acts as a place for tokens, but only lets one consumer get a particular token or tokens. It has one consume method that is synchronized, so it will only let one consume at a time, and all the others have to wait.

Figure 7 TokenSpace5.1.6. OutputListeners

An OutputListener is a design pattern that will permit attaching multiple outputs to a single output.

Figure 8 OutputListeners

5.1.7. TokenSubscriberA TokenSubscriber is a design pattern that will permit attaching multiple outputs to a TokenSpace. Since each TokenSubscriber knows about the arrival and departure of each token, it can know what tokens are in the TokenBuffer, and can wait for the right combination of tokens to arrive before attempting to consume them. There will be many flavors of these intelligent TokenSubscribers.

Figure 9 TokenSubscriber5.1.8. TaskListSubscriber

A TaskListSubscriber is a specialized subscriber that recognizes the availability of a token. When needed tokens are available, it adds a task to the Owner’s Task List. When the needed tokens are consumed, it cancels the task. When the Owner Starts the task, it consumes the needed tokens, and passes a token along.

Figure 10 TaskListSubscriber5.1.9. Task on a TaskList

The TaskList will be owned by someone (Owner)The Task will have a state.

Figure 11 State Transition Diagram

5.2. Workflows for Early Adopters5.2.1. Submit for Review

Figure 12 Submit For ReviewAny of the Tasks can have 0 or more people responsible for the task.If there are 0, then the task is skipped.If there are more than 1, then 1 will accept the task, and the others will automatically have the task removed from their TaskList.User Submit Item

Validate User and ItemPass along Review token(s)

ReviewReplicate Review token into number of Reviewers needed tokensAdds or removes tasks for the reviewer when appropriate tokens are available.When reviewer N clicks on a task on their task list

Present Item to Review N for review GUIWhen user Starts Task

Accept Review TokenGet Review decision from GUI and add it to Review tokenPass along Review Token

Edit Metadata Before ApprovalIs triggered by editor starting a task on their task listPass along Edited Token

Approve ItemAccept a Review tokenPresent Item to approver1 for approval GUIGet Approve decision from GUI and add it to Review tokenIf rejected pass along rejected review tokenelsepass along accepted review token

Edit Metadata After ApprovalIs triggered by editor starting a task on their task listPass along Edited Token

Manage ItemIf approved associate item with archiveElse if edit then remove offending doc from itemElse if reject remove item from workspace

Notify SubmitterIf approved send submitter acceptance notificationElse if edit then send submitter notification of what to editElse if reject then send submitter rejection notification

5.2.2. Batch Submit Pipeline

Figure 13 Batch Submit PipelineSubmit ID / Marc

Submit token that has ID / MarcValidate ID / Marc

Reject all but ID / MarcSubmit ID / Docs

Submit token that has ID / DocsValidate ID / Docs

Reject all but ID / DocsA

Is a TokenSpace with 1 TokenSubscriberMatch Metadata with Doc

Is a TokenSubscriberWhen there is a matching ID in 2 tokens {Marc and Docs} then consume them both, and combine them into 1 token.When there are no more tokens left to match then send an endWhenSubmitted token.

Marc to D.CTransform Marc into Dublin Core.

Submit item to ArchiveOr even Submit to the Collection’s Review Workflow

End Workflow(endWhen submitted and submitted)

AbortValidate Aborter

Reject all but isMember(aborter, “WorkflowAdmin”, collection)

5.3. Workflow State Change TableThis will keep track of every Workflow and sub-Workflow’s state change.It may be used to recover the state of a Workflow after a crash.It will be used by the TaskList GUI to present the TaskList to a user.WorkflowID Name ParentWorkflowID UserID Date Status Result

6. Requirements TraceabilitySorry, we didn’t write this section. You can look at the requirements Docs.

7. Notes7.1. Internal Representation of WorkflowTemplates

How it is stored and retrieved makes a big difference !7.1.1. XML Workflow Descriptions

There should be a longer life to our Workflow Descriptions if we can describe them in XML. If we can pick a dialect of XML that is being used by other workflow tools, then we may get even more benefits.

7.1.2. Database representation7.2. TaskList Preferences:

7.2.1. Default reminders for all incoming tasks ?Filters

Indicators for filter conditions

8. Software Item-Wide Test StrategyTesting the Workflow will probably be a combination of a set of automated Unit tests, and a manual process, because it involves using several browsers (possibly at the same time), and depending on the URL, and the step in a sequence, make sure the web page visible has the correct information on it.

8.1. Unit Tests:Each class shall be unit tested using JUnit.

8.2. Functional Tests:8.2.1. Submit for Review Workflow

Test SubmittingTest Review SubmissionTest Edit Metadata before ApprovalTest Approve SubmissionTest Edit Metadata after ApprovalTest Manage ItemTest Notify SubmitterTest Crash Recovery

8.2.2. Test Batch SubmitTest it using the MIT Press Data