Design of a Real-Time Web Application for collaborative … · 15-05-15 16-11-15 Scenarios Functional Requirements Evaluation Implement Implement Quality Attributes Expert survey

Software Engineering for Business Information Systems (sebis)

Department of Informatics

Technische Universität München, Germany

wwwmatthes.in.tum.de

Design of a Real-Time Web Application for

collaborative annotations in telemedicine.Master’s Thesis Final Presentation

Junus Ergin

http://wwwmatthes.in.tum.de

http://wwwmatthes.in.tum.de

1. Introduction & Scenarios

2. Research questions

3. Requirements and their importance

4. Mock-up design

5. Architecture and implementation

6. Evaluation & Conclusion

7. Future work

Overview

© sebis161115 Matthes Slides sebis 2016 2

Telemedical challenge


Dentist Bob Radiologist John

Patient


2. Research questions & Proceeding


4. Mock-up design



7. Future work

Overview


Research questions

RQ1: What is the state of the art for real-time collaborative web based

annotation tools?

RQ2: What are the requirements for web based component design based on

the given scenarios in order to create real-time annotations over the internet?

RQ3: How can the tool be designed to be understandable,

and fulfill all functional requirements and quality attributes?

RQ4: How far does the implemented prototype satisfy the requirements

derived in RQ2 und how is an additional benefit guaranteed against other

typical solutions?


Timeline


15-05-15 16-11-15

Scenarios

Functional

RequirementsEvaluation

ImplementImplement

Quality

Attributes

Expert

survey

Mock-

up

design

Expert

validation

Architect

ure

State of the

Art

State of the Art

RQ1: What is the state of the art for real-time collaborative web based

annotation tools?

What strategies for synchronizing annotations exist?

How to lock critical sections, prevent mutual exclusion?

What frameworks for collaborative annotations exist?

How is complex content, as videos, annotated?

How to deal with different user types?


Annotations

Lock critical sections Deal with different users Common solutions




4. Mock-up design



7. Future work

Overview


Requirements

• 18 Functional Requirements

• Importance rated with survey

• Medical experts in teleconsultation were asked


Synchronize in real-time Highlight parts in files

Turn annotation screen off Consistent color scheme




4. Mock-up design



7. Future work

Overview


Mock-up design

2 different mock-ups based on survey


All functions displayed Only important functions

Only menus

Everything

Only tools

Only frequent options

Mock-up 1

Mock-up 2




4. Mock-up design



7. Future work

Overview


Implementation


Scenarios


Scenario

How to transmit data?

Implementation


( )

Implementation


Implementation – Live Demo

Live Demo





4. Mock-up design



7. Future work

Overview


Evaluation

• „Field test“ with real users

• Different Scenarios

• First usage was unaccustomed

• After 1-3 Minutes they understood everything

• They solved all tasked from the scenarios without experience

• Even people who had reservations before, solved all tasks

Rare functions confused the users

• Turning screen black

• Replay annotations

• Conformity with the survey results





4. Mock-up design



7. Future work

Overview


Future Work

• More research about understandable UI design

• Make functions more understandable

• Reuse the tool in phone apps

• Apply tool to other consultation fields

• Generic rendering in several file formats by manipulating the binaries


Technische Universität München

Department of Informatics

Chair of Software Engineering for

Business Information Systems

Boltzmannstraße 3

85748 Garching bei München

Tel

wwwmatthes.in.tum.de

Junus Ergin

B.Sc.

+49 157 37033036

[email protected]

Thanks for your attention!

http://wwwmatthes.in.tum.de/

Proceeding

State of the art


Scenarios

Functional Requirements Quality Attributes

Expert feedback and importance

Mockup design

Experts validation

Architecture Design and implementation

Field test + results

Introduction (Backup)


Annotations

Support teleconsultation

with collaborative annotations

on different files

Scenarios (Backup)


Scenario 1

Scenarios (Backup)


Scenario 2

Scenarios (Backup)


Scenario 3

Requirements (Backup)


Highlight a part of a file Add text field to a file

Synchronize the view Draw a line on a file



Turn screen off See other collaborators

Add shapes to a file Delete objects



Support DICOM files Support video annotation

Support different image types Support PDF files



Download annotated file Replay the annotations

Support files embedded via

pluginStore author information



Consistent color scheme Display participant status

Quality Attributes (Backup)

1. Modularity / Reusability in all connected systems

2. Run within the browser environment

3. Deal with bad connection quality

4. Optimized data structure

5. Different screen sizes and devices

6. Robustness

7. Scalability

8. Don't store data on hard drive

9. Encrypted data transfer

10.End-to-end connection

11.Frontend in TypeScript / Angular 2

12.Backend in Python

13.Low effort of implementation

14.High Efficiency

15.Separation of concerns


Why Typescript (Backup)

• All Javascript features

• + ES6

• + Types

• Very similar to Java

• Architecture much better


Validation in backendDeal with conflicts

Validation in frontend

• Annotations can overlay

• Editing and deleting only possible

by one person

• How to prevent locks?

• -> Revert after a time Δ if there is

no update from the current editor

• Mutual exclusion?

• -> Not possible using Firebase

Implementation (Backup)


annotation.currentEditor = 1

annotation.currentEditor = null

annotation.currentEditor = 2

NO! annotation.currentEditor = 1

Implementation Class Diagramm (Backup)


Binary Semepahone (Backup)


Source: http://flylib.com/books/en/1.410.1.108/1/

Implementation – Turn the arrow (Backup)


Implementation – Show current state (Backup)


Implementation – Draw line (Backup)


Implementation – Dynamic Source (Backup)


Implementation – File Service (Backup)


Implementation – Annotation Service (Backup)


Implementation – Consistent color (Backup)


• Array with unique colors

• Doctor ID used in order to get a color

𝐶𝑜𝑙𝑜𝑟𝐴𝑟𝑟𝑎𝑦 𝑈𝑠𝑒𝑟𝐼𝐷 % 𝐶𝑜𝑙𝑜𝑟𝐴𝑟𝑟𝑎𝑦. 𝑙𝑒𝑛𝑔𝑡ℎ

UserId % length Hex Code

0 FF6E6E

1 EF2121

2 FFA26E

3 EF6B21

4 92A4D7

5 465DA2

6 57B7C4

7 198B9B

8 19B264

9 FDCBCC

10 C36567

Literature reseach keywords (Backup)

Sources were searched at mediaTUM

-> Includes all important indices, as Google Scholar and Deutsche Nationalbibliothek

• Annotations

• Annotation framework

• Collaboration conflicts

• Collaboration framework

• Collaboration internet

• Collaborative annotations

• Collaborative workspace

• Critical section data lock

• Critical section problem

• DICOM annotation

• Medical annotation

• Real-time collaboration

• Real-time lock

• Real-time synchronization

• Synchronize data

• Telemedicine annotation

• Telemedicine collaboration

• Telemedicine real-time


State of the art sources (Backup)

[Mich 14] P. N. R. K. M. J. Michael Derntl, Stephan Erdtmann. Echtzeitmetamodellierung

im Web-Browser. RWTH Aachen, Chair of Informatics 5, 2014.

https://pdfs.semanticscholar.org/0829/cada43a4b57612366885763ebb555578a6f0.pdf

[Qi] G.-J. Qi, X.-S. Hua, Y. Rui, T. Mei, and H.-J. Zhang. \Correlative multi-label video

annotation". http://dl.acm.org/citation.cfm?id=1291245

[Kell 07] M. A. Kelley and M. S. Wengrovitz. Real-time collaboration center. 2007.

https://www.google.com/patents/US8737596

[Rajk 12] R. Rajkuma. Synchronization in real-time systems: a priority inheritance approach. 2012.

https://books.google.de/books?hl=delr=id=jYICAAAQBAJoi=fndpg=PP10dq=

real+time+synchronizationots=sqMyEPcGA1sig=

LMTMWXPXpLeYWoh1hMIcPsB3wG8v=onepageq=real

[Sha 02] L. Sha, R. Rajkumar, and J. Lehoczky. Priority inheritance protocols: an approach to

real-time synchronization. 2002. http://ieeexplore.ieee.org/document/57058/

Fox 15] Fox, Wu, Uyar, and Bulut. \A Web Services Framework for Collaboration and

Audio/Videoconferencing". 2015.

[Herl 93] M. Herlihy and J. E. B. Moss. Transactional memory: architectural support for lock-free

data structures. ISCA '93 Proceedings of the 20th annual international symposium on

computer architecture, Pages 289-300, 1993. http://dl.acm.org/citation.cfm?id=165164

[Teva 87] A. Tevanian. MACH threads and the UNIX kernel:

the battle for control. Carnegie Mellon University, 1987.

http://repository.cmu.edu/cgi/viewcontent.cgi?article=2728context=compsci


https://pdfs.semanticscholar.org/0829/cada43a4b57612366885763ebb555578a6f0.pdf

http://dl.acm.org/citation.cfm?id=1291245


http://ieeexplore.ieee.org/document/57058/


http://repository.cmu.edu/cgi/viewcontent.cgi?article=2728context=compsci

State of the art sources 2 (Backup)

[Maek 85] M. Maekawa. An algorithm for mutual exclusion in decentralized

systems. ACM

Transactions on Computer Systems Volume 3 Issue 2, May 1985 Pages 145-159,

1985. http://dl.acm.org/citation.cfm?id=214445.

[Chen 05] T. Chen, R. Lai, and A. Chen. System and method for setting

user-right, and recording medium. US Patent 20050144060 A1, 2005.


[Vond 11] C. Vondrick and D. Ramanan. Video annotation and tracking with active

learning. Advances in Neural Information Processing Systems 24 (NIPS 2011),

2011. http://web.mit.edu/vondrick/vatic/videoalearn.pdf

[Holz 14] M. Holzer. Konzeption und Realisierung eines Frameworks f•ur verteiltes

Rechnen in

Webbrowsern mittels WebRTC. Master's thesis, 2014.

[S 13] M. de Sa, D. A. Shamma, and E. F. Churchill. \Live mobile collaboration for

video production: design, guidelines, and requirements". 2013.




http://web.mit.edu/vondrick/vatic/videoalearn.pdf

Documents

Design of a Real-Time Web Application for collaborative … · 15-05-15 16-11-15 Scenarios Functional Requirements Evaluation Implement Implement Quality Attributes Expert survey