TimeOP: Automated System for PC Activity Tracking and User Productivity Analysis

.UNIVERSITATEA POLITEHNICA BUCUREŞTI FACULTATEA DE AUTOMATICĂ ŞI CALCULATOARE

DEPARTAMENTUL CALCULATOARE

PROIECT DE DIPLOMĂ

TimeOP: Sistem automat de monitorizare a activitatii PC si analiza a productivitatii utilizatorilor

Coordonator ştiinţific: Conf. Dr. Ing. Răzvan Rughiniş

Absolvent: Bocșe Bogdan

BOGDAN BOCȘE

2

BUCUREŞTI

2010

POLITECHNICA UNIVERSITY OF BUCHAREST FACULTY OF AUTOMATIC CONTROL AND COMPUTERS

COMPUTER SCIENCE DEPARTMENT

DIPLOMA PROJECT

TimeOP: Automated System for PC Activity Tracking and User Productivity Analysis

Thesis supervisor: Assoc. Prof. Dr. Răzvan Rughiniş

Author: Bocșe Bogdan


3

Table of Contents

1 Introduction ............................................................................................................................................... 4 1.1 Thesis Objectives .............................................................................................................................. 4 1.2 Motivation......................................................................................................................................... 4 1.3 Thesis Overview ............................................................................................................................... 5

2 Solutions for Measuring User Productivity ............................................................................................... 6 2.1 RescueTime ...................................................................................................................................... 6 2.2 Cyclope ............................................................................................................................................. 6 2.3 Livigent ............................................................................................................................................. 7

3 Technologies and Standards for TimeOP .................................................................................................. 8 3.1 PHP and CodeIgniter ........................................................................................................................ 8 3.2 MySQL ............................................................................................................................................. 8 3.3 .NET C# ............................................................................................................................................ 8 3.4 JavaScript and jQuery ....................................................................................................................... 9 3.5 HighCharts ........................................................................................................................................ 9

4 Architecture for TimeOP ......................................................................................................................... 10 4.1 Overview ........................................................................................................................................ 10 4.2 Desktop Tracker .............................................................................................................................. 10 4.3 Server Gateway ............................................................................................................................... 12 4.4 Data store ........................................................................................................................................ 13 4.5 Administration Area ....................................................................................................................... 13 4.6 Web Application ............................................................................................................................. 13 4.7 Reporting Features .......................................................................................................................... 15 4.7.1 Application Reports ........................................................................................................................ 15 4.7.2 Productivity Reports ....................................................................................................................... 16 4.7.3 Focus Reports ................................................................................................................................. 18 4.8 Email notifications .......................................................................................................................... 19

5 Implementation Challenges for TimeOP ................................................................................................. 21 5.1 Collecting Data ............................................................................................................................... 21 5.1.1 Active process ................................................................................................................................. 21 5.1.2 Active window title ......................................................................................................................... 21 5.1.3 Collecting URLs ............................................................................................................................. 21 5.1.4 Keyboard and mouse hooks ............................................................................................................ 22 5.2 Storing Data .................................................................................................................................... 22 5.3 Accounts, Subscriptions and Billing ............................................................................................... 25 5.4 PayPal Instant Payment Notification .............................................................................................. 26 5.5 Defining, Quantifying and Computing User Involvement .............................................................. 28 5.6 In-house Storage and License Management ................................................................................... 28

6 Testing and User Feedback ...................................................................................................................... 29 6.1 TimeOP Statistics ........................................................................................................................... 29 6.2 User Survey .................................................................................................................................... 30

7 Conclusion and Further Developments ................................................................................................... 34

BOGDAN BOCȘE

4

1 Introduction

1.1 Thesis Objectives

The current project aims at describing, analyzing and building TimeOP (http://www.timeop.com), an integrated environment for tracking the computer activity, productivity and efficiency of the individual users and teams, with respect to the time and effort they spend on various desktop and web applications.

The desktop tracker responsible for acquiring and safely sending data does so in a seamless transparent manner, which does not disrupt the users’ workflow. The activity reports, time sheets and productivity charts are made available online to authorized users through an interface that is easy to use, responsive and compliant with web standards.

In order to maximize the commercial potential of TimeOP, the system integrates an interface with online payments for subscription-based access to the service. Also, it incorporates a server-side licensing mechanism for easy in-house deployment to larger customers.

1.2 Motivation

Workplace productivity is a key performance for any business employing knowledge workers. However, with the increasing amount of content made available online, computer users are often distracted from fulfilling their roles in respective business processes and are constantly drawn to unproductive, time-consuming activities. On the other hand, the resources needed for increasing amount of jobs often are online, thus making the restriction or complete banning of Internet access an unfeasible solution for business-owners and managers. Moreover, as shown in (Coker, 2009), allowing up to 15% of time for leisure browsing at the office can actually improve overall productivity by 9%.

While other solutions for tracking worker activity exist, they often overload managers and system administrators with immense lists of visited websites for each user, making it almost impossible to discern who is productive and who isn’t. Moreover, existing solutions such as keylogging and screenshot retrieval often risk to needlessly sacrifice the users’ right to privacy in order to measure productivity

TimeOP is built to automatically asses productivity over the whole set of used application, presenting only relevant data and conclusions to authorized parties. Additionally, user privacy is protected to a large degree since private content, such as passwords, typed text and conversations with third parties is never acquired, stored or revealed.

Therefore, the presented solution does not disrupt the users’ workflow, protects user privacy, while providing managers, business-owners and even the users themselves with relevant conclusions and visualizations on productivity.

Moreover, TimeOP aims at surrounding workplace productivity into a competitive metric, rather than just enforcing manager-surveys-employee mindset. As suggested in (Kompier Michiel, 1999), having a centralized surveillance authority in the workplace increases perceived stress and lack of privacy. Through various information sharing policies, TimeOP opens up activity data within the team environment, thus raising awareness without causing additional stress.

The current thesis aims at analyzing existing solutions, emphasizing their strengths and weaknesses and building a solution which is qualitatively superior, focusing on solving specific real world problems. Ever since the beginning of the project, we have adopted an easily extensible

http://www.timeop.com/


5

architecture, so that TimeOP data can be securely made available to other services through a REST API. Realizing the importance of data privacy, a significant part of the development effort has been directed toward right management and sharing policies that guarantee complete isolation between independent users. In order to maximize the commercial potential of this project, the user management component has built-in support for handling subscription types, payment gateway instant notifications, and account upgrading and downgrading policies.

In the rest of the thesis we’ll present the major components that allow for TimeOP business processes to run correctly.

1.3 Thesis Overview

This thesis is structured as follows: the first chapter described the reason for choosing this subject and the second chapter analysis and compares existing software solutions which measure worker productivity. In the third chapter we focus on the technologies used in developing the application. The forth chapter focuses on the application architecture and the relationships between components. The fifth chapter describes the challenges arisen in collecting and storing data, user and subscription management, receiving and processing payments, rights management and licensing and in-house storage. In the final two chapters the author shall point out conclusions driven from using and testing TimeOP, user feedback and future developments.

BOGDAN BOCȘE

6

2 Solutions for Measuring User Productivity

Tracking employee activity and measuring productivity is an increasing need for businesses employing knowledge workers, so there already are several solutions and approaches on the market. In the following chapter, we’ll highlight the most relevant ones in order to emphasize the advantages of TimeOP.

2.1 RescueTime

RescueTime is an automated time tracking and management solution which is disturbed as a free or premium service. It requires users to install a desktop tracker on their machines, that transparently sends all the data to the service.

Following data collection, the user is presented with activity and productivity reports. Application usage is categorized and efficiency is computed as correlation between application categories and the profile the user has chosen.

One of the downsides of RescueTime is that its web interface is not HTML5 compliant, as it relies heavily on Adobe Flash Player. As a consequence, the reports are not available for platforms which do not support the Flash plug-in, such as iOS mobile devices.

Also, RescueTime offers limited support for project and task management. Projects cannot be assigned individual categories, thus making it harder to distinguish between productive and unproductive activities in some contexts. Moreover, there is no clear scheme to define rights and privileges of users over certain projects, which in turn allows for any user to change the productivity index for any application. Therefore, there is no palpable difference between privileged users (managers) and non-privileged users (employees).

The RescueTime tracker does not allow for users to choose which project you’re working on without interrupting their workflow. Instead, users are required to separately login to the web application and manually assign time periods to their respective projects. This in itself can result in unnecessary time being wasted.

This solution does not allow for breaking down projects into smaller units (tasks), a feature essential when dealing with more than a few employees.

Last but not least, RescueTime is only available as an in-cloud service. Licensing is no possible to date, although medium businesses often need in-house storage to accommodate their need for privacy and data protection.

2.2 Cyclope

Cyclope is a time tracking solution that is best suited for the needs of enterprises with all computers and employees in the same LAN network. It does provide a web interface for generating reports and charts on user activity, but it does not allow for the interaction of the user with the tracking agent. Therefore, Cyclope is focused exclusively on surveillance, rather than including a time management component. This solution is not available as a service, but only as licensed software, stored on the customer’s infrastructure. The solution is developed in Microsoft ASP .NET by a Romanian company.


7

2.3 Livigent

Livigent is a LAN-based solution, monitoring and filtering the network traffic. It is installed as a router in the core network and it provides live filtering of content served (mainly HTTP and HTTPS) and live statistics. Although this solution can track and efficiently filter the amount and type of content members of an organization are exposed to, it cannot properly measure time they spent interacting with the content.

Nevertheless, Livigent can offer additional features, such as bandwidth management and removal of unwanted traffic types. Running this type of solution however comes at a significant cost to businesses, considering real time content parsing is quite resource intensive does put a strain on the CPU resources of the routing machine.

The solution is developed by R&D Software, a Romanian company and actively marketed in the USA. It includes some core kernel components developed in C, but it also provides a web interface developed in Python for policy management and traffic reports.

BOGDAN BOCȘE

8

3 Technologies and Standards for TimeOP

3.1 PHP and CodeIgniter

The server backend for TimeOP is developed using PHP 5, a high-level server-side scripting language. The reason for choosing PHP over more mature languages (such as ASP or Java) was strictly related to minimizing hosting costs. In spite of its being an open source technology, PHP offers extensive support for cryptography (RSA, SHA-1, AES, certificates), database interfaces, standard formats such as JSON and XML and data manipulation.

PHP (Hypertext Preprocessor) is a general-purpose scripting language that was originally designed for web development to produce dynamic web pages. PHP code can be embedded into the HTML source document and interpreted by a web server with a PHP processor module, which generates the web page document. PHP is available as a processor for most modern web servers and as a standalone interpreter on most operating systems and computing platforms.

On top of the main PHP environment, a CodeIgniter framework was installed. CodeIgniter significantly improves the development experience, by imposing a MVC (Model-View-Controller) design. Also, it improves on the underlying security of PHP, by offering automatic character escaping, string filtering. The libraries provided by CodeIgniter simply working with databases, HTTP responses, secure file uploading, email protocols and benchmarking code.

3.2 MySQL

MySQL is a relational database management system (RDBMS) that runs as a server providing multi-user access to a number of databases (L. Welling, 2008). The MySQL development project has made its source code available under the terms of the GNU General Public License, as well as under a variety of proprietary agreements. MySQL was owned and sponsored by a single for-profit firm, the Swedish company MySQL AB, now owned by Oracle Corporation.

Using MySQL was also prompted by its open-source license and the low hosting fees practiced for making it available. Using Oracle Database was also taken into consideration, but deemed unfit due to its complexity being unjustified within the confines of this startup project and its prohibitive cost for a commercial license. Other solutions considered include non-relational databases (also known as NoSQL), such as MongoDB, CouchDB and Cassandra. Although using such solution could ease scaling the datastore to several machines, it unnecessarily complicates the early development efforts.

For the purposes of this project, MySQL seemed to perform satisfactorily on a large enough dataset (over 2 million items), with some relevant optimization, such as indexing. Other optimization techniques presented in (Jeremy D. Zawodny, 2004) are also proposed.

3.3 .NET C#

Considering the targeted users for TimeOP are familiar with the Windows environment, Microsoft .NET C# was chosen for this desktop component (the tracker), as it has a extensive libraries available and it minimizes development life cycle by employing intuitive interface designers.

C# is a multi-paradigm programming language encompassing imperative, declarative, functional, generic, object-oriented (class-based), and component-oriented programming disciplines. It was developed by Microsoft within the .NET initiative and later approved as a standard by Ecma


9

(ECMA-334) and ISO (ISO/IEC 23270). C# is one of the programming languages designed for the Common Language Infrastructure (CLI). C# is intended to be a simple, modern, general-purpose, object-oriented programming language.

For additional portability, Java was also considered. However, the TimeOP tracker requires access to the operating system internals, such as the Windows API, for various purposes (detecting current process, detecting current web page viewed). Accessing these resources from the Java Runtime Environment (JRE) would have required using Java Native Interface (JNI), which needs recompilation on each platform. Thus, the apparent portability offered by Java would have been only apparent in this context.

3.4 JavaScript and jQuery

In order to embed a more dynamic and user-friendly experience in the web front-end of TimeOP, the conventional markup (HTML) and stylesheeting (CSS) languages were also combined with JavaScript. JavaScript is an implementation of the ECMAScript language standard and is primarily used in the form of client-side JavaScript, implemented as part of a web browser in order to provide enhanced user interfaces and dynamic websites. This enables programmatic access to computational objects within a host environment. It can be characterized as a prototype-based object-oriented scripting language that is dynamic, weakly typed and has first-class functions. It is also considered a functional programming language like Scheme and Haskell because it has closures and supports higher-order functions.

For a more robust cross-browser experience, the jQuery framework was also added (J. Chaffer, 2007). This framework offers simple methods for binding events to HTML DOM elements, as well as for safely and consistently handling these events over all major browsers (Chrome, Firefox, Opera, Safari, Internet Explorer). jQuery is a cross-browser JavaScript library designed to simplify the client-side scripting of HTML. Used by over 41% of the 10,000 most visited websites, jQuery is the most popular JavaScript library in use today.

jQuery is free, open source software, dual-licensed under the MIT License and the GNU General Public License, Version 2. jQuery's syntax is designed to make it easier to navigate a document, select DOM elements, create animations, handle events, and develop Ajax applications. jQuery also provides capabilities for developers to create plugins on top of the JavaScript library. Using these facilities, developers are able to create abstractions for low-level interaction and animation, advanced effects and high-level, theme-able widgets. This contributes to the creation of powerful and dynamic web pages.

3.5 HighCharts

HighCharts is a library written in pure JavaScript which helps render interactive, high quality, cross browser charts and data visualizations. This library was chosen as it solves compatibility issues, rendering challenges in various browsers and it minimizes the data footprint. As shown in (HighCharts Reference), the library is able to draw a wide range of visualization types, including line, bar, column, pie, scatter and pie charts.

Unlike Flash based data visualization libraries, HighCharts provides a lower loading time and it also works on a wide variety mobile systems (including Android, iOS and Symbian). Moreover, this library offers other relevant features, such as exporting charts as image files and dynamically loading data points via Ajax.

BOGDAN BOCȘE

10

4 Architecture for TimeOP

4.1 Overview

In this chapter, we will review the five components that allow TimeOP to function as whole, together with the specific problem that every one of them faces. In the figure below you can notice the dependencies between various parts of the system.

Figure 1 – Component overview for the TimeOP system

4.2 Desktop Tracker

The desktop tracker is responsible for acquiring real time activity data from the users’ computers, buffering in an encrypted data store until an Internet connection is available and consequently uploading it to the main TimeOP server via HTTPS. Although it might seem a trivial tasks, there are particular challenges which justify the design complexity of the tracker:


11

Acquiring URLs from browsers. Considering that users spend up to 40% of the time in a browser environment, together with the rising importance of web application (Google Docs, Google Apps, Salesforce, Basecamp, PivotalTracker and so on), marking browser usage by simply mentioning the name of the browser is highly unpractical. Using the same browser, an user can perform both productive activities (updating information in a web-based CRM, for instance) and unproductive activities (watching videos online and reading blogs). T o this avail, it is necessary to acquire the active URL in the browser environment. Considering the fundamentally different structure of browsers, three approaches are used depending on browser type: OLE components for acquiring data from Internet Explorer, the Windows API for acquiring data from Chrome and DDE (Dynamic Data Exchange) client-server for Mozilla, Firefox and Opera. These technologies are only viable on a Windows environment, thus making porting the tracker to Linux operating systems a project in itself.

Safely storing and encrypting activity information for later transfer. It is reasonable to assume that the user will not always have an Internet connection available. For this purpose, activity being tracked offline must be stored in an encrypted fashion on the computer hard drive for later retrieval and sending to server. The data is stored in XML format and consequently encrypted with Rijndael (AES) symmetric algorithm. The key used for the encryption process can be a constant, a user-dependent hash, a machine-dependent hash or a combination of the three.

Acquiring keyboard and mouse statistics. Time is not the only metric TimeOP tracks. Keyboard and mouse activity statistics may often prove useful in order to track user involvement, namely the level of effort put into interacting with each application. Usage profiles may vary greatly, since text editors are likely to use more character keys, internet browsing is likely to produce more mouse clicks and graphical design usually requires a lot of cursor movement. In order to capture all this activity, Windows API hooks have been used for key press, mouse click and mouse move events. In order to guarantee user privacy, the actual keys being pressed are not being recorded, only the number of keys pressed under each context.

Figure 2 – The main screen of the TimeOP tracker

Figure 3 – Users can add new tasks directly from the tracker

BOGDAN BOCȘE

12

The desktop tracker can run either in visible mode or in stealth mode (invisible to the user, as a Windows Service). Provided the tracker is visible to the user, it presents a set of features which include:

Selecting the project and task the user is working on, as well as observing the total time spent on each of them (Figure 2).

Defining new tasks on the selected project provided the authenticated user has the proper rights to do so (Figure 3).

Seeing the time spent on each task/project.

Pausing and resuming time tracking.

Automatically noticing periods of inactivity and allowing the use to mark activities done while away from computer (such as meetings, phone calls and breaks) (Figure 4).

Receiving notifications about new tasks or projects being added (possibly by some other user with the necessary rights) and about time spent away from the computer.

Figure 4 – Time spent away from the computer is also tracked

4.3 Server Gateway

The tracker interface is a Web component which periodically receives activity data from the desktop tracker via HTTPS protocol, verifies user identity and properly inserts the data in the database for later retrieval. This component has been written in PHP and its main roles include:

Receiving reports from the desktop tracker in XML format

Verifying the user credentials (username and password hashes)

Writing the date, time, IP and the unique key of the machine from which the report was sent.

Matching the projects and tasks the user worked on for the period of time specified within the XML report with existing data on defined projects and tasks.


13

Verifying that the user has the rights to sent data marked for those particular projects and tasks.

Writing the activities marked during the time spent on each task/project. An activity is a either a desktop process or a web page domain (such as “http://facebook.com”)

Optionally, writing the states of each activity, for more insight on the users’ activities. An activity state is the title of the active window for desktop application and it is the pair made up by full webpage URL and webpage title for web applications and pages.

4.4 Data store

We use the MySQL database management system for storing acquired data, user information, account information, user-defined projects and tasks, user rights and roles. As you’ll notice from the “Implementation Challenges for TimeOP” chapter, the most challenging part of storing TimeOP data is efficiently indexing user activity data,

The link between the application server and the data store is done by using conventional PHP-MySQL connectors.

4.5 Administration Area

The administration area from TimeOP is used to manage users and global system settings. The operations allowed in this section allow system administrators:

To create new user

To edit the profile information of existing users

To disable accounts of particular users

To define and edit the subscription type of each user

To define and edit new subscription types

To define, renew or expand TimeOP license

4.6 Web Application

The main purpose of the TimeOP web application is to show detailed activity, involvement and productivity reports for users, projects, tasks, for various time intervals and contexts. The main view of the web application is called the dashboard (Figure 5) and it allows the user to access the most necessary actions:

1. Browse active projects. By clicking on any of the projects, an activity summary will be displayed in the center panel, including time recently spent on a project, the average focus registered on a project, the status of tasks associated with that project. Also, an activity report will be automatically generated in the lower region of the dashboard (see chapter “Reporting features” for details).

2. Define new projects. This will take the users to the new project flow, allowing to set a project name and description, a start/end period, the project category, invite other people to work on the project and set tasks for them.

BOGDAN BOCȘE

14

3. Manage tasks for the selected project. 4. Manage the productivity of specific applications in a certain project. This feature is

useful for defining custom productivity factors 5. Manage the users (team) who have access to a project and the actions they can

perform (register activity, view reports, add tasks, decide to close tasks, invite other users and change the rights of existing users).

6. Close a project or permanently delete it. Closing a project will cause it to disappear from the tracker application of all those involved, but to remain in the project list in the web application (at the bottom of the list). Deleting it will completely remove it (including all activity data recorded on the project).

Figure 5 – The main view ("Dashboard") of the web application


15

4.7 Reporting Features

Activity reporting is one of the core features of TimeOP. It allows for user, account and system managers to metrics of activity, involvement and focus for all employees.

At the time of writing this paper, there are three types of reports available in TimeOP, which shall be described in the sub-chapters below:

1. Application Reports 2. Productivity Reports 3. Focus Reports

The metrics over which reports can be rendered include:

1. Time actively spent at the computer. 2. Involvement. The method for determining involvement will be described in full

detail in the “Implementation Challenge of TimeOP” chapter. 3. Clicks performed. 4. Keys pressed. 5. Mouse motion, as estimated in inch traveled by the mouse pointer on a 15”

displaying device with a resolution of 1280x800. Other variables which can be set for reports include:

1. The start and the end period of the report 2. Whether the report is detailed (with data points every time step) or summarized

(with overall data on the selected period. 3. The time step. This attribute is only available for detailed reports. 4. The specific project, task and/or user for which the report is generated. By default,

the report is generated for all projects/tasks of the current user. 5. Activity type defines whether offline activities (such as “Meeting”, “Coffee break”,

“Phone Call”) are included. 6. Application type defines whether web pages, web application and/or desktop

applications are included in the report. 7. The chart elements attribute defines the number of application included. For

instance, top 20 most used applications. This attribute is only available for application reports.

8. Drawing preferences, such as chart size and type (bar, column, line, area, spline).

4.7.1 Application Reports

Application Reports (also known as Activity Reports) emphasize the activities which took up the most of the user(s)’ time or involvement. The report can be either summarized as overall application usage over a defined period (a week, for example) or as a detailed look at application usage with a specified time step over a defined period (everyday, for a week).

In Figure 6 you can notice an activity reports emphasizing the 6 most used applications, over a period of two months, a weekly time step. Also, you can notice that the rest of the applications are neatly grouped in Desktop, Web Apps, Web Pages and Away.

BOGDAN BOCȘE

16

Figure 6 – Activity report, weekly, over two months

4.7.2 Productivity Reports

Productivity Reports automatically categorize all applications used in the specified period in 6 categories

a. Very Productive (colored green) b. Productive c. Somewhat Productive d. Somewhat Distracted e. Distracted f. Very Distracted (colored red)

In Figure 7 below, you’ll notice a productivity report rendered for a week, with a daily time step.


17

Figure 7 – Productivity reports, weekly, daily time step

Applications are automatically included in one of the abovementioned categories

considering several factors, by using an escalation policy. The escalation policy is applied considering that if very particular information is missing (how important a particular application is for a particular project) more general information is used. The 5 levels of escalation are:

1. How productive was that specific application marked for the specific project during which it was used.

2. How productive was that specific application marked for the category to which the project during which it was used pertains to. This information is determined by averaging votes of TimeOP users.

3. How productive an application is generally considered. This information is determined by averaging votes of TimeOP users.

4. If none of the above information is available, the application is by default marked as “Somewhat Distracting”.

The voting system cascades across these categories as well. For instance, let’s say a user votes for “Microsoft Word” as being “Very Productive” for a particular project registered in the “Office and Utilities” category. Three votes are registered at this point:

1. Microsoft Word is marked as “Very Productive” for the particular project. 2. Microsoft Word is marked as “Very Productive” for the “Office and Utilities”

category. 3. Microsoft Word is marked as “Very Productive” as a general application.

BOGDAN BOCȘE

18

The voting process also has several policies:

In a particular project, an application may have one productivity rating (the last vote counts).

In each category, a user may only vote an application once. If the user revotes an application, the last vote counts.

In general purpose votes, a user may only vote once for an application.

The votes for particular projects are effective immediately.

The category votes and the general votes are recomputed automatically every 24 hours, over all users.

There are 10 project categories currently available in TimeOP. Other categories may be added later on in the project, but this task can only be performed by a system administrator:

1. Business 2. Communication 3. Social Networking 4. Design and Creativity 5. Entertainment and Games 6. News and Buzz 7. Learning 8. Software Development 9. Office and Utilities 10. Miscellaneous

4.7.3 Focus Reports

Focus Reports provide a scalar measure over time of the balance between productivity and unproductivity. The equation (1) is used for calculating productivity over a period of time and over a given metric is:

(1)

In the above equation, sgn() is the signum function. It can be easily noticed, that if all

metric is spent unproductively, the focus is -1 (negative one), if all metric is spent productively, the focus is 1 (positive one) and for equal amounts of the metric being spent productively and unproductively, the focus is 0 (zero).

As mentioned before, the considered metric, can be time, involvement, mouse motion, number of clicks or keys pressed.


19

4.8 Email notifications

Email notifications are an essential part of keeping in touch with the user base. The TimeOP system emails users automatically for several purposes:

Communicating promotional offers

Delivering automatic productivity reports (such as the one in Figure 8)

Delivering account confirmations (on sign-up, on account upgrade)

Delivering password recovery notifications

Although this process may seem trivial, there are several issues and best practices that

have been considered into designing an emailing system:

Emails must not be unsolicited. All emails must include the address of the sender and the reason for sending them (sign-up for a particular service).

All emails should include an unsubscribe link, allowing the user the renounce a subscription whenever they see fit.

Massive amounts of messages must be delivered in stages. The hosting service chosen for TimeOP enforces a throttle rate of 500 emails per hour. Messages which are sent over that limit are discarded by the hoster.

Emails should contain a link to a web version of the message, in case the user is not using an HTML-enabled email client. For this reason, emails should be at least temporarily stored on the web server, for later retrieval.

Figure 8 – Automatic email report

BOGDAN BOCȘE

20

In order to deliver on the business requirements, TimeOP includes an emailing service, which queues and sends email, support subscribe/unsubscribe options for various types of messages and also permits the viewing of messages in the web browser.

When various services from the TimeOP system emit emails for several purposes (user management, account management, automatic reporting, marketing), the following sequence of steps occurs:

1. Messages are queued with a given priority. 2. A queue flush is then attempted, provided the throttle rate has not been

exceeded. High priority messages are sent first 3. If throttle rate has been exceeded, the mailing service will reattempt a flush every

60 minutes until the queue is empty.


21

5 Implementation Challenges for TimeOP

In the following chapter, we will discuss in more detail the implementing challenges posed by the development of TimeOP. The exposed problems include client-side data collection, efficient data retrieval and storage, user subscription management and secure online payment processing.

5.1 Collecting Data

The data collection is done by the tracker, which uses various operating system calls to determine the active process, the active URL (in case the active process is a browser), the number of keys, clicks and estimated distance traveled by the mouse pointer.

5.1.1 Active process

One of the most important roles of the TimeOP tracker is to determine the time each desktop application was active for. In order to do this, the tracker must periodically and automatically detect the application which sits in the foreground of the operating system.

The following method returns the PID (process identifier) of the active process via the handle of the active window.

public static Int32 GetWindowProcessID(IntPtr hwnd) {

Int32 pid; GetWindowThreadProcessId(hwnd, out pid); return pid;

}

The handle of the active window is, in turn, returned by Windows API method, with the following signature:

private static extern IntPtr GetForegroundWindow();

This method is imported from the “user32.dll” library.

5.1.2 Active window title

Another challenge is determining the title of the aforementioned active window. This can be done with another Windows API method: public static extern int GetWindowText(IntPtr hwnd, string lpString, int cch);

This method is imported from the “user32.dll” library.

5.1.3 Collecting URLs

Considering that a significant number of applications and business processes are available on the web, just assigning the time spent online to the browser process (Chrome, Mozilla, Internet Explorer and so on) actually relays no information about the activity being performed. Rather, the domains being visited (“http://techcrunch.com” or “http://twitter.com”) bares a lot more relevance.

BOGDAN BOCȘE

22

Tracking this URL addresses is done differently for each browser, so TimeOP implemented this mechanisms only for the most popular browsers, thus offering coverage of over 95% for Windows users, as reported by (W3Schools, 2011). The browser for which URL tracking was implemented are:

Chrome

Internet Explorer

Mozilla / Firefox

Opera

As an example, we’ll present the URL tracking mechanism for Chrome, using Windows API calls.

public static string getChromeURL() {

uint MAX_PATH=255; IntPtr hChromeWindow, hAddressBox; // Get the window handle of the active window hChromeWindow=GetForegroundWindow(); // Get the handle of the address box, identified by class name // “Chrome_AutocompleteEditView, within the // Chrome active window, hAddressBox = FindWindowEx(hChromeWindow, IntPtr.Zero,

"Chrome_AutocompleteEditView", IntPtr.Zero); StringBuilder sb = new StringBuilder(256); // Get the content of the control identified by the previously // determined handle by sending it the WM_GETTEXT Windows message // The results is returned within a StringBuilder object SendMessage(hAddressBox, WM_GETTEXT, (IntPtr)MAX_PATH, sb); string s = sb.ToString().Trim(new Char[] { ' ', '\0', '\n' }); // If the address cannot be retrieved (the Chrome window isn’t active, // some system error occurs and so on), the empty string is returned return s;

}

5.1.4 Keyboard and mouse hooks

Input activity (using keyboard, mouse or equivalent devices) is very relevant in assessing the involvement and the effort of the user in using applications. Also, taking into account the keyboard activity is very relevant for determining idle periods of time, when the user does not interact with the foreground window.

In order to effectively access these activity indicators, the TimeOP tracker installs keyboard and mouse hooks, with are essentially asynchronous callback mechanisms. Essentially, when any of these events occur the operating system calls specific methods from the TimeOP instance, which in turns allows to increment counters and perform the business logic associated with each event.

5.2 Storing Data

The most significant challenge in handling the data acquired by TimeOP is working with significant volumes, optimizing read efficiency and making the data available with little delay. At the


23

time of writing this material, the TimeOP deployment available at “www.timeop.com” contains 3.1 million records, totaling 600MB of stored information. Naturally, over 95% of data describes user activity, all of the other information (user information, projects, tasks) being modest in scale.

The first method used in order to tackle this issue, was indexing the tables with high volumes of data according to fields relevant for filtering: UserID, TaskID, Timestamp, Activity Name.

Figure 9 details the core database tables storing information on user activity.

Processes

PK ProcessID

FK1,I1 UserID

FK2,I2 TaskID

FK3,I3 SID

I6 Name

I4 StartTime

I5 LastTime

Units

IdleUnits

Switches

Clicks

Kys

Length

Turns

Scroll

CharKeys

IsWebApp

IsAway

Users

PK,FK1 UserID

FK1 AccountID

role_id

U1 username

password

U2 email

banned

ban_reason

newpass

newpass_key

newpass_time

last_ip

last_login

created

modified

Projects

PK ProjectID

Title

StartDate

EndDate

Comments

U1 PK

IsClosed

FK1 AccountID

Tasks

PK TaskID

FK1,U1,I1 ProjectID

U1 Name

Description

StartDate

EndDate

U1 UserID

Progress

State

Submissions

PK SID

IP

Serial

SubmitedAt

UserID

ProjectID

TrackedFrom

TrackedTo

UserProjects

PK,FK1 UserID

PK,FK2 ProjectID

CanSend

CanView

CanAddTask

CanClose

CanInvite

Accounts

PK AccountID

FK1,FK2 UserID

PlanID

UserAccounts

PK UserID

PK AccountID

CanInvite

CanDisinvite

CanSendAll

CanViewAll

CanAddTaskAll

CanCloseAll

CanInviteAll

ProcessNames

PK NameID

U1 Name

PerformanceIndex

Description

URL

LongName

IsWebApp

Categories

PK CateogoryID

CategoryName

ProcessCategories

PK CategoryID

PK,FK2 NameID

ProductivityIndex

ProductivityWeight

States

PK StateID

FK1,I7 ProcessID

I1 UserID

I2 TaskID

I3 SID

I6 Name

Units

IdleUnits

Switches

Clicks

Kys

Length

Turns

Scroll

CharKeys

IsWebApp

IsAway

URL

Title

Figure 9 – Overview on the TimeOP database core entities and relations

BOGDAN BOCȘE

24

The highlighted tables are very data intensive. The indexed information is marked with the letter “I”, primary keys are marked “PK” and foreign keys are market “FK”.

Let us now focus on the particular role of each table:

Processes stores activity data on the used applications and web pages. The indexed fields

includes user ID, task ID, submission ID, the name of the application (process) and the time

interval which it was used in. The non-indexed fields include: the active time spent on the

application floating point seconds (Units), the idle time spent on the application, the number

of context switches (Switches), number of keys pressed (Kys), number of clicks

performed(Clicks), number of mouse wheel turns (Turns), the number of character keys

(CharKeys), the number of scroll keys (Scroll) and the distance covered by the mouse pointer

(estimate in screen-inch, floating point).

States stores data on the states undergone by the application used. A state is identified by a

window title for a particular application or by a URL, in case of a web application.

Submissions stores data on each individual XML report sent by the tracker to the web

application. This information includes the key of the machine, the IP, the timestamp of the

submission (of the local machine and of the server) and the ID of the logged in user.

ProcessNames keeps a unique list of all desktop applications and web pages used. This is

useful for ranking activities as either productive or unproductive.

Projects include a list of all projects created by users.

Tasks include a list of all tasks created by users.

Users keep a list of all active and inactive users, with authentication credentials and an

unique indetifier.

Accounts keep a list of all accounts, including the subscription type of each account. Each

user is assigned one account for which they are account holder. However, a single user can

have usage and management rights on several other accounts. The account entity is mostly

used for subscription management and billing purposes.

Categories contain a list of types which can be assigned to a project. Depending on the

category, the applications used in a project can be deemed as either productive or

unproductive.

UserProjects defines the rights of each user on a particular project.

UserAccounts defines the rights of each user on a particular account (for which they are not

account holder).

ProcessCategories defines the relevance for each application in a particular category. For

instance, “Facebook” is very productive for “Social Networking”, but very unproductive for

“Software Development”.

At this stage of the project, the database response is satisfactory, with activity reports generated in 0.1 to 0.5 seconds. However, as the volume of available data increases, additional database optimization techniques shall be employed, as presented in (Jeremy D. Zawodny, 2004):

Logarithmic-timestamp table partitioning. Due to practical considerations, the most

accessed data is the data which is most recent. However, using a simple B-Tree index for


25

timestamp has limited performance. Therefore, tables will be dynamically partitioned using

a variable time step. For instance, the several non-overlapping partitions can contain data

from the last week, the last two weeks, the last month, the last six months, the last year, the

rest of the data. Periodically (once a week or once a month, during the weekend), a process

will be run to remake the partitions.

Database master-slave replication would allow for increasing the number of queries served

by TimeOP. This method works by having several MySQL slave machines replicating the

content from the master MySQL instance. All read operations are performed on slave

machines, while write operations are performed by the master machine. Data inserts and

updates are logged by the master and consequently transmitted to the workers. The only

downside with replication scaling is that data consistency is temporarily lost. Nevertheless,

minor inconsistencies are unnoticeable in the TimeOP use cases.

5.3 Accounts, Subscriptions and Billing

TimeOP is also available in the Software-as-a-Service model, so the subscription management and billing features are very important. For this purpose, the accounts were defined as an entity separate from users and are exclusively used for billing purposes. Each account has at least one associated user, also known as the account holder, but it can have several other users associated with diminished privileges.

Thus, in this model, the account represents a particular business and the account holder is the business administrator. The business administrator can delegate account rights to other trusted users (such a project managers), which can use the privileges of the account to track the activity of employees.

The main privilege associated with an account is the number of users whose work can be tracked, across an unrestricted number of projects. This privilege is defined by association with an existing plan (subscription). The existing plans have a maximum number of users defined, a number of days of availability (billing life cycle) and a price.

The relationships between the mentioned entities are emphasized in Figure 10. Table fields haven’t been exhaustively listed, due to their not being relevant for the subject matter.

BOGDAN BOCȘE

26

Accounts

PK AccountID

AccountName

AccountHolderID

FK1 PlanID

LastPaymentDate

...

Plans

PK PlanID

SubscriptionName

MaxUsers

Price

DaysAvailable

Users

PK UserID

Username

.... UserAccounts

PK,FK1 UserID

PK,FK2 AccountID

...

Projects

PK ProjectID

FK1 AccountID

...

PlansUpgrade

PK UpgradeID

FK3 AccountID

Timestamp

FK1 OldPlanID

FK2 NewPlanID

ExpiresOn

...

Figure 10 – Relations between users, accounts, plans and projects

Each projects which has more than one participant will be associated with the account to which it belongs. For privilege enforcement purposes, the number of user tracked from an account is counted by listing all the users who have rights on the projects pertaining to that account. When a user reaches the plan limit (subscription limit) for number of users tracked from his/her account, they will not be permitted to invite other users until the account is upgraded to a plan allowing more users.

The payment transactions are not directly processed by the TimeOP system, because that would require an unreasonable number of authorizations, certifications and increased security measures. Instead, transaction processing is operated by PayPal. The IPN (instant payment notification) interfacing between TimeOP and PayPal will be described in the following chapter.

5.4 PayPal Instant Payment Notification

The TimeOP system needs to interface with the PayPal system, in order to receive notification of all payments performed by users and operate the account upgrade as soon as possible. The protocol enforced by PayPal for this communication is called Instant Payment Notification. In Figure 11, the main steps involved in asking for a payment, making the payment and then confirming it is shown as a thee-way communication between user agent (browser), TimeOP and PayPal.


27

User Agent (Browser) TimeOP PayPal

Choose Subscription Plan Type

Redirect to PayPal

POST (HTTPS) PlanID to PayPal Checkout URL

Return Page with Subscription Confirmation, Price

POST (HTTPS) Name, Address, Payment Details

Redirect to TimeOP Upgrade Succesful Page

GET Success page Internal Operations

Return Success Page

POST (HTTPS) Payment Notification

OK

POST (HTTPS) Ask for Payment Confirmation

Granted

Record transaction, update account, email user

Figure 11 – The PayPal Instant Payment Notification (IPN) Protocol

The communication process involves 6 essential steps:

1. User asks for a subscription plan upgrade. 2. TimeOP securely redirects user to PayPal (via HTTPS POST), specifying the account

number for which the upgrade is performed. 3. PayPal securely collects the payment data from the user. 4. After processing the data (and charging the debit/credit card), PayPal issues a

notification to TimeOP (via HTTPS POST). 5. TimeOP reads the notification and issues a confirmation request to PayPal (via

HTTP POST). If the initial notification is legitimate (i.e. not forged by a third party), PayPal confirms it. This step is essential for guaranteeing the authenticity of the payment notifications.

6. TimeOP consequently records the transaction in its logs, upgrades the account for which the payment has been performed and notifies the user via email.

BOGDAN BOCȘE

28

5.5 Defining, Quantifying and Computing User Involvement

User involvement, in a given context, is defined as the amount of relevant interaction between the user and an application, in that context. In order to measure involvement, TimeOP tracks several components, including:

Time actively spent in an application

Time inactively spent in an application (without performing any action)

Number of clicks performed

Number of mouse wheel turns performed

Number of character keys pressed

Number of scroll keys pressed

Number of other keys pressed (control and function keys) It is however obvious that not all of these events bare the same amount of relevance.

5.6 In-house Storage and License Management

Although TimeOP has been initially designed as a Software-as-a-Service solution, there have been customer request for in-house storage. The reasons for having the TimeOP server and data store in house include:

Better data security and privacy. The TimeOP developers no longer have access to the activity data of the employees or to other privileged information.

Secure network traffic. No longer does the application network traffic (including HTTP, email) have to go through unsecure points.

Uptime and system availability. Hosting the solution on the company infrastructure brings the uptime guarantee to the internal company SLA, no longer depending on a third party hoster.

Lower response time. Having the server inside the local network decreases response time of TimeOP system, as data no longer has to travel to the TimeOP data center.

Higher capacity. The system load induced by 50 or more users is better handled by a dedicated server machine than by server shared with other users/customers.

In order to offer TimeOP as an on premises solution, licensing issues has to be considered, including total number of users allowed per deployment and the licensing period.

All data pertaining to a particular license is hashed and encrypted with a private key. The local TimeOP distribution decrypts the data with a public key and periodically checks if it matches data on the server. For more security, the license is also bound to several machine specific elements, such as motherboard key and processor key. This way, if the TimeOP is copied to another machine, a new licensing key needs to be obtained.

As any such mechanism, the licensing mechanism is not full proof. There are ways to crack it, including reverse engineering the code and removing the license verification hooks.


29

6 Testing and User Feedback

6.1 TimeOP Statistics

The chart in Figure 12 shows the number of signed up users on TimeOP.com, over 48 weeks. The sudden increase at around week 36 corresponds to a very successful social campaign we ran for TimeOP on Facebook, with over 12.000 impressions and 350 registered users during that week. As you can see, TimeOP is at 800 registered users at the time of writing this paper.

Figure 12 – The number of user signed-up on TimeOP.com, by week

The chart below shows the number of total tracked activity hours, over 48 weeks.

Figure 13 – Total tracked hours, weekly time step

0

100

200

300

400

500

600

700

800

900

0 12 24 36 48

User Sign-up on TimeOP.com

0

2000

4000

6000

8000

10000

12000

14000

16000

0 12 24 36 48

Total Hours Tracked

BOGDAN BOCȘE

30

In the chart below you’ll be able to notice the weekly number of hours of activity recorded

by TimeOP. You’ll notice that the peak values are also around week 36, during the Facebook campaign.

Figure 14 – Hours recorded every week by TimeOP

6.2 User Survey

This survey focuses on how users and consumers see the threat of interruptions, distractions, lost focus and declining productivity while using computer applications. By answering these questions, we were able to gauge how relevant the problem TimeOP solves is to end-users. One hundred and eighty one (181) TimeOP users responded to the test, with an age between 18 and 40, without any relevant gender bias. Respondents are considered to have average and above average training with using computers, online and offline applications.

Respondents were asked to answer the five questions below, by ticking an answer on a scale from 1 (one) to 5 (five). One means the respondent considers the issue does not apply to him/her or it applies to an insignificant extent. Five means that the respondent considers the issue very relevant to himself/herself.

1. How much do you think improving your focus on work-related activities would improve your

career / professional results?

(1 – very little; 5 – very much)

2. How concerned are you about spending too much time in front of the computer?


3. How much time do you spend with unproductive applications and websites?


4. Do you think knowing/visualizing how you much time you spend on unproductive

applications and website and can help improve your focus?


0

200

400

600

800

1000

1200

0 12 24 36 48

Weekly Hours Tracked


31

5. Do you think having daily/weekly information on your productivity would help you improve

it?


Respondents realize that keeping focus on work related tasks is essential for achieving satisfactory results in their careers.

89% of respondents said that

keeping focus is highly correlated with achieving good results in their professional lives.

On the other hand only 45% percent considered themselves concerned with spending too much time on their computers.

This means that people realize

that working on computers has become a critical part of their lives, both personally and professionally.

At the same time, this translates

to the fact that productivity cannot be increased by simply turning off the computer or Internet access.

0%1%

10%

34%55%

1. How much do you think improving your focus on work-related activities

would improve your career / professional results?

1-Very Little

2-

3-

4-

5-Very Much

10%

19%

26%

26%

19%

2. How concerned are you about spending too much time in front of

the computer?

1-Very Little

2-

3-

4-

5-Very Much

BOGDAN BOCȘE

32

49% of respondents consider they spent too much time with unproductive applications and websites.

This means that the lost

productivity is not only identified by business owners, but also by students and employees.

54% of users consider

visualizing how they spend their time on the computer can help them improve their focus and consequently minimize the time they spent unproductively.

This shows that the service

provided by TimeOP also serves regular employees, not only business owners and managers.

2%

20%

29%29%

20%

3. How much time do you spend with unproductive applications and

websites?

1-Very Little

2-

3-

4-

5-Very Much

11%

15%

20%31%

23%

4. Do you think knowing/visualizing how you much time you spend on

unproductive applications and website and can help improve your

focus?

1-Very Little

2-

3-

4-

5-Very Much


33

Even more respondents, 64%

consider that having information on their productivity delivered periodically (daily/weekly) can provide the support and motivation to improve one’s productivity.

This shows that even potential

TimeOP users would be motivated more by having automatic reports delivered (instead of having to actively open them).

8%

9%

19%

36%

28%

5. Do you think having daily/weekly information on your productivity

would help you improve it?

1-Very Little

2-

3-

4-

5-Very Much

BOGDAN BOCȘE

34

7 Conclusion and Further Developments

TimeOP (available at http://www.timeop.com) provides a state of the art method for monitoring workplace productivity and application usage, without restricting the rights of users or disrupting their workflow. We have shown you that our SaaS implementation makes this solution viable for small businesses, while our in-house deployment mechanism meets the particular security and availability needs of larger employers.

The project management behind envisioning and developing TimeOP focused on quick releases, agile development and continual improvement based on the feedback received. Our assertions of business requirements have been constantly reevaluated and reassessed with the help of our user base and potential customers.

By implementing a web standards compliant, mobile compatible interface, TimeOP raised the awareness of 800 users who signed up for the service. We have defined new metrics for gauging user and employee productivity and involvement as well as intuitive ways of visualizing those metrics.

Improvements are still being made to TimeOP, including a method of automatically determining the active project based on the user’s working patterns, a feature for suggesting team members for new projects based on experience in old projects, better tracking of user activity across multiple computers and more visualization for overall team performance and engagement. While the present version of the tracker is only available for Windows, a Debian Linux version is under development.

The complexity of the application, the integration of desktop and web components, optimizing the database structure and delivering an engaging user experience while keeping up with industry standards have presented remarkable challenges in delivering this project.

Figure 15 – The TimeOP logo

http://www.timeop.com/


35

F I G U R E L I S T

Figure 1 – Component overview for the TimeOP system ........................................................ 10

Figure 2 – The main screen of the TimeOP tracker ................................................................. 11

Figure 3 – Users can add new tasks directly from the tracker................................................. 11

Figure 4 – Time spent away from the computer is also tracked ............................................. 12

Figure 5 – The main view ("Dashboard") of the web application ............................................ 14

Figure 6 – Activity report, weekly, over two months .............................................................. 16

Figure 7 – Productivity reports, weekly, daily time step ......................................................... 17

Figure 8 – Automatic email report ........................................................................................... 19

Figure 9 – Overview on the TimeOP database core entities and relations ............................. 23

Figure 10 – Relations between users, accounts, plans and projects ....................................... 26

Figure 11 – The PayPal Instant Payment Notification (IPN) Protocol ...................................... 27

Figure 12 – The number of user signed-up on TimeOP.com, by week .................................... 29

Figure 13 – Total tracked hours, weekly time step .................................................................. 29

Figure 14 – Hours recorded every week by TimeOP ............................................................... 30

Figure 15 – The TimeOP logo ................................................................................................... 34

BOGDAN BOCȘE

36

B I B L I O G R A P H Y

I. Coker, B. (2009). Freedom to Surf: Workplace Internet Liesure Browsing. Melbourne.

II. G. Alonso, F. C. (2010). Web Services: Concepts, Architectures and Applications

(Data-Centric Systems and Applications). Springer.

III. HighCharts Reference. (n.d.). Retrieved from www.highcharts.com

IV. J. Chaffer, K. S. (2007). Learning jQuery: Better Interaction Design and Web

Development with Simple JavaScript Techniques. Packt.

V. Jeremy D. Zawodny, D. J. (2004). High Performance MySQL. O'Reily Media.

VI. Kompier Michiel, C. C. (1999). Preventing Stress, Improving Productivity. Routledge.

VII. L. Welling, L. T. (2008). PHP and MySQL Web Development (4th Edition). Addison-

Wesley Professional.

VIII. W3Schools. (2011, May). Browser Stats. Retrieved may 2011, from W3Schools:

http://www.w3schools.com/browsers/browsers_stats.asp

Education

TimeOP: Automated System for PC Activity Tracking and User Productivity Analysis