New Devices and Interfaces for Intuitive Manipulation of ...indigo.diginext.fr/EN/Documents/I4.4.2-MobileDevicesSOTA.V1.3.pdf · I4.4.22 - New Devices and Interfaces for Intuitive

I4.4.2 – NEW DEVICES AND INTERFACES FOR INTUITIVE

MANIPULATION OF 2D/3D MAPS

Contractual Delivery Date: 15/08/2010 Actual Delivery Date: 21/10/2010

Nature: Report Version: 1.4

Public Deliverable

Abstract

This document is the state-of-the-art about mobile interfaces for the INDIGO project.

I4.4.22 - New Devices and Interfaces for Intuitive Manipulation of 2D/3D Maps – V1.4

Grant Agreement 242341 PUBLIC Page 1

Preparation Slip

Name Company Date Signature

From F. Birot IMM 23/10/2010

Approved by

For delivery



Document Log

Issue Date Comment Author

V1.0 10/08/2010 First Draft C. Kervégant (IMM)

V1.1 22/08/2010 Second Draft C. Kervégant (IMM)

V1.2 23/10/2010 Third Draft C. Kervégant (IMM)

V1.3 23/10/2010 Final Version F. Birot (IMM)

Document Change Record

Issue Item Reason for Change

V1.2 All document Diginext part integration



Content

CONTENT ............................................................................................................................. 3

1 INTRODUCTION ............................................................................................................ 5

2 INITIAL HARDWARE CLASSIFICATION ...................................................................... 6

2.1 Ultra Mobile Devices ......................................................................................................... 6

2.1.1 Smartphone ............................................................................................................................... 6

2.1.2 Tactile Phone ............................................................................................................................. 7

2.1.3 UMPC (Ultra Mobile PC) ............................................................................................................ 8

2.1.4 Ultra Mobile Devices Pros ......................................................................................................... 9

2.1.5 Ultra Mobile Devices Cons ........................................................................................................ 9

2.1.6 Ultra Mobile Devices Recommendations ................................................................................... 9

2.2 Mobile Devices ............................................................................................................... 10

2.2.1 Tablet ....................................................................................................................................... 10

2.2.2 Tablet PC ................................................................................................................................. 11

2.2.3 Mobile Devices Pros ................................................................................................................ 12

2.2.4 Mobile Devices Cons ............................................................................................................... 12

2.2.5 Mobile Devices Recommendations ......................................................................................... 12

2.3 Classical Hardware Devices ........................................................................................... 13

2.3.1 Laptop ...................................................................................................................................... 13

2.3.2 Workstation (with mobile stand)............................................................................................... 14

2.3.3 Large Screen Mobile Stand ..................................................................................................... 15

2.3.4 Classical Hardware Devices Pros............................................................................................ 16

2.3.5 Classical Hardware Devices Cons........................................................................................... 16

2.3.6 Classical Hardware Devices Recommendations ..................................................................... 16

2.4 Advanced and Specialized Devices ................................................................................ 17

2.4.1 Mobile Tabletop ....................................................................................................................... 17

2.4.2 Large Mobile Projection Screen............................................................................................... 18

2.4.3 Mobile Control Center .............................................................................................................. 19

2.4.4 Visioconference Systems ........................................................................................................ 20

3 COMMERCIALLY AVAILABLE PRODUCTS LIST ...................................................... 21

3.1 Sensors .......................................................................................................................... 21

3.1.1 Global Positioning and Global orientation ............................................................................... 21

3.1.2 Local orientation ...................................................................................................................... 22

3.1.3 Touch Screen Systems ............................................................................................................ 23

3.1.4 Camera .................................................................................................................................... 25

3.2 Communication ............................................................................................................... 25

3.2.1 Cellular ..................................................................................................................................... 25

3.2.2 Wireless ................................................................................................................................... 26

3.3 Display technologies ....................................................................................................... 26

3.4 Integrated hardware ........................................................................................................ 27

3.4.1 iPhone 4 ................................................................................................................................... 28

3.4.2 Epic 4G .................................................................................................................................... 29

3.4.3 Galaxy Tab ............................................................................................................................... 30

3.4.4 RIM Bold 9700 ......................................................................................................................... 31

3.4.5 Evo 4G ..................................................................................................................................... 32

3.4.6 iPad .......................................................................................................................................... 33

3.4.7 Touchbook ............................................................................................................................... 34



3.4.8 Motion Computing J3500 ......................................................................................................... 35

3.4.9 EeePc T91 ............................................................................................................................... 36

3.4.10 N900 ........................................................................................................................................ 37

4 SOFTWARE ................................................................................................................. 39

4.1 Operating System ........................................................................................................... 39

4.2 Cross-Platform development ........................................................................................... 39

4.2.1 Sencha Touch .......................................................................................................................... 39

4.2.2 Titanium ................................................................................................................................... 40

4.2.3 Phone Gap ............................................................................................................................... 40

4.2.4 Rhodes ..................................................................................................................................... 40

4.2.5 Corona ..................................................................................................................................... 41

4.2.6 Unity 3D ................................................................................................................................... 41

5 MOBILE DEVICES CONCLUSION AND RECOMMENDATIONS ................................ 42

6 RELEVANT DEVICES AND CONCEPTS EXTENDED LIST ........................................ 43

6.1 PQ Labs G3 Plus ............................................................................................................ 43

6.1.1 Pros .......................................................................................................................................... 43

6.1.2 Cons ......................................................................................................................................... 43

6.1.3 Recommendations ................................................................................................................... 43

6.2 Interactive WhiteBoard eBeam ....................................................................................... 44

6.2.1 Pros .......................................................................................................................................... 44

6.2.2 Cons ......................................................................................................................................... 44


6.3 TangiSense Interactive Tabletop [KLA10] ....................................................................... 45

6.3.1 Pros .......................................................................................................................................... 45

6.3.2 Cons ......................................................................................................................................... 45


6.4 SixthSense [MM09] ......................................................................................................... 46

6.4.1 Pros .......................................................................................................................................... 46

6.4.2 Cons ......................................................................................................................................... 46


7 RECOMMENDATIONS FOR THE INDIGO PROJECT ................................................. 47

8 BIBLIOGRAPHY .......................................................................................................... 48



1 Introduction

This document deals about Human – Computer interaction systems ) that could be relevant for crisis management, either for control centers or first responders

No crisis management devices will be found in this technology review. Indeed, as other partners pointed out in their state of the art, such devices do not seem to exist. There is therefore quite some room for improvement in this area, which definitely validates the importance of this topic within the Indigo project.

This state of the art starts with a general list of hardware that could be used in such environment from ultra-mobility devices to large transportable and fix systems. The ultra-mobile part is thorough in order to define which specific capabilities could be relevant for the indigo project. The final part completes this state of the art explicating some potential devices that could bring more functionality to the control centers.



2 Initial Hardware classification

The first approach is a classification of generic potential useful devices that could be used in the Indigo project.

2.1 Ultra Mobile Devices

Ultra mobile devices could be kept inside a pocket or put on the garment of the first responder.

2.1.1 Smartphone

Smartphones are cellular phones that run Operating Systems (Windows Mobile, Symbian, Android ...) which allow the programming of custom applications. It is thus possible to add new functionalities to them, adapt them to first responders‟ needs and make them communicate with control centers.

Examples of products

- Blackberry Curve,

- Nokia E5,

- Samsung Omnia Pro B7330.



2.1.2 Tactile Phone

This kind of devices is generally an evolution of smartphones (previously cited) and runsimilar Operating Systems. The only difference is that they permit natural interaction, thanks to their tactile screens.


- Iphone,

- Android based phones,

- Symbian OS 6 based phones.



2.1.3 UMPC (Ultra Mobile PC)

These devices are based on classical embedded Operating Systems, which means that it is possible to run light applications made for PCs. .Thus, UMPCs provide a very simple way to prototype and test functionalities on field for nomad users.


- Samsung Q1,

- Sony UX50,

- Asus R2H.



2.1.4 Ultra Mobile Devices Pros

- Easy to realize applications (Java, Windows, IOS…),

- Ergonomic form factor,

- Lightweight,

- Autonomy,

- Outdoor usable,

- Almost everybody use it every day,

- Full connectivity possible (Bluetooth, Wi-Fi, 3G),

- Integrated camera for Augmented Reality,

And especially for tactile devices:

- Tactile and/or stylus capable,

- Natural interactions,

- Large fingers induce less precision.

2.1.5 Ultra Mobile Devices Cons

- Very little screen (4 or 5”),

- Some screens require bare hands to work,

- Minimal computing power,

- Reflective screens can be uncomfortable in very bright environments (outdoor).

And for non-tactile devices:

- Need to use a keyboard for interactions.

2.1.6 Ultra Mobile Devices Recommendations

This kind of devices could be used for first responder, but we need to be aware that it could be difficult for a person who wears gloves to directly interact with such small devices. Yet even if first responders won‟t be able to easily interact with it, these devices could be used for video communication between first responders and the control centers.

We also have to keep in mind that these devices are not very computing powerful. It will indeed not be possible to use it for heavy computing embedded applications.

Not every screen is really usable in bright daylight, but manufacturers now tend to include quite efficient screens in their high end units.



2.2 Mobile Devices

This type of device usually hasa larger screen than smartphones which permits to use more complete applications (large screen, more functionality).

2.2.1 Tablet

Such devices have almost standard Operating Systems (Windows7 starter, Android, ISO, …) and can run almost any application. Their main limitation is probably their weak batteries which prevent them from heavy computing.The tactile screen (minimum 7”) can provide a large, comfortable and intuitive interface.


- Fujitsu Stylistic,

- LG UX10,

- MSI Windpad,

- IPad.



2.2.2 Tablet PC

These devices are considered like classical little laptops but bring more nomad experience. It is indeed possible to hold it in one hand and interact with the other. The other interesting aspect is that it is possible to use a digitizer pen and provide a precise interface.


- Dell XT2,

- HP TX2,

- Toshiba M750.



2.2.3 Mobile Devices Pros

- Can be rugged for outdoor use,

- Full connectivity possible (Bluetooth, Wi-Fi, 3G),

- Can come with a real CPU for computation,

- Almost same Operating system as a standard computer (except iPad),

- Tactile and/or stylus

- Very precise with digitizer.

2.2.4 Mobile Devices Cons

- Needs a dedicated application for good tactile experience,

- The 3D capabilities are generally missing,

- Minimal computing power (not powerful as laptop).

2.2.5 Mobile Devices Recommendations

Even if it is sometimes possible to run PC applications on mobile devices directly, it is really important to rethink application interfaces and adapt them to each type of interaction used. Not doing so can lead to a frustration of the users and a non-acceptance of the system.



2.3 Classical Hardware Devices

These devices are the most common type of computing hardware. They are used by everybody and can be found in any computer shop.

2.3.1 Laptop

Laptops take a large place in the computer market. They‟re really useful for nomad use, and permit their owners to work on the same computer environment anywhere they go. It is also important to enlighten the fact that it is possible to change components and adapt laptops to different needs.


- Dell Precision,

- Lenovo Thinkpad,

- Sony Vaio.



2.3.2 Workstation (with mobile stand)

Any Desktop Workstation can easily be made mobile with dedicated mobile stands. They are now robust enough to provide all the comfort both in terms of ergonomics and mobility.


- Dell Precision,

- HP Z800

- Lenovo Think Centre.



2.3.3 Large Screen Mobile Stand

It is also possible to use adapted mobile stands when large screens (more than 30”) are required. Some of those stands also permit to hold the workstation.


- Mobile nesting1,

- Mobile TV Floor Stand2.

1 http://www.ergoindemand.com/67-h-mobile-nesting-tv-pole-stand-with-rotational-mount-for-37-to-61-screens.html

2 http://www.ergoindemand.com/65-mobile-tv-floor-stand-for-37-to-61-screens.html

http://www.ergoindemand.com/67-h-mobile-nesting-tv-pole-stand-with-rotational-mount-for-37-to-61-screens.html

http://www.ergoindemand.com/65-mobile-tv-floor-stand-for-37-to-61-screens.html



2.3.4 Classical Hardware Devices Pros

- Conventional workstation (Linux, PC, Mac ...),

- Can use large LCD screens,

- Can be tactilized using a layer (6.1),


- Can use tactile screens configurations (single and multitouch),

- Configuration is very flexible (any device could be put inside),

- Real Computer with any application,

- Full connectivity possible (Bluetooth, wifi, 3G),

- Some very powerful configurations are available,

- 3D Graphic cards are available.

2.3.5 Classical Hardware Devices Cons

- Need power outlet (not for laptop),

- Definitely not mobile during use,

- Heavy weight,

- Not designed for outdoor use,

- Tactile, even if enabled, is not usable in mobility (poor stability in this configuration for laptop),

- Mobile workstations could lose their mobility (heavy weight, size, heating).

2.3.6 Classical Hardware Devices Recommendations

Even though laptops are mobile workstations, they‟re only transportable and must be considered as non-moving system in use-case (i.e. easy to move but difficult to interact while moving). Conventional workstations (either with large screen or not) can handle very powerful configurations (high 3D capabilities or computing performances) but need power outlets and are unusable while moving.



2.4 Advanced and Specialized Devices

These devices are commercially available advanced and specialized hardware that could be relevant for the Indigo project. We chose to extend the state of the art to such devices that can be moved on-site thanks to dedicated vehicles (i.e. trucks), since early discussions with end-users showed that a trade-off will have to be found between mobility and functionalities.

2.4.1 Mobile Tabletop

Tabletops are extraordinary devices for information and application sharing. They also provide good interface for widget manipulation and exploration.


- Immersion IlIght Table3,

- TangiSense4,

- Microsoft Surface5.

Pros

- Full multitouch support,

- More than touch sensitive capable (hands, patterns),

- Projection System (scalable),

- Intuitive interaction and collaborative visualization.

Cons

- Heavy weight, need adaptation for easy move,

- Need power outlet,

- Non usable outside (Projection and detection).

Recommendations

Such systems are known to be efficient in case of collaborative multiuser setup for sharing information between multiple decision makers (very natural interactions, tactile). Furthermore, it is especially suitable to display 2D / 3D maps since it replicates both the interaction and visualization to which everyone is used with classical paper maps.

3 http://ilight-immersion.blogspot.com/

4 http://multicom.imag.fr/recherche/spip.php?article26

5 http://www.microsoft.com/surface

http://ilight-immersion.blogspot.com/

http://multicom.imag.fr/recherche/spip.php?article26

http://www.microsoft.com/surface



2.4.2 Large Mobile Projection Screen

Large screen are very useful but take lots of space. The following products could yet be stored in small rooms when not used. Some products could also be tactilized with the immersion IlIght technology and be used in the control centers during crisis.


- Immersion‟s customizable mobile screen projection system,

- Nobo Mobile Projection Screen6.

Pros


- Scalable System (Immersion‟s solution),

- Can be tactilized using IlIght.

Cons

- Not mobile during use,

- Needs power outlet,

- Needs calibration for projection (a solution could be found for automatic calibration),

- Not usable outside (Projection and detection).

Recommendations

This projection system is very powerful in order to realize large visualization or interactive walls. Easy to move and put away, but requires a calibration to use it after moving. This is a great solution for using very large screens and saving space.

6 http://www.amazon.co.uk/Nobo-Projection-Castors-W1500xH1000mm-33938440/dp/B000J6BYCI

http://www.amazon.co.uk/Nobo-Projection-Castors-W1500xH1000mm-33938440/dp/B000J6BYCI



2.4.3 Mobile Control Center

Crisis control centers must have nomad parts to be in contact with crisis and support first responders. The solution consists in deploying useful crisis hardware devices inside a dedicated truck. They can integrate multiples numeric, communication and analysis interfaces.


- Lynch7,

- Moblie Control Center8.

Pros

- Can integrate almost all of the above hardware systems,

- Traveling the system around a large geographical zone.

Cons

- Not mobile during use,

- Needs power management if no power source is available (powered by batteries or generators).

Recommendations

This type of vehicle can integrate any kind of electronic devices, including previously mentioned hardware, in order to get an office or a command center deployed in very short time. Such solutions may provide a good trade-off between functionality and mobility. This would indeed allows to embark feature-full relevant but cumbersome hardware that would still be movable on-site.

7 http://www.lynchexhibits.com

8 http://www.meridiansv.com

http://www.lynchexhibits.com/

http://www.meridiansv.com/



2.4.4 Visioconference Systems

Such systems allow several people to organize a meeting between distant places just as if they were at the same place. The best systems try to simulate important clues such as eye contact and human scale visualization to increase the presence feeling (telepresence).


- Cisco TelePresence 30009,

- TPT400010.

Pros

- Provides meeting information sharing without travelling,

- Can connect multiple headquarters.

Cons

- Need broadband connection,

- Could need large space.

Recommendations

This kind of system needs space, but are known to be very efficient to bring distant sites close to each other. Combined with the mobile centers cited above, they could be a fantastic addition to the crisis control hardware user interface to allow people in distant control centers to collaborate closely.

9 http://www.cisco.com/en/US/prod/collateral/ps7060/ps8329/ps8330/ps8333/ps7087/product_data_sheet0900aecd80543f46_ps8333_Products_Data_Sheet.html

10 http://www.telepresencetech.com/prod/Telepresence4000.html

http://www.cisco.com/en/US/prod/collateral/ps7060/ps8329/ps8330/ps8333/ps7087/product_data_sheet0900aecd80543f46_ps8333_Products_Data_Sheet.html

http://www.telepresencetech.com/prod/Telepresence4000.html



3 Commercially available products list

Modern mobile devices require fast and easy worldwide communication. Not only the data is gathered one way from the internet for example, but also the device has to communicate local information such as localisation in order to provide the service required. One of the best known services that necessitate such communications is maps, where the trajectory computation can be achieved on distant computer while the mobile device provides localisation and display features. This is done with sensors as described in Section 3.1. The communication means are described in Section 3.2. Section 3.4 shows high-end mobile devices giving pros and cons for the use in the INDIGO project.

3.1 Sensors

3.1.1 Global Positioning and Global orientation

GPS stands for “Global Positioning System”11. This service provides localisation information with the help of satellites. The accuracy is today from 5 to 15 meters for the mass market. There is no specific mobile hardware that can improve this accuracy, except changing satellites. Indeed, the European satellite Galileo, to be launched in 2010, coupled with EGNOS, is supposed to refine the localisation with a horizontal accuracy estimated less than 2 meters.

Other GPS-like systems exists (GLONASS, Beidou), but either they provide less accuracy, are confined to one country (e.g. China) or a still in development (India, Russia).

GPS accuracy can be enhanced using Differential GPS12. Fixed antennas are used to improve the accuracy, which for the best systems can be fewer than 10 cm. Nonetheless, this requires specific antennas on the field; hence this system is not suited for “world-wide” mobile devices.

For a mobile device, the first computation of the localisation can last several minutes because some data need to be downloaded from the satellites. Hence, most mobile devices use A-GPS (Assisted GPS13), which can reduce downloading time to 10 seconds using an internet connection. The following pros & cons only refer to A-GPS since available in most if-not-all high-end mobile devices.

Pros

In 2010, without the use of additional sensors, this is the most accurate global positioning technology;

Low-cost hardware;

Will be improved with new satellites, no hardware to change.

Cons

Build from and for the USA Army. They control the technology, and decide the accuracy they offer to the mass market. Before the year 2000, they restricted the accuracy to 100 meters;

This technology has flaws. Storms, temporal alignment of satellites, electrical field are some incidents that may affect the GPS system, which then may provide wrong localisation. Then, one cannot fully rely on the system and must be careful;

Does not work indoors.

11 http://en.wikipedia.org/wiki/Global_Positioning_System

12 http://en.wikipedia.org/wiki/Differential_GPS

13 http://en.wikipedia.org/wiki/Assisted_GPS

http://en.wikipedia.org/wiki/Global_Positioning_System

http://en.wikipedia.org/wiki/Differential_GPS

http://en.wikipedia.org/wiki/Assisted_GPS



Recommendations

A-GPS is definitely a must in the case of INDIGO for the localisation of first responders. The accuracy seems relevant outdoors. Unfortunately, this technology does not work indoors; hence, in this particular case, another technology should be used, such as Wi-Fi or cellular localisation. This feature is available on a few mobile devices.

Compasses14 are used to indicate the north direction, that is, global orientation. In mobile devices, compasses are solid state electronics built out of two or three magnetic field sensors. The data is sent to the processing unit which computes the device‟s heading using trigonometry.

Compass pros

Mandatory to indicate the north for maps orientation;

Low-cost hardware.

Compass cons

More stable around the equator (not the case for Europe);

May provide wrong direction when accelerated in cars for example;

Lags when turning from east to west.

Compass recommendations

This sensor is a must to provide the device user information regarding his global orientation. Maps may be aligned to the north or aligned to the user‟s orientation.

3.1.2 Local orientation

There exist two systems in mobile devices used to indicate the local orientation of the device. The first system makes use of a 3-axis accelerometer15, often coupled with a compass. The second system includes a gyroscopic16 sensor which captures motion along six degrees of freedom.

In mobile device usage, these sensors determine how the user carries the phone, compared to an initial reference. A typical application is the rotation of the device in landscape or portrait mode. Another practical application for accelerometers is to detect crash-strength G-forces and call for assistance unless manually cancelled.

Accelerometer pros

Useful to display information regarding device holding;

Low-cost hardware.

Accelerometer cons

Middle-class accuracy orientation;

14

http://en.wikipedia.org/wiki/Compass 15

http://en.wikipedia.org/wiki/Vibrating_structure_gyroscope 16

http://en.wikipedia.org/wiki/Accelerometer



Accelerometer recommendations

This sensor has definitely an added value for the INDIGO project in order to align the displayed map on the mobile device with respect to the user‟s orientation. The middle-class accuracy positioning seems to be sufficient for most applications.

Gyroscopic sensor pros

Useful to display information regarding device holding;

High-class accuracy positioning;

Low-cost hardware.

Gyroscopic sensor cons

More expensive than accelerometers.

Gyroscopic recommendations

The gyroscopic sensor provides a better positioning accuracy and thus is very appealing. While such accuracy might not be required at a first glance for the INDIGO usage, it could prove to be useful not to worry about the sensor accuracy.

3.1.3 Touch Screen Systems

The market trend of mobile device is to have the screen as a user input interface, that is, a “touchable screen”. This one touch feature has a major usage drawback where the user may not either “tap” two letters on a virtual keyboard. Also, the screen technology would require a special device, that is, a stylus. Thus, high-end mobile devices have multi-touch -with fingers- screens, leading to a new era of user-interfaces. Multi-touch refers often to the ability to register at most three simultaneous contacts.

Different touch screen technologies exist. The most popular are the capacitive17 and the resistive18 approach. The table 1 Comparison table of Touch Screen technologies19 is an overview comparison of the most used technologies.

17

http://en.wikipedia.org/wiki/Capacitive_sensing 18

http://en.wikipedia.org/wiki/Resistive_touchscreen 19

http://en.wikipedia.org/wiki/Capacitive_touchscreen



Comparison table of Touch Screen technologies

Technology 4-Wire

Resistive

Surface Acoustic

Wave 5-Wire Resistive Infrared Capacitive

Durability 3 years 5 years 5 years 5 years 2 years

Stability High High High High OK

Transparency Bad Good Bad Good OK

Installation Built-in/Onwall Built-

in/Onwall Built-in/Onwall Onwall Built-in

Touch Anything Finger/Pen Anything Finger/Pen Conductive

Intense light-resistant

Good Good Good Bad Bad

Response time <10ms 10ms <15ms <20ms <15ms

Following Speed

Good Low Good Good Good

Excursion No Small Big Big Big

Monitor option CRT or LCD CRT or LCD CRT or LCD CRT or LCD CRT or LCD

or LED

Waterproof Good Ok Good OK Good

1 Comparison table of Touch Screen technologies

Moreover, in mobile phones, touch screens are always enhanced with proximity sensors; they indicate the proximity of the user and deactivate the screen. This is achieved using an electromagnetic radiation or electrostatic field, and looks for changes in the return signal. A typical usage is to deactivate the screen when the user is having a phone conversation and holds the phone close to his ears; the ear touches the screen but does not interact with the phone. Moreover, as the screen is turned off, this mechanism saves battery.

The following pros & cons only refer to the capacitive and resistive technologies since available in most mobile devices.

Resistive touch screens Pros

Allows intuitive interaction with the device;

With multi-touch screens, maps interaction is made simple;

Can be used with gloves;

Exists with rugged devices.

Resistive touch screens Cons

Fingers or stylus hide part of the visual elements;

Touching the screens leaves dirt and needs cleaning.

http://en.wikipedia.org/wiki/Electromagnetic_field

http://en.wikipedia.org/wiki/Electrostatic



Capacitive touch screens Pros

Allows intuitive interaction with the device;

With multi-touch screens, maps interaction is made simple;

Capacitive technology exists with LED screens, which are more power-friendly;

Exists with rugged devices.

Capacitive touch screens Cons

Fingers or stylus hide part of the visual elements;

Touching the screens leaves dirt and needs cleaning.

Recommendations

For the INDIGO projects, it is more suitable for the first responders and/or the crisis managers to interact with their fingers rather than using a small and fragile stylus. Moreover, first responders might wear gloves which forbid the use of capacitive touch screens because most gloves and styluses prevent the necessary electrical conductivity. The 5-Wire Resistive technology seems the most appropriate.

3.1.4 Camera

Modern mobile device include two cameras, one facing front and a rear camera. This is very useful if one wants to take photos (rear camera) and when one wants to do video conferencing (front camera). Having only one camera implies a rotation of the mobile device for video conferring which then forbids the visualisation of the screen.

There is no need for pros and cons for camera; videoconferencing between first responders and crisis managers could be an important added value for the INDIGO project. Therefore, two cameras are mandatory. While videoconferencing is still a real challenge, even more when mobile, this might not be possible within the time interval of the project. But looking ahead in the future, it could be an everyday use.

In any case, at least one camera is mandatory to allow one to take photos from the field and transmit the information to the Common Operational Picture.

3.2 Communication

3.2.1 Cellular

Below is a list of the most popular cellular network:

UMTS/HSDPA/HSUPA (850, 900, 1900, 2 100 MHz)

GSM/EDGE (850, 900, 1 800, 1 900 MHz)

3G

4G

A state of the art of the cellular networks is beyond the scope of this document. Nonetheless, it is important to note that they cover much of the earth. 3G networks enable fast wireless communication of data. 4G is even faster, but is not today yet available. High end mobile devices claim to be 4G compatible, even though there is no network to rely on.

For the INDIGO project, 3G seems the most appropriate, since it allows phone and data communication. Unfortunately, the bandwidth is a bit too small to allow videoconferencing. Thus, vendors propose this functionality only when using a Wi-Fi network.

http://en.wikipedia.org/wiki/Stylus_%28computing%29



3.2.2 Wireless

The two most popular wireless networks are listed below:

Wi-Fi a/b/g/n

Bluetooth

The state of the art of wireless networks is beyond the scope of this document. Nonetheless, it is important to highlight the main limitation of the Bluetooth technology. Most of the mobile devices include a Bluetooth range which is at most 20 meters. This means peer to peer data communication within that range. At a first glance, this peer to peer aspect does not seem relevant for the INDIGO use.

To access internet using a non-cellular wireless network, most if-not-all mobile software use Wi-Fi. This is then the wireless network technology to use, if available, for the INDIGO project.

3.3 Display technologies

The two main display technologies available on mobile devices are:

TFT-based

LED-based

TFT was the only technology used before LED screens were affordable.

Thus this section is a comparison between TFT, LED and AMOLED (LED-based).

LED Pros (compared to TFT)

Better brightness of the screen;

Better viewing angle;

Less power consumption;

Vividness of and colours and blacks.

LED Cons (compared to TFT)

Sunlight legibility;

Lifespan;

More expensive, even though on a small screen, the difference is negligible.

AMOLED Pros (compared to TFT)

Same as LED;

Sunlight legibility.

AMOLED Cons (compared to TFT)

Lifespan;

More expensive, even though on a small screen, the difference is negligible.



Recommendations

LED/AMOLED screens are better than TFT screens. They are thus recommended for the INDIGO project. An important feature of the AMOLED display technology is the sunlight legibility, which makes the screen visible even in shiny conditions. For first responders on the field, this feature is almost mandatory.

3.4 Integrated hardware

This section lists high-end mobile devices which could be used for the INDIGO projects, showing the pros and cons. Also, table 2 Mobile Device Hardware Comparison shows a comparison of the hardware features of these mobile devices. The list hereunder shows the state of the art mobile devices available (or soon) on the market.

iPhone 4 (Apple);

Epic 4G (Samsung);

Galaxy Pad (Samsung);

RIM Bold 9700 (BlackBerry);

Evo 4G (HTC);

iPad (Apple);

Touchbook (Always Innovating);

Motion Computing J3500 (Motion Computing);

EeePc T91 (Asus) ;

N900 (Nokia).

The pricing of each device is not indicated because the market is rapidly changing and the price varies across Europe. Moreover, all the devices except the Motion Computing J3500 (due to rugged casing property) are in the same price range.



3.4.1 iPhone 4

The iPhone devices, produced by Apple, are probably the most popular phone on the mass market. Besides the company‟s marketing force, this mobile device includes very sophisticated technologies which are often publically available before rivals. For example, the latest model, the iPhone 4, is the only mobile phone including a gyroscopic sensor.

Part of what makes the iPhones perform so well is that Apple controls hardware and software. Thus, they always show applications exploiting the device on the first day available.

Moreover, Apple takes much care in the screen quality and user interface; hence the device shows clear information and intuitive usage.

Pros

Good hardware for accurate localisation;

Good hardware for accurate positioning;

High quality screen;

Multi-touch screen

Front and rear cameras;

Powerful graphics capabilities;

Fits in the pocket.

Cons

Capacitive Touch screen, does not work with gloves;

Middle screen size;

Wi-Fi only video Conferencing;

Average batteries.

Recommendations



Besides the middle-sized capacitive touch screen, this device includes useful and accurate hardware for maps manipulation. The gyroscopic is a real added value for accurate orientation sensing. Moreover, the graphics possibilities are high with the powerful embedded graphical processor. It could be possible to display 3D content in the palm of one‟s hand. This device is thus the state of the art mobile device that could definitely be used in the INDIGO project.

3.4.2 Epic 4G

Pros


Multi-touch screen


3G/4G Video Conferencing;


Integrated keyboard;

Fits in the pocket.

Cons


Middle screen size;

Average batteries.

Recommendations

This device is comparable to the iPhone 4, besides a screen and a battery capacity a bit bigger. An added value for the INDIGO project is the video conferencing feature on Cellar networks which could connect first responders on the field to the decision makers elsewhere. Also, some users prefers a physical keyboard, here one has the choice between a virtual and a physical keyboard.



3.4.3 Galaxy Tab

Pros


Middle screen size;

Multi-touch screen;



Good batteries;

Fits in the pocket.

Cons


Recommendations

This device is categorized as a tablet because of the size of the screen. Even though big, it seems that this device could fit in one‟s pocket. This device has appealing features, even though there does not seem to be Cellular nor Wi-Fi indoor localisation.



3.4.4 RIM Bold 9700

Pros

Resistive touch screen, compatible with gloves;


Fits in the pocket.

Cons

Small screen size;

Poor graphics capabilities;

No position sensor;

LCD Screen;

No video conferencing;

Average batteries.

Recommendations

This device has a resistive screen which could be used on the field with gloves. Besides that, the other features are not interesting; hence this device is not recommended.



3.4.5 Evo 4G

Pros

Multi-touch screen


3G/4G Video Conferencing;


Fits in the pocket.

Cons


LCD display;

Middle screen size;

Average batteries.

Recommendations

This device is comparable to the iPhone 4, besides a screen and a battery capacity a bit bigger. An added value for the INDIGO project is the video conferencing feature on Cellar networks which could connect first responders on the field to the decision makers elsewhere.



3.4.6 iPad

This device, which is not a computer neither a phone, is defined as a “tablet”.

Pros


Big screen size;

Multi-touch screen



Good batteries.

Cons

No camera;

Does not fit in the pocket;


No phone calls;

Capacitive Touch screen, does not work with gloves.

Recommendations

This tablet has a big but mobile screen that can display much information while staying mobile. Moreover, the great battery capacity makes this device definitely interesting for long exercises on the field. Unfortunately, the lack of camera and phoning capability will limit the communication between first responders and crisis managers to “data” exchange. If “on the field photos”



exchange, video conferencing and audio conversation are mandatory for the INDIGO project, then this device is not recommended.

3.4.7 Touchbook

This net-top has a removable screen which then turns into a tablet.

Pros

Can be used as a tablet or a net-top

Big screen size;



Internal USB connectors (GPS, …);

Magnetic back-screen;

Good batteries;

Optional keyboard.

Cons

One touch screen;

Poor quality screen;

No camera;

Poor positioning;



No phone calls by default;



Recommendations

This net-top has an attractive concept with a removable screen. The battery life is also interesting. Unfortunately, the poor positioning capabilities, the lack of camera makes this device not so appealing for the INDIGO project.

3.4.8 Motion Computing J3500

The rugged property of this device makes it more professional oriented.

Pros

Big screen size;

Dual-touch screen;



Good batteries;

Rugged;

Optional keyboard;

Optional phoning;

Optional GPS.

Cons

Poor positioning;

No front camera;


Expensive;

No video conferencing.



Recommendations

Besides being rugged, this device shows attractive battery life. The possibility to have optional GPS, keyboard and mobile broad band capabilities is definitely interesting. Unfortunately, there is no compass no front camera. Also, this device is expensive, mainly because it is rugged.

3.4.9 EeePc T91

This net-top has a screen that can be turned and put over the keyboard, transforming it into a tablet.

Pros

Big screen size;

Dual-touch screen;


Resistive touch screen;

Good batteries;


Optional phoning;

Optional GPS.

Cons

Poor positioning;

No front camera;

Recommendations

This net-top computer has attractive features. The resistive touch screen is definitely a plus. The lack of a compass might be an issue.



3.4.10 N900

Pros


Multi-touch screen;





Fits in the pocket.

Cons

Small screen size;

Middle quality screen;

No video Conferencing;

Average batteries.

Recommendations

This device is comparable in features with the iPhone 4, but includes an integrated keyboard and most of all a resistive touch screen. This could be the mobile device to be used for first responders.


Grant Agreement 242341 PUBLIC Page 38 2 Mobile Device Hardware Comparison

Device Name Operating

System

Cellular

Networks

Wireless

Networks Location Position

Screen

Size (cm)

Touch

technology

Screen Keyboard Camera

Teleconferencing

Graphics

Chip Battery

iPhone 4 iOS 4 GSM/EDGE

3G

Wi-Fi b/g/n

Bluetooth 2.1

A-GPS

Wi-Fi

Tri-band cellular

3-axis gyro,

3-axis accel

Compass

8.9

960x640

Capacitive

LED IPS No

Rear : 5MP

HD Video recording

Front: 0.3MP

Wi-Fi only Yes 1200mAH

Epic 4G Android 2.1 3G, 4G, CDMA CDMA2000, EVDO

Wi-Fi b/g/n

WiMAX

Bluetooth 2.1

A-GPS 3-axis accel

Compass

10.2

480x800

Capacitive

AMOLED Yes

Rear: 5MP

HD Video recording

Front 0.3 MP

Yes Yes 1500mAH

Galaxy Tab Android 2.2 3G Wi-Fi b/g/n

Bluetooth 2.1 A-GPS

3-axis accel

Gyroscope

Compass

17.8

1024x600 Capacitive AMOLED None

Rear: 3MP

Front: 1.3 MP Yes Yes 4000mAH

RIM Bold 9700 BlackBerry OS

3G, EDGE, GPRS, GSM

Wi-Fi b/g

Bluetooth 2.1 A-GPS NC

6.2

480x360 Resistive TFT LCD Yes 3.2 MP No No 1500mAH

Evo 4G Android 2.1 3G, 4G, CDMA CDMA2000, EVDO

Wi-Fi b/g

WiMAX

Bluetooth 2.1

A-GPS 3-axis accel

Compass

10.9

480x800 Capacitive TFT LCD No

Rear: 8MP

HD Video recording

Front: 1.3MP

Yes Yes 1500mAH

iPad iOS 3.2.2 Optional 2G, 3G (Data only, no phone calls)

Wi-Fi b/g/n

Bluetooth 2.1

Optional A-GPS

Skyhook Wireless

Cellular network

3-axis accel

Compass

24.6

1024x768

Capacitive

LED IPS Optional None No Yes 6700mAH

Touchbook TouchOS (Linux)

Optional 3G Wi-Fi b/g/n

Bluetooth 2.1 Optional GPS 3-axis accel

22.6

1024x600 Resistive TFT LCD Optional None No Yes

Screen: 6000 mAH Keyboard: 12000 mAH

Motion Computing J3500

Windows 7

Windows XP

Optional Mobile Broadband

Wi-FI a/b/g/n

Bluetooth 2.1 GPS Optional accel

30.7

1280x800

Capacitive Dual-touch

LED AFFS+

Optional 3 MP Wi-Fi Yes 2x2000mAH

EeePc T91MT Windows 7 External 3G Wi-FI b/g/n

Bluetooth 2.1 External GPS NC

22.6

1024x600 Resistive LED Yes 0.3 MP Wi-Fi NC NC

N900 Maemo 5 3G, EDGE, GPRS, GSM

Wi-Fi b/g

BlueTooth 2.1

A-GPS

Cell-based receivers

3-axis accel

Compass

8.9

800x480 Resistive TFT LCD Yes

Rear: 5MP

Front: 0.3 MP No Yes 1320mAH

I4.4.2 – New Devices and Interfaces for Intuitive Manipulation of 2D/3D Maps – V1.4


4 Software

4.1 Operating System

The list below shows the most used and popular operating systems on the market:

iOS

Android

Windows 7

Symbian

MeeGo (the future of Maemo and Moblin)

BlackBerry OS

Each operating system has its particularities. This state of the art does not focus on one operating system. The reason is that each operating system requires specific developments. Moreover, since the mobile market is rapidly changing, one needs to follow the operating system evolutions to adapt his software. Instead, it would be preferable to use cross-platform libraries, which enables the development of one application deployable on every mobile device.

However, it is important to note that the Android and MeeGo operating systems are open source, and that more and more mobile device Vendors have adopted these technologies. For example Nokia, which has bought Trolltech to enhance the user interfaces, has joined Intel for the development of MeeGo. Also, the iPhone/iPad software development kit has some limitations regarding developments; firstly, one needs to develop on the Mac OS X platform with the XCode development environment; secondly, to upload a program on the mobile device, one needs to use iTunes.

Indeed, most rugged devices are computers and run on the Windows 7 and XP operating systems.

4.2 Cross-Platform development

This section covers a few cross platform libraries that could be used to develop INDIGO software.

4.2.1 Sencha Touch

Pros

Javascript, HTML and css;

Commercial support.

Cons

GPL open source;

Only iOS and Android;

No sensor access;

No 3D (OpenGL | ES) module;

Not native.



Recommendations

This software development kit offers a simple way to develop cross platform applications with widely known languages. Unfortunately, the lack of hardware access makes this library unsuitable for the INDIGO needs.

4.2.2 Titanium

Pros

Native applications;

JavaScript language;

Hardware access (compass, GPS, camera);

Social sharing (facebook, twitter, email, …);

Open source Apache 2.0 licence;

Can build applications for computers;

Commercial support and training services.

Cons

No 3D (OpenGL | ES) module.

Recommendations

Multi-platform development environment, native application, javascript based, export to mobile and non-mobile devices, all major mobile device supported, open source, access to device hardware, all these features makes this software development kit definitely the tool for the INDIGO project. The lack of a 3D module could be an issue.

4.2.3 Phone Gap

Pros


Sensor access (compass, GPS, camera);

Open source MIT licence;

Major smartphones supported;

Commercial support and training services.

Cons

No 3D (OpenGL | ES) module;

Web applications, non-native applications.

Recommendations

Despite the fact that the application programmable interface allows access to the hardware, the fact that this produces web applications – non-native applications – can lead to slow execution of the software.

4.2.4 Rhodes

Pros

Transform into native application;

Support for all major smartphones;

Sensor access (Camera and GPS);

PIM card control;



Free and MIT open source;

Commercial support;

Cons

No 3D (OpenGL | ES) module.

Recommendations

This software development kit is a real candidate for the INDIGO mobile developers.

4.2.5 Corona

Pros



Sensor access (compass, GPS, camera);

OpenGL | ES module.

Cons

iOS and Android support only;

non-free $99, none open-source;

Mac OS X development environment only.

Recommendations

Compared to the Titanium library, Corona offers the ability to develop 3D applications. If this feature is mandatory, Corona could be used for INDIGO. This software is unfortunately not free and not open source. The Mac OS X development environment only is also constraining for software developers.

4.2.6 Unity 3D

Pros



OpenGL | ES module;

Web browser plugin for Windows and MacOS.

Cons

No sensor access (compass, GPS, camera);

iOS and Android support only;

not free;

Recommendations

This is a game development environment. This cross-platform library is available on Windows, Mac OS, on the Nintendo‟s Wii and now goes mobile with iOS and Android. Unfortunately, the lack of sensor access makes this software not very suitable for the INDIGO project.



5 Mobile devices conclusion and recommendations

Mobile devices are fantastic technologies. They gather hardware that most computers do not include. Moreover, applications are user friendly and enable users to communicate easily across many networks. What is the future of all this? High-end mobile hardware devices are more or less equivalent. The small differences doesn‟t make any big gap at the end. Location and position information are pretty much in place, with GPS being improved from the satellite perspective.

Gaming is a hot topic and an important market share industry. User interfaces must be appealing to catch one‟s eye. Let‟s not forget that in average we change our mobile device every 4 years.

Many industrial visionaries predict that the future is graphics. Hence much attention is given to the graphics processor, in terms of capabilities and power consumption. Moreover, the screen size and display properties also evolve thanks to this graphic trend [CPA08].

What perfect device does INDIGO need in terms of on-the-field visualization? The answer is tricky since no perfect devices are on the market yet. A good candidate could for example be the Galaxy Tab with a resistive touch technology, an integrated keyboard and indoor localization when possible.

It is the same issue for the development library. The ideal one would be a cross-platform development environment coupled with a gaming library, for example the Titanium library with a 3D module.

Picking out what is best and available on the market, the

N900

Galaxy Tab

iPhone 4

are serious candidates, coupled with the Titanium library, this could be the chosen ones for the INDIGO project.

Despite all their advantages, those devices are nevertheless dedicated to a single user and take all their meaning when used on the field. They indeed offer smaller screens, have to be always hold, and are based on inputs that are also reduced in terms of size (compared to a standard keyboard for instance). Other devices and hardware interfaces may therefore be required on the control center side to propose a more collective and comfortable visualization and interaction.



6 Relevant devices and concepts extended list

6.1 PQ Labs G3 Plus

This device is a multitouch layer that you can put on a classical LCD Screen or a rear projection system. Available from 32” to 65” and custom sizes.

6.1.1 Pros

- Can be used with all non-tactile screens,

- Detects contacts on the surface,

- USB Device, almost plug‟n‟play,

- Windows 7 ready multitouch.

6.1.2 Cons

- Some touch configurations (2 fingers on the same column or row) could confuse the system,

- Not directly in contact with the display, the glass layer is around 6mm thick,

- Detects before contact,

- Need to create a mounting kit.

6.1.3 Recommendations

Very easy to place over a screen and use it natively with Windows 7, or use the SDK to use the multitouch technology in your own application.



6.2 Interactive WhiteBoard eBeam20

This is a small device that allows interactions on a white projection screen when it is fixed at the corner of the display.

6.2.1 Pros

- Very simple to use,

- Works with classical projection screen and computers,

- Option for using on a real whiteboard with content copied on computer screen.

6.2.2 Cons

- Need a projection setup, but maybe could be used on a large LCD screen,

- Can only interact with eBeam Markersleeve or eBeam Stylus,

- No Multitouch.


A great setup could be a large LCD screen or projection system, protected with an acrylic glass, coupled with eBeam Marker and stylus to interact at two different levels - real tracing provide information with discrete feedback, and virtual tracing interact with the GUI.

20

http://www.speechi.net/us/

http://www.speechi.net/us/



6.3 TangiSense Interactive Tabletop [KLA10]

This is an interactive tabletop system that integrates an advanced RFID location system to track tangible objects on the surface.

6.3.1 Pros

- Can track tangibles objects,

- Interactive LED display.

6.3.2 Cons

- Need a direct projection setup for better resolution and image quality,

- No touch capabilities,

- Pretty slow detection,

- Changes in the scale factor may affect the props/map association.


Very Interesting device that allows to interact with tangibles objects, the missing tactile part can be fixed with eBeam (6.2) or PQ Labs G3 (6.1) and could bring to the user a good feeling in interaction and a direct projection system for visualisation.



6.4 SixthSense [MM09]21

„SixthSense‟ is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

6.4.1 Pros

- Wearable,

- Augmented reality.

6.4.2 Cons

- Outdoor not possible, projection needs darkness,

- Need to wear equipment for interaction.


Great concept as “Minority Report”, a prototype could be developed and improved for the project.

21

http://www.pranavmistry.com/projects/sixthsense

http://www.pranavmistry.com/projects/sixthsense



7 Recommendations for the INDIGO project

Our first exchanges led us to understand that there are several levels of mobility involved in the Indigo system. On the one hand, people on the field need ultra-mobile devices to be able to read crisis or simulation-related information. On the other hand, the people in the control centers need feature-full hardware interfaces that have the capability to be moved on-site.

A solution could be a tablet as soon as ultra-mobility tasks are required. Specific devices have already been identified in the previous pages, while new devices are announced frequently nowadays.

Tactile tables that are capable to be embarked into mobile control centers seem a good solution for the control center interface issue. They may have to be completed with vertical screens to ease the communication sharing with the people around. Besides the mobility issue, collaboration is indeed a strong issue to be taken into account. While the large tactile tables seem a strong step in that direction, our initial user requirements exchanges and state of the art highlighted that telepresence solutions may be the best way to bring together commanders that are on different sites through the mobile control centers. It remains to be found how those different solutions can be integrated together.



8 Bibliography

[CPA08] The State of the Art in Mobile Graphics Research. T. Capin, K.Pulli, T. Akenine-Möller. IEEE

Computers and Graphic Applications, vol 2,8 p74-84, 2008.

[KLA10] Manipulation d’objets tangibles et/ou virtuels en lien avec la table TangiSense. S. Kubicki, Y.

Lebrun, E. Adam, C. Kolski, S. Lepreux and R. Mandiau. To appear in Workshop of Forum sur l‟interaction tactile et gestuelle, 2010.

[MM09] SixthSense – A Wearable Gestural Interface. P. Mistry and P. Maes. To be appeared in

SIGGRAPH Asia 2009, Sketch. Yokohama, Japan. 2009.