Make Smartphones Last A Day: Pre-processing

Based Computer Vision Application Offloading

Jiwei Li*, Zhe Peng*, Bin Xiao*, Yu Hua**

The Hong Kong Polytechnic University*Huazhong University of Science and Technology**

Outline• Background & Motivation• Pre-processing Based Offloading• Evaluation• Conclusion

Background & Motivation

Computer Vision Applications

Object Recognition

3D Reconstruction

Augmented Reality

Power hungry Compute intensive

Offloading Techniques• To tackle limitations of smartphones• Limited computational power• Limited battery life

• Offloading compute-intensive jobs to remote servers via wireless networks• Improve response time• Improve energy efficiency

Offloading Image Processing

Original Image (8 Mb)

Processed Image(8+ Mb)

Transferring images consumes a lot of energy.

Original Image (8+ Mb)

Original Image (2+ Mb)

Pre-processing helps

Processed Image(2+ Mb)

Why pre-processing?• Less active time on the network interface• Save energy

• Drawbacks• Compromise the image processing accuracy• Pre-processing incurs energy cost

Challenges• Need to know how exactly pre-processing will

affect the energy consumption and final processing accuracy• Important for determining the right pre-processing level

• Need to work at both static and dynamical environments. • Predication-based techniques should be avoided

Motivating Experiments• Understand the effect of pre-processing

• Face detection experiments:• We adopt downscaling as the pre-processing technique• Correct detection rate, false positive rate• File size

Effect of Downscaling

As the downscaling factor decreases from 1 to 0.1:• the average correct detection rate drops• the average false positive rate drops• the file size drops linearly

Smaller image file sizes indicate lower result accuracy.

The Studied Problem• Energy consumption as a constraint• E.g., one day

• Overall response time as a constraint

• Problem: maximize the final processing accuracy subject to energy and time constraints

People charge their phones every day

Pre-processing Based Offloading

Overview

Two decisions:• When to start data transmission via which network interface• Determine the pre-processing level

Decisions are made based on current channel information and accumulated delay time.

Single Task Offloading• Calculate the condition check deadline• Determine when to perform condition check

periodically (two parameters: D(0) and ) • Determine the image pre-processing level

Maximize Final Processing Accuracy

• (1) is time constraint that the task must be finished before completion deadline • (2) is power constraint that the task consumes no more energy than using cellular

Image file size

Minimum image file size

Remaining time till conditioncheck deadline

Minimize pre-processing level

Analytical Framework• Study how Smax is affected by and D(0)• Assume Bw follows an exponential distribution with

rate λ• Calculate expected Smax

Expected Smax

Multiple Task Offloading• Task execution order: FIFO• Recalculate the new condition check deadline

Evaluation

Implementation• Client side (Android smartphone):• Estimate the TCP throughput of current connected Wi-Fi• Downscale images

• Server side (Windows Server 2012 R2):• Process images using OpenCV library

Experiment Methodology• Face detection

• Static environment• Compare different pre-processing levels• Compare Wi-Fi and 3G environments

• Dynamic environment• Compare different D(0) and β• Compare against other approaches

Static Environment – Different Wi-Fi Environments

• The energy consumption decreases when the downscaling factor decreases.

• Some exceptions• Better Wi-Fi results in lower energy consumption.

Static Environment –Comparison of Wi-Fi & 3G

Dynamic Environment –Image Processing Accuracy

Dynamic Environment –Comparison with Other Approaches

• Compare against• Instant upload • Instant check

Conclusion• Understand how the image pre-processing level would

affect accuracy and energy consumption on smartphones

• Propose a Pre-Processing Based Computer Vision Application Offloading approach

• Achieves near-optimal image processing accuracy while satisfying energy and time constraints

Q&A

Thank you