Remote Ray Tracing for Mobile

ApplicationsGregory Fotiades

Global illumination techniques are highly desirable for realistic interaction due to their high level of accuracy and photorealism

Mobile devices are typically too weak in terms of computational abilities to perform global illumination techniques

Introduction

For real time performance we need rendering speeds in the order of < 40ms (25fps)

Low powered devices by their nature do not have access to high performance hardware

Introduction – Cont.

Pre-rendering the scene using tools such as “RenderMan on Demand”[5] is not feasible as we do not know what commands the user will send

Lorio et al.[6] did work in using cloud computational power to model urban environments in a realistic time frame

Background

Low powered device will offload intensive global illumination computations to the cloud

Cloud will perform computations on distributed parallel architecture

Rendered scenes will be sent back to client

Solution

Solution

Smart phone/PDA

Tablet

Real Time RenderingSystem(s)

Low Powered Laptop

Commands

Renderings

RenderingsCommands

Renderings

Commands

Renderer in cloud must perform operations real time (< 40ms) – even less

Renderer will be hardware accelerated to achieve said real time

Renderer will need to scale in future works

Solution - Renderer

Solution - Renderer

RAW formats are too large even if we use convert colorspaces to make use of subsampling

Solution – Network Communication

Height Width Format Bytes Per Pixel (BPP)

Frames per Second (FPS)

Required Bandwidth

432 432 RGBA 4 24 137Mbps1080 1920 RGBA 4 24 1519Mbps432 432 YUV444 3 24 103Mbps432 432 YUV420 1.5 24 52Mbps432 432 YUV420 1.5 20 43Mbps1080 1920 YUV420 1.5 20 475Mbps

Video compression must be utilized◦ Compression codec is rather irrelevant, but

VP8/VP9 come to mind first[29][30]

Real time encoding/decoding in software is very common (ie. video conferencing)

Network transmission takes a lot of time

Solution – Network Communication

Renderer is the ray tracing core Renderer makes use of Nvidia GTX 280 GFX CUDA API[24] is used for HW acceleration 432x432 resolution used

◦ 16x16 threads used across 27x27 blocks Only modified objects updated to lower

overhead (copy scene to GPU) QT Toolkit[27] used for UI

Implementation - Renderer

Recursion not supported by GFX card◦ No stack pointer or concept of frames◦ Iterative conversion done

Static frame stack implemented and used◦ Recursion supported on newer HW, future work

may investigate possible port to recursion Stack/recursion depth of 8 used

◦ Higher than 8 had no noticeable improvement Some basic optimizations performed, but all

trivial

Implementation – Ray Tracer

Triangle intersection detection not implemented, so complex meshes cannot be rendered◦ This is topic to analyze in future work as triangular

meshes will drastically increase render time◦ Multiple client implementation may be necessary

Ray tracer based on CG2 ray tracer, so lacks advanced shading techniques

Ray tracer uses static scene intersection detection, needs some form of dynamic detection in future

Implementation – Ray Tracer Cont.

Implementation video compression is outside the scope of project

RDP clients fill niche of transmission assuming they perform well enough for UX to be smooth as they also handle cmds to server

Kinoni Remote Desktop[33] used for transmission

Future work should explore its own compression or direct compression control and color space conversion

Implementation – Network Communication

Implementation – Network Communication Cont.

Results

CPU GPUMin Render Time 87.177 14.089Max Render Time 92.629 17.576Avg Render Time 89.441 15.989

Rendering speedup of ~5.6x◦ More important than speedup is scalability

200Kbits/sec bandwidth usage

4.4Mbit/sec download easily feasible over 4g◦ Not enough for raw transfers

Results – Cont.

Commands to server Rendering to ClientAvg bytes / second 2454 26967Avg Kbits / second 19.172 210.680Avg Mbits / second 0.019 0.206

Upload DownloadAvg bytes / second 919.62 577540.72Avg Kbits / second 7.185 4512.037Avg Mbits / second 0.007 4.406

Real time interactive global illumination is feasible if parallel resources and a network connection are available

Video compression is necessary to transmit images

Conclusions

Scalability across multiple systems and multiple GPUs

Implement triangular mesh intersection detection

Implement dynamic object management In-house video compression

Future Work Summary

Demo

Q&A