● The overview of holographic development

● Create holographic apps by Unity

● Case study from developers

设置 安全 管理 更新

自然的用户输入 一套SDK和工具

云服务一个商店一个开发者中心

适应式的设计

Holographic XboxDevices

+IoT Surface HubMobilePC

Universal Windows Platform

一个统一的Windows 内核和应用开发模型

https://developer.microsoft.com/en-us/windows/holographic https://github.com/Microsoft&query=Holo

● Visual Studio 2015 Update 3

● Windows 10 SDK (10.0.10586 or newer)

● HoloLens Emulator

o build 10.0.14342.1018o Holographic templated is includedo Hyper-V supported

● Unity for 3D Holographic view

o Not need for 2D UWP appso 5.4f3 HoloLens Technical Preview o UWP runtimeo http://unity3d.com/partners/windows/hololens

● No Device needed

https://developer.microsoft.com/enus/windows/holographic/install_the_tools

https://developer.microsoft.com/en-us/windows/holographic/using_the_hololens_emulator

• Close• Minimize• Human Input• Keyboard• Fit to Screen• Zoom• Help• Open Device Portal• Tools

Walk: W,A,S, D/left stick Look: Arrow/right stick Air tap: Enter/Space/A button Bloom: Windows/B buttonHand: Alt key/right trigger

Control

Mapping

HOME

Performance

App Mgr

2D View

Holographic View

3 Ways UWP upgrade for HoloLens

DirectX 11 App with Holographic template

Unity for Hologram

UWP Application life cycle

Install/uninstall/update, version control

Limit APIs and Concepts

Tiles

Notification

Toast

Share Contract

https://developer.microsoft.com/en-us/windows/holographic/current_limitations_for_apps_using_apis_from_the_shell

https://developer.microsoft.com/en-us/windows/holographic/updating_your_existing_universal_app_for_hololens

●Same process as UWP apps to Store1. Prepare image assets and Live tiles2. Specifying target device families

3. Associate app with the Store4. Creating an upload package5. Testing your app on WACK6. Submitting an app to the Store

2D

Holo

Step 1: Create a new project

Step 1: Create a new project

Step 2: Set the Main Camera

Select the ‘Main Camera’ in the Hierarchy

Reset the Transform

Change the Clear Flags from Skybox to Solid

Color

Change the RGBA values to (0,0,0,0) of the

Background

Change the Near Clipping Planes field to 0.85.

Step 3: Adjust the Quality Settings

Select Edit > Project Settings > Quality

Select the dropdown under the Windows Store

logo and select Fastest.

Step 4: Unity Build Settings

Select File > Build Settings...

Click Add Open Scenes to add the scene

Select Windows Store in the Platform list.

Click Switch Platform button

Set SDK to Universal 10

Set Build Type to D3D.

Step 5: Set the VR Support of Windows Holographic

Open Player Settings...

Select the Settings for Windows Store tab

Expand the Other Settings group

In the Rendering section, check the Virtual

Reality Supported checkbox

Confirm "Windows Holographic"

Step 6: Export the Visual Studio solution

Return to the Build Settings window.

Check Unity C# Projects under "Windows Store"

build settings.

Click Build Button

Click New Folder and name the folder "App".

With the App folder selected, click the Select

Folder button.

Export the Visual Studio solution

Step 7: Deploy to the Emulator

Open the generated Visual Studio solution in

App folder

Select Release in target list

Select x86 in architecture list

Select HoloLens Emulator in device list

Click green arrow to run

A collection of scripts and components

Intended to accelerate development of holographic applications targeting Windows Holographic

Open Sourcehttps://github.com/Microsoft/HoloToolkit-Unity

6 feature areas

Modules Components

InputHoloLens input features namely Gaze, Gesture and Voice: Cursor.prefab, CursorWithFeedback.unity, FocusedObjectKeywords.unity, GazeStabilizer.cs, Cursor.cs, CursorManager.cs, GazeManager.cs, GestureManager.cs, KeyboradManager.cs

SharingSharing and networking components: Sharing.prefab, Sharing.unity, SharingMenu.cs

Spatial MappingSpatial Mapping related features: RemoteMapping.prefab, SpatialMapping.prefab, SpatialMappingManager.cs

Spatial SoundAudio features: AudioEmitter.cs, AudioOccluder.cs, IAudioInfluencer.cs

UtilitiesUseful common concepts: FPSDisplay.prefab, MainCamera.prefab, StandardFast.shader Billboard.cs, DirectionIndicator.cs, Interpolator.cs, TextToSpeechManager.unity

BuildBuild and deploy automation window for building your VS solution, APPX, installing, launching: BuildDeployPortal.cs, BuildDeployTools.cs, BuildSLNUtilities.cs, BuildDeployWindow.cs

All contents are in the different directories by category: Scripts, Prefabs, Shaders, Models, Materials, Tests

https://github.com/Microsoft/HoloToolkit-Unity/blob/master/GettingStarted.md

1. Clone the HoloToolkit-Unity code from github and open the project in the Unity

2. Select the assets folder in the Unity and export the package

3. Create new project and import the HoloToolKit-Unity package

4. Finish the basic configuration Use MainCamera.prefab Apply Scene Setting Apply Project Setting Apply Capability Setting

5. Use Build Window

Metric Target

Frame Rate 60 fps

Power consumption 1-minute average in orange and green area. See System Performance Tool information.

Memory < 900 MB Total Commit

● Batch draw calls and use instancing to minimize DirectX overhead● Avoid full screen effects such as FXAA and SSAO● Use mip-maps wherever possible● Use 16bit depth buffers● Turn off physics if your project doesn't use it● Perform frustum culling against the combined left and right eye frustum● Lower LOD for holograms that are far away● Avoid Geometry, Hull and Compute Shaders● Hand-writing Shaders in HLSL● Move work from Pixel Shader to Vertex Shader when possible(move lighting

into Vertex Shader and interpolate)● Use bilinear whenever possible● Use trilinear sparingly● Anisotropic filtering should only be used when absolutely necessary

Frustum Culling Occlusion Culling

Stop objects from being rendered if they cannot be seen○ outside the camera’s viewing area○ obscured (occluded) by other objects○ https://docs.unity3d.com/Manual/OcclusionCulling.html

Instead of rendering each eye separately, this uses a wider render target and alternating draw calls to render both eyes with a single scene traversal.

Option in Player Settings from 5.4.0.

Standard Two Pass Rendering Single-Pass Stereo Rendering

Totally support for Forward rendering and Deferred rendering

Can move jobs from the main thread to worker threads for significant rendering performance improvements especially on multi-core CPUs

Currently in experimental status 0 2 4 6 8 10 12 14

5.3

5.4a

5.4b

5.4c

Time (ms)

Main Thread Rendering Render Thread

Two Pass VR

Single Pass VR

Graphics Jobs + Single Pass VR

https://unity3d.com/cn/unity/beta/unity5.4.0b6

Open Player Settings...

Check Graphics Jobs checkbox

Check Single-Pass Stereo Rendering checkbox

Setting

Unity Assets for Terrain

https://blogs.unity3d.com/cn/2016/10/28/introducing-holographic-emulation-2/

Developers creating applications for Microsoft HoloLens will immediately benefit by being able to prototype,

debug, and iterate on design directly from the Unity Editor without getting bogged down by long build and

deploy times.

Requirements• Windows 10 Anniversary update (or later)• Unity 5.5

Emulation mode• Simulation• Remoting

Challenges & Workarounds在整个项目开发过程中还算是比较顺利的，大概综合整理为以下几点：

渲染性能问题：毕竟需要在一个相对小型的移动设备上同时完成定位计算和立体渲染工作，对于设备计算单元以及电池的

压力非常大。目前发现，其渲染能力略高于主流移动设备的，在开发过程仍然按照移动设备开发的标准进行，但依赖于

DirectX11，有很多有效的优化让整体应用获得不错的性能。

内存限制问题，开发的应用一般建议用户开销的内容不大于900MB。这种限制完全可以理解，对于目前以展示为目的的

应用来说，在一定优化后完全可以接受。当然考虑到将来更加复杂或者针对更大数据集合进行开发时，需要特别留意。

目前较好的开发平台是Unity的 Technical Preview版本。可能由于一切都还是处于预览状态，有一些诸如

KeywordManager，OnPressed/OnReleasedHolotoolkit以及ManipulationOffset等细节处有些与常规的开发习惯不

符。但好在 Holotoolkit 是开源的，按照项目的需求对其进行一些修正并不困难。

我们在项目开发过程中所使用HoloTookit与UGUI自带的事件配合不太好，当然基于物体碰撞的触发工作的很好，所以我

们仍然可以顺利完成我们需要交互操作。

混合现实的应用，无法限制用户与虚拟物体的位置关系，当用户因为任何原因需要非常靠近目标时，近切平面的影响会造

成不少困扰。所以一些按照物体深度进行渐隐或者如Fragment应用中将几何体以一种漂亮的方式炸裂开来，是一种很好

的解决手段。

专家面对面

12/01 Zone1 10:00-11:30