Sumit& archit osi nov-2011-displays-in-mobile-devices

  • View

  • Download

Embed Size (px)

Text of Sumit& archit osi nov-2011-displays-in-mobile-devices

  • 1. Displays in Mobile Devices OSI Days 2011, Bangalore Archit Taneja ( Sumit Semwal ( 1LinuxDevelopment Centre

2. Agenda Historical look at mobile devices Non-phone mobile devices Types of display panels Display Subsystems In PC World SoC World PC vs SoC Example Data Flow Mobile Display Interface Standards Interesting Display Use Cases Example of SoC DSS: OMAP2+ Display Subsystem Current software design DSS2 framework Linux Kernel Display Frameworks What Next? Q&A 2LinuxDevelopment Centre 3. Under CC License from Linux Development Centre 4. A Historical Look at Mobile Displays LG Optimus 3D 3D Stereoscopic LCD; 800x480; 4.3 [4]Nokia 9000 Communicator:First business phoneGoogle Galaxy NexusMonochrome display, WideSuper AMOLED plus640x200: [3]720x1280; 4.65Nokia 1011First mass One of the firstproduced color screenGSM phone phonesMonochrome Sony Ericsson T68i[1] 101x80, 256color [2] Apple iPhone 4Google Nexus One Retina displayAMOLED: 480x800, 3.7960 x 640; 3.5 [5] All trademarks and copyrights are properties of and attributed to their respective companies Sources: in references 5. non-phone Mobile Displays Device PositioningScreenResolution Technology size SamsungTablet 5.31280 x 800 HD sAMOLED Galaxy NoteSmartphone Dell Phone5800 x 480TFT LCDTablet 7800 x 480Capacitative 10.1 1366 x 768 TFT Amazon eBook Reader 6600 x 800eInk, 16 level Kindle 3 grayscale B&N Nook eBook Reader 6800 x 600eInk, 16 levelgrayscale Archos PMP7800 x 480TFTTablet 81024 x 768Tablet 10.1 1024 x 600 Apple iPad Tablet 9.71024 x 768 IPS LED SamsungTablet 71024 x 600 TFT Galaxy Tab8.91280 x 800 10.1 1280 x 800 Only an indicative list; certainly not exhaustive!All trademarks and copyrights are properties of and attributed to their respective companies LinuxSources: in references Development Centre 6. Types of Display Panels Retina Apple iPhone 4 In Plane Switching (IPS) LCD Highest density: 326 ppi (3.5 screen, 960x640) AMOLED (Active Matrix Organic LED) No Backlight needed Higher contrast ratios, more power efficient. Pen Tile arrangement RGBG arrangement of sub-pixels, where G sub- pixel is shared by 2 pixels. So the display has only 2 sub-pixels per real pixel. Super AMOLED Same as AMOLED, but touchscreen is integrated onto the same glass, thus increasing visibility on the screen.All trademarks and copyrights are properties of and attributed to their respective companies LinuxSources: in references Development Centre 7. Types of Display Panels Super AMOLED Plus Integrated touchscreen, and no Pen tile arrangement much brighter, thinner and more efficient. regular RGB, so 50% more subtiles, hence lesser resolution. eInk Panel Based on actual electronic ink that is coated on some ultra-thin plastic. eInk contains microcapsules filled with ink, and pigmented, charged chips. Each microcapsule acts as a pixel; when an electric charge is applied, either the light or dark pigmented chip will come on top, thus switching the pixel on or off.All trademarks and copyrights are properties of and attributed to their respective companies LinuxSources: in references Development Centre 8. Display Subsystems: PC World 2D/3D HDMI Accelerators Pixel Gen 1 DisplayPortPort Video Pixel Gen 2Decoders InterconnectDisplay(s)LVDSScene information Overlay / Pixel Gen M orAnalog Blending Encoded framesVideoGPUDisplay I/F Display SubsystemShared/DedicatedMemoryLinuxDevelopment Centre 9. Generic SoC Hardware BlocksVideoDisplay SubMPU GPUAcceleratorSystem Display(s)interconnectEMIF DSPImagingOther Peripherals MemoryOnly display related blocks are shown; there are many other blocks too, which are out of scopefor this discussion.LinuxDevelopment Centre 10. Display Subsystem in SoCs Serial Plane 1 Compositor 1Interface& (MIPI DSI)Pixel Gen Plane 2Parallel Interface(MIPI DPI) Compositor 2 Plane 3& Interconnect RemoteDisplay(s)Pixel Genframebuffer Interface(MIPI DBI)GPU composed frames orDecoded Video frames Compositor M HDMI/&Display Port Plane NPixel Gen OverlayDisplay I/FDisplay Subsystem Linux Development Centre 11. Display Subsystems: PC vs SoC Differences: PC: GPUs are generally responsible for composing different windows/layers to get one single frame. Display Subsystem only fetches and display this composed frame . SoC: Display Subsystem is also responsible for composing layers generated by the GPU and Video accelerators. PC: Display Subsystem may only fetch frame data from the dedicated memory for the graphics card/controller. SoC: Display Subsystem fetches frame data from the main memory connected to the SoC. Reasons: Legacy: GPUs and Video Accelerators have existed in SoCs as separate entities for a while. Making it important for the Display Subsystems to compose the output of each into one frame. Area: SoCs (and mobile devices) cant afford dedicated memory for GPUs. IP reuse: Different Vendors of GPUs and Display Subsytems in SoCs, easier to interface with a standard interconnect. Power: Overlaying different layers is more optimal for SoCs in some cases. Linux Development Centre 12. Example data flow 3. Display frame on panel1. Video frames to decode1. Commands to draw UI 2.DoneVideo Display SubMPU GPU Accelerator System 2. Done interconnect2.Soc 2.3.DisplayControl pathData pathMemoryOnly display related blocks are shown; there are many other blocks too, which are out of scope Linuxfor this discussion. Development Centre 13. Mobile Display Interface StandardsMIPI- Display Working GroupPrimary (fixed) panels:- Display Serial Interface (DSI):- serializes all pixel data, commands, and events.- low power(SLVS)- Works with Smart / Dumb Panels.- Display Parallel Interface (DPI):- Legacy parallel interface.- Lots of lanes needed for pixel data and control events.- used with Dumb Panels.- Display Bus Interface (DBI):- Parallel Interface for smart panels.Pluggable panels:- HDMI- Display Port- SDTV / Composite- DVIAll trademarks and copyrights are properties of and attributed to their respective companies LinuxSources: in references Development Centre 14. Interesting Display Use cases Device Dock Dynamic configurability, parallel usage Switch output paths, using multiple paths simultaneously Handle audio interfacing additionally HDMI support DSI support in future Support for virtual pipelines when we run out of video pipelines Co-use of 2D/3D graphics accelerator. Multi-display configurations Clone, Virtual.Display security14 Linux Development Centre 15. OMAP2+ DSS hardware: Overview OMAP2,3,4Display OMAP3,4Panels OMAP4 DISPCDSI1 encoderGFXGfx, vid1-vid3 are Primary DSI overlays, which color- convert, scale each frameVID1 LCD1RFBImgr RFBI Managers can compose,VID2 overlay multiple streams LCD2DSI2 encoder into one.VID3mgr Secondary DSI Not all paths are shown:DPIAll overlays connect to all TV overlay managers. mgrDPIHDMI encoderVENC HDMI Display InterfacesSDTVDSS hardware15 Linux Development Centre 16. Current software design DSS2 frameworkUI // GraphicsUI GraphicsVideo playback Video playback Userspace app Userspace appControl Userspace app Userspace appUserspaceFramebuffer APIs V4L2 APIs /dev/fb0 /dev/video1 sysfs /dev/fb1 /dev/video2 Panelomapfb driverdrivers omap_vout driverKernelspacegfxvid1vid2 vid3DSS2 DSS2lcd1-mgrlcd2-mgr tv-mgrOMAP DSS hardwareDisplayhardware HDMI TV LCD panels16 Linux Development Centre 17. Linux Kernel Display Frameworks Frame Buffer Oldest of them all - first public release in 1999. Linux Kernel Abstraction for graphics hardware represents framebuffer of some video hardware, so the software doesnt need to know the low-level (hardware) stuff. allows emulate text console without hardware text-mode support. is a possible graphic output method for X server; independent of video adapter hardware and drivers. allows direct graphics programs without X; popular in embedded world [GTK+, Qt Extended are some examples] No inherent window or buffer management DirectFB [6] Specifically designed for embedded systems. Is a thin library providing hardware graphics acceleration. Integrated windowing system with support for translucent windows and multiple display layers. Input device handling. Is a complete hardware abstraction layer, with software fallbacks to every graphics operation not supported by an underlying hardware. Provides for X-less graphical systems: gtk+, LinuxTV, HP WebOS to name a few. Linux Development Centre 18. Linux Kernel Display Frameworks Video For Linux (V4L) Kernel interface for streaming capture and output drivers. Provides framework for buffer management and streaming very relevant for streamingkind of use-cases [video record, playback, TV, even analog radio! :)] Buffers are tied to each device, and allocated before streaming starts. Direct Rendering Manager (DRM) The modern Linux (and *BSD) display driver framework The kernel component of DRI (Direct Rendering Infrastructure) But not just DRI: Hotplug XRandR/KMS EDID parsing Kernel Mode Setting (KMS) Setting screen resolution/timings Hotplug, and retrieving EDID Multi-display configurations Buffers are independent of overlays Security inherentLinuxDevelopment Centre 19. What Next?- HDMI 3D- eInk Color (Triton) [7]LinuxDevelopment Centre 20. Q and A? 20 Linux Development Centre 21. References 3430 public Technical Reference Manual 4430 public Technical Reference Manual[1]:[2]:[3]:[4]:[5]:[6]:[7]: Centre 22. Thank You! 22 Linux Development Centre 23. Backup 23 Linux Development Centre 24. Acronyms SoC: System on Chip DSI