Sioux Hot-or-Not: The future of Linux (Alan Cox)

The future of Linux

Arnoud de Geus

March 5th 2008

© Sioux Embedded Systems 2008 | Confidential | 2

Welcome @Sioux

Agenda18:00 hrs Welcome

Alan Cox I19:30 – 20:00 hrs Break

Alan Cox II21:00 hrs Social

Introduction

Arnoud de Geus Business Development Manager Responsible for Linux @ Sioux


Hot-or-Not @Sioux

Why Hot-or-Not? Increase productivity by innovation

We invite experts from all around the world to inspire us in improving the productivity of our region.

Is it ‘Hot’? or is it ‘Not’?


Innovation & Open Source

Open source accelerates innovation Open innovation is the only way for certain

commodities such as operating systems Embedded appliance of open source layers

accelerates time-to-market enormously OEM: focus on core technologies One of the best examples: Linux


Linux @Sioux

Linux frequently used in projects @Sioux since end 90’s

Stable open platform used in both embedded environments and on application level

Examples Consumer Electronics (like high end TV’s, ADSL

modems, domotics, Wifi routers, ..) Professional Equipment (like copiers, electron

microscope’s, wafer scanners, ..) Fri-Jado Food vending machine > demo


Linux Knowledge @Sioux

Kernel related Linux kernel tweaking, Linux device driver

development, USB, Linux system boot, Linux system configuration, ..

Development / Test environment GNU, MontaVista, WindRiver, Eclipse, ..

Linux user space Graphics subsystems like DirectFB, VNC, ..


Employee Programs @Sioux

Embedded Linux Courses (MontaVista) Device Driver Courses (MontaVista) Linux Knowledge Expertise Group ~25

members (both Belgium & Netherlands) Linux conference attendance Actively acquiring Linux projects (frequently

we choose Linux for projects @ Sioux Development Centre)


The future of Linux

Alan Cox


Welcome to HTS valley

High Tech Systems Industry 2004-2007 20-24 Billion € growth, average > 6%

growth Consumer Electronics / Telecom,

Professional equipment with high accuracy, Medical systems, Automotive, Semiconductor

E.g. ASML, Bosch, FEI, NXP, Océ, Medtronic, TomTom, Thomson, Philips, Vanderlande, …


Alan Cox

Linux entrepreneur Early Linux adopter at

University of Wales (Rewriting the network layer)

Alan Cox branches –ac (2.4 branch)(Very stable platform for vendor branches)

Supports programming freedomIs against software patents

Honorary degree CS, MBA University of Wales

Advisory board FIPR, ORG Working @Red Hat Living in Wales with Telsa

Gwynne

THE FUTURE OF LINUXpresented at Hot-or-Not powered by Sioux

March 5th 2008Alan Cox

March 5th 2008 The future of Linux Alan Cox

How do we predict the future

Analyze The Past Identify Trends Propose The Result

We cannot predict the future of the kernel by considering the kernel alone


What Influences Linux Development

Business Evolution Hardware Is Evolving Software Methodologies Are Improving The Security Situation Microsoft Law and Politics


Why Put Business First

Most Technology Benefits Come Through Exploiting Business Opportunities Enabling People and Processes

Focus on the code and you are walking blindly


What Is Business Driving

Small Footprint Embedded Devices Have Memory Constraints Flash Size Determines Profits Virtualization: Memory Is No Longer Free

Power Battery Limitations In Laptop/Embedded Cost Of Power For Server Rooms State and Economic Pricing Pressure


What Is Business Driving

Real Time Embedded Uses Large Servers Needing Predictible Response Trading Floor

Scalability More Storage More Memory More Processors More Systems To Manage


Hardware Evolution

Storage Solid State Storage Disk Approaches Flat Cost Scaling File Systems Scaling Search/Index/Backup

Price Ever Downward Enabling New Opportunities (EeePC) Almost No Margin For Software


Hardware Evolution

Performance Single Thread Of Execution Ceiling Multi-threaded, Multi-core Challenges Of Parallelism

Dynamic Configuration Hardware Used To Change At Power Cycle Dynamic Devices Object Lifetimes Dynamic Application Events


Software

Validation New Better Tools Find Bugs At Compile Time Verifiers At Runtime

Languages And Tools More Interpreters Stronger Security Models Faster Locking Models


Security

Liability (Notably For Data Loss) Interest In Real Security Systems Which Fail Safe Compartments Strong Encryption Smart Cards


Legal Evolution

Liability Law EU Determined To "Fix" Software Liability Standards Of Design Standards Of Operation Properly Stating What Software Is Fit For

Patents EU Software Patents War Goes On


Big Picture

Everyone wants security power management real time performance/efficiency

There are clashes between footprint and scalability dynamic devices and footprint security and performanc


Dynamic Configuration

Historical Unix System Device configuration at compile Device configuration at boot Object lifetime is kernel Administrator manages /dev

User Interface User driven No event model (User requests status) Dynamic reconfiguration



Modern Systems Device hotplug System responds to events User application events

New Rules Object lifetimes matter Kernel to user events exist Device namespace is dynamic



KObject A basic kernel refcount and name Refcounted Can be grouped No garbage collectors

Used to build other objects Device objects Sysfs - generic configuration And many many more



Sysfs Built on kobject and devices Heirarchical file system Namespace for configuration Replaces much of /proc/sys and sysctl Dynamically extensible

Allow user space to configure and explore the object heirarchy in the kernel.



Triggering User Actions /sbin/hotplug netlink

User Space HAL Dbus Desktop events



The /dev Problem MAKEDEV Devfs (policy in the wrong place) Use events Udev

Udev can also drive user space via HAL.



Modern Desktop Is Complicated

Inserting An SD Card Kernel: New hardware found Hotplug: Load module Kernel: New device UDev: Make device node Kernel: New volume found HAL: Mount it

...



Inserting an SD Card (2) Kernel: Key Event HAL: Password Prompt, Send key Kernel: Mounted HAL: Dbus message to apps Desktop: Add new icon Desktop: Umm "dcim/"... Desktop: Start photo manager

And at any point the card might be removed!


Usual Implementation

Usage Counts foo_get and foo_put operators destroy is delayed until users = 0 internal implementation via locks/atomic types

Problems Easy to get use counts wrong Scaling of use counters across processors Fine for most object types


Speeding Updates

Read Copy Update Self consistent object view Less synchronization No or little locking Some memory overhead

Objects are copied before update and then the pointer atomically switched. Old objects are discarded at a point we know they are no longer in use.


Driver Mentality

Historically Driver entry point scans for devices Driver creates devices

Today Devices are enumerated by bus drivers Device match functions Bus matches device to drivers Bus calls driver probe/remove methods Everything assumed to be dynamic


Power Management

Saving Power increases battery life reduces cooling cost saves money and planets

It is not just about hardware just about kernels


Power Management

Full Stack Optimisation Hardware supports power levels Kernel makes best use of them Governors set strategy

User Space Fewerwatts.org Reduce unneccessary wakeup events Tools to identify power hogs

Open source means we can fix the entire stack without being limited by a single badly performing vendor.


Power Management

We need to know How to turn stuff off When to turn stuff off The order to do it

Old Linux kernels had a simple power event model, unsuited to anything but firmware managed power.

Today we use bus and device heirarchies to sequence power events.


Mainframes and Phones

Why are mainframes and phones alike ?

Virtualisation needs reduced wakeup events in order to be efficient. Phones need reduced wakeup events to save power.

We need to identify Common goals Different goals with a common solution

Sometimes this is hard with embedded Power specific memory


Tickless Kernel

Our most rapid wakeup becomes the clock

Avoid the clock tick Initial ’stop clock when idle’ hack Proper timer based methods Identify timeouts Maintainer a timer for the timeout point Aggregate nearby time events

Still Needed Ability to indicate required accuracy


Real Time Linux

Historically Soft (very soft) real time Memory locking Adding Pre-emption points Pre-emptible kernel

Approaches Guest OS on RT uKernel (RTLinux) Add on patches (many) Fix the kernel cleanly (hard)


Real Time Linux

Systematic Approach Replace most semaphores with mutexes Support schedulable interrupt handlers Preserve most existing behaviour guarantees

Compromises Reader/writer lock is actually single reader

Linux is developed by evolution. Sudden large changes are generally opposed. Thus the realtime patches must not break existing code.


Real Time Linux

Priority Inversion Multireader locks cause problems Fixable for mutexes and single reader locks RCU never causes priority inversion Schedulable interrupts permit sleeping IRQs

Switch any problematic reader/writer locks to RCU and boost performance for all.

Can deliver 100uS - 1mS guarantees dependant on hardware.


The Trouble With Disks

Disk Scaling Capacity Keeps Doubling Seek Time Unchanged Data Rate Rises Too Slowly

Disks typically perform under 300 ops/second.



Filesystems How to lay out the file store Block pointers and bitmaps Scalable error recovery Linear runs are important

Short Term Approaches Grow the on the disk pointer sizes Adjust layout for faster checking Add Encryption



Long Term Support partial disk checking Integrity and self healing Media level assistance Flash addons for power management Will we be using disks ?



User Space Back up times Find/Search times Versioning data Visualising disk usage


Memory Scaling

Page Size Stuck at 4K on PC systems Very large number of pages Lot of meta data Makes paging and I/O decisions costly

Memory Management Free memory management Virtualisation Caches and SMP


Management

Drowning in PCs Want to reduce data centre cost Want to save power PC systems mostly idle resource

Virtualisation Dynamic resource assignment Higher total utilisation Less hardware Isolate software from hardware


Virtualisation

Impacts Wasted resources are now real waste How to manage memory dynamically Disk optimisation between guests Too many cooks

Approaches Xen - microkernel plus special guest KVM - Linux as hypervisor


Virtualisation

User Space Management interfaces (libvirt) Removing waste Visualising and controlling Integration with older tools Software licensing

Research Areas Disk management Timer management across guests Gang scheduling RT and Hypervisors


Multiprocessing

Threaded Software Poor C language support More threads more bugs Cost of locking Cache affinity management

Approaches Kernel futex locks NUMA and affinity aware scheduler Reconsidering software models


Multiprocessing

Avoiding Kernel Bugs Validation tools for locking Debug aids Well defined locking policies We can’t change language

User Space Language choice Threading or messaging Give hard jobs to tools


Software Quality And Approach

Once Upon A Time Well written tight code Programmers wrote programs Programmers fixed inefficiencies Kernel developers told users to fix sillyness

Nowdays Non programmers put together code PC CPU is cheaper than fixing code Systems must be robust to poor coding Efficiency under stupid loads


Video Support

Hardware Changes 2D to 3D Fast programmable 3D engines Fast main memory paths Much harder to program

Fortunately there are few players left who matter


Video Support

Software Demands Alpha blending for truetype Video playback overlay and compositing OpenGL 3D Robustness with speed Compositing for effects

User Space Cairo X EXA Acceleration Model RandR Xv


Video Support

In The Pipeline Kernel side video mode setting Oops reporting in graphics mode Improved 3D and DRI integration

Hardware Direction Closer CPU and 3D integration From 3D engine to parallel CPU Expect on CPU video


Different Hardware

Unix Model Processor with MMU Memory Disks Block/Char devices tty device for console

Linux Today MMU optional Disks or Flash or RAM Block devices optional Still need memory 8)


Different Hardware

Currently Unsupported 36bit systems (no PDP 10) Very small embedded Processors without atomic pointer_t writes Really warped hardware designs


And finally..

To understand the future we need to understand the processes involved

Nobody understands how free software happens ?

1995 - True2008 - False


Systems Theory

Systems Theory Models to explain how self organising systems behave. Hard mathematics to analyse those models Understanding of emergent behaviour

We thus finally have the tools to make important breakthroughs in understanding how free software communities form, why they are robust against people dropping out and how they scale.

For both good and bad uses that may prove to be the most important development yet.


The future of Linux


More Hot-or-Not

Erlang - is functional programming the newest trend after OO?Joe ArmstrongMay 8th

Agile Manifesto - the essentials unveiledAlistair Cockburn June 26th

The evolution of testing - xUnit Test PatternsGerard Meszaros September 10th

Taking Software Architecting into the next decadeBran Selic October 8th

www.jouwontwikkelingtelt.nl


Alan Cox and Linux...

Sioux will donate € 3.000,- to

1. Foundation for a Free Information Infrastructure

2. Free Software Foundation Europe

3. Electronic Frontier Foundation Europe

You will choose the destination(s)!


Alan Cox and Linux...

Sioux will donate € 3.000,- to

1. Foundation for a Free Information Infrastructure

2. Free Software Foundation Europe

3. Electronic Frontier Foundation Europe

You will choose the destination(s)!


Alan Cox and trains...

Alan has a passion for software, and also a passion for model trains. This train (scale 1:160), which carries the city of Eindhoven logo, is a present for Alan to remember his visit to the Netherlands.

[email protected]+31(0)40 2677100