Virtualization the Rabbit Hole Goes Deep Spotlight... · Virtualization – the Rabbit Hole Goes Deep. The principles Beneﬁts Shortcomings Technicalities Use cases Examples Roundup

The principles Benefits Shortcomings Technicalities Use cases Examples Roundup

Virtualization – the Rabbit Hole Goes Deep

Leonid Bloch

[project21] TheAlternative

October 25, 2018

Leonid Bloch [project21] TheAlternative



A virtual computer?

What are computers, really?

◮ All computers are machines for solving logical problems using simple operations




A virtual computer?

Logic is logic

◮ Which problems a computer can solve?

◮ Any solvable logical problem! (Can be mathematically proven!)




A virtual computer?

Logic is logic

◮ Which problems a computer can solve?

◮ Any solvable logical problem! (Can be mathematically proven!)

Turing completeness:

If a machine answers to some very simple requirements:

◮ Can perform logical operators (IF, AND, NOT)

◮ Random access to as much memory as needed

Then it can be proven that it can solve any solvable logical problem!




A virtual computer?

A sea of possibilities

◮ But the hardware is a Turing machine etched on asemiconductor, and (most of) the programminglanguages that run on it are Turing complete, can“hardware” be written in a programming language?




A virtual computer?

A sea of possibilities

◮ But the hardware is a Turing machine etched on asemiconductor, and (most of) the programminglanguages that run on it are Turing complete, can“hardware” be written in a programming language?

◮ Sure!!!




A virtual computer?

A simulation inside a simulation... inside a simulation???




A virtual computer?





A virtual computer?





What is it good for?

A truly unbreakable computer

Physical hardware:

◮ Overheats

◮ Corrodes

◮ Wears

◮ Burns

◮ Gets outdated!

Virtual hardware:

◮ In most of the cases to the left canbe migrated to another physicalhardware, in many cases whileoperating!





A truly unbreakable computer

Physical hardware:

◮ Overheats

◮ Corrodes

◮ Wears

◮ Burns

◮ Gets outdated!

Virtual hardware:

◮ In most of the cases to the left canbe migrated to another physicalhardware, in many cases whileoperating!

◮ Except the burn case. Well, then itsstate can be backed up – easily!





Totally predictable

◮ Sometimes it’s important for hardware to be totally predictable (scientificcalculations, traffic control...)

◮ Physical hardware, even of the same model, can vary!

◮ Virtual hardware is always identical





Transparent hardware

◮ Most of the hardware is not open-source

◮ Regardless, it’s impossible to verify its functionality in non-destructive methods.





Transparent hardware

◮ Most of the hardware is not open-source

◮ Regardless, it’s impossible to verify its functionality in non-destructive methods.

◮ However... many hypervisors∗/emulators are open-source, and have the samebenefits as other FOSS.

∗ – The components which provide the virtual machines with their hardware assets.





Write your own hardware!

Example: Intel gigabit ehternet card – physical:

◮ Multi-billion Dollar facilities

◮ Extremely complicated manufacturing process

◮ Thousands of engineers











Example: Intel gigabit ehternet card – emulated:

◮ Three months of work for two engineers◮ One engineer and an apprentice, actually

◮ Works exactly according to Intel specifications◮ However, it is impossible to know the unspecified behavior of the physical card!

Therefore if a misfit with the specs is known, it has to be emulated!





Write your own hardware!hw/net/e1000e.c :

[...]

static uint64_t

e1000e_mmio_read ( void *opaque , hwaddr addr , unsigned size)

{

E1000EState *s = opaque ;

return e1000e_core_read (&s->core , addr , size);

}

static void

e1000e_mmio_write ( void *opaque , hwaddr addr ,

uint64_t val , unsigned size)

{

E1000EState *s = opaque ;

e1000e_core_write (&s->core , addr , val , size);

}

[...]





Total control by the user

◮ Did you ever modify the configuration of your home PC?

◮ Well now you can also modify the internals of your CPU!




No perfect solution

Real hardware overhead

Running VMs comes at a cost:

◮ The physical hardware has to run another kernel (OS).

◮ The physical hardware has also to run the hypervisor

◮ The emulated hardware has to use the logic of the physical hardware throughabstractions

... And obviously the performance degrades.




No perfect solution

Passthrough & other solutions

But fear not – modern VMs work at close to native speeds!

◮ Modern CPUs “know” that they are likely to be used for running VMs, and supplyhypervisors with optimized logic for that (Intel VT-x, AMD-V, ...).

◮ If a truly native performance is desired, physical devices can be assigned directlyto a VM.

◮ There are virtual devices that are optimized for VM usage – do not emulate anyreal-world device (paravirtualized). They are designed to work fast in VMs, andusually use some hardware acceleration.

◮ There are physical devices which incorporate optimizations for VM usage(KVMGT, scalable I/O, ...).




No perfect solution

What about containers?

Containers:

◮ Run natively on the hardware

◮ Use the host kernel

◮ Provide isolated environment on asoftware level (namespaces, etc...)

◮ Very fast to launch

◮ Smaller image sizes

◮ Fit for single application needs

Virtual machines:

◮ Run on emulated hardware (may bedifferent architecture than the host)

◮ Use their own kernels

◮ Stronger isolation from the host

◮ Slower to launch (entire boot process)

◮ More overhead on the host

◮ Provide more complete environments

∗ Lately it’s fashionable to mix VMs and containers, and a “hot trend” now is to usethe same management tools for both.




Tools

Many tools...




What can it be used for, in practice?

The trivial example – the clouds around us

◮ When you go to google.com do you think that you connect to a physical server?◮ “Do you think it’s air you’re breathing now, Neo?”

◮ All the major cloud providers use VMs, for their flexibility and fault-tolerance.

◮ You can actually rent the same VMs that Amazon and Google use for your ownneeds.

◮ Need a beast with 128 cores and 1 TB of RAM for some data crunching? For justa few hours? No problem, under $100!

◮ Need a small webserver for your own VPN solution? No problem – few $ a year!





Migration between servers? How about migration between clouds?






But what if... the virtualization layer was running on a virtualization layer itself?

◮ Cloud infrastructure is very difficult for an organization to disconnect from.

◮ Yet cloud providers (Amazon, Google, Microsoft) often change prices and terms.

◮ The solution?






But what if... the virtualization layer was running on a virtualization layer itself?

◮ Cloud infrastructure is very difficult for an organization to disconnect from.

◮ Yet cloud providers (Amazon, Google, Microsoft) often change prices and terms.

◮ The solution?

◮ Work with a cloud infrastructure that can migrate your infrastructure,transparently to you, between different cloud providers!

◮ We mentioned migration between servers? How about migration between clouds?





Seamless VM on the desktop – the perfect control





Seamless VM on the desktop – the perfect control

Control employee computing at sensitive organizations (banks, government, ...)

◮ No dedicated bulky hardware

◮ Full control and tracking by the organization – mouse, keyboard, screen◮ Even if you take a screenshot with a camera!

◮ On boot an underlying (host) OS starts transparently to the user, and the OSwhich the user ever sees runs inside a VM!





Your ideas?

???




A short migration demo

Live migration with QEMU demo

1. Start source VM

2. Start destination VM (idle while waiting for the incoming “ghost”)

3. Go to the monitor interface of the first VM

4. migrate -d tcp:<destination IP>:<destination port>◮ ...And follow the progress with: info migrate

5. Done!




And finally...

What now?

◮ Q&A – now

◮ 10 min. break

◮ Hands on session!Please see the hands-on walkthrough here:https://github.com/blochl/pVM/blob/demo/LinuxDays.md




And finally...

Credits

◮ The template for these slides was written by Christian Horea, and is availablehere: https://bitbucket.org/TheChymera/ld_foss/src

◮ Some images used in this presentation are copyrighted. They are used here underthe protection of the Fair Use Act, as this presentation is non-commercial, andmade for educational purposes only.



Documents

Virtualization the Rabbit Hole Goes Deep Spotlight... · Virtualization – the Rabbit Hole Goes Deep. The principles Beneﬁts Shortcomings Technicalities Use cases Examples Roundup