UML and other Linux Virtualization Techniques

Marcus Kraft

Product Manager

Novell, Inc.

Abstract

The trend continues:

User Mode Linux is one of the important virtualization technologies that customers benefit from.

See what other technologies are available to host virtual instances of SUSE Linux Enterprise Server (SLES) and how other OS's can be deployed on SLES.

Learn about SLES technologies supporting specific virtualization capabilities of different hosts and the benefits of different virtualization technologies across different hardware platforms.

Introduction

Marcus Kraft, Novell Inc / SUSE, Product Manager SUSE LINUX Enterprise Server with a technical focus.

Being with SUSE since 1998, he worked with several departments of SUSE (support, R&D, now Product Management). With a strong background of mainframe technologies, he and others innovated SUSE LINUX Enterprise Server for x86 and IBM S/390, the first enterprise class Linux OS released in 2001.

With a focus on cross platform deployment of SLES, he promotes the different existing and emerging virtualization technologies for SUSE LINUX Enterprise Server.

Agenda

Overview

Definitions

Technology and Products Overview

Generic Benfits And Drawbacks Of Virtual Systems

Product Specific Exploits

Things to Remember

Q&A

Definitions

Hardware Related Terminology

Architecture

Distinct processor architecture (eg. IA32, AMD64, POWER, ...)

System

Physical machine or virtual entity consiting of CPU, memory, IO building blocks to deploy an architecture (eg. PC, MAC, IBM JS20) or Operation System (OS) for data processing

Hardware Platform

Family of systems (PC, IBM POWER, HP Integrity, ...)

Hardware Related Terminology

Processing Capacity

Full or partial performance a CPU can deliver (shared, dedicated, weighted, uncapped)

Memory

Volatile system memory (RAM)

Storage, IO path / channel

Persistent system storage (disk, tapes, ...)

Connection to a storage unit

Related Terminology

Software Platform

OS or application offering a common set of identical interfaces or APIs for application deployment or interaction

Single System Image

Hardware layer, virtual machine monitor or OS spanning multiple physical machines

Architecture / System / Platfom

Testing

I/O

Memory

System

I/O

Memory

System

Platform

Storage

CPU

CPU

CPU

Process of replacing a direct interface tying a resource (often hardware) and its user with an indirect, software-mediated connection

What is Virtualization?

Resource
User

Resource
User

Physical
Resource

Virtualized
Resource

Virtualization Layer

Physical
Resource

Unvirtualized

Virtualized

Virtualization Of Software Resources

Overview

Threads

Shared storage

Shared code

Shared libraries within an OS

Shared working subset across systems

Simulator

Emulator

Virtual machine

Virtualization Of Software Resources

Threads

Process splitting up in multiple execution pathes, context data is distinct to each thread, joint access to application data needs sychronization by semaphores

Shared Memory

Common memory of one or more processes or threads Needs sychronization by semaphores

Shared Storage

Multiple systems can access unique data atomically
Needs sychronization

Virtualization Of Software Resources

Shared Code

Reuse of same code segments among different processes or applications

Shared Libraries

Multiple processes can execute a unique code segment

Shared Working Subset

Multiple systems and their processes can access and execute a unique code segment

Aspects Of Virtualization

Testing

Aggregation - Aggregate pools of resources for allocation to users as virtual resources (Virtual Iron, ScaleMP)

Partitioning - Separate presentation of resources to users from actual physical resources (VMWare XEN, Virtual Iron)

Full Virtualization

The ablity to host unmodified guest OS binaries

Depending on the hardware architecture
requires the need to trap and emulate all privileged instructions
(eg. Vmware, qemu)

Performance will then be negatively
impacted
(eg. resolved by hardware support with
IBM zSeries)

Virtual Machine Monitor

Platform

Virtual Machine

Operating System

Applications

Paravirtualization

Requires hardware dependent portions of the guest OS to be modified to become aware of the virtualization layer

Allows you to avoid hard-to-virtualize processor instructions by replacing them with a procedure call that provides that functionality

Higher performance than full virtualization

A technique in which the VMM is supplemented by an API that provides an assist for certain situations

Virtual Machine Monitor

Platform

Virtual Machine

Operating System

Applications

A

P

I

A

P

I

Types Of Hardware Abstraction

Simulator

Hardware or software reproducing the exact actions of a physical processor or whole system

Emulator

Hardware or software reproducing an equivalent behavior of a physical or virtual system

Virtual Machine

Hardware and/or software rbased environments simulating, emulating or abstracting from a system

Definitions / Terminology

Host system

Hardware or OS to run the guest system

Guest system

Machine or operating system instance running on top of a host system

2nd Level Guest

A guest in a guest

Instance

Gest environment ready to run or running an OS

Host and Guest

Host System

Memory

Guest

vCPU

Memory

Guest

vCPU

Network

Memory

Guest

vCPU

vI/O

vNet

vCPU

vNet

vI/O

vNet

vNet

vI/O

vCPU

vCPU

vCPU

Net

Net

Net

Storage

Disk

Tape

I/O

Disk

Disk

CPU

CPU

CPU

I/O

Net

CPU

I/O

I/O

Net

Maturity States

Beta

Initial version, concept implemented, some or more of intended functionality is already working

Developing

Already multiple versions or fixes released, active development, progress can be expected or is ongoing

Stable

Stable for test and use (production might be an issue)

Maturity States

Production ready

Stable for production deployment

Mission-critical

Stable for production and SLA available

Stalled

Development stalls, no progress since some time

Technology And Product Overview

User Mode Linux - UML

Description / classification

Software based virtual Linux instance / Linux kernel running as a user space application on a Linux host system

Origin

Open source community / Jeff Dike http://user-mode-linux.sourceforge.net/

Maturity

Stable / ongoing development / included and supported with SLES 9

User Mode Linux - UML

Benefit / Drawback

Host and guest basically run same code

No licence charge

Performance impact / needs modified & re-complied kernel / only Linux available as guests / x86 only

Use case

Small instances with dedicated tasks (mail server / web server in separated instances)

Debugging Linux device drivers

Guest instance with access to host file system data

bochs

Description / Classification

Software based x86 system emulator

Origin

open source community / Kevin Lawton

Debugging Linux device drivers

http://bochs.sourceforge.net/

Maturity

Developing / stable

bochs

Benefit / Drawback

Full PC type system emulation capable to run different types of x86 based OS on any target hardware

Guest OS experiences heavy performance loss

Use Case

Test and educational deployment of different x86 based OS

VMware

Description / Classification

Software based virtual host

Full virtualization

Origin

VMware, Inc.

http// www.vmware.com

Maturity

Mission-critical deployment on SLES 8 / SLES 9 / SLA by VMware available

VMware

Benefit / Drawback

High performance / product portfolio from desktop to server / supports multiple OS as guests (Linux, MS Windows, NetWare, DOS, etc, ... not OS/2)

File system behavior regarding modification can be chosen (eg. dump changes on guest restart)

Software & support license /x86 hardware only / AMD64/Intel EM64T in development

Use Case

Consolidate multiple OS environments to few or single machine / smooth fail over and instance migration on standby host (geo cluster possible)

Virtual Iron

Description / Classification

Software based grid capable virtual machine monitor and frame work / paravirtualization

Origin

Virtual Iron, Inc.,

http://www.virtualiron.com

Maturity

Stable / developing (GA announced for May 2005)

Virtual Iron

Benefit / Drawback

Single system image capable to build a partionable host out of aggregated x86 based systems

Hot add & remove of CPUs, memory, IO channels and storage

Needs modified Linux kernel for the guest

Software & support license / x86 hardware only / AMD64/Intel EM64T in development

Use Case

Flexible and scalable OS and application deployment infrastructure

Xen

Description / classification

Software based visual machine monitor / paravirtualization

Origin

Open source project / University Of Cambridge, UK

http://www.cl.cam.ac.uk/netos/xen

Maturity

Stable / ongoing development / not ready for mission critical use

Xen

Benefit / Drawback

Start / stop of instances and migration capability to second host / no license charge for monitor

Commercial support offering by Xensource

Near native performance

Needs modified guest kernel / only open source OS available as guest right now

Use Case

Partitioning of a single machine

Hot migration of instances / HA & fail over

Xen Architecture

NIC, USB, ...

CPUs, Mem, IRQ

Xen

Linux

(domain 0)

Linux
(domain n)

Linux
(domain 1)

...

Userspace

Userspace

Userspace

System hardware

qemu

Description / classification

Software based processor or system emulation using dynamic translation

Full virtualization

Origin

Open source community / Fabrice Bellard http://fabrice.bellard.free.fr/qemu/

Maturity

Stable / ongoing development / not ready for mission critical use (no SLA available)

qemu

Benefit / Drawback

Good emulation speed due to dynamic translation but still large performance impact

Two operation modes: full system emulation or user mode emulation (Linux only, 2x faster)

Can emulate several processor architectures x86, ppc, arm

Use Case

Test and educational deployment of different OS

Consolidate OS environments to few or single machine

pearpc

Description / classification

Software based POWER4 and POWER5 processor and system emulator

Origin

Open source community / Sebastian Biallas

http://pearpc.sourceforge.net/

Maturity

Developing / stable for test / boots eg. MAC OS X, PPC Linux

pearpc

Benefit / drawback

Deployment of POWER based OS on non-POWER hardware

Guest experiences heavy performance loss

Not included or supported by SLES 9

Use case

Cross platfrom development and test

Educational deployment (Mac OS X, PowerPC Linux)

hercules/390

Description / classification

Software based IBM S/390 ( 31 bit/ESA) and
IBM zSeries (64 bit/ESAME) emulator

Origin

Open source community

http://www.conmicro.cx/hercules

Maturity

Stable for test / ongoing developments

hercules/390

Benefit / drawback

Runs all types of S/390 based OS on a different hardware platform

Non license charge / guest OS experiences heavy performance loss (1/150 of host native performance)

Not IBM supported / not all S/390 architectured features are implemented

Use case

Historical, test, educational S/390 based OS and application deployment (OS/360, MVS, zVM, zOS needs license) or zLinux (no license charge)

Cross hardware platform development and test possible

Education and training of S/390 architecture specific features

FLEX-ES

Description / classification

Software based IBM S/390 (31bit/ESA) and IBM zSeries (64bit/ESAME) emulator with dynamic translation

Origin

Fundamental Software, Inc. Fremont, California

http://www.funsoft.com/

Maturity

Mission-critical use / SLA available from fundamental Software

FLEX-ES

Benefit / drawback

Runs all types of S/390 based OS on x86 hardware (with performance impact for guest system)

License and support charge / IBM supported

Reuse of mainframe type IO hardware / available on different host OS including Linux

Use case

Migrate legacy applications with low computational needs to alternative environment

Access to host type IO attachments and data structures

Logical Partitions

Description / classification

Hardware and microcode based and hypersior virtualized machine

Origin

IBM / IBM zSeries and IBM POWER5

Maturity

Secure (EAL5) and mission-critical ready / support and SLA available by IBM

Logical Partitions

Benefit / drawback

Efficient use of statistical workload / capacity on demand (hotplug memory and CPUs)

Up to 60 partitions (IBM z990) or 640 partition (64way IBM POWER5)

License charge / support charge

Use case

Partition a machine to run multiple types of OS: zOS, zVM, VSE, zLinux, OS/400

Assign dedicated logical machines to different workloads / very secure separation of workloads

zVM

Description / classification

Hardware, microcode, and software based virtual machine for S/370 and successors up to current IBM zSeries

Origin

IBM zSeries

Maturity

Mission-critical ready / support and SLA available by IBM

zVM

Benefit / drawback

Efficient use exploiting statistical workload

Capacity on demand (CoD)

License charge and support charge

Only available on ESA / ESA/ME architecture type machines

Use case

Hosting up to thousands of instances

Run different types and levels of OS: zOS, zVM, VSE, zLinux on one machine

Flexible provisioning of machines

Generic Benfits And Drawbacks Of Virtual Systems

Virtual Instances Benefits

Hardware abstraction layer

Processing resources, memory, IO pathes and storage are provided by host system and thus reduce the variety of required device drivers

Fast system deployment or restore

Copying customed or saved instances within seconds

Mass deployment of predefined setups

Centralized backup

Resource Manageing and Control

grant full or partial resource access and use

stop on request and resume operation later

Virtual Instances Benefits

Test system, test network and isolation

Deployment without extra hardware cost & lead times

Test machines or networks reside inside host

Minimized OS and single applications setup reduces complexity and interdependency of the managed system

Small special purpose systems

Memory and storage footprint of customized systems can be significantly reduced

Virtual Instances Benefits

Application separation

Minimized OS and single applications setup reduces complexity and interdependency of the managed system

Reduced amount of hardware

Reduces amount of physical machine to be serviced

Reduced amounted airconditioned floor space

Spare phyical networking components due to intra system connections

Virtual hardware is the same for all instances thus reducing system variety

Virtual Instances Drawbacks

Administration

Each system contributes to administrative and planning efforts

Resource Monitoring

A fine granular resource monitor is required to identify resource abusing guests

Needs increased host availability

Host system crashes will also kill guest systems

Product Specific Exploits

Hotplug CPU

SLES 9 IBM POWER and IBM zSeries:

Add CPUs dynamically while system is running to a Logical Partition or zVM instance thus increasing processing capacity

Redirecting unused processing capacity to where its needed to increase throughput and responsiveness of applications

Hotplug Storage

SLES 9:

Add and remove persistent disk space at runtime

Dynamical reaction or proactive provisioning of temporary or permanent disk space

Add and migrate data to file system with different characteristics (eg. stripping from performance, raid5 for more data integrity)M

Multipathing support prevents IO path outage on failure or service

CKRM

SLES 9:

Class Based Kernel Resource Management included into the kernel

Soft partitioning of processes and process groups within the OS

Resource control of processing capacity, IO throughput, memory consumption of processes

CPUset / PAGG / IPF

SLES 9 IPF SP1 benefit for SGI machines:

Grouping of CPUs to sets and assigning them to one dedicated application

Separate multiple instances of identical application but with different amount of processors

Improved resource usage accounting for processes or process sets

Hardware partitioning support in OS included

IBM zSeries and IBM POWER processing resource managment

SLES 8 / 9 IBM benefit from zSeries capabilities:

Shared CPUs: processing capacity shared equally

Dedicated CPUs: CPU does not share capacity, even if idle to preserve response time

Weighted CPUs: receive dedicated percentage of one CPU or multiple CPUs

LPAR and zVM virtual CPUs: both can define more CPUs than available physical CPUs, physical CPUs can be added later contributing to overall processing capacity

IBM POWER virtual 10 devices

SLES 8 / 9 IBM benefit from POWER capabilities:

Shared network within machine connecting partitions and real network cards

Access and (exclusively) shared virtual SCSI disks or partitions on these drives

Hot swap CPU to react on dynamic workloads

Uncapped CPU: dedicated CPU granting its processing capacity to others if idle

IBM zSeries zVM CMM

SLES 8 / 9 IBM zSeries

zCMM = collaborative memory management

OS guest shares kernel and active page information with zVM host to identify kernel and unused user space pages

If host OS needs to page guest OS unsued pages get paged out first

Improved overall zVM host performance

IBM zSeries DCSS

SLES 9 IBM zSeries

Shared readonly code segments between zVM guest

Dramatical decrease in overall memory foot print of participating instances

Update of master instance requires needs all guests to be restarted

Example SUSE development

SUSE / IBM zSeries and IBM POWER

Logical partitioning into production/test/development instances

Different weights for production (batch) / test (batch and interactive) / deveoplement (interactive)

Intra system virtual network connection for fast instance setup

Using multipath feature for increased IO bandwidth and bypass OS IO queue bottle necks

Examples SLES 9 IBM zSeries

Large Telecom Customer (Germany)

Cloning of instances by copying preinstalled systems

On-the-fly customizing of instances by manipulating guest config files inserting predefined customized configuration data

Time delay from customized system request to production: < 15min

Example VMware

VMware and Friedrich Scharr KG

Server consolidation of serveral machines

Use of dedicated application servers: Citrix, Samba and email server

Centralized backup of hosting server

http://www.vmware.com/customers/stories/

Example Itanium

IPF / Unisys ES7000 with SLES 9

Static partitioning and aggregation of system resources to logical servers / partitions

Dynamic CPU assigment possible by build in load balancer (dynamic partioning)

http://www.unisys.com/products/es7000__linux/ \	features/index.htm

Things to Remember

Many virtualization technologies are available in the market place: maturity, performance and manageability varies from case to case

Evaluate benefit of virtualization deployment technologies vs. system admin efforts: have a close look what you really need, then decide

SUSE LINUX Enterprise Server 9 is able to exploit various different virtualization technologies today: scenario & benefit depending on customer use case

1

2

3

Thank you. Question & Answers.

Backup Slides

Definitions / Terminology

SLA

service level agreement

UP

uni or single processor

SMP

symmetric multiple processor

IBM zSeries Hipersockets

SLES 8 / 9 IBM zSeries

Intra host based network / 4 separate networks available

Any LPAR or zVM guest can connect to these networks

Data transfer rates benefit from zSeries memory speed and data protection

Fast TCP/IP based interconnect between different instances and OS types (eg. data on z/OS available to Linux instance)

Fast routing between instances

UML and other Linux virtualization Technologies

2004-01-28T12:03:58