This page is part of my virtualization context i.e. from my point of view talking/doing virtualization includes

• the OS (Operating System) part e.g. OpenVZ, Xen, Linux-VServer, VMware, KVM, etc. and
• the storage part e.g. LVM (Logical Volume Manager), a world-class solution for doing storage virtualization.

  

• The URL (Uniform Resource Locator) for Linux-VServer: http://linux-vserver.org/Welcome_to_Linux-VServer.org.
• As ever, it certainly is a good idea to read the FAQs (Frequently Asked Questions) at first.
• Aside from Linux-VServer, I also use Xen ...
• Hint: Join the IRC (Internet Relay Chat) channel on #vserver@irc.oftc.net. The IRC support of Linux-VServer is outstanding!

What is Linux-VServer?

Virtualization is a framework or methodology of dividing the resources of a computer into multiple execution environments. As a perceived hot term, virtualization has been claimed by IT (Information Technology) marketers to refer to everything from virtual machines to systems management software, so as to become nearly meaningless. Some common applications of virtualization are listed below -– this list reflects the extreme diversity that the term has come to encapsulate.

Emulation

A software emulator allows computer programs to run on a platform (computer architecture and/or operating system) other than the one for which they were originally written. Unlike simulation, which only attempts to reproduce a program’s behavior, emulation attempts to model to various degrees the state of the device being emulated. The virtual machine simulates the complete hardware, allowing an unmodified OS (Operating System) for a completely different CPU to be run. This is also known as Dynamic Recompilation.

Paravirtualization

Paravirtualization is a virtualization technique that presents a software interface to virtual machines that is similar, but not identical to that of the underlying hardware. This requires operating systems to be explicitly ported (as can be seen below) to run on top of the VMM (Virtual Machine Monitor) also known as Hypervisor but may enable the VMM itself to be simpler and for the virtual machines that run on it to achieve higher performance. Xen for example, is an example for paravirtualization.

Native Virtualization

Native virtualization is a virtualization technique where the virtual machine only partially simulates enough hardware to allow an unmodified Operating System to be run in isolation, but the guest Operating System must be designed for the same type of CPU.

Operating System-Level Virtualization

Operating System-level Virtualization is a server virtualization technology which virtualizes servers at the operating system (kernel) layer. It can be thought of as partitioning a single physical server into multiple small computational partitions. Each such partition looks and feels like a real server, from the point of view of its owner. On Unix systems, this technology can be thought of as an advanced extension of the standard chroot mechanism.

The Linux-VServer approach

At a basic level, a Linux server consists of three building blocks: hardware, kernel and applications. The hardware usually depends on the provider or system maintainer, and, while it has a big influence on the overall performance, it cannot be changed that easily, and will likely differ from one setup to another.

The main purpose of the kernel is to build an abstraction layer on top of the hardware to allow processes (applications) to work with and operate on resources (data) without knowing the details of the underlying hardware. Ideally, these processes would be completely hardware agnostic, by being written in an interpreted language and therefore not requiring any hardwarespecific knowledge. Given that a system has enough resources to drive ten times the number of applications a single Linux server would usually require, why not put ten servers on that box, which will then share the available resources in an efficient manner? Most server applications (e.g. httpd) will assume that it is the only application providing a particular service, and usually will also assume a certain filesystem layout and environment.

This dictates that similar or identical services running on the same physical server, but for example, only differing in their addresses, have to be coordinated. This in turn typically requires a great deal of administrative work which can lead to reduced system stability and security.

The basic concept of the Linux-VServer — as well as OpenVZ — solution is to separate the userspace environment into distinct units (sometimes called VPSs (Virtual Private Servers) in such a way that each VPS looks and feels like a real server to the processes contained within. Although different Linux distributions use (sometimes heavily) patched kernels to provide special support for unusual hardware or extra functionality, most Linux distributions are not tied to a special kernel. Linux-VServer uses this fact to allow several distributions, to be run simultaneously on a single, shared kernel, without direct access to the hardware, and share the resources in a very efficient way.

Installation

The operation system kernel is often likened to the conductor in an orchestra. Among other things, it makes sure that all other processes in the system work together coherently. Though it is only a small part of the operating system, the kernel has the most important job of keeping everything else synchronized. Since many processes can be running on the computer at the same time, and since the hardware resources are limited, the kernel decides when and how long a program should be able to make use of a piece of hardware.

Why Rebuild?

Why rebuild the kernel? The main reason is to optimize the kernel to an environment (hardware and usage patterns) or to gain better performance. Another purpose is, if there is a particular feature that one needs (e.g. real-time, virtualization, support for a particular file system, etc.). Reasons why one may want to or should rebuild a kernel can be classified:

Testing:

A kernel hacker has to rebuild in order to see if everything works as expected.

Hardware:

Rebuild to blend with specific hardware for performance reasons.

Size:

Configure and compile in order to get a smaller kernel binary file than the off-the-shelf version.

Security

People, governments as well as many companies insist that parts or the whole source code, the kernel should be build of, gets reviewed by humans –- people might for example make sure, that there is no backdoor built-in, which for example might allow secret services to spy on them. What if, for example, there are two competing companies. One of them gets managed to put a backdoor onto its competitors computers? Such things happen, but no person in management knows such things are possible (sure it is not their business — in most cases they have a very limited computer knowledge).

Have you ever controlled the computers of your superiors, read all their mail, listened to all their VoIP calls and redirected a copy of their computer screen to your notebook 1300 miles away, while you were sitting in your parents garden and watched them gaping because you managed to control their webcam as well? If you explain all those dangers, those folks are getting all ears in the blink of an eye! It is best, a companies employee (a guardian angel) takes care of that an builds a protection shield in order to protect the people instead of a cracker does all those things and really causes harm.

The very good thing about Linux and proficient people with knowledge about it is, all of those dangers can be eliminated¹. While operating systems following the closed source model (e.g. Microsoft Windows) deliver all those dangers with only limited capabilities for people or companies to patronize themselves. Unfortunately, the majority of companies (even global players) underestimate the danger although they become more and more reliant on their IT (Information Technology) every day. Sometimes it really scares me –- it is like, as if companies and people do not put their money to a bank but leave it all on its own in a train crowded with people and then go away for dinner.

Enhance Capabilities or change existing Functionality

One wants capabilities which are not carried out by the kernel or wants to change built-in functionality – therefor a person has source code available (written by the person himself or someone else).

Personal Interest

You want to be informed about the subject.

Other reasons could have various intentions — philosophical, emotional, political or economical for example.

Mistakes and Dangers

As everywhere else, one can make mistakes and these could lead to errors, malfunctions and therefore it is necessary to point out possible dangers and sticking points. I will mention them throughout the reminder of this page.

Wrong parameters could cause a computer system to fail to boot (very bad if someone is remotely connected), software to malfunction, or hardware peripherals to become unavailable. It is a good idea to always practice on a test system and keep a backup copy of the old kernel. Whenever possible, ask or hire someone with kernel experience to help, and use decent literature to learn about the subject and be prepared for what to expect.

Creating a boot floppy

Before building a new kernel, the first thing to do is making a boot disk for the kernel we are running. This way, if anything gets screwed up we will be able to boot the machine and fix it. To create a GRUB boot floppy, we need to take the files stage1 and stage2 from the image directory, and write them to the first and the second block of the floppy disk. So, everybody get a floppy disk he does not mind erasing now and do this (as the root user):

sa@pc1:~$ locate stage{1,2} | egrep stage1$\|2$
/boot/grub/stage1
/boot/grub/stage2
/usr/lib/grub/i386-pc/stage1
/usr/lib/grub/i386-pc/stage2
sa@pc1:~$ cd /usr/lib/grub/i386-pc/
sa@pc1:/usr/lib/grub/i386-pc$ su
Password:
pc1:/usr/lib/grub/i386-pc# dd if=stage1 of=/dev/fd0 bs=512 count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 1.21664 seconds, 0.4 kB/s
pc1:/usr/lib/grub/i386-pc# dd if=stage2 of=/dev/fd0 bs=512 seek=1
206+1 records in
206+1 records out
105704 bytes (106 kB) copied, 7.46012 seconds, 14.2 kB/s
pc1:/usr/lib/grub/i386-pc# exit
exit
sa@pc1:/usr/lib/grub/i386-pc$

The device file name may be different. One should consult the manual for his OS (Operating System).

Making a GRUB bootable CD-ROM

GRUB supports the no emulation mode in the El Torito specification². This means that we can use the whole CD-ROM from GRUB and therefore we do not have to make a floppy or hard disk image file, which can cause compatibility problems. For booting from a CD-ROM, GRUB uses a special Stage 2 file called stage2_eltorito. The only GRUB files we need to have in our bootable CD-ROM are stage2_eltorito and optionally a config file menu.lst. We don’t need to use ‘stage1’ or ‘stage2’, because El Torito is quite different from the standard boot process. Here is an example of procedures to make a bootable CD-ROM image. First, we make a top directory for the bootable image, say, ‘iso’. If desired, one could put the config file ‘menu.lst’ under ‘iso/boot/grub’, and copy any files and directories for the disc to the directory ‘iso/’. Finally, we make a ISO9660 image file like this:

sa@pc1:/tmp/test$ mkdir -p iso/boot/grub
sa@pc1:/tmp/test$ cp /usr/lib/grub/i386-pc/stage2_eltorito iso/boot/grub/
sa@pc1:/tmp/test$ mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot
-input-charset iso8859-1 -boot-load-size 4 -boot-info-table -o grub.iso iso
Size of boot image is 4 sectors -> No emulation
Total translation table size: 2048
Total rockridge attributes bytes: 760
Total directory bytes: 4096
Path table size(bytes): 34
Max brk space used 0
231 extents written (0 MB)
sa@pc1:/tmp/test$ ls -l
total 472
-rw-r--r-- 1 sa sa 473088 2006-10-08 13:16 grub.iso
drwxr-xr-x 3 sa sa 4096 2006-10-08 13:15 iso
sa@pc1:/tmp/test$

This produces a file named ‘grub.iso’, which then can be burned onto a CD (or a DVD). mkisofs has already set up the disc to boot from the ‘boot/grub/stage2_eltorito’ file, so there is no need to setup GRUB on the disc. (Note that the ‘-boot-load-size 4’ bit is required for compatibility with the BIOS on many older machines.) Then, when booting from CD, we can use the device ‘(cd)’ to access a CD-ROM in our config file. This is not required – GRUB automatically sets the root device to ‘(cd)’ when booted from a CD-ROM. It is only necessary to refer to ‘(cd)’ if we want to access other drives as well.

Memory Management