This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
docs:vserver [2012-10-28 12:39] glen cleanup dead links, update desc |
docs:vserver [2015-10-05 15:07] (current) glen syn on |
||
---|---|---|---|
Line 14: | Line 14: | ||
===== Installing Vserver host on PLD Linux ===== | ===== Installing Vserver host on PLD Linux ===== | ||
- | Build and install the kernel: | ||
- | |||
- | <file>$ ./builder -bb kernel.spec | ||
- | # poldek -u kernel | ||
- | </file> | ||
+ | Ensure you have appropriate [[packages:kernel]] installed. | ||
+ | You can check this from kernel config: | ||
+ | <code> | ||
+ | # modprobe configs | ||
+ | # zgrep CONFIG_VSERVER /proc/config.gz | ||
+ | CONFIG_VSERVER=y | ||
+ | </code> | ||
===== Installing guest PLD Linux Vserver ===== | ===== Installing guest PLD Linux Vserver ===== | ||
==== Preparing userspace tools ==== | ==== Preparing userspace tools ==== | ||
- | First, install the tools: | ||
- | <file># poldek -u util-vserver | + | First, install the tools: |
+ | <file> | ||
+ | # poldek -u util-vserver | ||
</file> | </file> | ||
- | Then configure //poldek.conf//: | ||
- | <file># vim /etc/vservers/.distributions/pld-XX/poldek/repos.d/pld.conf | + | If you need to review poldek repo sources, then the configs are in |
- | </file> | + | ''/etc/vservers/.distributions/pld-*/poldek/'' |
- | Where XX is: ac, ti, th | + | where ''*'' can be ''ac'' or ''th'' depending which guest you wish to install. |
At this point you should have booted into vserver-enabled kernel. You must start ''vprocunhide'' or none of your Vservers can start. | At this point you should have booted into vserver-enabled kernel. You must start ''vprocunhide'' or none of your Vservers can start. | ||
Line 39: | Line 41: | ||
To start ''vprocunhide'': | To start ''vprocunhide'': | ||
- | <file># /sbin/service vprocunhide start | + | <file> |
+ | # /sbin/service vprocunhide start | ||
</file> | </file> | ||
+ | ==== Guest creation ==== | ||
- | ==== Actual guest creation ==== | + | Build the guest system. |
- | Build the guest system: | + | |
+ | <file> | ||
+ | # a guest name (not hostname) | ||
+ | NAME=test | ||
+ | # <num> must be a number within 2-32767 range. | ||
+ | CTX=2 | ||
- | <file># vserver test build --context <num> -m poldek -n test | + | vserver $NAME build --context $CTX -m poldek -n $NAME |
</file> | </file> | ||
- | <num> must be a number within 2-32767 range. | ||
- | There are two versions of PLD available for guest systems: | + | This defaults installing guest same ARCH and VERSION that your host is. |
+ | |||
+ | If you need to use another combination, then there are two versions of PLD available for guest systems: | ||
+ | * pld-ac - [[:ac|PLD 2.0 (Ac)]] | ||
+ | * pld-th - [[:th|PLD 3.0 (Th)]] | ||
- | * pld-ac - [[:AcInfo|PLD 2.0 (Ac)]] | ||
- | * pld-th - [[:ThInfo|PLD 3.0 (Th)]] | ||
You may choose one using ''-d'' option: | You may choose one using ''-d'' option: | ||
+ | <file> | ||
+ | DIST=pld-th | ||
- | <file># vserver test build --context <num> -m poldek -n test -- -d pld-th | + | vserver $NAME build --context $CTX -m poldek -n $NAME -- -d $DIST |
</file> | </file> | ||
+ | |||
using ''util-vserver >= 0.30.214-2'' from ac-updates, ''util-vserver >= 0.30.215-2'' from from th you can build other arch or distro or using own mirror: | using ''util-vserver >= 0.30.214-2'' from ac-updates, ''util-vserver >= 0.30.215-2'' from from th you can build other arch or distro or using own mirror: | ||
- | <file># vserver test build --context <num> -m poldek -n test -- -m http://ftp.pld-linux.org/dists/ac | + | <file> |
- | # vserver test build --context <num> -m poldek -n test --personality linux_32bit --machine i686 -- -d pld-th | + | MIRROR=http://ftp.pld-linux.org/dists/ac |
+ | |||
+ | vserver $NAME build --context $CTX -m poldek -n $NAME -- -m $MIRROR | ||
</file> | </file> | ||
+ | |||
+ | To build 32bit guest on 64bit host: | ||
+ | <file> | ||
+ | vserver $NAME build --context $CTX -m poldek -n $NAME --personality linux_32bit --machine i686 -- -d $DIST | ||
+ | </file> | ||
+ | |||
To build vserver from template (archive containing whole filesystem): | To build vserver from template (archive containing whole filesystem): | ||
- | <file># vserver test build --context <num> -m template -n test -- -t image.tar.bz2 | + | <file> |
+ | # vserver $NAME build --context $CTX -m template -n $NAME -- -t image.tar.bz2 | ||
</file> | </file> | ||
To see other ''build'' command options: | To see other ''build'' command options: | ||
- | <file># vserver test build --help | + | <file> |
+ | # vserver test build --help | ||
</file> | </file> | ||
+ | |||
Install ''rc-scripts'' to the new system using ''vpoldek'': | Install ''rc-scripts'' to the new system using ''vpoldek'': | ||
- | <file># vpoldek test -- -u rc-scripts | + | <file> |
+ | # vpoldek test -- -u rc-scripts | ||
</file> | </file> | ||
+ | |||
you should consider installing ''vserver-packages'' rpm package to satisfy packages dependency which have no use inside vserver. | you should consider installing ''vserver-packages'' rpm package to satisfy packages dependency which have no use inside vserver. | ||
And then start the guest system: | And then start the guest system: | ||
- | <file># vserver test start | + | <file> |
+ | # vserver test start | ||
</file> | </file> | ||
+ | |||
To enter that vserver, type: | To enter that vserver, type: | ||
- | <file># vserver test enter | + | <file> |
+ | # vserver test enter | ||
</file> | </file> | ||
- | Note, however, that if you don't run //plain// init style you must have at least one daemon running inside your guest vserver or it will be shut down shortly. | ||
+ | Note, however, that if you don't run //plain// init style you must have at least one daemon running inside your guest vserver or it will be shut down shortly. | ||
Line 363: | Line 392: | ||
[[http://www.solucorp.qc.ca/howto.hc?projet=vserver&id=72|http://www.solucorp.qc.ca/howto.hc?projet=vserver&id=72]] | [[http://www.solucorp.qc.ca/howto.hc?projet=vserver&id=72|http://www.solucorp.qc.ca/howto.hc?projet=vserver&id=72]] | ||
- | You can use //lcap// program to see available capatabilities: | + | You can use //lcap// program to see available capabilities: |
Line 547: | Line 576: | ||
==== Running 32 bit vserver on an 64 bit host ==== | ==== Running 32 bit vserver on an 64 bit host ==== | ||
- | With recent PLD util-vserver package you can create 32-bit guest systems inside a 64-bit host. First you need to prepare a new distribution definition skeleton: | ||
+ | With recent [[package>util-vserver]] package you can create 32-bit guest systems inside a 64-bit host. | ||
+ | To specify arch during guest creation, use ''-d'' option, and to change what ''uname'' returns, use arguments ''%%--personality linux_32bit --machine i686%%'': | ||
- | <file># mkdir -p /etc/vservers/.distributions/pld-th-i686/poldek/repos.d | + | <file># vserver test build --context <num> -n test -m poldek -- -d pld-th-i686 --personality linux_32bit --machine i686 |
</file> | </file> | ||
- | Then copy your repository configuration to ''/etc/vservers/.distributions/pld-th-i686/poldek/repos.d/pld.conf'' and change the architecture and source paths to your liking. When configuration is ready, create a new guest vserver using the ''-d'' command line option: | ||
- | + | If you need to set ''uts'' parameters afterwards, you can just echo them: | |
- | + | <file> | |
- | <file># vserver test build --context <num> -n test -m poldek -- -d pld-th-i686 | + | # echo linux_32bit >> /etc/vservers/test/personality |
+ | # echo i686 > /etc/vservers/test/uts/machine | ||
</file> | </file> | ||
- | Later to force i686 32bit use: | ||
- | |||
- | |||
- | <file># echo linux_32bit >> /etc/vservers/test/personality | ||
- | # echo i686 > /etc/vservers/test/uts/machine | ||
- | </file> | ||
- | however, you can do that at vserver build time using arguments ''--personality linux_32bit --machine i686''. | ||
Line 687: | Line 710: | ||
+ | ==== Running auditd inside guest ==== | ||
+ | |||
+ | You need ''CAP_AUDIT_CONTROL'' in ''bcapabilities'' and lower ''priority_boost'' to ''0'' in ''/etc/audit/auditd.conf'' | ||
+ | |||
+ | ==== XFS filesystem - kernel upgrade causes xfs related oops (xfs_filestream_lookup_ag) ==== | ||
+ | |||
+ | After upgrading from 2.6-3.4 kernels (possibly other versions) to 3.18 (tested, possibly other versions) kernel ooppses | ||
+ | almost immediately after accessing some files on xfs filesystem with ''xfs_filestream_lookup_ag'' visible in stack trace | ||
+ | (or other filestream related function). | ||
+ | |||
+ | That's because vserver patch for kernels earlier than 2.6.23 patched xfs filesystem to introduce new flag: | ||
+ | |||
+ | <file c> | ||
+ | #define XFS_XFLAG_BARRIER 0x00004000 /* chroot() barrier */ | ||
+ | </file> | ||
+ | |||
+ | and files/dirs with such flag got saved on your filesystem. | ||
+ | |||
+ | Starting with kernel 2.6.23 kernel introduced filestreams which are using 0x00004000 bit, thus causing conflict with vserver. | ||
+ | |||
+ | <file c> | ||
+ | #define XFS_XFLAG_FILESTREAM 0x00004000 /* use filestream allocator */ | ||
+ | </file> | ||
+ | |||
+ | Vserver stopped adding such xfs xflag in 3.13 BUT your existing filesystem can still have XFS_XFLAG_BARRIER (0x00004000) set | ||
+ | causing oops in newer kernels. | ||
+ | |||
+ | How to find out if I'm affected? | ||
+ | |||
+ | IIF you don't use filestream feature then modify http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=blob_plain;f=src/bstat.c;hb=HEAD to show only files containing XFS_XFLAG_FILESTREAM | ||
+ | |||
+ | <file diff> | ||
+ | diff --git a/src/bstat.c b/src/bstat.c | ||
+ | index 4e22ecd..887512f 100644 | ||
+ | --- a/src/bstat.c | ||
+ | +++ b/src/bstat.c | ||
+ | @@ -34,19 +34,21 @@ dotime(void *ti, char *s) | ||
+ | void | ||
+ | printbstat(xfs_bstat_t *sp) | ||
+ | { | ||
+ | - printf("ino %lld mode %#o nlink %d uid %d gid %d rdev %#x\n", | ||
+ | - (long long)sp->bs_ino, sp->bs_mode, sp->bs_nlink, | ||
+ | - sp->bs_uid, sp->bs_gid, sp->bs_rdev); | ||
+ | - printf("\tblksize %d size %lld blocks %lld xflags %#x extsize %d\n", | ||
+ | - sp->bs_blksize, (long long)sp->bs_size, (long long)sp->bs_blocks, | ||
+ | - sp->bs_xflags, sp->bs_extsize); | ||
+ | - dotime(&sp->bs_atime, "atime"); | ||
+ | - dotime(&sp->bs_mtime, "mtime"); | ||
+ | - dotime(&sp->bs_ctime, "ctime"); | ||
+ | - printf( "\textents %d %d gen %d\n", | ||
+ | - sp->bs_extents, sp->bs_aextents, sp->bs_gen); | ||
+ | - printf( "\tDMI: event mask 0x%08x state 0x%04x\n", | ||
+ | - sp->bs_dmevmask, sp->bs_dmstate); | ||
+ | + if (sp->bs_xflags & XFS_XFLAG_FILESTREAM) { | ||
+ | + printf("ino %lld mode %#o nlink %d uid %d gid %d rdev %#x\n", | ||
+ | + (long long)sp->bs_ino, sp->bs_mode, sp->bs_nlink, | ||
+ | + sp->bs_uid, sp->bs_gid, sp->bs_rdev); | ||
+ | + printf("\tblksize %d size %lld blocks %lld xflags %#x extsize %d\n", | ||
+ | + sp->bs_blksize, (long long)sp->bs_size, (long long)sp->bs_blocks, | ||
+ | + sp->bs_xflags, sp->bs_extsize); | ||
+ | + dotime(&sp->bs_atime, "atime"); | ||
+ | + dotime(&sp->bs_mtime, "mtime"); | ||
+ | + dotime(&sp->bs_ctime, "ctime"); | ||
+ | + printf( "\textents %d %d gen %d\n", | ||
+ | + sp->bs_extents, sp->bs_aextents, sp->bs_gen); | ||
+ | + printf( "\tDMI: event mask 0x%08x state 0x%04x\n", | ||
+ | + sp->bs_dmevmask, sp->bs_dmstate); | ||
+ | + } | ||
+ | } | ||
+ | </file> | ||
+ | |||
+ | and then run it with mounted directory of each filesystem (bstat /; bstat /home etc). It will print "ino ..." information for filestream files. | ||
+ | |||
+ | |||
+ | How to clean up? | ||
+ | |||
+ | rsync files to other partition, recreate problematic partition and then copy files back. | ||
===== Debian or Ubuntu guest installation ===== | ===== Debian or Ubuntu guest installation ===== | ||
Line 960: | Line 1060: | ||
</file> | </file> | ||
- | * load the ''vroot'' module and add it to your ''/etc/modules'' | + | * load the ''vroot'' module and add it to your ''/etc/modules''. you can optionaly increase max vroot number of devices by putting the limit in your ''/etc/modprobe.conf'': <file>options vroot max_vroot=64 |
- | * you can optionaly increase max vroot number of devices by putting the limit in your ''/etc/modprobe.conf'': <file>options vroot max_vroot=64 | + | |
</file> | </file> | ||
- | assing a free vroot node for the device in question: <file># vrsetup /dev/vroot3 /dev/space/vserver1_home | + | * assing a free vroot node for the device in question: <file># vrsetup /dev/vroot3 /dev/space/vserver1_home |
</file> | </file> | ||
- | copy the vroot device to the guest: <file># cp -af /dev/vroot3 /vservers/test/dev/ | + | * copy the vroot device to the guest: <file># cp -af /dev/vroot3 /vservers/test/dev/ |
</file> | </file> | ||
- | add to ''/etc/vservers/test/apps/init/mtab'': <file>/dev/vroot3 /home/ xfs defaults,usrquota 0 0 | + | * add to ''/etc/vservers/test/apps/init/mtab'': <file>/dev/vroot3 /home/ xfs defaults,usrquota 0 0 |
</file> | </file> | ||
- | add ''quota_ctl'' to ''/etc/vservers/test/ccapabilities'': | + | * add ''quota_ctl'' to ''/etc/vservers/test/ccapabilities'': |
- | restart your vserver and run ''edquota'' inside | + | * restart your vserver and run ''edquota'' inside |
===== Network namespace in vservers ===== | ===== Network namespace in vservers ===== | ||
+ | |||
+ | Starting from util-vserver 0.30.216-1.pre3054 there is basic support for creating network namespaces with interfaces inside. | ||
+ | |||
+ | Enabling netns and two capabilities: NET_ADMIN (allows interfaces in guest to be managed) and NET_RAW (makes iptables working). | ||
+ | |||
+ | |||
+ | <file>mkdir /etc/vservers/test/spaces | ||
+ | touch /etc/vserver/test/spaces/net | ||
+ | echo NET_ADMIN >> /etc/vservers/test/bcapabilities | ||
+ | echo NET_RAW >> /etc/vservers/test/bcapabilities | ||
+ | echo 'plain' > /etc/vservers/test/apps/init/style | ||
+ | </file> | ||
+ | |||
+ | Avoid context isolation since it makes little sense when using network namespaces: | ||
+ | <file>touch /etc/vserver/test/noncontext</file> | ||
+ | |||
+ | Configure interfaces: | ||
+ | |||
+ | 0 - arbitrary directory name, just for ordering | ||
+ | |||
+ | myiface0 will be interface name inside of guest (optional, default geth0, | ||
+ | geth1 and so on) | ||
+ | |||
+ | veth-host - interface name on the host side | ||
+ | |||
+ | <file> | ||
+ | mkdir -p /etc/vservers/test/netns/interfaces/0 | ||
+ | echo myiface0 > /etc/vservers/test/netns/interfaces/guest | ||
+ | echo veth-host > /etc/vservers/test/netns/interfaces/host | ||
+ | </file> | ||
+ | |||
+ | !!! FINISH ME. FINISH ME. FINISH ME. !!! | ||
+ | |||
+ | ===== Network namespace in vservers (OLD WAY) ===== | ||
Enabling netns and two capabilities: NET_ADMIN (allows interfaces in guest to be managed) and NET_RAW (makes iptables working). | Enabling netns and two capabilities: NET_ADMIN (allows interfaces in guest to be managed) and NET_RAW (makes iptables working). | ||
Line 985: | Line 1117: | ||
- | <file>mkdir /etc/vserver/test/spaces | + | <file>mkdir /etc/vservers/test/spaces |
- | touch /etc/vserver/test/spaces/net | + | touch /etc/vservers/test/spaces/net |
echo NET_ADMIN >> /etc/vservers/test/bcapabilities | echo NET_ADMIN >> /etc/vservers/test/bcapabilities | ||
echo NET_RAW >> /etc/vservers/test/bcapabilities | echo NET_RAW >> /etc/vservers/test/bcapabilities | ||
Line 1008: | Line 1140: | ||
ip link add name "${VSERVER_HOST_IFACE}" type veth peer name "${VSERVER_GUEST_IFACE}" | ip link add name "${VSERVER_HOST_IFACE}" type veth peer name "${VSERVER_GUEST_IFACE}" | ||
- | vserver ${VSERVER_NAME} exec sh -c 'exec sleep 60' & | + | vserver ${VSERVER_NAME} exec sleep 60 & |
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do | for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do | ||
pid=$(vserver ${VSERVER_NAME} exec pidof -s sleep) | pid=$(vserver ${VSERVER_NAME} exec pidof -s sleep) | ||
Line 1014: | Line 1146: | ||
usleep 100000 | usleep 100000 | ||
done | done | ||
+ | if [ -z "$pid" ]; then | ||
+ | echo "vserver guest $VSERVER_NAME: failed to find guest net namespace" >&2 | ||
+ | fi | ||
ip link set "${VSERVER_GUEST_IFACE}" netns $pid | ip link set "${VSERVER_GUEST_IFACE}" netns $pid | ||
sysctl -q -w net.ipv4.conf.${VSERVER_HOST_IFACE}.forwarding=1 | sysctl -q -w net.ipv4.conf.${VSERVER_HOST_IFACE}.forwarding=1 | ||
Line 1074: | Line 1209: | ||
<file># cat /proc/mounts |grep cgroup | <file># cat /proc/mounts |grep cgroup | ||
- | cgroup /dev/cgroup/blkio cgroup rw,relatime,blkio 0 0 | + | cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0 |
- | cgroup /dev/cgroup/cpu cgroup rw,relatime,cpu 0 0 | + | cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0 |
- | cgroup /dev/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0 | + | cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0 |
- | cgroup /dev/cgroup/cpuset cgroup rw,relatime,cpuset 0 0 | + | cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0 |
- | cgroup /dev/cgroup/devices cgroup rw,relatime,devices 0 0 | + | cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0 |
- | cgroup /dev/cgroup/freezer cgroup rw,relatime,freezer 0 0 | + | cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0 |
- | cgroup /dev/cgroup/memory cgroup rw,relatime,memory 0 0 | + | cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0 |
- | cgroup /dev/cgroup/net_cls cgroup rw,relatime,net_cls 0 0 | + | cgroup /sys/fs/cgroup/net_cls cgroup rw,relatime,net_cls 0 0 |
</file> | </file> | ||
For these to work you need at least util-vserver-0.30.216-1.pre2955.3 (that .3 is important) and turn on per subsys support by doing: | For these to work you need at least util-vserver-0.30.216-1.pre2955.3 (that .3 is important) and turn on per subsys support by doing: | ||
- | |||
<file># mkdir /etc/vservers/.defaults/cgroup | <file># mkdir /etc/vservers/.defaults/cgroup | ||
# touch /etc/vservers/.defaults/cgroup/per-ss | # touch /etc/vservers/.defaults/cgroup/per-ss | ||
+ | </file> | ||
+ | |||
+ | ===== cgroups mountpoint ===== | ||
+ | |||
+ | if you have cgroups mounted somewhere else, you can inform vserver of that (it searching in ''/sys/fs/cgroup'' by default) | ||
+ | |||
+ | <file> | ||
+ | none /dev/cgroup cgroup cpuset,cpu,cpuacct,devices,freezer,net_cls 0 0 | ||
+ | </file> | ||
+ | |||
+ | you need to tell vserver where it mounted: | ||
+ | <file> | ||
+ | # cat /etc/vservers/.defaults/cgroup/mnt | ||
+ | /dev/cgroup | ||
</file> | </file> |