Differences

This shows you the differences between two versions of the page.

--- docs:vserver [2012-05-03 13:40]
hawk
+++ docs:vserver [2015-10-05 14:40]
arekm [XFS filesystem - kernel upgrade causes xfs related oops (xfs_filestream_lookup_ag)]
@@ Line 1: / Line 1: @@
-/* page was renamed from Vserver */
-/* pragma: section-numbers 2 */
 ====== Linux VServer Project ======
-Linux-VServer is a virtual private server implementation for the Linux kernel
+Linux-VServer provides virtualization for GNU/Linux systems. This is accomplished by kernel level isolation. It allows to run multiple virtual units at once. Those units are sufficiently isolated to guarantee the required security, but utilize available resources efficiently, as they run on the same kernel.
@@ Line 11: / Line 7: @@
   * [[http://linux-vserver.org/|Project homepage]]
-  * [[http://linux-vserver.org/short+presentation|short presentation]]
+  * [[http://linux-vserver.org/Overview|Vserver Overview]]
   * [[http://www.nongnu.org/util-vserver/doc/conf/configuration.html|doc of configuration items]]. you should change the stylesheet in your browser if you care for your eyes ;-)
   * [[http://www.solucorp.qc.ca/miscprj/s_context.hc?prjstate=1|Introduction to vserver]] an older document (for < 2.x), but still useful
   * [[http://www.gentoo.org/doc/en/vserver-howto.xml|Gentoo Linux-VServer Howto]] you'll need this if you want to setup Gentoo guest vserver
-  * [[http://linux-vserver.org/index.php?page=Linux-VServer-Paper-08|Field of Application paper]] reasons why to use vserver.
-===== Contents =====
-/* UndefinedMacro: TableOfContents(None) */
 ===== Installing Vserver host on PLD Linux =====
-Build and install the kernel:
-<file>$ ./builder -bb kernel.spec
-# poldek -u kernel
-</file>
+Ensure you have appropriate [[packages:kernel]] installed.
+You can check this from kernel config:
+<code>
+# modprobe configs
+# zgrep CONFIG_VSERVER /proc/config.gz
+CONFIG_VSERVER=y
+</code>
 ===== Installing guest PLD Linux Vserver =====
 ==== Preparing userspace tools ====
-First, install the tools:
-<file># poldek -u util-vserver
+First, install the tools:
+<file>
+# poldek -u util-vserver
 </file>
-Then configure //poldek.conf//:
-<file># vim /etc/vservers/.distributions/pld-XX/poldek/repos.d/pld.conf
+If you need to review poldek repo sources, then the configs are in
-</file>
+''/etc/vservers/.distributions/pld-*/poldek/''
-Where XX is: ac, ti, th
+where ''*'' can be ''ac'' or ''th'' depending which guest you wish to install.
 At this point you should have booted into vserver-enabled kernel. You must start ''vprocunhide'' or none of your Vservers can start.
@@ Line 49: / Line 41: @@
 To start ''vprocunhide'':
-<file># /sbin/service vprocunhide start
+<file>
+# /sbin/service vprocunhide start
 </file>
+==== Guest creation ====
-==== Actual guest creation ====
+Build the guest system.
-Build the guest system:
-<file># vserver test build --context <num> -m poldek -n test
+<file>
+# a guest name (not hostname)
+NAME=test
+# <num> must be a number within 2-32767 range.
+CTX=2
+vserver $NAME build --context $CTX -m poldek -n $NAME
 </file>
-<num> must be a number within 2-32767 range.
-There are two versions of PLD available for guest systems:
+This defaults installing guest same ARCH and VERSION that your host is.
+If you need to use another combination, then there are two versions of PLD available for guest systems:
+  * pld-ac - [[:ac|PLD 2.0 (Ac)]]
+  * pld-th - [[:th|PLD 3.0 (Th)]]
-  * pld-ac - [[:AcInfo|PLD 2.0 (Ac)]]
-  * pld-th - [[:ThInfo|PLD 3.0 (Th)]]
 You may choose one using ''-d'' option:
+<file>
+DIST=pld-th
-<file># vserver test build --context <num> -m poldek -n test -- -d pld-th
+vserver $NAME build --context $CTX -m poldek -n $NAME  -- -d $DIST
 </file>
 using ''util-vserver >= 0.30.214-2'' from ac-updates, ''util-vserver >= 0.30.215-2'' from from th you can build other arch or distro or using own mirror:
-<file># vserver test build --context <num> -m poldek -n test -- -m http://ftp.pld-linux.org/dists/ac
+<file>
-# vserver test build --context <num> -m poldek -n test --personality linux_32bit --machine i686 -- -d pld-th
+MIRROR=http://ftp.pld-linux.org/dists/ac
+vserver $NAME build --context $CTX -m poldek -n $NAME -- -m $MIRROR
 </file>
+To build 32bit guest on 64bit host:
+<file>
+vserver $NAME build --context $CTX -m poldek -n $NAME --personality linux_32bit --machine i686 -- -d $DIST
+</file>
 To build vserver from template (archive containing whole filesystem):
-<file># vserver test build --context <num> -m template -n test -- -t image.tar.bz2
+<file>
+# vserver $NAME build --context $CTX -m template -n $NAME -- -t image.tar.bz2
 </file>
 To see other ''build'' command options:
-<file># vserver test build --help
+<file>
+# vserver test build --help
 </file>
 Install ''rc-scripts'' to the new system using ''vpoldek'':
-<file># vpoldek test -- -u rc-scripts
+<file>
+# vpoldek test -- -u rc-scripts
 </file>
 you should consider installing ''vserver-packages'' rpm package to satisfy packages dependency which have no use inside vserver.
 And then start the guest system:
-<file># vserver test start
+<file>
+# vserver test start
 </file>
 To enter that vserver, type:
-<file># vserver test enter
+<file>
+# vserver test enter
 </file>
-Note, however, that if you don't run //plain// init style you must have at least one daemon running inside your guest vserver or it will be shut down shortly.
+Note, however, that if you don't run //plain// init style you must have at least one daemon running inside your guest vserver or it will be shut down shortly.
@@ Line 373: / Line 392: @@
 [[http://www.solucorp.qc.ca/howto.hc?projet=vserver&amp;id=72|http://www.solucorp.qc.ca/howto.hc?projet=vserver&id=72]]
-You can use //lcap// program to see available capatabilities:
+You can use //lcap// program to see available capabilities:
@@ Line 557: / Line 576: @@
 ==== Running 32 bit vserver on an 64 bit host ====
-With recent PLD util-vserver package you can create 32-bit guest systems inside a 64-bit host. First you need to prepare a new distribution definition skeleton:
+With recent [[package>util-vserver]] package you can create 32-bit guest systems inside a 64-bit host.
+To specify arch during guest creation, use ''-d'' option, and to change what ''uname'' returns, use arguments ''%%--personality linux_32bit --machine i686%%'':
-<file># mkdir -p /etc/vservers/.distributions/pld-th-i686/poldek/repos.d
+<file># vserver test build --context <num> -n test -m poldek -- -d pld-th-i686 --personality linux_32bit --machine i686
 </file>
-Then copy your repository configuration to ''/etc/vservers/.distributions/pld-th-i686/poldek/repos.d/pld.conf'' and change the architecture and source paths to your liking. When configuration is ready, create a new guest vserver using the ''-d'' command line option:
+If you need to set ''uts'' parameters afterwards, you can just echo them:
+<file>
-<file># vserver test build --context <num> -n test -m poldek -- -d pld-th-i686
+# echo linux_32bit >> /etc/vservers/test/personality
+# echo i686 > /etc/vservers/test/uts/machine
 </file>
-Later to force i686 32bit use:
-<file># echo linux_32bit >> /etc/vservers/test/personality
-# echo i686 > /etc/vservers/test/uts/machine
-</file>
-however, you can do that at vserver build time using arguments ''--personality linux_32bit --machine i686''.
@@ Line 697: / Line 710: @@
+==== Running auditd inside guest ====
+You need ''CAP_AUDIT_CONTROL'' in ''bcapabilities'' and lower ''priority_boost'' to ''0'' in ''/etc/audit/auditd.conf''
+==== XFS filesystem - kernel upgrade causes xfs related oops (xfs_filestream_lookup_ag) ====
+After upgrading from 2.6-3.4 kernels (possibly other versions) to 3.18 (tested, possibly other versions) kernel ooppses
+almost immediately after accessing some files on xfs filesystem with ''xfs_filestream_lookup_ag'' visible in stack trace
+(or other filestream related function).
+That's because vserver patch for kernels earlier than 2.6.23 patched xfs filesystem to introduce new flag:
+<file>
+#define XFS_XFLAG_BARRIER     0x00004000      /* chroot() barrier */
+</file>
+and files/dirs with such flag got saved on your filesystem.
+Starting with kernel 2.6.23 kernel introduced filestreams which are using 0x00004000 bit, thus causing conflict with vserver.
+<file>
+#define XFS_XFLAG_FILESTREAM   0x00004000      /* use filestream allocator */
+</file>
+Vserver stopped adding such xfs xflag in 3.13 BUT your existing filesystem can still have XFS_XFLAG_BARRIER (0x00004000) set
+causing oops in newer kernels.
+How to find out if I'm affected?
+IIF you don't use filestream feature then modify http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=blob_plain;f=src/bstat.c;hb=HEAD to show only files containing XFS_XFLAG_FILESTREAM
+<file>
+diff --git a/src/bstat.c b/src/bstat.c
+index 4e22ecd..887512f 100644
+--- a/src/bstat.c
++++ b/src/bstat.c
+@@ -34,19 +34,21 @@ dotime(void *ti, char *s)
+ void
+ printbstat(xfs_bstat_t *sp)
+ {
+-       printf("ino %lld mode %#o nlink %d uid %d gid %d rdev %#x\n",
+-               (long long)sp->bs_ino, sp->bs_mode, sp->bs_nlink,
+-               sp->bs_uid, sp->bs_gid, sp->bs_rdev);
+-       printf("\tblksize %d size %lld blocks %lld xflags %#x extsize %d\n",
+-               sp->bs_blksize, (long long)sp->bs_size, (long long)sp->bs_blocks,
+-               sp->bs_xflags, sp->bs_extsize);
+-       dotime(&sp->bs_atime, "atime");
+-       dotime(&sp->bs_mtime, "mtime");
+-       dotime(&sp->bs_ctime, "ctime");
+-       printf( "\textents %d %d gen %d\n",
+-               sp->bs_extents, sp->bs_aextents, sp->bs_gen);
+-       printf( "\tDMI: event mask 0x%08x state 0x%04x\n",
+-               sp->bs_dmevmask, sp->bs_dmstate);
++       if (sp->bs_xflags & XFS_XFLAG_FILESTREAM) {
++               printf("ino %lld mode %#o nlink %d uid %d gid %d rdev %#x\n",
++                               (long long)sp->bs_ino, sp->bs_mode, sp->bs_nlink,
++                               sp->bs_uid, sp->bs_gid, sp->bs_rdev);
++               printf("\tblksize %d size %lld blocks %lld xflags %#x extsize %d\n",
++                               sp->bs_blksize, (long long)sp->bs_size, (long long)sp->bs_blocks,
++                               sp->bs_xflags, sp->bs_extsize);
++               dotime(&sp->bs_atime, "atime");
++               dotime(&sp->bs_mtime, "mtime");
++               dotime(&sp->bs_ctime, "ctime");
++               printf( "\textents %d %d gen %d\n",
++                               sp->bs_extents, sp->bs_aextents, sp->bs_gen);
++               printf( "\tDMI: event mask 0x%08x state 0x%04x\n",
++                               sp->bs_dmevmask, sp->bs_dmstate);
++       }
+ }
+</file>
+and then run it with mounted directory of each filesystem (bstat /; bstat /home etc). It will print "ino ..." information for filestream files.
+How to clean up?
+rsync files to other partition, recreate problematic partition and then copy files back.
 ===== Debian or Ubuntu guest installation =====
@@ Line 970: / Line 1060: @@
 </file>
-  * load the ''vroot'' module and add it to your ''/etc/modules''
+  * load the ''vroot'' module and add it to your ''/etc/modules''. you can optionaly increase max vroot number of devices by putting the limit in your ''/etc/modprobe.conf'': <file>options vroot max_vroot=64
-    * you can optionaly increase max vroot number of devices by putting the limit in your ''/etc/modprobe.conf'': <file>options vroot max_vroot=64
 </file>
-    assing a free vroot node for the device in question: <file># vrsetup /dev/vroot3 /dev/space/vserver1_home
+  * assing a free vroot node for the device in question: <file># vrsetup /dev/vroot3 /dev/space/vserver1_home
 </file>
-    copy the vroot device to the guest: <file># cp -af /dev/vroot3 /vservers/test/dev/
+  * copy the vroot device to the guest: <file># cp -af /dev/vroot3 /vservers/test/dev/
 </file>
-    add to ''/etc/vservers/test/apps/init/mtab'': <file>/dev/vroot3     /home/    xfs     defaults,usrquota        0       0
+  * add to ''/etc/vservers/test/apps/init/mtab'': <file>/dev/vroot3     /home/    xfs     defaults,usrquota        0       0
 </file>
-    add ''quota_ctl'' to ''/etc/vservers/test/ccapabilities'':
+  * add ''quota_ctl'' to ''/etc/vservers/test/ccapabilities'':
-    restart your vserver and run ''edquota'' inside
+  * restart your vserver and run ''edquota'' inside
 ===== Network namespace in vservers =====
+Starting from util-vserver 0.30.216-1.pre3054 there is basic support for creating network namespaces with interfaces inside.
+Enabling netns and two capabilities: NET_ADMIN (allows interfaces in guest to be managed) and NET_RAW (makes iptables working).
+<file>mkdir /etc/vservers/test/spaces
+touch /etc/vserver/test/spaces/net
+echo NET_ADMIN >> /etc/vservers/test/bcapabilities
+echo NET_RAW >> /etc/vservers/test/bcapabilities
+echo 'plain' > /etc/vservers/test/apps/init/style
+</file>
+Avoid context isolation since it makes little sense when using network namespaces:
+<file>touch /etc/vserver/test/noncontext</file>
+Configure interfaces:
+- arbitrary directory name, just for ordering
+myiface0 will be interface name inside of guest (optional, default geth0,
+geth1 and so on)
+veth-host - interface name on the host side
+<file>
+mkdir -p /etc/vservers/test/netns/interfaces/0
+echo myiface0 > /etc/vservers/test/netns/interfaces/guest
+echo veth-host > /etc/vservers/test/netns/interfaces/host
+</file>
+!!! FINISH ME. FINISH ME. FINISH ME. !!!
+===== Network namespace in vservers (OLD WAY) =====
 Enabling netns and two capabilities: NET_ADMIN (allows interfaces in guest to be managed) and NET_RAW (makes iptables working).
@@ Line 995: / Line 1117: @@
-<file>mkdir /etc/vserver/test/spaces
+<file>mkdir /etc/vservers/test/spaces
-touch /etc/vserver/test/spaces/net
+touch /etc/vservers/test/spaces/net
 echo NET_ADMIN >> /etc/vservers/test/bcapabilities
 echo NET_RAW >> /etc/vservers/test/bcapabilities
@@ Line 1018: / Line 1140: @@
 ip link add name "${VSERVER_HOST_IFACE}" type veth peer name "${VSERVER_GUEST_IFACE}"
-vserver ${VSERVER_NAME} exec sh -c 'exec sleep 60' &
+vserver ${VSERVER_NAME} exec sleep 60 &
 for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15; do
         pid=$(vserver ${VSERVER_NAME} exec pidof -s sleep)
@@ Line 1024: / Line 1146: @@
         usleep 100000
 done
+if [ -z "$pid" ]; then
+        echo "vserver guest $VSERVER_NAME: failed to find guest net namespace" >&2
+fi
 ip link set "${VSERVER_GUEST_IFACE}" netns $pid
 sysctl -q -w net.ipv4.conf.${VSERVER_HOST_IFACE}.forwarding=1
@@ Line 1084: / Line 1209: @@
 <file># cat /proc/mounts |grep cgroup
-cgroup /dev/cgroup/blkio cgroup rw,relatime,blkio 0 0
+cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
-cgroup /dev/cgroup/cpu cgroup rw,relatime,cpu 0 0
+cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
-cgroup /dev/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
+cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
-cgroup /dev/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
+cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
-cgroup /dev/cgroup/devices cgroup rw,relatime,devices 0 0
+cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
-cgroup /dev/cgroup/freezer cgroup rw,relatime,freezer 0 0
+cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
-cgroup /dev/cgroup/memory cgroup rw,relatime,memory 0 0
+cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
-cgroup /dev/cgroup/net_cls cgroup rw,relatime,net_cls 0 0
+cgroup /sys/fs/cgroup/net_cls cgroup rw,relatime,net_cls 0 0
 </file>
 For these to work you need at least util-vserver-0.30.216-1.pre2955.3 (that .3 is important) and turn on per subsys support by doing:
@@ Line 1101: / Line 1225: @@
 </file>
+===== cgroups mountpoint =====
+if you have cgroups mounted somewhere else, you can inform vserver of that (it searching in ''/sys/fs/cgroup'' by default)
+<file>
+none        /dev/cgroup     cgroup  cpuset,cpu,cpuacct,devices,freezer,net_cls  0 0
+</file>
+you need to tell vserver where it mounted:
+<file>
+# cat /etc/vservers/.defaults/cgroup/mnt
+/dev/cgroup
+</file>

PLD Linux Distribution

User Tools

Site Tools

Differences

Page Tools