From: Luca Filipozzi Date: Thu, 31 May 2012 09:28:33 +0000 (+0000) Subject: improve ganeti documentation X-Git-Url: https://git.adam-barratt.org.uk/?p=mirror%2Fdsa-wiki.git;a=commitdiff_plain;h=2b91ec2c2778b0e78efa0a29d432456563ad4dae improve ganeti documentation --- diff --git a/input/howto/install-ganeti.creole b/input/howto/install-ganeti.creole index 7270537..b46038e 100644 --- a/input/howto/install-ganeti.creole +++ b/input/howto/install-ganeti.creole @@ -1,11 +1,37 @@ == How To Install Ganeti Clusters and Instances == -=== How To Install Ganeti Clusters === +=== suppositions === -Suppose that you have two identical hosts running squeeze, foo.debian.org and -bar.debian.org, that will be used as nodes in a ganeti cluster. +Suppose that there are two identical hosts: foo.debian.org and bar.debian.org. -==== install required packages ==== +They are running squeeze and have been integrated into Debian infrastructure. + +They will serve as nodes in a ganeti cluster named foobar.debian.org. + +They have a RAID1 array exposing three partitions: c0d0p1 for /, c0d0p2 for +swap and c0d0p3 for lvm volume groups to be used by ganeti via drbd. + +They have two network interfaces: eth0 (public) and eth1 (private). + +The public network is A.B.C.0/24 with gateway A.B.C.254. + +The private network is E.F.G.0/24 with no gateway. + +Suppose that the first instance to be hosted on foobar.debian.org is +qux.debian.org. + +The following DNS records exist: + +{{{ + foobar.debian.org. IN A A.B.C.1 + foo.debian.org. IN A A.B.C.2 + bar.debian.org. IN A A.B.C.3 + qux.debian.org. IN A A.B.C.4 + foo.debprivate-hoster.debian.org. IN A E.F.G.2 + bar.debprivate-hoster.debian.org. IN A E.F.G.3 +}}} + +=== install required packages === On each node, install the required packages: @@ -14,9 +40,9 @@ On each node, install the required packages: apt-get install -t squeeze-backports drbd8-utils ganeti2 ganeti-instance-debootstrap ganeti-htools }}} -==== configure kernel modules ==== +=== configure kernel modules === -On each node, ensure that required kernel modules are loaded at boot: +On each node, ensure that the required kernel modules are loaded at boot: {{{ ainsl /etc/modules 'drbd minor_count=255 usermode_helper=/bin/true' @@ -24,19 +50,42 @@ On each node, ensure that required kernel modules are loaded at boot: ainsl /etc/modules 'tun' }}} -==== configure networking ==== +=== configure networking === + +On each node, ensure that br0 (not eth0) and eth1 are configured. -Determine a hostname for the cluster (foobar.debian.org) and assign an A record. +The bridge interface, br0, is used by the guest virtual machines to reach the +public network. -On each node, ensure that all nodes and the cluster itself are listed in -{{{/etc/hosts}}}. +If the guest virtual machines need to access the private network, then br1 +should be configure rather than eth1. -On each node, create a bridge interface: br0. +To prevent the link address changing due to startup/shutdown of virtual +machines, explicitly set the value. + +{{{ + # /etc/networking/interfaces for foo.debian.org + + auto br0 + iface br0 inet static + bridge_ports eth0 + bridge_maxwait 0 + bridge_fd 0 + address A.B.C.2 + netmask 255.255.255.0 + gateway A.B.C.254 + up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE + + auto eth1 + iface eth1 inet static + address E.F.G.2 + netmask 255.255.255.0 +}}} -==== create storage ==== +=== configure lvm === On each node, configure lvm to ignore drbd devices and to prefer -{{{/dev/cciss}}} devices names to {{{/dev/block}}} device names +{{{/dev/cciss}}} devices names over {{{/dev/block}}} device names ([[http://code.google.com/p/ganeti/issues/detail?id=93|why?]]): {{{ @@ -44,22 +93,36 @@ On each node, configure lvm to ignore drbd devices and to prefer -e 's#^\(\s*filter\s\).*#\1= [ "a|.*|", "r|/dev/drbd[0-9]+|" ]#' -e 's#^\(\s*preferred_names\s\).*#\1= [ "^/dev/dm-*/", "^/dev/cciss/" ]#' \ /etc/lvm/lvm.conf + service lvm2 restart }}} +=== create lvm volume groups === + On each node, create a volume group: {{{ vgcreate vg_ganeti /dev/cciss/c0d0p3 }}} -==== create kvmU symbolic links ==== +=== create kvmU symbolic links === + +On each node, create vmlinuz and initrd symlinks as expected by the kvm +hypervisor: {{{ ln -s -T /boot/vmlinuz-2.6.32-5-amd64 /boot/vmlinuz-2.6-kvmU ln -s -T /boot/initrd.img-2.6.32-5-amd64 /boot/initrd.img-2.6-kvmU }}} -==== instantiate the cluster ==== +=== exchange ssh keys === + +ipsum loren bacon puppet + +=== configure iptables (via ferm) === + +ipsum loren bacon puppet + +=== instantiate the cluster === On the master node (foo) only: @@ -67,29 +130,41 @@ On the master node (foo) only: gnt-cluster init \ --master-netdev br0 \ --vg-name vg_ganeti \ - --secondary-ip 192.168.2.213 \ + --secondary-ip E.F.G.2 \ --enabled-hypervisors kvm \ --nic-parameters link=br0 \ --mac-prefix 00:16:37 \ --no-ssh-init \ --no-etc-hosts \ - --hypervisor-parameters kvm:initrd_path=/boot/initrd.img-2.6-kvmU \ + --hypervisor-parameters kvm:initrd_path=/boot/initrd.img-2.6-kvmU,kernel_path=/boot/vmlinuz-2.6-kvmU \ foobar.debian.org }}} -==== add slave nodes === +Note the following: + +* the master network device is set to br0, matching the public network bridge interface created above +* the volume group is set to vg_ganeti, matching the volume group created above +* the secondary IP address is set to the value of the master node's interface on the private network +* the nic parameters for instances is set to use br0 as default bridge +* the MAC prefix is registered in the dsa-kvm git repo -For each slave node (bar), on the master node (foo): +=== add slave nodes === + +For each slave node (only bar for this example), on the master node (foo): {{{ gnt-node add \ - --secondary-ip 192.168.2.217 \ + --secondary-ip E.F.G.3 \ --no-ssh-key-check \ --no-node-setup \ bar.debian.org }}} -==== verify cluster ==== +Note the following: + +* the secondary IP address is set to the value of the slave node's interface on the private network + +=== verify cluster === On the master node (foo): @@ -97,18 +172,23 @@ On the master node (foo): gnt-cluster verify }}} -=== How To Install Ganeti Instances === +If everything has been configured correctly, no errors should be reported. + +=== create the 'dsa' variant === -Suppose that the first guest on will be qux.debian.org. +ipsum loren bacon deboostrap+dsa -==== define the instance ==== +---- -# add qux.debian.org entry in LDAP to establish A record -# ask hosting partner to establish PTR record as appropriate -# add qux.debian.org entry in puppet -# add qux.debian.org entry in nagios +== How To Install Ganeti Instances == -==== create the instance ==== +Suppose that qux.debian.org will be an instance (a virtual machine) hosted on +the foobar.debian.org ganeti cluster. + +Before adding the instance, an LDAP entry must be created so that an A record +for the instance (A.B.C.4) exists. + +=== create the instance === On the master node (foo): @@ -119,7 +199,79 @@ On the master node (foo): --os-size 4GiB \ --os-type debootstrap+dsa \ --hypervisor-parameters kvm:initrd_path=,kernel_path= \ - --net 0:ip=1.2.3.4 \ + --net 0:ip=A.B.C.4 \ qux.debian.org }}} +Note the following: + +* the primary and secondary nodes have been explicitly set +* the operating system type is 'debootstrap+dsa' +* the network interfarce 0 (eth1 on the system) is set to the instance's interface on the public network + +---- + +== Variations == + +If the instances require access to the private network, then there are two modifications necessary. + +=== re-configure networking === + +Ensure that br1 is configured (rather than eth1). + +{{{ + # /etc/networking/interfaces for foo.debian.org + + auto br0 + iface br0 inet static + bridge_ports eth0 + bridge_maxwait 0 + bridge_fd 0 + address A.B.C.2 + netmask 255.255.255.0 + gateway A.B.C.254 + up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE + + auto br1 + iface br1 inet static + bridge_ports eth1 + bridge_maxwait 0 + bridge_fd 0 + address E.F.G.2 + netmask 255.255.255.0 + up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE +}}} + +=== create or update the instance === + +When creating the instance, indicate both netwokrs: + +{{{ + gnt-instance add \ + --node foo:bar \ + --disk-template drbd \ + --os-size 4GiB \ + --os-type debootstrap+dsa \ + --hypervisor-parameters kvm:initrd_path=,kernel_path= \ + --net 0:ip=A.B.C.4,1:ip=E.F.G.4 \ + qux.debian.org +}}} + +Please note that the dsa bootscript will only configure eth1 (nic0). If +specifying a second internface, the guest operating system must be updated +manually. + +When updating an existing instance, add the interface: + +{{{ + gnt-instance shutdown qux.debian.org + gnt-instance modify \ + --net add:ip=E.F.G.4 \ + qux.debian.org + gnt-instance startup qux.debian.org +}}} + +Please note that the hook scripts are run only at instance instantiation. When +adding interfaces to an instance, the guest opearting system must be updated +manually. +