== How To Install Ganeti Clusters and Instances ==
-=== How To Install Ganeti Clusters ===
+=== suppositions ===
-Suppose that you have two identical hosts running squeeze, foo.debian.org and
-bar.debian.org, that will be used as nodes in a ganeti cluster.
+Suppose that there are two identical hosts: foo.debian.org and bar.debian.org.
-==== install required packages ====
+They are running squeeze and have been integrated into Debian infrastructure.
+
+They will serve as nodes in a ganeti cluster named foobar.debian.org.
+
+They have a RAID1 array exposing three partitions: c0d0p1 for /, c0d0p2 for
+swap and c0d0p3 for lvm volume groups to be used by ganeti via drbd.
+
+They have two network interfaces: eth0 (public) and eth1 (private).
+
+The public network is A.B.C.0/24 with gateway A.B.C.254.
+
+The private network is E.F.G.0/24 with no gateway.
+
+Suppose that the first instance to be hosted on foobar.debian.org is
+qux.debian.org.
+
+The following DNS records exist:
+
+{{{
+ foobar.debian.org. IN A A.B.C.1
+ foo.debian.org. IN A A.B.C.2
+ bar.debian.org. IN A A.B.C.3
+ qux.debian.org. IN A A.B.C.4
+ foo.debprivate-hoster.debian.org. IN A E.F.G.2
+ bar.debprivate-hoster.debian.org. IN A E.F.G.3
+}}}
+
+=== install required packages ===
On each node, install the required packages:
apt-get install -t squeeze-backports drbd8-utils ganeti2 ganeti-instance-debootstrap ganeti-htools
}}}
-==== configure kernel modules ====
+=== configure kernel modules ===
-On each node, ensure that required kernel modules are loaded at boot:
+On each node, ensure that the required kernel modules are loaded at boot:
{{{
ainsl /etc/modules 'drbd minor_count=255 usermode_helper=/bin/true'
ainsl /etc/modules 'tun'
}}}
-==== configure networking ====
+=== configure networking ===
+
+On each node, ensure that br0 (not eth0) and eth1 are configured.
-Determine a hostname for the cluster (foobar.debian.org) and assign an A record.
+The bridge interface, br0, is used by the guest virtual machines to reach the
+public network.
-On each node, ensure that all nodes and the cluster itself are listed in
-{{{/etc/hosts}}}.
+If the guest virtual machines need to access the private network, then br1
+should be configure rather than eth1.
-On each node, create a bridge interface: br0.
+To prevent the link address changing due to startup/shutdown of virtual
+machines, explicitly set the value.
+
+{{{
+ # /etc/networking/interfaces for foo.debian.org
+
+ auto br0
+ iface br0 inet static
+ bridge_ports eth0
+ bridge_maxwait 0
+ bridge_fd 0
+ address A.B.C.2
+ netmask 255.255.255.0
+ gateway A.B.C.254
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+
+ auto eth1
+ iface eth1 inet static
+ address E.F.G.2
+ netmask 255.255.255.0
+}}}
-==== create storage ====
+=== configure lvm ===
On each node, configure lvm to ignore drbd devices and to prefer
-{{{/dev/cciss}}} devices names to {{{/dev/block}}} device names
+{{{/dev/cciss}}} devices names over {{{/dev/block}}} device names
([[http://code.google.com/p/ganeti/issues/detail?id=93|why?]]):
{{{
-e 's#^\(\s*filter\s\).*#\1= [ "a|.*|", "r|/dev/drbd[0-9]+|" ]#'
-e 's#^\(\s*preferred_names\s\).*#\1= [ "^/dev/dm-*/", "^/dev/cciss/" ]#' \
/etc/lvm/lvm.conf
+ service lvm2 restart
}}}
+=== create lvm volume groups ===
+
On each node, create a volume group:
{{{
vgcreate vg_ganeti /dev/cciss/c0d0p3
}}}
-==== create kvmU symbolic links ====
+=== create kvmU symbolic links ===
+
+On each node, create vmlinuz and initrd symlinks as expected by the kvm
+hypervisor:
{{{
ln -s -T /boot/vmlinuz-2.6.32-5-amd64 /boot/vmlinuz-2.6-kvmU
ln -s -T /boot/initrd.img-2.6.32-5-amd64 /boot/initrd.img-2.6-kvmU
}}}
-==== instantiate the cluster ====
+=== exchange ssh keys ===
+
+ipsum loren bacon puppet
+
+=== configure iptables (via ferm) ===
+
+ipsum loren bacon puppet
+
+=== instantiate the cluster ===
On the master node (foo) only:
gnt-cluster init \
--master-netdev br0 \
--vg-name vg_ganeti \
- --secondary-ip 192.168.2.213 \
+ --secondary-ip E.F.G.2 \
--enabled-hypervisors kvm \
--nic-parameters link=br0 \
--mac-prefix 00:16:37 \
--no-ssh-init \
--no-etc-hosts \
- --hypervisor-parameters kvm:initrd_path=/boot/initrd.img-2.6-kvmU \
+ --hypervisor-parameters kvm:initrd_path=/boot/initrd.img-2.6-kvmU,kernel_path=/boot/vmlinuz-2.6-kvmU \
foobar.debian.org
}}}
-==== add slave nodes ===
+Note the following:
+
+* the master network device is set to br0, matching the public network bridge interface created above
+* the volume group is set to vg_ganeti, matching the volume group created above
+* the secondary IP address is set to the value of the master node's interface on the private network
+* the nic parameters for instances is set to use br0 as default bridge
+* the MAC prefix is registered in the dsa-kvm git repo
-For each slave node (bar), on the master node (foo):
+=== add slave nodes ===
+
+For each slave node (only bar for this example), on the master node (foo):
{{{
gnt-node add \
- --secondary-ip 192.168.2.217 \
+ --secondary-ip E.F.G.3 \
--no-ssh-key-check \
--no-node-setup \
bar.debian.org
}}}
-==== verify cluster ====
+Note the following:
+
+* the secondary IP address is set to the value of the slave node's interface on the private network
+
+=== verify cluster ===
On the master node (foo):
gnt-cluster verify
}}}
-=== How To Install Ganeti Instances ===
+If everything has been configured correctly, no errors should be reported.
+
+=== create the 'dsa' variant ===
-Suppose that the first guest on will be qux.debian.org.
+ipsum loren bacon deboostrap+dsa
-==== define the instance ====
+----
-# add qux.debian.org entry in LDAP to establish A record
-# ask hosting partner to establish PTR record as appropriate
-# add qux.debian.org entry in puppet
-# add qux.debian.org entry in nagios
+== How To Install Ganeti Instances ==
-==== create the instance ====
+Suppose that qux.debian.org will be an instance (a virtual machine) hosted on
+the foobar.debian.org ganeti cluster.
+
+Before adding the instance, an LDAP entry must be created so that an A record
+for the instance (A.B.C.4) exists.
+
+=== create the instance ===
On the master node (foo):
--os-size 4GiB \
--os-type debootstrap+dsa \
--hypervisor-parameters kvm:initrd_path=,kernel_path= \
- --net 0:ip=1.2.3.4 \
+ --net 0:ip=A.B.C.4 \
qux.debian.org
}}}
+Note the following:
+
+* the primary and secondary nodes have been explicitly set
+* the operating system type is 'debootstrap+dsa'
+* the network interfarce 0 (eth1 on the system) is set to the instance's interface on the public network
+
+----
+
+== Variations ==
+
+If the instances require access to the private network, then there are two modifications necessary.
+
+=== re-configure networking ===
+
+Ensure that br1 is configured (rather than eth1).
+
+{{{
+ # /etc/networking/interfaces for foo.debian.org
+
+ auto br0
+ iface br0 inet static
+ bridge_ports eth0
+ bridge_maxwait 0
+ bridge_fd 0
+ address A.B.C.2
+ netmask 255.255.255.0
+ gateway A.B.C.254
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+
+ auto br1
+ iface br1 inet static
+ bridge_ports eth1
+ bridge_maxwait 0
+ bridge_fd 0
+ address E.F.G.2
+ netmask 255.255.255.0
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+}}}
+
+=== create or update the instance ===
+
+When creating the instance, indicate both netwokrs:
+
+{{{
+ gnt-instance add \
+ --node foo:bar \
+ --disk-template drbd \
+ --os-size 4GiB \
+ --os-type debootstrap+dsa \
+ --hypervisor-parameters kvm:initrd_path=,kernel_path= \
+ --net 0:ip=A.B.C.4,1:ip=E.F.G.4 \
+ qux.debian.org
+}}}
+
+Please note that the dsa bootscript will only configure eth1 (nic0). If
+specifying a second internface, the guest operating system must be updated
+manually.
+
+When updating an existing instance, add the interface:
+
+{{{
+ gnt-instance shutdown qux.debian.org
+ gnt-instance modify \
+ --net add:ip=E.F.G.4 \
+ qux.debian.org
+ gnt-instance startup qux.debian.org
+}}}
+
+Please note that the hook scripts are run only at instance instantiation. When
+adding interfaces to an instance, the guest opearting system must be updated
+manually.
+
projects / mirror / dsa-wiki.git / commitdiff