On each node, install the required packages:
{{{
- apt-get install fai-client ssed extlinux
- apt-get install -t squeeze-backports drbd8-utils ganeti2 ganeti-instance-debootstrap ganeti-htools
+ # maybe: apt-get install drbd8-utils
+ # maybe: apt-get install ganeti-instance-debootstrap
+ apt-get install ganeti2 ganeti-htools qemu-kvm
}}}
=== configure kernel modules ===
ainsl /etc/modules 'drbd minor_count=255 usermode_helper=/bin/true'
ainsl /etc/modules 'hmac'
ainsl /etc/modules 'tun'
+ ainsl /etc/modules 'ext3'
+ ainsl /etc/modules 'ext4'
}}}
=== configure networking ===
public network.
If the guest virtual machines need to access the private network, then br1
-should be configure rather than eth1.
+should be configured rather than eth1.
To prevent the link address changing due to startup/shutdown of virtual
machines, explicitly set the value.
-{{{
- # /etc/networking/interfaces for foo.debian.org
+This is the interfaces file for foo.debian.org:
+{{{
auto br0
iface br0 inet static
bridge_ports eth0
netmask 255.255.255.0
}}}
+This is the interfaces file for bar.debian.org:
+
+{{{
+ auto br0
+ iface br0 inet static
+ bridge_ports eth0
+ bridge_maxwait 0
+ bridge_fd 0
+ address A.B.C.3
+ netmask 255.255.255.0
+ gateway A.B.C.254
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+
+ auto eth1
+ iface eth1 inet static
+ address E.F.G.3
+ netmask 255.255.255.0
+}}}
+
=== configure lvm ===
On each node, configure lvm to ignore drbd devices and to prefer
{{{
ssed -i \
- -e 's#^\(\s*filter\s\).*#\1= [ "a|.*|", "r|/dev/drbd[0-9]+|" ]#'
+ -e 's#^\(\s*filter\s\).*#\1= [ "a|.*|", "r|/dev/drbd[0-9]+|" ]#' \
-e 's#^\(\s*preferred_names\s\).*#\1= [ "^/dev/dm-*/", "^/dev/cciss/" ]#' \
/etc/lvm/lvm.conf
service lvm2 restart
vgcreate vg_ganeti /dev/cciss/c0d0p3
}}}
-=== create kvmU symbolic links ===
+=== exchange ssh keys ===
-On each node, create vmlinuz and initrd symlinks as expected by the kvm
-hypervisor:
+on each node:
{{{
- ln -s -T /boot/vmlinuz-2.6.32-5-amd64 /boot/vmlinuz-2.6-kvmU
- ln -s -T /boot/initrd.img-2.6.32-5-amd64 /boot/initrd.img-2.6-kvmU
+ mkdir -m 0700 -p /root/.ssh &&
+ ln -s /etc/ssh/ssh_host_rsa_key /root/.ssh/id_rsa
}}}
-=== exchange ssh keys ===
-
-ipsum loren bacon puppet
-
=== configure iptables (via ferm) ===
-ipsum loren bacon puppet
+the nodes must connect to each other over the public and private networks for a number of reasons; see the ganeti2 module in puppet
=== instantiate the cluster ===
--mac-prefix 00:16:37 \
--no-ssh-init \
--no-etc-hosts \
- --hypervisor-parameters kvm:initrd_path=/boot/initrd.img-2.6-kvmU,kernel_path=/boot/vmlinuz-2.6-kvmU \
+ --hypervisor-parameters kvm:initrd_path=,kernel_path= \
foobar.debian.org
}}}
=== add slave nodes ===
-For each slave node (only bar for this example), on the master node (foo):
+For each slave node (only bar for this example):
+
+on the slave, append the master's /etc/ssh/ssh_host_rsa_key.pub to
+/etc/ssh/userkeys/root. This is only required temporarily - once
+everything works, puppet will put it/keep it there.
+
+on the master node (foo):
{{{
gnt-node add \
=== create the 'dsa' variant ===
-ipsum loren bacon deboostrap+dsa
+Ensure that the debootstrap+dsa variant has been installed by puppet (see ganeti2 module in dsa-puppet).
----
* the primary and secondary nodes have been explicitly set
* the operating system type is 'debootstrap+dsa'
-* the network interfarce 0 (eth1 on the system) is set to the instance's interface on the public network
+* the network interfarce 0 (eth0 on the system) is set to the instance's interface on the public network
+* If qux.d.o does not yet exist in DNS/LDAP, you may need --no-ip-check --no-name-check. Be careful that the hostname and IP address are not taken already!
----
=== re-configure networking ===
-Ensure that br1 is configured (rather than eth1).
+On the nodes, ensure that br1 is configured (rather than eth1).
-{{{
- # /etc/networking/interfaces for foo.debian.org
+This is the interfaces file for foo.debian.org:
+{{{
auto br0
iface br0 inet static
bridge_ports eth0
up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
}}}
+This is the interfaces file for bar.debian.org:
+
+{{{
+ auto br0
+ iface br0 inet static
+ bridge_ports eth0
+ bridge_maxwait 0
+ bridge_fd 0
+ address A.B.C.3
+ netmask 255.255.255.0
+ gateway A.B.C.254
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+
+ auto br1
+ iface br1 inet static
+ bridge_ports eth1
+ bridge_maxwait 0
+ bridge_fd 0
+ address E.F.G.3
+ netmask 255.255.255.0
+ up ip link set addr $(cat /sys/class/net/$IFACE/address) dev $IFACE
+}}}
+
=== create or update the instance ===
-When creating the instance, indicate both netwokrs:
+When creating the instance, indicate both networks:
{{{
gnt-instance add \
--os-size 4GiB \
--os-type debootstrap+dsa \
--hypervisor-parameters kvm:initrd_path=,kernel_path= \
- --net 0:ip=A.B.C.4,1:ip=E.F.G.4 \
+ --net 0:ip=A.B.C.4 \
+ --net 1:link=br1,ip=E.F.G.4 \
qux.debian.org
}}}
-Please note that the dsa bootscript will only configure eth1 (nic0). If
-specifying a second internface, the guest operating system must be updated
-manually.
+* If qux.d.o does not yet exist in DNS/LDAP, you may need --no-ip-check --no-name-check. Be careful that the hostname and IP address are not taken already!
When updating an existing instance, add the interface:
{{{
gnt-instance shutdown qux.debian.org
gnt-instance modify \
- --net add:ip=E.F.G.4 \
+ --net add:link=br1,ip=E.F.G.4 \
qux.debian.org
gnt-instance startup qux.debian.org
}}}
adding interfaces to an instance, the guest opearting system must be updated
manually.
+
+* If you are importing an instance from libvirt with LVM setup, you can adopt LVs:
+
+{{{
+ gnt-instance add -t plain --os-type debootstrap+dsa-wheezy \
+ --disk 0:adopt=lully-boot \
+ --disk 1:adopt=lully-root \
+ --disk 2:adopt=lully-swap \
+ --disk 3:adopt=lully-log \
+ --hypervisor-parameters kvm:initrd_path=,kernel_path= \
+ --net 0:ip=82.195.75.99 -n clementi.debian.org lully.debian.org
+}}}
+
+And you want to convert it to use DRBD afterwards and start it on the other cluster node, so we can ensure that DRBD is correctly working.
+{{{
+ gnt-instance shutdown lully.debian.org
+ gnt-instance modify -t drbd -n czerny.debian.org lully.debian.org
+ gnt-instance failover lully.debian.org
+ gnt-instance startup lully.debian.org
+}}}
+
+* Some instances NEED ide instead of virtio
+
+{{{
+ gnt-instance modify --hypervisor-parameters disk_type=ide fils.debian.org
+}}}
+
+* To import instances with SAN volumes
+
+{{{
+ gnt-instance add -t blockdev --os-type debootstrap+dsa \
+ --disk 0:adopt=/dev/disk/by-id/scsi-reger-boot \
+ --disk 1:adopt=/dev/disk/by-id/scsi-reger-root \
+ --hypervisor-parameters kvm:initrd_path=,kernel_path= \
+ --net 0:ip=206.12.19.124 -n rossini.debian.org reger.debian.org
+}}}
+
+* How to add new LUNs to Bytemark Cluster
+
+** Add new LUN to MSA and export to all blades
+
+{{{
+ Log into MSA controller
+
+ Choose which vdisk to use, use "show vdisks" to list
+
+Add the volume:
+ # create volume vdisk msa2k-2-500gr10 size 5G donizetti
+
+Find a free LUN:
+
+ # show lun-maps
+ or (if we assume they are all the same)
+ # show host-maps 3001438001287090
+
+Make a note of the next free LUN
+
+Generate map commands for all blades, all ports, run locally:
+
+ $ for bl in 1 2 3 4 5 6 ; do for p in 1 2 3 4; do echo "map volume donizetti lun 27 host bm-bl$bl-p$p" ; done ; done
+
+Paste the output into the MSA shell
+
+Find the WWN by doing show host-maps and looking for the volume name.
+Transform it using the sed run at the top of /etc/multipath.conf:
+
+echo "$WWN" | sed -re 's#(.{6})(.{6})0000(.{2})(.*)#36\1000\2\3\4#'
+}}}
+
+{{{
+
+ then:
+ gnt-cluster command "echo 1 > /sys/bus/pci/devices/0000:0e:00.0/cciss0/rescan"
+
+ then:
+ reload multipath-tools on gnt-master (normaly bm-bl1):
+ service multipath-tools reload
+ add the WWNs to dsa-puppet/modules/multipath/files/multipath-bm.conf and define the alias and commit that file to git.
+
+ then:
+ gnt-cluster command "puppet agent -t"
+
+ This will update the multipath config on all cluster nodes. WITHOUT doing this, you can't migrate VMs between nodes.
+}}}
projects / mirror / dsa-wiki.git / blobdiff