X-Git-Url: https://git.adam-barratt.org.uk/?a=blobdiff_plain;f=input%2Fdsablog%2F2014%2FThe_Debian_DNA_universe.mdwn;fp=input%2Fdsablog%2F2014%2FThe_Debian_DNA_universe.mdwn;h=0000000000000000000000000000000000000000;hb=03494baf8275a389b6c30419ef775a32ddf86877;hp=a5105d2fe3dcc3d5044f83efa448e5c45aa6da76;hpb=ada4d3cc46579694ca8086850f717b059fdd1cee;p=mirror%2Fdsa-wiki.git diff --git a/input/dsablog/2014/The_Debian_DNA_universe.mdwn b/input/dsablog/2014/The_Debian_DNA_universe.mdwn deleted file mode 100644 index a5105d2..0000000 --- a/input/dsablog/2014/The_Debian_DNA_universe.mdwn +++ /dev/null @@ -1,158 +0,0 @@ -[[!meta author="Peter Palfrader"]] - -# Abstract - -I recently moved our primary nameserver from `orff.debian.org`, which is -an aging blade in Greeze, to a VM on one of our ganeti clusters. In the -process I rediscovered a lot about our DNS infrastructure. In this post -I will describe the many sources of information and how it all comes -together. - -# Introduction - -The [Domain Name System][DNS] is the hierarchical database and query -protocol that is in use on the Internet today to map hostnames to IP -addresses, the reverse thereof, lookup relevant servers for certain -services such as mail, and a gazillion other things. Management and -authority in the DNS is split into different zones, subtrees of the -global tree of domain names. - -Debian currently has a bit over a score of zones. The two most -prominents clearly are `debian.org` and -`debian.net`. The rest is made up of `debian` -domains in various other top level domains and reverse zones, which -are utilized in IP address to hostname mappings. - -# Types and sources of information - -The data we put into DNS comes from a wide range of different systems: - - * [Classical zonefiles maintained in git][domains]. This represents - the core of our domain data. It maps services like - `blends.debian.org` to `static.debian.org` or - specifies the servers responsible for accepting mail to - `@debian.org` addresses. It also is where all the - `ftp.CC.debian.org` entries are kept and maintained together with the - mirror team. - * Information about `debian.org` hosts, such as `master`, is maintained - in Debian's [userdir LDAP][db], queryable using - LDAP[^ldap]. - * This includes first and foremost the host's IP addresses (v4 and v6). - * Additionally, we store the server responsible for receiving a host's - mail in LDAP (`mXRecord` LDAP attribute, DNS `MX` record type). - * LDAP also has some specs on computers, which we put into each host's - `HINFO` record, mainly because we can and we are old-school. - * Last not least, LDAP also has each host's public ssh key, which we - extract into [SSHFP][rfc4255] records for DNS. - * LDAP also has per-user information. Users of debian infrastructure - can attach limited DNS elements as `dnsZoneEntry` attributes to their - user[^ldap2]. - * The auto-dns system (more on that below). - * Our [puppet] also is a source of DNS information. Currently it - generates only the [`TLSA`][rfc6698] records that enable clients to - securely authenticate certificates used for mail and HTTPS, similar - to how `SSHFP` works for authenticating ssh host keys. - -# Debian's auto-dns and geo setup - -We try to provide the best service we can. As such, our goal is that -for instance user access to [`www`][www] or [`bugs`][bugs] should always -work. These services are, thus, provided by more than one machine on -the Internet. - -However, HTTP did not specify a requirement for clients to re-try a -different server if one of those in a set is unavailable. This means -for us that when a host goes down, it needs to be removed from the -corresponding DNS entry. -Ideally, the world wouldn't have to wait for one of us to notice and -react before they can have their service in a working manner. - -Our solution for this is our auto-dns setup. We [maintain a -list][auto-dns] of hosts that are providing a service. We [monitor them -closely][mini-nag]. Whenever a server goes away or comes back we -automatically rebuild the zone that contains the element. - -This setup also lets us reboot servers cleanly — since one of the -things we monitor is "is there a shutdown running", we can, simply by -issuing a `shutdown -r 30 kernel-update`, de-rotate the machine in -question from DNS. Once the host is back it'll automatically get -re-added to the round-robin zone entry. - -The auto-dns system produces two kinds of output: - -* In *service*-mode it generates a file with just the address records - for a specific service. This snippet is then included in its zone - using a standard bind `$INCLUDE` directive. Services that work like - this include [`bugs`][bugs] and `static` ([service definition for - static][static]). -* In *zone*-mode, auto-dns produces zonefiles. For each service it - produces a set of zonefiles, one for each out of a set of different - geographic regions. These individual zonefiles are then transferred - using `rsync` to our [GEO-IP enabled][geoip] nameservers. This - enables us to give users a list of `security` mirrors closer to them - and thus hopefully faster for them. - -# Tying it all together - -![The Debian DNS Rube Goldberg Machine.](../debian-dns.png) - -Figure 1: The Debian DNS Rube Goldberg Machine. - -Once all the individual pieces of source information have been -collected, the `dns-update` and `write_zonefile` scripts from our -[dns-helpers] repository take over the job of building complete -zonefiles and a bind configuration snippet. Bind then loads the zones -and notifies its secondaries. - -For geozones, the zonefiles are already produced by auto-dns' -`build-zones` and those are pulled from the geo nameservers via rsync -over ssh, after an ssh trigger. - -# and also DNSSEC - -All of our zones are signed using DNSSEC. We have a script in -[dns-helpers] that produces, for all zones, a set of rolling signing -keys. For the normal zones, bind 9.9 takes care of signing them -in-process before serving the zones to its secondaries. For our -geo-zones we sign them in the classical `dnssec-signzone` way before -shipping them. - -The secure delegation status (DS set in parent matches DNSKEY in child) -is monitored by a set of nagios tests, from both [dsa-nagios] and -[dns-helpers]. Of these, `manage-dnssec-keys` has a dual job: not -only will it warn us if an expiring key is still in the DSset, it can -also prevent it from getting expired by issuing timly updates of the -keys metadata. - -# Relevant Git repositories: - -* [domains] -* [auto-dns] -* [mini-nag] -* [dns-helpers] -* [puppet] - ---- - -[^ldap]: `ldapsearch -h db.debian.org -x -ZZ -b dc=debian,dc=org -LLL 'host=master'` - -[^ldap2]: `ldapsearch -h db.debian.org -x -ZZ -b dc=debian,dc=org -LLL 'dnsZoneEntry=*' dnsZoneEntry` - - -[DNS]: https://en.wikipedia.org/wiki/Domain_Name_System -[db]: https://db.debian.org/ -[rfc4255]: https://tools.ietf.org/html/rfc4255 -[rfc6698]: https://tools.ietf.org/html/rfc6698 -[www]: https://www.debian.org/ -[security]: http://security.debian.org/ -[bugs]: http://bugs.debian.org/ -[geoip]: https://en.wikipedia.org/wiki/Geolocation_software - -[puppet]: http://anonscm.debian.org/gitweb/?p=mirror/dsa-puppet.git;a=tree -[domains]: http://anonscm.debian.org/gitweb/?p=mirror/domains.git;a=tree -[auto-dns]: http://anonscm.debian.org/gitweb/?p=mirror/dsa-auto-dns.git;a=tree -[mini-nag]: http://anonscm.debian.org/gitweb/?p=mirror/dsa-mini-nag.git;a=tree -[dns-helpers]: http://anonscm.debian.org/gitweb/?p=mirror/dns-helpers.git;a=tree -[static]: http://anonscm.debian.org/gitweb/?p=mirror/dsa-auto-dns.git;a=blob;f=services/static.debian.org.service;hb=HEAD - --- Peter Palfrader