From: Martin Zobel-Helas Date: Fri, 20 Jul 2018 13:53:33 +0000 (+0200) Subject: Merge remote-tracking branch 'origin/master' into staging X-Git-Url: https://git.adam-barratt.org.uk/?p=mirror%2Fdsa-puppet.git;a=commitdiff_plain;h=76ca91bce24ecbcbcc4e62a37aa06fd0fb9f96c7 Merge remote-tracking branch 'origin/master' into staging * origin/master: Add sallinen to blacklist_acpi_power_meter list We only have puppets >= 3.0 now run puppet every 1.5 hrs instead of every 2 Remove wheezy-supporting cruft fix apache version our cipher suite is still the one recommended by mozilla retire smetana Apparently, no quoting and use template after setting var fix template Use update-ca-certificates to update ca-global on stretch and later Give us longer to notice degraded boot only run /usr/local/sbin/update-ca-certificates-dsa if it exists Allow debadmin to sudo to codesign Make salsa.d.o the default ssl vhost on godard so lame clients can get to it Comment out rate-limiting of https traffic on security-tracker Increase https bandwidth for security-tracker Keep things cached for at least 10min Fix apache module name Use mod_cache_disk on security-tracker Fix typo in comment drop things from 66.170.99.[12] fix rule disable deflate on security-tracker. we are cpu bound do some basic traffic shaping on soriano enable expires module for security-tracker move apache config for security-tracker.debian.org.conf to puppet Kill planet.debian.net (RT#7019) The git user's sudo entries should be NOPASSWD (RT#7316) fix rule name snapshot - drop traffic from 61.69.254.110 Also give the git user sudo access to salsa-* on godard (RT#7316) More users for salsa (RT#7316) Add registry.salsa.debian.org vhost config (RT#7316) unicamp renumbering remove parth, re: RT#7334 setup-all-dchroots: wheezy is gone, jessie is limited to LTS architectures get arm-arm-01 out of broken_rtc set Install ganeti-reboot-cluster Update my home ip ranges yet again set Expires to 1 week also for .gz files Enable HTTP/2 on sources.d.o http rate limiting for dynamic hosts also on v6 snapshot: allow 6 requests per minute even to clients that we think are excessive snapshot_web dynamic rules snapshot_web dynamic rules Drop apache2deb9 variable Add data-protection@d.o to various exim config bits port 6081 should be allowed via snapshot try apache rate limiting on snapshot hosts, 2 try apache rate limiting on snapshot hosts add template parts of the nagios setup nagios: install some packages and define service debian nagios service does not use digest auth nagios: we do not need proxy_http add apache::authn_anon and apache::auth_digest nagios master: apache vhost start using nagios::server again, move cert setup there remove obsolete stuff from nagios::server restart stale icinga automatically wider regex for clearing failed rsyncd service to catch rsyncd-snapshot-farm@ ignore ruby-dbi ruby-deprecated ruby-dbd-pg on snapshot hosts ignore ruby-dbi ruby-deprecated ruby-dbd-pg on snapshot hosts set expires: headers on alioth-archive Add a few pointers on the anonscm index page index page for anonscm, 2 index page for anonscm put an /srv/anonscm.debian.org/htdocs in place vhost cleanup vhost update non-SSL is on 80 Use anonscm.map try to put anonscm.map onto host, 3 try to put anonscm.map onto host, 2 try to put anonscm.map onto host prepare anonscm vhost set hsts on snapshot Try to put haproxy on snapshot hosts Add a logging device for haproxy Add haproxy module from tor a haproxy facter More verbose setup-all-dchroots when run in a terminal install snapshot cert sallinen: retire 443->5473 dnat Fetch sallinen.debian.org snapshot backups from port 5473 pg ssh auth: danzi: remove read for sibelius; lw07: switch read sibelius to read sallinen; sallinen: remove read sibelius backup sallinen pg sallinen has a pg server pg firewalling add lw07 to snapshot_web group start varnish only after network is online Try an network_online target for stretch hosts And setup ferm, 2 And setup ferm add -j unix,user=vcache -F to varnishd call and use array for listening ports varnish on stretch now takes several -a arguments instead of one with multiple addresses sallinen varnish, 2 sallinen varnish a very basic generic varnish module rename varnish to varnish_pkgmirror module rename varnish to varnish_pkgmirror module allow archvsync to trigger snapshot imports block mails from @qq.com fix kanboard role (php wants mpm_prefork) add a kanboard role kanboard group members can run stuff as kanboard on kantuser Fixup previous commit, log directory permissions were already defined pybuildd: ensure that the build and logs dir have the correct permissions New IP ranges for jcristau Fix acquire-reboot-lock molly-guard hook to actually keep the lock until shutdown retire old basic-ssh_known_hosts setup put initial ssh_known_hosts in place and run ud-replicate by puppet and indexes on alioth-archive alioth-archive needs mod rewrite fix path alioth-archive apache site snapshot: rewrite module Add apache vhost put apache on sallinen sudo for alioth-archive create /srv/alioth-archive on alioth-archive host dedication for grabbe install apache on alioth archive prepare alioth archive puppet role fix grabbe-lvm volume name add grabbe volumes Fixup buildd manifest for jessie pkg-ruby-extras.alioth.d.o on static Give up on the distinction between /etc/ssl/certs and /etc/ssl/ca-debian Remove CAs we no longer use from /etc/ssl/ca-debian/ Also remove /usr/local/share/ca-certificates/debian.org Get rid of /etc/ssl/servicecerts check-libs: ignore all access to /srv/salsa/repos by user git, regardless of process name Decommission zemlinsky.d.o (RT#7208) Remove buildd package on pybuildds based buildds buildd: use a different configuration for buildd and pybuildd Reorganize buildd module into different sections buildd: drop old compat code, make more jessie code conditional buildd: remove buildd-schroot-aptitude-kill.squeeze Allow ftp-masters access to the dak-code user Add video.debconf.org redirect on static (RT#7186) Cleanup roles::signing some more Delete scripts for code signing buildd lingering: remove a bashism buildd lingering: setup XDG_RUNTIME_DIR in .profile Set up lists.alioth.debian.org to alioth-lists.debian.net redirect buildd lingering: ensure /var/lib/systemd/linger directory exists Enable lingering and persistent journal on buildds Deploy ssl cert for bugs-devel.d.o on bugs-master Fix logic in cleanup-watcher-pause-file: clean out files *after* they should be deleted Only set headers in apache if they don't exist buildd.d.o: update archive key 99porterbox-extra-sources: Enable debug archive for buster and beyond 99porterbox-extra-sources: Update security blacklist post-stretch smaller timeout before we attempt to restart hpasmcli restart hp-health on lobos and villa if they are broken Do ignore raid controller cache failures on lw08 Try to make dsa-check-hpssacli cron entry setup code easier to read raise warn-age for pg base backups to 11 days There is no ferm-restart Exec to notify postgres-make-base-backups: fix () formatting format days differently postgres-make-base-backups: and print seconds as times postgres-make-base-backups: print more values postgres-make-base-backups: rename variables to make them more obvious postgres-make-base-backups: re-order logic for consistency also print cutoff times Format time deltas in a readable way instead of in seconds Try to escape things differently running every half hour should also suffice easily, with a semicolon running every half hour should also suffice easily postgres-make-base-backups: locks and logs run postgres-make-base-backups every 10 minutes not only on Sunday sane mode for state dir And create state dir for postgres-make-base-backups run base backups spread over time. This also should help us to recover from failures or reboots better Have postgres-make-base-backups use postgres-make-one-base-backup Make a postgres-make-one-base-backup script with the logic from postgres-make-base-backups ferm::conf - include ferm start ferm config with a 00-init and start SSH*SOURCES there ferm::conf - merge with tor version Revert "The debian.ch domain is obsolete" Revert "Revert "massage log messages"" The debconf13.ch domain is obsolete Revert "massage log messages" The debian.ch domain is obsolete massage log messages massage log messages Run our own bacula scheduler from cron Update ntp init script to the stretch version (RT#6907) Drop alioth zone from named config Fix /etc/repro/radius-servers more Fix /etc/repro/radius-servers Configuration item "hashsize" is deprecated Configuration item "allowmultiplekeys" is deprecated Configuration item "ignorenislike" is deprecated And fixup another path Fix path to template Disable default freeradius sites I don't think we want Attempt to pull in some of the freeradius config from rtc.d.o Also put bacula messages into syslog Disable scheduling for backup jobs in preparation of deploying our own scheduler Only add host to bacula dsa client list if we do backups for it Update (c) year Be more defensive when removing potentially obsolete pools collect backup client list in a plain text file bacula: remove obsolete pools Redirect all of *.pages to https (re: RT#7072) mirror-health: set User-Agent http header Revert "Make security -> security-cdn redirect global, not just for the linux package" Make security -> security-cdn redirect global, not just for the linux package Drop security-cdn.d.o on stretch storace also makes ACPI noises about power_meter we do not need to backup clamav-unofficial-sigs files push empty /var/lib/varnish/.nobackup mirror-conova also does lots of ACPI power-meter dmesg noise Decommission mirror-bytemark Fix check url for security mirror health Run dsa-check-openmanage on schumann and wieck mirror-bytemark no longer a fastly backend for /debian/ make schumann a fastly backend for security Remove /srv/ftp.root from security mirrors Serve security mirrors from /srv/mirrors/debian-security Import facts from schumann Drop m68k@buildd.debian.org -> m68k-build@nocrew.org rewrite Add schumann to the security_mirror role Remove lobos from fastly security backends for now dupload.conf: fix a thinko in the security upload hostname buildd: do security uploads using SSH rsync-ssh-wrap: force the permissions of uploaded files planet-master.d.o: fix a thinko in my previous commit planet-master.d.o: only allow access from localhost and local IP 99builddsourceslist: access the security archive using https lintian.d.o: fix deflate output filter Mock more certificates RT#7092: Apache on godard adds an additional X-Xss-Protection Import facts from godard octocatalog: add dummy file for LE service certs Mock ldapinfo during octocatalog runs static_mirror: enable deflate and filter modules Install ca-certificates in the buildd chroots lintian.d.o: Move svg compression to the resources directory lintian.d.o: Remove redundant + incorrect IfModule mod_userdir Revert "99builddsourceslist: access the security archive using https" 99builddsourceslist: access the security archive using https Fully retire spontini.d.o Also drop security anycast-test mirrors snapshot storage nodes want the toolchain to build the snapshot fsck utility setup-dchroot: fix a typo Install apt-transport-https in the buildd chroots Drop anycast-test mirrors from apt More kfreebsd removal setup-all-dchroots: get rid of kfreebsd and ppc64 nagios: use dsa-check-systemd-services instead of systemctl is-system-running Also systemctl reset-failed failed session-nnn.scope Move failed rsync cleanup into systemd module octocatalog: add dummy file for LE service certs Fixup local-mirror.cdbuilder sites-enabled symlink name Add {deb,security}.d.o aliases to local-mirror.cdbuilder use ttyS1 for the serial console on casulana Get trailing slashes right for aliases First go at cdbuilder local mirror export (re: RT##7101) Add a apache_not_public role where we do not add ferm allow rules and put casulana into it no more experimental_apache (previously cgi-grnet-01, pejacevic, petrova) Add cdbuilder-logs static component (re: RT##7101) Add casulana as a static source for cdbuilder-logs (re: RT##7101) RT#7092: Apache on godard adds an additional X-Xss-Protection Test with Puppet 4.8 Update facts Move nagios stuff Move generated cert files to new location Update octocatalog job Test with Puppet 4.8 Update facts Move nagios stuff Move generated cert files to new location Update octocatalog job rsync on lw09,lw10 update lw autotab update lw autotab do nfs server setup on lw09/lw10 no more 10/8 network at leaseweb remove sgran from root keys remove sgran IP range. he can hop via master if needed puppet does not have any mail config in /srv/puppet.debian.org/mail backgrounding does not really work remotely dsa-restart-all-idle-postgres: only restart pg instances that show up in dsa-check-libs dsa-restart-all-idle-postgres: and do not keep fds open dsa-restart-all-idle-postgres: disown background jobs instead of waiting for them in practice make the sleep longer fix filename Add script to restart postgres clusters ignore wb-buildd.more on buildd_master role hosts samhain ignore /etc/ssh/userkeys/buildd-uploader on ssh upload hosts Use "restrict" key option for buildd access to upload hosts Use "restrict" key option for buildd access to wanna-build Use "restrict" key option for storace's da-backup keys Use "restrict" key option in debbackup authorized_keys Simplify portforwarder authorized_keys options Put ganeti VMs into their own systemd scope modules/postgres/manifests/backup_source: add a comment re docs Add a comment header to /etc/ssh/userkeys/debbackup Do samhain checks only half as often Update private IP range at leaseweb Add debconf18.debconf.org config on debussy (rt#7089) update sudo for new dsa-check-libs call Clean up failed rsyncs every few minutes ignore salsa fd leak in sidekiq for dsa-check-lib purposes and log checksums correctly also log failed target pg-backup-file: continue after failures and only report at the end Decommission fano and finzi mirror-anu should not actually have an onion address Improve kpartx rule Disable default kpartx udev rule Get rid of obsolete vsftpd::site→absent resources No more conntrackd in bm, so drop firewall opening Retire ftp.d.o role, it is unused Clean up debugging foo steve probably does not care about samhain mails very much Get rid of unused role Get rid of some intermediate variables Move onion IP addresses into hiera Simplify debian_mirror for hiera-hash Whitespace Move debian_mirror over to being a hash Use .dig to dig into hiera structs Debugging Cleanup obsolete absent resource Get rid of security_mirror_onion role in favour of just keying off the ip address in hiera sshd: Raise MaxStartups on ssh upload hosts Decommission fils and fayrfax sshd_config: Remove UsePrivilegeSeparation yes. on stretch the default is sandbox which seems better sshd_config: remove commented out options and options where we just use the default value (according to the stretch manpage) Simplify lookups now that security_mirror is a hash Switch the security mirror role over to using a hash Add support to hashes for has_role Whitespace fixups Add localhost listens when listen-addresses is set Whitespace Pull out listen addresses from hiera again Set service-hostname for mirror-conova too mirror-conova is a fastly backend, mark it as such Fix typo Hard code listen IPs while I debug hiera again Avoid redeclaring the mirror-health file resource Stop hard coding host list for debian_mirror and use the same code we use for security Refactor hiera lookup for security mirrors slightly Gah, puppet! Use notify, not notice for debugging Revert "Correct hiera function call syntax" Fix has_role to handle richer data structures properly Hard code deb.d.o backend hosts while debugging Revert "Debugging" Debugging Debugging Correct hiera function call syntax Also redirect mips64el to the mips port family page Fix a thinko in previous commit lobos and villa do not have a battery on their raid controller More debugging Fix typo More gunking around to see if we can make this work Make all entries in security_mirror into hashes More syntax fixing YAML is hard Use hiera data for pulling health check data for security hosts Typos-r-us Pull list of hosts to health check from hiera Remove backup access from franck.d.o dsa-check_puppet_agent was renamed to dsa-check-puppet_agent get rid of pizzetti Move listen-address information out of manifest and into hiera Use ensure_packages to avoid problems with puppet redeclaring resources remove falla and fischer Pull listen addresses for apache mirrors from hiera remove bendel/lists blackhole rules that are probably long obsolete Fix yaml syntax Add extra metadata for debian_mirror hosts remove busoni Import cron entries from dsa-nagios-check package Use the right path to health checks on security hosts Decommission ubc-bl*.debian.org Publish security mirror health on _health Fix hiera function call syntax Start setting up mirror health checking for security too Cut down a tiny bit on exim config distributed everywhere Remove obsolete block Try harder at handling connection timeouts for mirror-health fasolo, klecker: blacklist acpi power meter. rt#6974 systemd: do not reload journald godard: enable persistent journald storage. rt#7049 wafer: only ask for client certs on the login page Django sites rely on Referrer headers for XSS protection wafer wants to be able to write its log, make it run with the debconf-web gid wafer config uses expires apache module debussy wants sso_rp for wafer fixup debconf_wafer role apache config for wafertest.debconf.org Use a specific IP address for pages.d.n's vhost Add debussy to the insecure_ssl role fix pages port once more fix port for pages ssl cert for pages.debian.net do proxypass for pages SSL for pages.debian.org ProxyPass everything so we can set nocanon (re: RT#7057) change redirections about policy manual to 302, since a change back to the multi-page format is under consideration 79.124.75.18 sends us hotel booking spam update recursors for grnet Decommission asachi, arm-linaro-01 and arm-linaro-03 (RT#6895) use ttyS1 for the kernel console on fasolo Try to get ipsec between storace and fasolo And ensure wsgi module gets loaded Switch debtags to wsgi python3 lower heartbeat intervals Set Heartbeat Interval in the Director resource instead of each client's Client resource only manage grub if we have it --- 76ca91bce24ecbcbcc4e62a37aa06fd0fb9f96c7