mwsv3/misc.git
3 years agoMerge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc master
Jon Warbrick [Wed, 26 Jul 2017 09:43:17 +0000 (10:43 +0100)]
Merge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc

3 years agoCommit last batch of changes
Jon Warbrick [Wed, 26 Jul 2017 09:42:39 +0000 (10:42 +0100)]
Commit last batch of changes

3 years agoIgnore distribution tarballs for Nagios plugin.
Ben Harris [Mon, 10 Oct 2016 15:02:14 +0000 (16:02 +0100)]
Ignore distribution tarballs for Nagios plugin.

3 years agoAdd a script for updating Pacemaker for per-node scores.
Ben Harris [Mon, 10 Oct 2016 15:00:11 +0000 (16:00 +0100)]
Add a script for updating Pacemaker for per-node scores.

3 years agoAdd %Prep section to RPM specfile
Ben Harris [Mon, 3 Oct 2016 15:41:36 +0000 (16:41 +0100)]
Add %Prep section to RPM specfile

3 years agoFix spelling of 'License'
Ben Harris [Mon, 3 Oct 2016 15:40:23 +0000 (16:40 +0100)]
Fix spelling of 'License'

3 years agoAdd my makedist script for building RPMs of this (indirectly).
Ben Harris [Mon, 3 Oct 2016 15:38:18 +0000 (16:38 +0100)]
Add my makedist script for building RPMs of this (indirectly).

3 years agoRPM specfile for Nagios plugins
Ben Harris [Mon, 3 Oct 2016 14:51:35 +0000 (15:51 +0100)]
RPM specfile for Nagios plugins

3 years agocheck_mws3_pacemaker: print something on success.
Ben Harris [Mon, 3 Oct 2016 14:25:14 +0000 (15:25 +0100)]
check_mws3_pacemaker: print something on success.

3 years agoAdd a (not yet deployed) script to cross-check VMs between Pacemaker and panel.
Ben Harris [Fri, 30 Sep 2016 18:33:42 +0000 (19:33 +0100)]
Add a (not yet deployed) script to cross-check VMs between Pacemaker and panel.

3 years agoSnapshot after 2016-09-30 emailing.
Jon Warbrick [Fri, 30 Sep 2016 11:11:36 +0000 (12:11 +0100)]
Snapshot after 2016-09-30 emailing.

4 years agoCommit 2016-09-19 mailing (wich actually went out on 2016-09-22)
Jon Warbrick [Thu, 22 Sep 2016 12:43:04 +0000 (13:43 +0100)]
Commit 2016-09-19 mailing (wich actually went out on 2016-09-22)

4 years agoCumilative changes following 20160901 and 20160912 mailings (and
Jon Warbrick [Fri, 16 Sep 2016 15:54:44 +0000 (16:54 +0100)]
Cumilative changes following 20160901 and 20160912 mailings (and
subsequent updates)

4 years agoSnapshot 2016-08-12 emailing
Jon Warbrick [Wed, 31 Aug 2016 09:56:57 +0000 (10:56 +0100)]
Snapshot 2016-08-12 emailing

4 years agoAdd requirements.txt, revise .gitignore again
Jon Warbrick [Wed, 31 Aug 2016 09:56:09 +0000 (10:56 +0100)]
Add  requirements.txt, revise .gitignore again

4 years agoRework processing
Jon Warbrick [Wed, 31 Aug 2016 09:50:57 +0000 (10:50 +0100)]
Rework processing

* Use downloader.py to download data
* Add update-dns-status to manipulate DNS column
* Extract and flaten current MWS2 sites/names exlicitly

4 years agoOmit all .csv and API secrets
Jon Warbrick [Wed, 31 Aug 2016 09:45:01 +0000 (10:45 +0100)]
Omit all .csv and API secrets

4 years agoAdd support for new 'Cancelled' column
Jon Warbrick [Wed, 31 Aug 2016 09:42:57 +0000 (10:42 +0100)]
Add support for new 'Cancelled' column

4 years agoAdd bounce messages from 2016-06-24 mailing
Jon Warbrick [Fri, 24 Jun 2016 15:09:45 +0000 (16:09 +0100)]
Add bounce messages from 2016-06-24 mailing

4 years agoUpdate for plan-still-on-mws2 bulk emailing plan-still-on-mws2-2016-06-24
Jon Warbrick [Fri, 24 Jun 2016 14:55:44 +0000 (15:55 +0100)]
Update for plan-still-on-mws2 bulk emailing

4 years agoRemove unnecessary 'already' in '...you have already told...'
Jon Warbrick [Fri, 24 Jun 2016 14:55:07 +0000 (15:55 +0100)]
Remove unnecessary 'already' in '...you have already told...'

4 years agoUpdate expected fields in flattener
Jon Warbrick [Fri, 24 Jun 2016 14:53:38 +0000 (15:53 +0100)]
Update expected fields in flattener

4 years agoAdd copies of 2016-05-13 emailing bounces
Jon Warbrick [Mon, 16 May 2016 08:36:48 +0000 (09:36 +0100)]
Add copies of 2016-05-13 emailing bounces

4 years agoAdd year to final closedown date no-plan-on-mws2-2016-05-13
Jon Warbrick [Fri, 13 May 2016 14:22:19 +0000 (15:22 +0100)]
Add year to final closedown date

4 years agoModify .gitignore to include derived csv files as audit trail
Jon Warbrick [Fri, 13 May 2016 14:18:49 +0000 (15:18 +0100)]
Modify .gitignore to include derived csv files as audit trail

4 years agoAdd link to 'How to transition' document
Jon Warbrick [Fri, 13 May 2016 14:16:54 +0000 (15:16 +0100)]
Add link to 'How to transition' document

4 years agoAdd no-plan-not-on-mws2 & plan-still-on-mws2 templates
Jon Warbrick [Wed, 11 May 2016 13:17:25 +0000 (14:17 +0100)]
Add no-plan-not-on-mws2 & plan-still-on-mws2 templates

4 years agoMinor rewording
Jon Warbrick [Wed, 11 May 2016 13:16:19 +0000 (14:16 +0100)]
Minor rewording

4 years agoMisc changes to process.sql
Jon Warbrick [Wed, 11 May 2016 13:14:13 +0000 (14:14 +0100)]
Misc changes to process.sql

* Expect input in flattened_curent.csv
* Alow for Plan and DNS_status being 'Mixed' and generate
  extracts for each

4 years agoRework process-current.sh to run flattener on current.csv first
Jon Warbrick [Wed, 11 May 2016 13:11:51 +0000 (14:11 +0100)]
Rework process-current.sh to run flattener on current.csv first

Running flattener first means we can usefully detect when a site
has different plans or statuses for its various hostnames. When
this happens the relarent field in the combined record is set to
'Mixed' which can be matched later in the process.

4 years agoAdd support in flattener for mixed plans and statuses
Jon Warbrick [Wed, 11 May 2016 13:10:29 +0000 (14:10 +0100)]
Add support in flattener for mixed plans and statuses

4 years agoRevise no-plan-on-mws2-template wording
Jon Warbrick [Wed, 11 May 2016 09:35:05 +0000 (10:35 +0100)]
Revise no-plan-on-mws2-template wording

4 years agoMerge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc
Jon Warbrick [Tue, 10 May 2016 16:24:44 +0000 (17:24 +0100)]
Merge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc

4 years agoAdd initial bulk emailing tools
Jon Warbrick [Tue, 10 May 2016 16:24:09 +0000 (17:24 +0100)]
Add initial bulk emailing tools

4 years agobug fix
Abraham Martin [Thu, 17 Mar 2016 12:06:59 +0000 (12:06 +0000)]
bug fix

4 years agoturn up Debconf debugging
Matthew Vernon [Wed, 10 Feb 2016 13:46:29 +0000 (13:46 +0000)]
turn up Debconf debugging

The theory is, when we next have an installer failing, we'll have lots
of logs to look at...

4 years agoMerge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc
Jon Warbrick [Fri, 15 Jan 2016 15:33:01 +0000 (15:33 +0000)]
Merge branch 'master' of ssh://git.csx.cam.ac.uk/mwsv3/misc

4 years agoSuppress email for DNS updates to test and dev databases
Jon Warbrick [Fri, 15 Jan 2016 15:27:11 +0000 (15:27 +0000)]
Suppress email for DNS updates to test and dev databases

4 years agoScript to move from v4 replication to private v6 replication
Matthew Vernon [Tue, 5 Jan 2016 16:43:34 +0000 (16:43 +0000)]
Script to move from v4 replication to private v6 replication

4 years agoFurther updates to email sending by the API
Jon Warbrick [Wed, 23 Dec 2015 15:53:09 +0000 (15:53 +0000)]
Further updates to email sending by the API

1) Fix parsing of the --database argument
2) Make email sender the same as email target
3) Note which database is in use in the message

4 years agoAdd email notification to API
Jon Warbrick [Tue, 22 Dec 2015 17:42:53 +0000 (17:42 +0000)]
Add email notification to API

4 years agoFurther indentation fix
Jon Warbrick [Tue, 22 Dec 2015 16:36:48 +0000 (16:36 +0000)]
Further indentation fix

4 years agoFix (remove) a number of bogus tabs that seem to have sneaked in
Jon Warbrick [Tue, 22 Dec 2015 16:29:29 +0000 (16:29 +0000)]
Fix (remove) a number of bogus tabs that seem to have sneaked in

4 years agoFirst draft rollback script
Matthew Vernon [Thu, 3 Dec 2015 15:39:48 +0000 (15:39 +0000)]
First draft rollback script

4 years agoMerge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc
Matthew Vernon [Wed, 2 Dec 2015 16:18:04 +0000 (16:18 +0000)]
Merge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc

4 years agoCorrect typos, migrate mysql and wordpress
Matthew Vernon [Wed, 2 Dec 2015 16:16:15 +0000 (16:16 +0000)]
Correct typos, migrate mysql and wordpress

typos were with ssh invocations

We also had to migrate /etc/mysql and /etc/wordpress (as they
contained database passwords). This is a bug in our ansible setup,
which should get fixed in due course.

4 years agobug fixes
Abraham Martin [Wed, 2 Dec 2015 10:39:40 +0000 (10:39 +0000)]
bug fixes

4 years agofirst version of xen->xen migration script
Matthew Vernon [Tue, 1 Dec 2015 17:11:07 +0000 (17:11 +0000)]
first version of xen->xen migration script

4 years agoMerge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc
Matthew Vernon [Thu, 26 Nov 2015 18:04:09 +0000 (18:04 +0000)]
Merge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc

4 years agoFirst attempt at anautomatic deployment script
Matthew Vernon [Thu, 26 Nov 2015 18:02:43 +0000 (18:02 +0000)]
First attempt at anautomatic deployment script

4 years agoMove to using password authentication via Oralogin for API
Jon Warbrick [Thu, 26 Nov 2015 14:40:43 +0000 (14:40 +0000)]
Move to using password authentication via Oralogin for API

CH10's 'Oralogin' module alows Perl scripts to authentication using
usernames/passwords stored in ~/.oralogin (on jackdaw) and
/local/oracle/.oralogin/$USER (on Ent). This in turn means that
database accounts can be set to use password authentication and so can
be accessed across the network by suitably-authorised users.

This edit changes the script from using DButil.pm for authetication to
using Oralogin, and removes references to the local copy of DButil.pm.

4 years agoAllow targets in mws3.csx.private domains ans well as mws3.csx
Jon Warbrick [Tue, 24 Nov 2015 16:05:07 +0000 (16:05 +0000)]
Allow targets in mws3.csx.private domains ans well as mws3.csx

4 years agoCorrect error codes for several endpoints
Jon Warbrick [Tue, 24 Nov 2015 16:04:44 +0000 (16:04 +0000)]
Correct error codes for several endpoints

4 years agoMerge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc
Matthew Vernon [Tue, 24 Nov 2015 15:55:40 +0000 (15:55 +0000)]
Merge branch 'master' of git.csx.cam.ac.uk:mwsv3/misc

4 years agoMerge branch 'dns' -- incorporate API developments:
Jon Warbrick [Tue, 17 Nov 2015 11:01:51 +0000 (11:01 +0000)]
Merge branch 'dns' -- incorporate API developments:

* 'nameinfo' returns 'delegated' information
* 'put' will update existing as well as creating new (but you can't
  select one ot the other in advance)
* 'put cname' throws an error if hostname is in a delegated domain
  (and 'put sshfp' is limited to .mws3.csx.cam.ac.uk' names so
  implicitly has the same limitation)
* 'put' and 'delete' report wat they did, if anything
* 'find sshfp' renamed back to 'get'

4 years agoMove ipreg_api2 into ipreg_api dns
Jon Warbrick [Tue, 17 Nov 2015 11:00:29 +0000 (11:00 +0000)]
Move ipreg_api2 into ipreg_api

4 years agoFinal tidying of ipreg_api2.pl, API doc and tests
Jon Warbrick [Tue, 17 Nov 2015 09:26:21 +0000 (09:26 +0000)]
Final tidying of ipreg_api2.pl, API doc and tests

4 years agoAdapt 'put' to create or update; return change indicators
Jon Warbrick [Fri, 13 Nov 2015 17:07:32 +0000 (17:07 +0000)]
Adapt 'put' to create or update; return change indicators

* PUT methods now update an existing record if necessary, or create
  one. IF a matching record already exists they do nothing.

* PUT and DELETE methods now return an indication in the returned JSON
  to show what they did (action = 'I'/'U'/'' for PUT, changed =
  'Y'/'N'/'' for DELETE)

4 years agoUpdate API docs to match FIND SSHF --> GET SSHFP change
Jon Warbrick [Fri, 13 Nov 2015 15:57:41 +0000 (15:57 +0000)]
Update API docs to match FIND SSHF --> GET SSHFP change

4 years agoGratutiously change find_sshfp back to get_sshfp to better match REST
Jon Warbrick [Fri, 13 Nov 2015 15:55:37 +0000 (15:55 +0000)]
Gratutiously change find_sshfp back to get_sshfp to better match REST

It appears to be OK for a RESTfull 'GET' on a container to return a
list of matches, so we'll go back to using GET whan searching for
SSHFP records for clarity.

4 years agoApply fix to the delete_cname update clause supplied by FANF
Jon Warbrick [Fri, 13 Nov 2015 15:51:43 +0000 (15:51 +0000)]
Apply fix to the delete_cname update clause supplied by FANF

4 years agoAllow cnames pointing to non-mws3.csx.cam.ac.uk domains to be deleted
Jon Warbrick [Fri, 13 Nov 2015 15:45:18 +0000 (15:45 +0000)]
Allow cnames pointing to non-mws3.csx.cam.ac.uk domains to be deleted

4 years agoAdd copy of ipreg_api for development
Jon Warbrick [Fri, 13 Nov 2015 11:58:29 +0000 (11:58 +0000)]
Add copy of  ipreg_api for development

4 years agoDrop individual scripts in favour of ipreg_api.pl
Jon Warbrick [Fri, 13 Nov 2015 11:57:22 +0000 (11:57 +0000)]
Drop individual scripts in favour of ipreg_api.pl

4 years agoMerge branch 'dns'
Jon Warbrick [Wed, 11 Nov 2015 15:52:56 +0000 (15:52 +0000)]
Merge branch 'dns'

This merges into master the IP register API contained in the 'dns' directory

4 years agoUpdated API to work, pass manual testing; updated API doc to match
Jon Warbrick [Wed, 11 Nov 2015 15:45:03 +0000 (15:45 +0000)]
Updated API to work, pass manual testing; updated API doc to match

4 years agoemove bogus '=='
Jon Warbrick [Tue, 10 Nov 2015 18:10:17 +0000 (18:10 +0000)]
emove bogus '=='

4 years agoRename 'GET SSHFP' as 'FIND SSHFP'
Jon Warbrick [Tue, 10 Nov 2015 18:06:06 +0000 (18:06 +0000)]
Rename 'GET SSHFP' as 'FIND SSHFP'

The method for searching for SSHFP records doesn't take a primary key,
potentially returns more than one hit, and doesn't error if nothing is
found. As such it doesn't match the expeted behavious of 'GET' so
change the action to 'FIND'.

4 years agoAdd an Ansible cofig for installing the API on Jackdaw
Jon Warbrick [Tue, 10 Nov 2015 17:54:26 +0000 (17:54 +0000)]
Add an Ansible cofig for installing the API on Jackdaw

4 years agoAdjust DButil.pm to add the default prefix for local conenctions
Jon Warbrick [Tue, 10 Nov 2015 17:53:44 +0000 (17:53 +0000)]
Adjust DButil.pm to add the default prefix for local conenctions

4 years agoFix target daratabase to be jdawdev (for the time being)
Jon Warbrick [Tue, 10 Nov 2015 17:53:02 +0000 (17:53 +0000)]
Fix target daratabase to be jdawdev (for the time being)

4 years agoRemove /bin symlinks - will be recreated by Ansible on install
Jon Warbrick [Tue, 10 Nov 2015 17:17:11 +0000 (17:17 +0000)]
Remove /bin symlinks - will be recreated by Ansible on install

4 years agoUpdate dbpl.jackdaw and DButil.pm to do the right thing on Jackdaw
Jon Warbrick [Tue, 10 Nov 2015 17:15:58 +0000 (17:15 +0000)]
Update dbpl.jackdaw and DButil.pm to do the right thing on Jackdaw

4 years agoCreate new single script for the entire API; update API spec to match
Jon Warbrick [Sun, 8 Nov 2015 18:47:34 +0000 (18:47 +0000)]
Create new single script for the entire API; update API spec to match

4 years agoRevise API document to match reality
Jon Warbrick [Sun, 8 Nov 2015 18:07:44 +0000 (18:07 +0000)]
Revise API document to match reality

4 years agoAdd stand-alone versions of make/remove cname and sshfp scripts
Jon Warbrick [Sun, 8 Nov 2015 18:06:25 +0000 (18:06 +0000)]
Add stand-alone versions of make/remove cname and sshfp scripts

4 years agoRename api call paramaters
Jon Warbrick [Fri, 6 Nov 2015 16:10:26 +0000 (16:10 +0000)]
Rename api call paramaters

4 years agoRename hostinfo endpoint to nameinfo
Jon Warbrick [Fri, 6 Nov 2015 14:37:54 +0000 (14:37 +0000)]
Rename hostinfo endpoint to nameinfo

4 years agoAdd API spec document
Jon Warbrick [Thu, 5 Nov 2015 18:37:36 +0000 (18:37 +0000)]
Add API spec document

4 years agoAvoid uninitialised variable warning for names not in unique_name
Jon Warbrick [Thu, 5 Nov 2015 18:34:12 +0000 (18:34 +0000)]
Avoid uninitialised variable warning for names not in unique_name

4 years agoDon't truncate late_command
Matthew Vernon [Thu, 5 Nov 2015 18:26:13 +0000 (18:26 +0000)]
Don't truncate late_command

4 years agodevices need to be /dev/foo ; check for .../blockdev/loop
Matthew Vernon [Thu, 5 Nov 2015 18:18:34 +0000 (18:18 +0000)]
devices need to be /dev/foo ; check for .../blockdev/loop

The former is just the syntax you need, the latter catches the loop
devices that are in use (and so have non-zero size).

4 years agore-include partman-auto/method
Matthew Vernon [Thu, 5 Nov 2015 18:16:44 +0000 (18:16 +0000)]
re-include partman-auto/method

(removing it was an error)

4 years agoFix hostinfo.pl in respet of domains and cnames
Jon Warbrick [Thu, 5 Nov 2015 18:11:39 +0000 (18:11 +0000)]
Fix hostinfo.pl in respet of domains and cnames

* Correctly detect failing to find a domain matching the host name

* Avoid finding CNAMES in both unique_names and maildomain_or_cname

4 years agoCall find_disks.sh for disk setup
Matthew Vernon [Thu, 5 Nov 2015 17:02:56 +0000 (17:02 +0000)]
Call find_disks.sh for disk setup

4 years agoGenerate key for RSA key
Matthew Vernon [Thu, 5 Nov 2015 17:00:52 +0000 (17:00 +0000)]
Generate key for RSA key

4 years agoAdd initial hostinfo command
Jon Warbrick [Thu, 5 Nov 2015 16:53:59 +0000 (16:53 +0000)]
Add initial hostinfo command

4 years agoAdd main system Perl libary to get us JSON.pm
Jon Warbrick [Thu, 5 Nov 2015 16:53:27 +0000 (16:53 +0000)]
Add main system Perl libary to get us JSON.pm

4 years agoAdd symlinks to make dbtime work
J. Warbrick [Wed, 4 Nov 2015 16:04:12 +0000 (16:04 +0000)]
Add symlinks to make dbtime work

4 years agoAdd initial example files from main ipreg repository
Jon Warbrick [Wed, 4 Nov 2015 15:44:17 +0000 (15:44 +0000)]
Add initial example files from main ipreg repository

4 years agoFirst draft script to work out what RAID to do
Matthew Vernon [Wed, 4 Nov 2015 11:23:06 +0000 (11:23 +0000)]
First draft script to work out what RAID to do

4 years agomodified menu files
Matthew Vernon [Mon, 2 Nov 2015 17:33:11 +0000 (17:33 +0000)]
modified menu files

4 years agoSeed as much network as we can ; shut up popcon ; remove junk lv
Matthew Vernon [Thu, 22 Oct 2015 17:51:53 +0000 (18:51 +0100)]
Seed as much network as we can ; shut up popcon ; remove junk lv

4 years agoMake the surplus partition be an LVM LV
Matthew Vernon [Tue, 20 Oct 2015 17:04:25 +0000 (18:04 +0100)]
Make the surplus partition be an LVM LV

4 years agoTry upping the minimum size of the raid set
Matthew Vernon [Tue, 20 Oct 2015 16:32:58 +0000 (17:32 +0100)]
Try upping the minimum size of the raid set

4 years agousb-stick appearing as sdh -> use sda->g
Matthew Vernon [Tue, 20 Oct 2015 15:47:46 +0000 (16:47 +0100)]
usb-stick appearing as sdh -> use sda->g

4 years agoFirst attempt
Matthew Vernon [Tue, 20 Oct 2015 15:46:31 +0000 (16:46 +0100)]
First attempt