Transition Technology: Ticket #478: Import TN.org site from Quince to Puffin

jim — Sat, 26 Jan 2013 20:32:05 GMT

Steps to get site onto platform:

Created "Transition Network D6 002" platform based on latest Drush Make file
Create an empty site called 'transitionnetwork.org' with aliases for dev.newlive.puffin.webarch.net, newlive.puffin.webarch.net, and www.transitionnetwork.org
Use one-time login link sent to me but with dev.newlive.puffin.webarch.net as host in instead of transitionnetwork.org.
Enable Backup and Migrate module

SSH to Puffin and replace the default files folder with symlink to the one imported:

totn
cd static/transition-network-d6-002/sites/transitionnetwork.org
rm -R files
ln -s ~/static/sites/transitionnetwork.org-PROD/files/

jim — Sat, 26 Jan 2013 21:06:11 GMT

Use Backup and Migrate to restore the DB snapshot
(white screen of death, expected)
Run drush transitionnetwork.org rr to clear registry and caches
Run drush transitionnetwork.org updb to do updates present in new platform. This also disables the old modules we won't use like Memcache, Varnish, Session 443

Platform/site seem healthy this time...

NOTES/TO FIX: 1) Views Slideshow: is 2.4 on current site and platform is 3.0, checking to see any issues/damage on homepage slideshow. Needs libraries (now added to Make file and 002 platform) 2) Colorbox: on status page says "Colorbox plugin must be at least 1.3.18"... It should be, will check sites/all/libraries/colorbox folder 3) Broken links due to change in sites/default/files to sites/transitionnetwork.org/files can be dealt with by migrating the site twice in Aegir.

jim — Sat, 26 Jan 2013 21:37:08 GMT

Fixes: 1) Views Slideshow major version must match Views, I've updated Makefile to manage this. 2) Colorbox needs to be in colorbox/colorbox, Makefile reflects this. 3) Will try double-migrate per http://omega8.cc/import-your-sites-to-aegir-in-8-easy-steps-109#hint-8 now.

jim — Sat, 26 Jan 2013 22:04:05 GMT

Updates: 2) Colorbox needs specific version downloaded, rather than latest. Set to 1.3.18 in makefile. 3) Double migrate worked AWESOMELY. (Migrated to 'rename.tn.org', then back to 'tn.org'. Most file references are fixed, but some links refer to the original domain and are therefore broken until DNS changes are done.

New issues: 4) Warning: file_get_contents(sites/all/modules/contrib/gmap/thirdparty/markerclusterer_packed.js): failed to open stream: No such file or directory in _locale_parse_js_file() (line 1710 of /data/disk/tn/static/transition-network-d6-002/includes/locale.inc). -- This is because we need the TN Gmap version instead of the standard one. Will add to Github 5) Logo is set to wrong place at admin/build/themes/settings/transition2 sites/default/files/transition2_logo.jpg needs bold bit moved to transitionnetwork.org

jim — Sat, 26 Jan 2013 22:07:03 GMT

4) Will add our Gmap (rather than patching the shit out of the standard version) to Github and changing the makefile to use our version. 5) also affects favicon -- easy to fix.

So all good so far!!!

To do on this ticket:

Resolve issues 1-5 (mostly done)
Rewrite process in ticket description to final verison
Run though final version with final (hopefully) makefile

hours, totalhours changed

chris — Sun, 27 Jan 2013 17:14:37 GMT

hours changed from 0.0 to 0.15
totalhours changed from 0.0 to 0.15

Replying to jim:

Get the site secured using Aegir's inbuilt SSL handing

Anything I need to help with on this?

I note I need to tweak some Nginx settings regarding HTTPS:

This server is vulnerable to the BEAST attack. Chain issues: Incomplete

https://www.ssllabs.com/ssltest/analyze.html?d=newdev.transitionnetwork.org

jim — Sun, 27 Jan 2013 19:19:06 GMT

@Chris:

SSL should just need the NGINX SSL feature on master.puffin.webarch.net to be enabled, then the site's SSL setting put to 'automatic'. I'll try it on
I'll leave you to work out a fix for BEAST, but if we have found something that needs changing BOA stuff we should raise a ticket. How best to fix?

status, description changed

jim — Sun, 27 Jan 2013 19:19:38 GMT

status changed from new to accepted
description modified (diff)

Need run this Drush command to disable the modules that are installed on current but not needed on the new platform:

drush dis memcache memcache_admin session443 varnish

And do files import... Fleshing out process in description.

description changed

jim — Sun, 27 Jan 2013 23:57:57 GMT

description modified (diff)

Fixed: 4) Our copy of GMap in the repo, makefile updated. 5) Logo/favicon details added to process

As for 6) SSL:

Trying this now: http://community.aegirproject.org/content/content/administrator/post-install-configuration/using-ssl -- Will add to process or raise another ticket depending on sucess/fail when setting this up on http://newlive.puffin.webarch.net/
Chris does this look useful or needed?: http://drupalcode.org/project/barracuda.git/blob/HEAD:/docs/SSL.txt -- seems to be for an extra IP, does that match our needs?

NOTE: the 'Aborted connection' on the DB causing/caused by lots of 502 Bad Gateways is back... going back to #466 now.

description changed

jim — Mon, 28 Jan 2013 17:37:15 GMT

description modified (diff)

Tweaks to process.

description changed

jim — Mon, 28 Jan 2013 17:40:34 GMT

description modified (diff)

Domain name corrections as www is to be used rather than naked domain.

description changed

jim — Mon, 28 Jan 2013 18:47:50 GMT

description modified (diff)

Re SSL -- http://drupalcode.org/project/barracuda.git/blob_plain/HEAD:/docs/SSL.txt <-- this is the 'proper' way of doing things. DO NOT use aegirproject.org link posted in comment 9... Correct link says:

1. Use existing or deploy a new site as usual - don't enable SSL features in Aegir.
2. Create two extra configuration files with contents as shown further below.
    * Replace YO.UR.AEGIR.IP with your Aegir Hostmaster main IP address.
    * Replace YO.UR.EXTRA.IP1,2,3 etc with correct extra IP addresses.
    * Paste your SSL key in the file /etc/ssl/private/abc-ssl-enabled-domain.key
    * Paste your SSL certificate and all intermediate certificates (bundles)
      in the file /etc/ssl/private/abc-ssl-enabled-domain.crt
3. Restart Nginx with `service nginx reload` or `service nginx restart`. Done!

SOOOooo.... I have broken the aegir panel by enabling the SSL/Nginx SSL features, so I'm now backing out of this per http://drupal.org/node/1882078. Bugger. Once it's back to normal I'll follow the instructions above.

---

ALSO: corrections to Drush commands in process

AND: See http://tn.i-jk.co.uk/ <-- platform and site (minus files) installed per this process with no issues except for one Colorbox path thing which I've now fixed in the makefile -- WOOP! Process and makefile good and ready for L-Day!

SSL to go...

hours, totalhours changed

chris — Mon, 28 Jan 2013 19:24:14 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.15 to 0.4

Replying to jim:

Re SSL -- http://drupalcode.org/project/barracuda.git/blob_plain/HEAD:/docs/SSL.txt <-- this is the 'proper' way of doing things.

We are not adding an additional IP address since we don't need one, the directory suggested for creating the two new Nginx config files, /var/aegir/config/server_master/nginx/pre.d/ contains nginx_wild_ssl.conf which has the following:

  ssl_certificate              /etc/ssl/private/nginx-wild-ssl.crt;
  ssl_certificate_key          /etc/ssl/private/nginx-wild-ssl.key;

These are symlinks:

/etc/ssl/private/nginx-wild-ssl.crt -> ../transitionnetwork.org/transitionnetwork.org.crt
/etc/ssl/private/nginx-wild-ssl.key -> ../transitionnetwork.org/transitionnetwork.org.key

But the cert is wrong, Nginx needs the chained one, so I have fixed it:

cd /etc/ssl/private/
rm nginx-wild-ssl.crt
ln -s /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem nginx-wild-ssl.crt

However the certificate used at https://newlive.puffin.webarch.net/ is still wrong, I'm still not clear where the certs are that the server is using?

jim — Mon, 28 Jan 2013 20:15:02 GMT

OK I think I now understand much more about the guts of Aegir after that goose chase!

Nginx SSL entry removed manually from /data/disk/tn/config/server_master/nginx/vhost.d/transitionnetwork.org, which I backed up to /root/scratch/transitionnetwork.org. In there at the top is this section:

server {
  include      /data/disk/tn/config/includes/fastcgi_ssl_params.conf;
  limit_conn   gulag 32; # like mod_evasive - this allows max 32 simultaneous connections from one IP address
  listen       81.95.52.103:443;
  server_name  transitionnetwork.org dev.newlive.puffin.webarch.net newlive.puffin.webarch.net www.transitionnetwork.org;
  root         /data/disk/tn/static/transition-network-d6-002;
  ssl                        on;
  ssl_certificate            /data/disk/tn/config/server_master/ssl.d/transitionnetwork.org/openssl.crt;
  ssl_certificate_key        /data/disk/tn/config/server_master/ssl.d/transitionnetwork.org/openssl.key;
  ssl_protocols              SSLv3 TLSv1;
  ssl_ciphers                HIGH:!ADH:!MD5;
  ssl_prefer_server_ciphers  on;
  keepalive_timeout          70;
  # Extra configuration from modules:
  include      /data/disk/tn/config/includes/nginx_octopus_include.conf;
}

Which answers a number of questions, like where the cert was coming from.

Chris, I would be tempted to revert the wildcard certs symlinks and look around in /data/disk/tn/config/ for all the relevant config. Note that this is all auto-created by Aegir & Octopus, so I'd avoid editing things directly... Extra conf can be added here and there, but best to check the docs here: http://drupalcode.org/project/barracuda.git/tree/HEAD:/docs

So my mistake is fixed... SSL now needs setting up though. I'll research the RIGHT WAY now.

jim — Mon, 28 Jan 2013 22:06:24 GMT

AND Since I restarted Nginx after removing the broken HTTPS stuff, the cert at https://newlive.puffin.webarch.net is correct! Ghandi... WOOP! (or was that your doing Chris?)

Moving SSL stuff to ticket: #484

description changed

jim — Tue, 29 Jan 2013 20:14:46 GMT

description modified (diff)

reroute_email and environment_indicator notes.

description changed

jim — Tue, 12 Feb 2013 23:58:27 GMT

description modified (diff)

A few tweaks...

description changed

jim — Wed, 13 Feb 2013 00:35:03 GMT

description modified (diff)

robots! and testing environment

description changed

jim — Mon, 18 Feb 2013 12:00:17 GMT

description modified (diff)

Done for real, no issues found...

Tweaks for Prod vs Test/Dev?:

puffin_server_override_settings_set_environment('Production');

status changed; resolution set

jim — Mon, 18 Feb 2013 12:47:30 GMT

status changed from accepted to closed
resolution set to fixed

Done, closing.