Transition Technology: Ticket #631: Move Transition Culture onto PARROT

chris — Fri, 29 Nov 2013 13:49:39 GMT

Ed, have you spoken to Simon about this yet, by all means CC me in on emails regarding the size of the site and traffic.

ed — Mon, 02 Dec 2013 14:17:12 GMT

speaking to him on thurs 5/12 - will hook you up asap after that

chris — Fri, 06 Dec 2013 11:24:55 GMT

Replying to ed:

speaking to him on thurs 5/12 - will hook you up asap after that

How did that go?

owner, component changed

chris — Fri, 06 Dec 2013 11:25:32 GMT

owner changed from ed to chris
component changed from Unassigned to Parrot server

cc, status changed

chris — Fri, 06 Dec 2013 11:26:26 GMT

cc ed added
status changed from new to accepted

ed — Mon, 09 Dec 2013 10:41:20 GMT

Ed has spoken to Simon - handing over to Chris - if we can move it before Christmas 2013 that would be great. Depending on TC's DB size, we may not need to make PARROT any bigger or faster.

ed — Mon, 09 Dec 2013 10:42:34 GMT

TC has had approx 70K pageviews in last three months

hours, totalhours changed

chris — Mon, 09 Dec 2013 11:00:47 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.0 to 0.25

I have just sent this email to Simon:

This is where we are at space wise on the server:
Filesystem      Size  Used Avail Use% Mounted on
rootfs          9.9G  6.0G  3.5G  64% /
/dev/xvda3      8.9G  5.5G  3.0G  65% /home
The files will live in /home and the database plus the plain text
nightly dump in /
Simon, the easiest thing for me would be to simply rsync the files and a
recent db dump off the bytemark server to our server, if that is OK with
you? If it is you could add the attached public key to
~/.ssh/authorized_keys on an account the server and let me know the
account name and where things are and I'll get it sorted.
Alternatively I could create an account on our server and add your
public key / send you the password and you could rsync the files and a
db dump up.
I'm happy doing it either way, let me know what is best for you.

hours, totalhours changed

chris — Mon, 09 Dec 2013 14:42:30 GMT

hours changed from 0.0 to 0.45
totalhours changed from 0.25 to 0.7

I've sortied out ssh access with Simon.

Creating an account for the site, following wiki:ParrotServer#AddingaNewWordPressSite

sudo -i
curses-create-user

Copy the files over:

rsync -av tc:/srv/transitionculture.org/public/htdocs/ /home/tc/sites/default/

Dump the DB and copy it over and import it:

ssh tc
cd /tmp
mysqldump -uroot -pXXXXXX trans1t10ncult > trans1t10ncult.sql
exit
cd /home/tc/private
scp tc:/tmp/trans1t10ncult.sql .
cat trans1t10ncult.sql | mysql tc

Edit /root/webarch/accounts/sites.txt and rebuild apache config:

tc default tc.parrot.webarch.net transitionculture.org,www.transitionculture.org
buildapache tc
chown -R tc:tc /home/tc/sites/default

The wp-config.php was edited and we have the site up, cron and .htaccess needs sorting.

http://tc.parrot.webarch.net/

hours, totalhours changed

chris — Mon, 09 Dec 2013 15:26:42 GMT

hours changed from 0.0 to 0.1
totalhours changed from 0.7 to 0.8

This is basically complete, if we don't do the DNS switch today then another rsync of the files and redumping the database and copying that across and inserting it will be nedeed, otherwise this is all that needs doing:

Simon to update the DNS
Uncomment the tc crontab and test it's working.
Convert the nginx.conf rules, see below, into Apache .htaccess rules

if ( $request_filename ~ feed/ ) {
        rewrite ^ https://www.transitionnetwork.org/blogs/feed/rob-hopkins/ permanent;
}
# BEGIN W3TC Browser Cache
gzip on;
gzip_types text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;
location ~ \.(css|htc|js|js2|js3|js4)$ {
    add_header X-Powered-By "W3 Total Cache/0.9.3";
}
location ~ \.(html|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml)$ {
    add_header X-Powered-By "W3 Total Cache/0.9.3";
}
location ~ \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|json|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$ {
    add_header X-Powered-By "W3 Total Cache/0.9.3";
}
# END W3TC Browser Cache
# BEGIN W3TC Skip 404 error handling by WordPress for static files
if (-f $request_filename) {
    break;
}
if (-d $request_filename) {
    break;
}
if ($request_uri ~ "(robots\.txt|(sitemapindex|[a-z0-9_-]+)\.xml|[a-z0-9_\-]+-sitemap([0-9]+)?\.xml(\.gz)?)") {
    break;
}
if ($request_uri ~* \.(css|htc|js|js2|js3|js4|html|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml|asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|json|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$) {
    return 404;
}
# END W3TC Skip 404 error handling by WordPress for static files

ed — Mon, 09 Dec 2013 16:05:38 GMT

All clear to move from TN

hours, totalhours changed

chris — Mon, 09 Dec 2013 19:45:00 GMT

hours changed from 0.0 to 0.25
totalhours changed from 0.8 to 1.05

Simon and I have just crossed emails, he has moved transtionculture.org to the Gandi nameservers while I set it up on the Webarchitects ones, this is the zone file:

; addresses and other host information
;
$TTL 86400
@          IN   SOA   dns1.webarchitects.co.uk.    root.dns1.webarchitects.co.uk. (
                                2013120901      ; serial
                                      7200      ; refresh
                                      3600      ; retry
                                      1209600   ; expire
                                      86400 )   ; minimum
;
; define the nameservers and the mailservers
;
        IN      NS      dns1.webarchitects.co.uk.
        IN      NS      dns0.webarchitects.co.uk.
;       IN      MX      10 mx.webarch.net.
; define localhost
;
localhost  IN   A     127.0.0.1
;
;
; parrot.webarch.net
www                     IN      A       81.95.52.43
transitionculture.org.  IN      A       81.95.52.43

And the servers have updated:

dig @dns0.webarchitects.co.uk transitionculture.org +short
81.95.52.43
dig @dns1.webarchitects.co.uk transitionculture.org +short
81.95.52.43

I have enabled the crontab (actual key replaced with XXXX):

sudo -i
crontab -e -u tc
*/20 * * * * wget http://transitionculture.org/wp-content/plugins/wp-o-matic/cron.php?code=XXXX>/dev/null 2>&1

I have added the site to the front page of the wiki, wiki:WikiStart#ServersandWebsites

I have created a page for it, it can be documented next month, wiki:TransitionCultureWordPress

I still need to sort the .htaccess file out.

attachment set

chris — Tue, 10 Dec 2013 10:23:51 GMT

attachment set to parrot-2013-12-10_apache_vh_volume-week.png

Parrot Apache Volume by User Dec 10th 2013

hours, totalhours changed

chris — Tue, 10 Dec 2013 10:27:24 GMT

hours changed from 0.0 to 0.5
totalhours changed from 1.05 to 1.55

W3 Total Cache plugin was deleted using the web interface and then also the following files and directories were deleted:

rm -rf /home/tc/sites/default/wp-content/cache/*
rm /home/tc/sites/default/wp-content/w3-total-cache-config.php
rm -rf /home/tc/sites/default/wp-content/w3tc-config/

The wp command line tool didn't work, this HTML page is served when you try to use it:

wp
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!--< html xmlns="http://www.w3.org/1999/xhtml">-->
<head>
<title>HTTP Error 403</title>
</head>
<body>
<h1>Error 403</h1>
<p>We're sorry, but we could not fulfill your request for
/usr/local/wp-cli/bin/../php/boot-fs.php on this server.</p>
<p>You do not have permission to access this server.</p>
<p>Your technical support key is: <strong>0096-c0bd-29</strong></p>
<p>You can use this key to <a href="http://www.ioerror.us/bb2-support-key?key=0096-c0bd-29">fix this problem yourself</a>.</p>

This issue came up with other sites, see ticket:539#comment:4 and it was fixed by adding the following to http://transitionculture.org/wp-admin/options-general.php?page=bb2_whitelist

/usr/local/wp-cli/bin/../php/boot-fs.php

Now it's working and I have posted a list of plugins here wiki:TransitionCultureWordPress#Plugins

This is the basic ~/.htaccess file we have:

Redirect /feed/ http://www.transitionnetwork.org/blogs/feed/rob-hopkins/
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

You can see the volume of traffice in this graph:

I think this migration is basically done, some more documentation could be added to wiki:TransitionCultureWordPress but that isn't urgent.

hours, status, totalhours changed; resolution set

chris — Tue, 10 Dec 2013 11:03:51 GMT

hours changed from 0.0 to 0.1
status changed from accepted to closed
resolution set to fixed
totalhours changed from 1.55 to 1.65

Space wise there isn't a problem with wiki:ParrotServer at the moment, but it would be an issue if we need to add some other WordPress sites this size:

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda2      9.9G  6.2G  3.2G  67% /
/dev/xvda3      8.9G  6.8G  1.8G  80% /home

Closing this ticket now.