Transition Technology: Ticket #924: Sheffield Server Shutdown Timetable?

chris — Tue, 20 Sep 2016 11:52:37 GMT

We (Webarchitects) have just had a email from Ainslie asking for PuffinServer to be shutdown, so I'm doing that now and disabling things like the Munin stats and deleting the backups etc.

sam — Tue, 20 Sep 2016 12:00:27 GMT

Hi Chris
Could you send us a copy of the backups before deleting everything?
Sorry not seen Ainslie's mail yet..
Thanks
Sam
On 20 September 2016 at 12:52, Transition Technology Trac
<trac@tech.transitionnetwork.org> wrote:
> #924: Sheffield Server Shutdown Timetable?
> -----------------------------------+-----------------------------------
>            Reporter:  chris        |                      Owner:  chris
>                Type:  maintenance  |                     Status:  new
>            Priority:  minor        |                  Milestone:
>           Component:  Live server  |                 Resolution:
>            Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0            |                  Billable?:  1
>         Total Hours:  0            |
> -----------------------------------+-----------------------------------
>
> Comment (by chris):
>
>  We (Webarchitects) have just had a email from Ainslie asking for
>  PuffinServer to be shutdown, so I'm doing that now and disabling things
>  like the
>  [https://penguin.transitionnetwork.org/munin/transitionnetwork.org/puffin.transitionnetwork.org/index.html
>  Munin stats] and deleting the backups etc.
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/924#comment:1>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.

hours, totalhours changed

chris — Tue, 20 Sep 2016 12:12:54 GMT

hours changed from 0.0 to 0.4
totalhours changed from 0.0 to 0.4

I shutdown and deleted PuffinServer before I got the following, but I hadn't deleted the backups...

Replying to sam:

Could you send us a copy of the backups before deleting everything?

You can access them via SFTP, see the documentation here PuffinServer#Backups, this public key of yours is on this server:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdFOmECqe3NI0unEBGI/Bl6rIuAywK54/YzreZBwTbjDwr1oglGZix4wI3CCiCN3SoFzns4Q53pFGr0DSsqQL6GrErGzqx7bcByJSHqNR6C2Wc+JnLQKATr5JX1+a56gJFkN//vkQ28tagrOpBFT+C8M5zgaIpQ76lt2+0MRyVs6KHof7MP3Z4IGW8rLUoIksmpoOddPUUqsCftsln7b67j2R5vZuz16Z9ORc9p+Tt1DspxosVhIJFubTHWwXpDZ33QcmM2wcNqls8hPneD+a2HTPsw5BAKBhurCw/KRciD6aobtx7PAYSZNynxMnfCJ6pPIn5haX4ftOiw5kWlSa/ sam@bristolwireless.net

Is that your current one or would you like another one adding?

I have agreed with Totnes that their two sites on ParrotServer will be moved tomorrow morning:

Once that is done I think the main thing you need to find a home for is the Wagn site on PenguinServer:

https://patterns.transitionresearchnetwork.org/

The plan is to delete everything else like the archives for http://www.intransitionmovie.com/ and this Trac site etc?

chris — Wed, 21 Sep 2016 11:21:15 GMT

The Totnes sites have now been migrated:

I'd suggest that for static archives like http://www.intransitionmovie.com/ GitHub pages could be used to host them for free:

https://pages.github.com/

If you would like any help with sorting that out just let me know.

chris — Tue, 04 Oct 2016 10:34:39 GMT

Any news on this matter?

What is the plan for the two main sites running on PenguinServer and ParrotServer?

sam — Tue, 04 Oct 2016 10:44:13 GMT

Hi Chris
Hi Chris
We're in the process of migrating them to a Hetzner server. It should be
complete and tested by this time next week. I'll confirm with you when
we're ready to switch them off at your end.
With the trac and piwik sites could you make a copy of the databases of
both these sites and send them to the Totnes office on some appropriate
media? A USB stick or CF card maybe?
We'll obviously pay for your time and the media.
Or upload them to our Google Drive if I give you access to a folder?
Thanks
Sam
On 4 October 2016 at 11:34, Transition Technology Trac <
trac@tech.transitionnetwork.org> wrote:
> #924: Sheffield Server Shutdown Timetable?
> -----------------------------------+-----------------------------------
>            Reporter:  chris        |                      Owner:  chris
>                Type:  maintenance  |                     Status:  new
>            Priority:  minor        |                  Milestone:
>           Component:  Live server  |                 Resolution:
>            Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0            |                  Billable?:  1
>         Total Hours:  0.4          |
> -----------------------------------+-----------------------------------
>
> Comment (by chris):
>
>  Any news on this matter?
>
>  What is the plan for the two main sites running on PenguinServer and
>  ParrotServer?
>
>  * https://patterns.transitionresearchnetwork.org/
>  * http://reconomy.org/
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/924#comment:5>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.
>

hours, totalhours changed

chris — Tue, 04 Oct 2016 11:19:01 GMT

hours changed from 0.0 to 0.6
totalhours changed from 0.4 to 1.0

Replying to sam:

We're in the process of migrating them to a Hetzner server. It should be complete and tested by this time next week. I'll confirm with you when we're ready to switch them off at your end.

Good to hear it is all in hand, let me know if you need any help.

With the trac and piwik sites could you make a copy of the databases of both these sites and send them to the Totnes office on some appropriate media? A USB stick or CF card maybe?

Files to download from a password protected directory on this server would be OK I assume?

Following https://trac.edgewall.org/wiki/TracBackup

sudo -i
su - tracd -s /bin/bash
web/tech.transitionnetwork.org/trac hotcopy /web/tech.transitionnetwork.org/backup
  Hotcopying /web/tech.transitionnetwork.org/trac to /web/tech.transitionnetwork.org/backup ...
  Hotcopy done.
tar -zcvf trac.backup.tgz backup/

There is already a password protected directory on this server which is suitable (same usernames and passwords as for trac):

https://penguin.transitionnetwork.org/info/

So moving the file there:

mv /web/tech.transitionnetwork.org/trac.backup.tgz /web/penguin.transitionnetwork.org/www/info/

So you can now download the Trac backup from this URL:

https://penguin.transitionnetwork.org/info/trac.backup.tgz

Copying last nights Piwik database backup to the same directory and gz'ing it:

cp /var/backups/mysql/sqldump/stats.sql /web/penguin.transitionnetwork.org/www/info/
cd /web/penguin.transitionnetwork.org/www/info/
gzip stats.sql

The uncompressed stats.sql dump is 4.0G, compressed it is 1.5G and it can be downloaded from here:

https://penguin.transitionnetwork.org/info/stats.gz

To ensure that these files were password protected the following was added to /etc/nginx/sites-available/penguin:

        location = /info/trac.backup.tgz {
                alias   /web/penguin.transitionnetwork.org/www/info/trac.backup.tgz;
                fastcgi_param HTTP_PROXY "";
                autoindex  on;
                satisfy any;
                deny all;
                auth_basic "PHP Info";
                #auth_basic_user_file /etc/phpmyadmin/htpasswd;
                auth_basic_user_file /web/tech.transitionnetwork.org/.htpasswd;
        }
        location = /info/stats.gz {
                alias   /web/penguin.transitionnetwork.org/www/info/stats.tgz;
                fastcgi_param HTTP_PROXY "";
                autoindex  on;
                satisfy any;
                deny all;
                auth_basic "PHP Info";
                #auth_basic_user_file /etc/phpmyadmin/htpasswd;
                auth_basic_user_file /web/tech.transitionnetwork.org/.htpasswd;
        }

sam — Tue, 04 Oct 2016 11:35:06 GMT

Hi Chris
Thanks for this. When I visit https://penguin.transitionnetwork.org/info/
trac.backup.tgz I see a page of symbols in my browser (see attached)
And https://penguin.transitionnetwork.org/info/stats.gz is giving a 404
(see attached)
Do I need to wait for them to copy over? Or has something gone awry?
Thanks
Sam
Sam
On 4 October 2016 at 12:22, Chris Croome <chris@webarchitects.co.uk> wrote:
> Hi
>
> On Tue 04-Oct-2016 at 11:40:09AM +0100, Sam Rossiter wrote:
> >
> > With the trac and piwik sites could you make a copy of the databases of
> > both these sites and send them to the Totnes office on some appropriate
> > media? A USB stick or CF card maybe?
>
> This has been done, you can download them here:
>
> - https://penguin.transitionnetwork.org/info/trac.backup.tgz
> - https://penguin.transitionnetwork.org/info/stats.gz
>
> Using the same username / passwords as for Trac, details here:
>
> - https://trac.transitionnetwork.org/trac/ticket/924#comment:7
>
> All the best
>
> Chris
>
> --
> Webarchitects Co-operative
> http://webarchitects.coop/
> +44 114 276 9709
> @webarchcoop
>

file.png

attachment set

sam — Tue, 04 Oct 2016 11:35:06 GMT

attachment set to nofiles.png

Added by email2trac

attachment set

sam — Tue, 04 Oct 2016 11:35:06 GMT

attachment set to file.png

Added by email2trac

hours, totalhours changed

chris — Tue, 04 Oct 2016 11:43:30 GMT

hours changed from 0.0 to 0.15
totalhours changed from 1.0 to 1.15

Replying to sam:

Thanks for this. When I visit https://penguin.transitionnetwork.org/info/ trac.backup.tgz I see a page of symbols in my browser (see attached)

This means that the tgz is being served as text/plain so checking /etc/nginx/mine.types there is no entry for tgz files so the following lines were added and Nginx restarted:

    application/x-gtar-compressed                   tgz;
    application/gzip                                gz;

And https://penguin.transitionnetwork.org/info/stats.gz is giving a 404 (see attached)

Sorry about that, the following typo in the Ngnix config:

        location = /info/stats.gz {
                alias   /web/penguin.transitionnetwork.org/www/info/stats.tgz;

Was the cause, this has now been corrected.

sam — Tue, 04 Oct 2016 12:00:50 GMT

Hi Chris
Thanks, I've saved the Trac one.
This is still giving a 404 though: https://penguin.
transitionnetwork.org/info/stats.gz
Thanks
Sam
On 4 October 2016 at 12:43, Transition Technology Trac <
trac@tech.transitionnetwork.org> wrote:
> #924: Sheffield Server Shutdown Timetable?
> -----------------------------------+-----------------------------------
>            Reporter:  chris        |                      Owner:  chris
>                Type:  maintenance  |                     Status:  new
>            Priority:  minor        |                  Milestone:
>           Component:  Live server  |                 Resolution:
>            Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0.15         |                  Billable?:  1
>         Total Hours:  1.0          |
> -----------------------------------+-----------------------------------
> Changes (by chris):
>
>  * hours:  0.0 => 0.15
>  * totalhours:  1.0 => 1.15
>
>
> Comment:
>
>  Replying to [comment:8 sam]:
>  >
>  > Thanks for this. When I visit
>  https://penguin.transitionnetwork.org/info/
>  > trac.backup.tgz I see a page of symbols in my browser (see attached)
>
>  This means that the `tgz` is being served as `text/plain` so checking
>  `/etc/nginx/mine.types` there is no entry for `tgz` files so the following
>  lines were added and Nginx restarted:
>
>  {{{
>      application/x-gtar-compressed                   tgz;
>      application/gzip                                gz;
>  }}}
>
>  > And https://penguin.transitionnetwork.org/info/stats.gz is giving a 404
>  > (see attached)
>
>  Sorry about that, the following typo in the Ngnix config:
>
>  {{{
>          location = /info/stats.gz {
>                  alias
>  /web/penguin.transitionnetwork.org/www/info/stats.tgz;
>  }}}
>
>  Was the cause, this has now been corrected.
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/924#comment:9>
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.
>

hours, totalhours changed

chris — Tue, 04 Oct 2016 12:16:11 GMT

hours changed from 0.0 to 0.25
totalhours changed from 1.15 to 1.4

Replying to sam:

This is still giving a 404 though: https://penguin.transitionnetwork.org/info/stats.gz

Really sorry about that, the file is actually stats.sql.gz, so the URL is:

https://penguin.transitionnetwork.org/info/stats.sql.gz

However this generates a 403 and I think this is due to the size of the file, and this is the case, in /etc/nginx/nginx.conf there was:

        client_max_body_size 200M;

So that has been changed to:

        client_max_body_size 2G;

And Nginx has been restarted.

Also the file needed to be chmod'ed:

chmod 644 /web/penguin.transitionnetwork.org/www/info/stats.sql.gz

Sam, can you try again?

https://penguin.transitionnetwork.org/info/stats.sql.gz

sam — Tue, 04 Oct 2016 13:35:04 GMT

Hi Chris
This is now working, I've grabbed the backups.
Thanks.
Sam
On 4 October 2016 at 13:16, Transition Technology Trac <
trac@tech.transitionnetwork.org> wrote:
> #924: Sheffield Server Shutdown Timetable?
> -----------------------------------+-----------------------------------
>            Reporter:  chris        |                      Owner:  chris
>                Type:  maintenance  |                     Status:  new
>            Priority:  minor        |                  Milestone:
>           Component:  Live server  |                 Resolution:
>            Keywords:               |  Estimated Number of Hours:  0.0
> Add Hours to Ticket:  0.25         |                  Billable?:  1
>         Total Hours:  1.15         |
> -----------------------------------+-----------------------------------
> Changes (by chris):
>
>  * hours:  0.0 => 0.25
>  * totalhours:  1.15 => 1.4
>
>
> Comment:
>
>  Replying to [comment:12 sam]:
>  >
>  > This is still giving a 404 though:
>  https://penguin.transitionnetwork.org/info/stats.gz
>
>  Really sorry about that, the file is actually `stats.sql.gz`, so the URL
>  is:
>
>  * https://penguin.transitionnetwork.org/info/stats.sql.gz
>
>  However this generates a 403 and I think this is due to the size of the
>  file, and this is the case, in `/etc/nginx/nginx.conf` there was:
>
>  {{{
>          client_max_body_size 200M;
>  }}}
>
>  So that has been changed to:
>
>  {{{
>          client_max_body_size 2G;
>  }}}
>
>  And Nginx has been restarted.
>
>  Also the file needed to be chmod'ed:
>
>  {{{
>  chmod 644 /web/penguin.transitionnetwork.org/www/info/stats.sql.gz
>  }}}
>
>  Sam, can you try again?
>
>  * https://penguin.transitionnetwork.org/info/stats.sql.gz
>
> --
> Ticket URL: <https://tech.transitionnetwork.org/trac/ticket/924#comment:13
> >
> Transition Technology <https://tech.transitionnetwork.org/trac>
> Support and issues tracking for the Transition Network Web Project.
>

chris — Fri, 14 Oct 2016 09:41:38 GMT

Replying to chris:

What is the plan for the two main sites running on PenguinServer and ParrotServer?

https://patterns.transitionresearchnetwork.org/
http://reconomy.org/

Replying to sam:

We're in the process of migrating them to a Hetzner server. It should be complete and tested by this time next week. I'll confirm with you when we're ready to switch them off at your end.

Any update on this?

chris — Thu, 20 Oct 2016 09:42:50 GMT

Has any progress been made on this?

Replying to chris:

Replying to chris:

What is the plan for the two main sites running on PenguinServer and ParrotServer?

https://patterns.transitionresearchnetwork.org/
http://reconomy.org/

Replying to sam:

We're in the process of migrating them to a Hetzner server. It should be complete and tested by this time next week. I'll confirm with you when we're ready to switch them off at your end.

Any update on this?

chris — Tue, 25 Oct 2016 13:20:26 GMT

Any news on the progress made on the migration of https://patterns.transitionresearchnetwork.org/ and http://reconomy.org/ to new servers and when PenguinServer and ParrotServer are to be switched off?

Replying to chris:

Has any progress been made on this?

Replying to chris:

Replying to chris:

What is the plan for the two main sites running on PenguinServer and ParrotServer?

https://patterns.transitionresearchnetwork.org/
http://reconomy.org/

Replying to sam:

We're in the process of migrating them to a Hetzner server. It should be complete and tested by this time next week. I'll confirm with you when we're ready to switch them off at your end.

Any update on this?

chris — Fri, 04 Nov 2016 10:30:15 GMT

It's a month today since you said the sites would be migrated off ParrotServer and PenguinServer within a week, ticket:924#comment:6 -- is there any progress to report or a new date by which you think this work will be completed?

chris — Mon, 21 Nov 2016 10:50:50 GMT

I see that http://reconomy.org/ is no longer pointing to ParrotServer and that https://patterns.transitionresearchnetwork.org/ is now static HTML, (with a missing HTTPS certificate) and not running on PenguinServer -- does this mean that the Transition Network would like these two servers and the backups of them deleting?