kiwi.webarch.net / kiwi.transitionnetwork.org

This server is due to be switched off shortly (late January 2013), the Drupal development sites are being migrated to wiki:PuffinServer and all the other sites and services have been migrated to wiki:PenguinServer.

This is the development server for ​The Transition Network, there is a ​static html front page for this server which lists the sites and applications which are running on it. The live server, quince.transitionnetwork.org is documented on the wiki:NewLiveServer page.

The development server has a development, ​http://dev.transitionnetwork.org/, and a testing, ​http://test.transitionnetwork.org/, version of the live server set-up. The test, dev and live code bases were kept in ​this subversion repository but have now been switched to the Trac svn repo, browser:www via ticket:92. To mirror LIVE's set up, there are Drupal multi-sites for the workspaces sub-site at ​http://workspaces.dev.transitionnetwork.org/ and ​http://workspaces.test.transitionnetwork.org/.

The a development (and for now, testing) copy of the SharingEngine at ​http://news.dev.transitionnetwork.org/. It is a separate install of Drupal 6 with its own Trac svn repo at browser:sharing-engine.

The development server is ​kiwi.transitionnetwork.org also available at ​kiwi.webarch.net, it's a encrypted 40G debian virtual machine with 1Gb of dedicated RAM. The encryption key can be shared with others if needs be, there is a ssh account on the physical machine that can be used to reboot and decrypt the machine. The physical server that virtual machine is running on has quad Intel Xeon 2.00GHz 64bit processors, 16Gb RAM, the host OS debian is running on a pair of mirrored 73Gb SAS disks and the virtual machines on a pair of mirrored 2Tb SATA disks. / There are ​remote munin stats for the server available. The stats are also ​available from the server itself and the server hosts ​munin stats for the live server, transitiontowns.gaiahost.coop.

The original documentation for this server was at ​http://atrium.transitiontoday.org/dev/development-server the parts that still seem relevant have been copied here and updated as necessary.

In September 2011 the server was upgraded from LennyToSqueeze and tracked on ticket:301

Network and DNS

The server has 3 IP addresses, but only one is currently needed, 81.95.52.78, 81.95.52.79, 81.95.52.80

SSH access

Which ever developers need ssh access can have it, contact chris@…. When adding a new account don't forget to add the user to the sudo group -- ssh access is limited to only users in the sudo group.

Users can be added and also put in the sudo group with one command:

kiwi:~# adduser -G sudo newusername

/etc/sudoers is set up so that people don't need to type their passwords when sudo'ing.

If you want to create a user and allow them to have ssh access but not sudo then they need to be in the sshaccess group:

kiwi:~# adduser -G sshaccess newusername

If users set up ssh keys then this means that people can use very long, essentially uncrackable, passwords but never actually need to remember or use them unless they lose their local ssh key or they need to login via the consol, these can be generated using:

kiwi:~# pwgen -sn 25

live2dev

This is a script, run as root, which will update either the ​http://dev.transitionnetwork.org.webarch.net/ or the ​http://test.transitionnetwork.org.webarch.net/ sites with the last MySQL dump from the live site and also copy across backups of the uploaded files.

The script can be run interactively or by telling it which site to update on the command line (dev or test), for example to update the dev site:

kiwi:~# live2dev dev
Have you run /usr/local/bin/backup2kiwi on quince.webarch.net? (y):

The script will update using the latest data synced from the live server, this sync is done using the wiki:NewLiveServer#backup2kiwi script on kiwi.

After the script has run you need to Flush all caches to avoid error messages, ideally the script would drop the appropriate tables to make this unnecessary.

This script was sorted out via ticket:124 and the email settings via ticket:136 and there is still the outstanding matter of the https cert / domain name mismatch, see ticket:166.

A copy of the live2dev script is attached attachment:live2dev​

Varnish

See ticket:224.

Install Varnish 2.1 via the ​varnish-cache.org repository:

curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -
aptitude install lsb-release
echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c) varnish-2.1" >> /etc/apt/sources.list.d/varnish.list
aptitude update
aptitude install varnish

Edit these things in the main config file, /etc/default/varnish :

#DAEMON_OPTS="-a :6081 \
#             -T localhost:6082 \
#             -f /etc/varnish/default.vcl \
#             -S /etc/varnish/secret \
#             -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

DAEMON_OPTS="-a :80 \
             -T localhost:81 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,256M"

Make it listen on port 80 and have 256M RAM and also have the management interface on port 81, apache need it's configuration changing so it listens on port 8080.

Varnish 3.0 configuration:

backend default {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}
acl purge {
  "localhost";
  "127.0.0.1";
}

acl local {
  "localhost";         // myself
  "127.0.0.1";         // myself
  "81.95.52.78";       // this machines main ip address
  "81.95.52.79";       // this machines 2nd ip address
  "81.95.52.80";       // this machines 3rd ip address
}


sub vcl_recv {

    # remove all cookies
    unset req.http.Cookie;

    ## Pass cron jobs and server-status
    if (req.url ~ "cron.php") {
      if (client.ip ~ local) {
        return (pass);
      }
      else {
        error 403 "Access Denied";
      }
    }
    if (req.url ~ "/server-status$") {
      if (client.ip ~ local) {
        return (pass);
      }
      else {
        error 403 "Access Denied";
      }
    }
    if (req.url ~ "apc_info.php") {
      if (client.ip ~ local) {
        return (pass);
      }
      else {
        error 403 "Access Denied";
      }
    }

    # Normalize the Accept-Encoding header
    # as per: http://varnish-cache.org/wiki/FAQ/Compression
    if (req.http.Accept-Encoding) {
      if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
        # No point in compressing these
        remove req.http.Accept-Encoding;
      } elsif (req.http.Accept-Encoding ~ "gzip") {
        set req.http.Accept-Encoding = "gzip";
     } elsif (req.http.Accept-Encoding ~ "deflate") {
        set req.http.Accept-Encoding = "deflate";
      } else {
        # unkown algorithm
        remove req.http.Accept-Encoding;
      }
    }

    ## Default request checks
    if (req.request != "GET" &&
    req.request != "HEAD" &&
    req.request != "PUT" &&
    req.request != "POST" &&
    req.request != "TRACE" &&
    req.request != "OPTIONS" &&
    req.request != "DELETE" &&
    req.request != "PURGE") {
    # Non-RFC2616 or CONNECT which is weird.
    return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD" && req.request != "PURGE") {
    # We only deal with GET, PURGE and HEAD by default
    return (pass);
    }

    if (req.http.Authorization) {
          # Not cacheable by default
          return (pass);
    }

    # Check the incoming request type is "PURGE", not "GET" or "POST"
    if (req.request == "PURGE") {
      # Check if the ip coresponds with the acl purge
      if (!client.ip ~ purge) {
      # Return error code 405 (Forbidden) when not
        error 405 "Not allowed.";
      }
      # purge all objects from cache that match the incoming url and host
      #ban("req.url == " + req.url " && req.http.host == " + req.http.host);
      # purge all objects from cache that match the incoming url
      #ban("req.url = " + req.url);
      ban("req.http.host == " + req.http.host + "&& req.url == " + req.url);
      # Return a http error code 200 (Ok)
      error 200 "Purged.";
      }

    # Grace to allow varnish to serve content if backend is lagged
    set req.grace = 5m;

    return (lookup);
}

sub vcl_fetch {
    unset beresp.http.set-cookie;
    return (deliver);
}

sub vcl_deliver {
  if (obj.hits > 0) {
    set resp.http.X-Varnish-Cache = "HIT";
  }
  else {
    set resp.http.X-Varnish-Cache = "MISS";
  }
  return (deliver);
}

The security hole that resulted in resources that apache only made available to localhost being available to all via varnish was closed on ticket:357#comment:11

Piwik

The ​Piwik server is documented on a seperate PiwikServer page.

memcache

Already installed on live server, wiki:NewLiveServer#memcache, following ​this:

pecl install memcache
 1. Enable memcache session handler support? : yes
 
 1-1, 'all', 'abort', or Enter to continue: 1
 Enable memcache session handler support? [yes] : yes
echo "extension=memcache.so" > /etc/php5/conf.d/memcache.ini

The main config file is /etc/memcached.conf and the following was added to /web/dev.transitionnetwork.org.webarch.net/www/sites/default/settings.php:

$conf['cache_inc'] = './sites/all/modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
        'default' => array(
        'engine'  => 'memcache',
        'server'  => array('127.0.0.1:11211'),
        'shared'  => TRUE,
),
);

Mediawiki

There is a development copy of the ​wiki.transitionnetwork.org site at ​http://wiki.dev.transitionnetwork.org/Main_Page which is running on kiwi.transitionnetwork.org it is installed in /web/wiki.dev.transitionnetwork.org/www and the apache VirtualHost is configured in /etc/apache2/sites-available/wiki.dev.transitionnetwork.org

See also the NewLiveServer#mediawiki documentation.

To upgrade the site to the latest version of Mediawiki, from ​http://www.mediawiki.org/wiki/Download you could follow the instructions from ​http://www.mediawiki.org/wiki/Upgrade or use the mediawiki-upgrade script which takes the latest version of Mediawiki as an argument on the command line and then does everything for you:

kiwi:~# mediawiki-upgrade 1.16.0

• Mediawiki WYSIWYG extension was changed on ticket:432#comment:5
• Mediawiki was upgraded to 1.18.1 on ticket:394

wiki-live2dev

To sync the dev version of the wiki with a database dump and the latest uploaded files from wiki:NewLiveServer#Mediawiki run /usr/local/bin/wiki-live2dev on kiwi.transitionnetwork.org after you have run wiki:NewLiveServer#backup2kiwi on quince.transitionnetwork.org:

kiwi:~# wiki-live2dev
Have you run /usr/local/bin/backup2kiwi on quince.webarch.net? (y):

A copy of the wiki-live2dev script is attached attachment:wiki-live2dev​

extensions

You can view the installed list of extensions ​on the wiki, these are also update using the mediawiki-upgrade, using subversion.

The ​FCKeditor extension was installed via ticket:129.

Packages

The following packages and their dependencies have been installed using aptitude:

• rsync
• libapache2-mod-fastcgi
• mysql-server
• php5-imagick
• php5-mysql
• phpmyadmin
• php-pear
• subversion
• php5-suhosin
• php-apc
• php5-mcrypt
• php5-dev
• libmagic-dev
• make
• dnsutils
• whois
• munin
• libwww-perl
• lynx
• imagemagick
• trac
• git-core

Apache

After making any changes to the Apache configuration best do a configtest first to make sure the configuration is OK:

sudo /usr/sbin/apache2ctl configtest

And then to restart the apache server:

sudo /usr/sbin/apache2ctl restart

phpMyAdmin

The phymyadmin debian package was installed and directives from /etc/phpmyadmin/apache.conf and /etc/apache2/sites-available/default-ssl were copied into /etc/apache2/sites-available/phpmyadmin and then the VirtualServer was enabled, a2ensite phpmyadmin and apache restarted.

phpMyAdmin is available at: ​https://kiwi.transitionnetwork.org/phpmyadmin/

MySQL

There is a copy of the root passwd for MySQL in /root/.

The MySQL settings have been tweaked to increase memory usage, see /etc/mysql/my.cnf

MySQL users and databases were set up using the phpMyAdmin interface.

Postfix

Postfix has been installed for outgoing SMPT email, the only changes from the default configuration are related to enabling TLS and these changes are in /etc/postfix/main.cf.

Email to root is forwarded to chris@… via /root/.forward -- if anyone else needs to get a copy of the root emails then please add yourself to this file.

Backupninja

​Backupninja has been installed and set up -- it's set to backup files to another server in the same rack. The main configuration file is /etc/backupninja.conf and the files containing the list of things to be backed up are in /etc/backup.d/. 60 days worth of backups are saved. It is set to backup MySQL and the following directories:

include = /etc
include = /home
include = /var
include = /home
include = /usr
include = /srv
include = /web
 
exclude = /var/run

Mysql Backup

A MySQL Backup script from ​http://worldcommunitypress.com/opensource/mysql-backup is installed in /usr/local/bin and it's set to create backups in /var/backups/mysql/

It needs the libmime-lite-perl debian package to be installed.

To run it:

/usr/local/bin/mysql-backup

These lines have been changed from the original at ​http://worldcommunitypress.com/assets/files/opensource/utilities/mysql_backup.txt :

$admin_email_to              = 'chris@webarchitects.co.uk';
$admin_email_from            = 'root@kiwi.webarch.net';
$cnf_file                    = '/root/.my.cnf';
$site_name                   = 'kiwi.webarch.net';
$mysql_backup_dir            = '/var/backups/mysql';

DenyHosts

To prevent SSH brute force attacks ​DenyHosts has been installed -- if you make too many mistakes with your password you will be locked out! To whitelist IP addresses add them to /etc/hosts.allow -- blacklisted IP's are written to /etc/hosts.deny.

Munin

There are ​remote munin stats for the server available, they are also ​available from the server itself.

The munin client has been configured via /etc/munin/munin-node.conf

Note the ​missing iostat graph is caused by ​a debian munin xen bug.

Varnish stats: ticket:224#comment:27

APC stats: ticket:397#comment:9

Trac

This Trac site was installed via ticket:1 and it resides in /web/tech.transitionnetwork.org/trac and the linked Subversion repo is in /web/tech.transitionnetwork.org/svn and the XSLT to style web access to ​https://tech.transitionnetwork.org/svn/ is in /web/tech.transitionnetwork.org/www.

The install followed notes from ​Configuring Trac on Debian and ​Trac on Debian.

Further information and links to help pages can be found here: wiki:WikiStart#Trac

Trac was upgraded to 0.12 on ticket:364

bbPress

There is an archived copy of the old forum at ​http://2011.archive.transitionnetwork.org/forum/ this was upgraded to the latest version of bbPress when it was installed, the admin email was reset so the SQL upgrade could be run and the apache configuration generated.

The following things were done to lock the forum down as read-only, template edit, bb-templates/kakumei/header.php removed line 32:

<?php if ( !in_array( bb_get_location(), array( 'login-page', 'register-page' ) ) ) login_form(); ?>

In ./bb-templates/kakumei/front-page.php line 86 was edited to remove the Add Topic link:

<div class="bbcrumb"><a href="<?php bb_uri(); ?>"><?php bb_option('name'); ?></a> &raquo; <?php _e('Add New Topic'); ?></div>

In ./bb-includes/functions.bb-template.php a section around line 278 was commented out:

/*
        if ( empty( $h2 ) && false !== $h2 ) {
                if ( bb_is_topic() ) {
                        $h2 = __( 'Reply' );
                } elseif ( bb_is_forum() ) {
                        $h2 = __( 'New Topic in this Forum' );
                } elseif ( bb_is_tag() || bb_is_front() ) {
                        $h2 = __( 'Add New Topic' );
                }
        }
*/

And around line 310 another section was commented:

        } /* elseif ( !bb_is_user_logged_in() ) {
                echo '<p>';
                printf(
                        __('You must <a href="%s">log in</a> to post.'),
                        esc_attr( bb_get_uri( 'bb-login.php', null, BB_URI_CONTEXT_A_HREF + BB_URI_CONTEXT_BB_USER_FORMS ) )
                );
                echo '</p>';
        } */

And in bb-templates/kakumei/forum.php line 9:

        <th><?php _e('Topic'); ?> &#8212; <?php bb_new_topic_link(); ?></th>

And in bb-templates/kakumei/front-page.php line 17:

        <th><?php _e('Topic'); ?> &#8212; <?php bb_new_topic_link(); ?></th>

And in bb-templates/kakumei/tag-single.php line 11:

        <th><?php _e('Topic'); ?> &#8212; <?php bb_new_topic_link(); ?></th>

In bb-templates/kakumei the following files, forum.php front-page.php tags.php tag-single.php topic.php login.php register.php were edited to add:

<h2>These forums are now closed</h2>                                                  
<p>You cannot add any posts but all threads are readable. Please visit the                                                       
<a href="http://www.transitionnetwork.org/forum">Transition Network                                                              
forums</a> for current Transition-hosted discussions. Thanks.</p>

Apache configuration:

      RewriteRule ^bb-admin.*$ [G,L]
      RewriteRule ^bb-config.*$ [G,L]
      RewriteRule ^bb-edit.*$ [G,L]
      RewriteRule ^bb-cron.*$ [G,L]
      RewriteRule ^bb-login.*$ [G,L]
      RewriteRule ^bb-post.*$ [G,L]
      RewriteRule ^bb-reset-password.*$ [G,L]
      RewriteRule ^bb-settings.*$ [G,L]
      RewriteRule ^profile-edit.*$ [G,L]
      RewriteRule ^edit.*$ [G,L]
      RewriteRule ^register.*$ [G,L]

  <Directory /web/2011.archive.transitionnetwork.org/www/forum/bb-admin>
    Order deny,allow
    deny from all
  </Directory>

See ticket:234 for further information.

nginx

nginx was installed via ticket:357