[[PageOutline(2-9, Table of Contents)]]
= quince.transitionnetwork.org / quince.webarch.net =

This is the live server for '''[http://www.transitionnetwork.org/ www.transitionnetwork.org]''', '''[http://wiki.transitionnetwork.org/  wiki.transitionnetwork.org]''' and '''[http://static.transitionnetwork.org/  static.transitionnetwork.org]'''.

This is a debian Xen virtual server with 1GB ram, 32GB HDD, single partition, 4 processors and one IP address, 81.95.52.88.

Munin starts for the server are available on [http://nsa.rat.burntout.org/munin/webarch.net/quince.webarch.net.html the webarchitects monitoring server] and on [https://kiwi.transitionnetwork.org/munin/webarch.net/quince.webarch.net.html the transition network development server].

The notes about the old live server are here: LiveServer and the move to quince.webarch.net was done via ticket:147.

For admin related issues contact [mailto:chris@webarchitects.co.uk chris@webarchitects.co.uk].

== TODO ==

1. Optimise and monitor also what php accelerator should we use? Filecache for the moment because of problems encountered with both memcache and apc. Tweak MySQL defaults?

2. Install http://awstats.sf.net/ for generating nice usage graphs from the apache logs and exim logs, see ticket:160

3. After testing on the dev server, install Varnish, see ticket:161

== apache ==

The server is running the default debian apache2:

{{{
/usr/sbin/apache2 -v
  Server version: Apache/2.2.9 (Debian)
  Server built:   Apr 20 2010 15:40:17
/usr/sbin/apache2 -l 
  Compiled in modules:
    core.c
    mod_log_config.c
    mod_logio.c
    prefork.c
    http_core.c
    mod_so.c
}}}

The main configuration file is /etc/apache2/apache2.conf and the virtual hosts are sym linked from /etc/apache2/sites-enabled

The HTTPS !VirtualHosts have the following directives:

{{{
SSLEngine on
SSLCipherSuite HIGH
SSLProtocol all -SSLv2
SSLCertificateFile      /etc/ssl/transitionnetwork.org/transitionnetwork.org.pem
SSLCertificateChainFile /etc/ssl/transitionnetwork.org/gandi.pem
}}}

The transitionnetwork.org.pem file contains both the certificate and the key (these are the files from gandi.net):

{{{
cat transitionnetwork.org.crt > transitionnetwork.org.pem
cat transitionnetwork.org.key >> transitionnetwork.org.pem
}}}

And the gandi.pem contains the cert and the chain of root certificates:

{{{
wget http://crt.gandi.net/GandiStandardSSLCA.crt
wget http://crt.usertrust.com/UTNAddTrustServer_CA.crt
wget http://crt.usertrust.com/AddTrustExternalCARoot.crt
openssl x509 -inform DER -in GandiStandardSSLCA.crt -out GandiStandardSSLCA.pem
openssl x509 -inform DER -in AddTrustExternalCARoot.crt -out AddTrustExternalCARoot.pem
openssl x509 -inform DER -in UTNAddTrustServer_CA.crt -out UTNAddTrustServer_CA.pem
cat transitionnetwork.org.crt > gandi.pem
cat GandiStandardSSLCA.pem >> gandi.pem
cat AddTrustExternalCARoot.pem >> gandi.pem
cat UTNAddTrustServer_CA.pem >> gandi.pem
}}}

== apc ==

The php-apc package is installed and info about how it is preforming is at https://live.quince.webarch.net/info/apc.php it's protected using htauthentication, ask chris@webarchitects.co.uk for the username / password if you need it.

The configuration is in /etc/php5/conf.d/apc.ini and the settings have been taken from here http://www.innovatingtomorrow.net/2008/01/17/improve-php-performance-apc

{{{
extension=apc.so
apc.enabled = 1
apc.shm_size = 128
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX
}}}

The wiki:NewLiveServer#mediawiki site is set to use APC via this setting in /web/wiki.transitionnetwork.org/www/LocalSettings.php

{{{
$wgMainCacheType = CACHE_ACCEL;
}}}

Drupal can be set to use it via /web/transitionnetwork.org/www/sites/default/settings.php but it doesn't appear to improve performance over the filecache and also it generates lots of errors in the Drupal logs like this:

{{{
unlink(/tmp/cache_views_lock) [<a href='function.unlink'>function.unlink</a>]: No such file or directory in /web/transitionnetwork.org/www/sites/all/modules/cacherouter/Cache.php on line 124.
}}}

See this thread for more on this problem: http://drupal.org/node/588820


== mediawiki ==

The Mediawiki site at http://wiki.transitionnetwork.org/ is running on quince.webarch.net (see ticket:147 and ticket:148 for the move), it is also available at  http://wiki.quince.webarch.net/

There is also a wiki:DevelopmentServer#Mediawiki version of this site at http://wiki.dev.transitionnetwork.org/ -- when upgrading Mediawiki please first test the upgrade on the dev server first.

Mediawiki is installed in /web/wiki.transitionnetwork.org/www and the apache !VirtualHost configuration is in /etc/apache2/sites-available/wiki.transitionnetwork.org.conf.

To upgrade the site to the latest version of Mediawiki, from http://www.mediawiki.org/wiki/Download you could follow the instructions from http://www.mediawiki.org/wiki/Upgrade or use the '''mediawiki-upgrade''' script which takes the latest version of Mediawiki as an argument on the command line and then does everything for you:

{{{
kiwi:~# mediawiki-upgrade 1.16.0
}}}

The main configuration file for Mediawiki is /web/wiki.transitionnetwork.org/www/LocalSettings.php and this are the things that have been changed from their default values:

{{{
$wgScript           = "/index.php";
$wgRedirectScript   = "/redirect.php";
$wgArticlePath      = "/$1";

$wgLogo             = "/images/wiki.png";

$wgEmergencyContact = "wiki@transitionnetwork.org";
$wgPasswordSender = "wiki@transitionnetwork.org";

$wgRightsPage = "Copyright"; # Set to the title of a wiki page that describes your license/copyright
$wgRightsUrl = "http://creativecommons.org/licenses/by-sa/2.0/uk/";
$wgRightsText = "Creative Commons Attribution-Share Alike 2.0 UK: England & Wales";
$wgRightsIcon = "/images/cc-by-sa.png";
}}}

== cron ==

The cron job for the http://www.transitionnetwork.org/ site is set up for user chris and it contains:

{{{
# m h  dom mon dow   command
*/30 * * * * /usr/sbin/ab -n 1 http://www.transitionnetwork.org/cron.php >/dev/null 2>&1  
* */1 * * * /usr/sbin/ab -n 1 http://workspaces.transitionnetwork.org/cron.php >/dev/null 2>&1  
}}}

ab is [http://httpd.apache.org/docs/2.2/programs/ab.html apachebench].

== backup2kiwi ==

To backup the Mysql database and the files for the web sites to the wiki:DevelopmentServer run the /usr/local/bin/backup2kiwi script, it puts the files in /home/live/quince on kiwi.webarch.net and these files are used by the scripts on kiwi to update the Drupal and Mediwiki sites with the latest data from the live sites.

A copy of this script is attached to this page: attachment:backup2kiwi

== mysql-backup ==

A MySQL Backup script from http://worldcommunitypress.com/opensource/mysql-backup is installed in /usr/local/bin and it's set to create backups in /var/backups/mysql/

It needed the libmime-lite-perl debian package to be installed.

To run it:

{{{
/usr/local/bin/mysql-backup
}}}

These lines have been changed from the original at http://worldcommunitypress.com/assets/files/opensource/utilities/mysql_backup.txt :

{{{
$admin_email_to              = 'chris@webarchitects.co.uk';
$admin_email_from            = 'root@quince.webarch.net';
$cnf_file                    = '/root/.my.cnf';
$site_name                   = 'quince.webarch.net';
$mysql_backup_dir            = '/var/backups/mysql';
}}}

== backupninja ==

[https://labs.riseup.net/code/projects/show/backupninja/ Backupninja] has been installed and set up -- it's set to backup files to another server in the same rack and then this backups up the data to a server in another colo. The main configuration file is /etc/backupninja.conf and the files containing the list of things to be backed up are in /etc/backup.d/. 60 days worth of backups are saved. It is set to backup MySQL and the following directories:
{{{
include = /var/spool/cron/crontabs
include = /var/backups
include = /etc
include = /root
include = /home
include = /usr/local/*bin
include = /var/lib/dpkg/status*
include = /web
exclude = /home/*/.gnupg
exclude = /home/*/.local/share/Trash
exclude = /home/*/.Trash
exclude = /home/*/.thumbnails
exclude = /home/*/.beagle
exclude = /home/*/.aMule
exclude = /home/*/gtk-gnutella-downloads
}}}

== php ==

See https://live.quince.webarch.net/info/ for the php info, the php.ini file is /etc/php5/apache2/php.ini 

PECL Uploadprogress was installed as suggested here: http://www.joergfelser.at/content/howto-install-pecl-uploadprogress-debian-50-lenny 

{{{
aptitude install php5-dev
pecl install uploadprogress
}}}

And this was added to the php.ini file:

{{{
extension=uploadprogress.so
}}}

The, default php.ini files which had these changes:

{{{
expose_php = Off
memory_limit = 256M
extension=uploadprogress.so
}}}

Was moved to php.ini.dist.tweaked and then /usr/share/doc/php5-common/examples/php.ini-recommended was copied to /etc/php5/apache2/php.ini and a new /etc/php5/apache2/conf.d/uploadprogress.ini file was created with this in it:

{{{
extension=uploadprogress.so
}}}

And /etc/php5/apache2/php.ini was edited and these things were changed:

{{{
expose_php = Off
memory_limit = 256M      ; Maximum amount of memory a script may consume (128MB)
error_log = syslog
post_max_size = 12M
upload_max_filesize = 12M
display_errors = On
default_charset = "utf-8"
== TODO ==

1. Sort out drupal errors:

{{{
warning: array_merge() [function.array-merge]: Argument #2 is not an array in /web/transitionnetwork.org/www/includes/theme.inc on line 930.
}}}

{{{
warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'nodereference_autocomplete_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
}}}

2. Optimise, what php accelerator should we use? Tweak MySQL defaults.

3. Cron

4. Consider switching to http://www.dotdeb.org/ packages for php and mysql -- this should solve the GD problem...

mysql.default_socket = /var/run/mysqld/mysqld.sock
mysql.default_port = 3306
session.save_path = "N;/var/lib/php5"
}}}

== phpmyadmin ==

This is available here: https://live.quince.webarch.net/phpmyadmin/ it's protected using htauthentication because there are a lot of attacks launched against phpmyadmin, ask chris@webarchitects.co.uk for the username / password if you need it.

== memcache ==

The memcache configuration file is /etc/memcached.conf the settings which have been changed from the default are:

{{{
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 128
}}}

The use of memcache by Drupal is configured in /web/transitionnetwork.org/www/sites/default/settings.php:

{{{
$conf['cache_inc'] = './sites/all/modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
        'default' => array(
        'engine'  => 'memcache',
        'server'  => array('127.0.0.1:11211'),
        'shared'  => TRUE,
),
);
}}}

It's not clear if there is any gain from using memcache with one server, see this thread: http://groups.drupal.org/node/73513

== munin ==

In addition to the plugins available by default these were installed:

 * [http://exchange.munin-monitoring.org/plugins/multimemory/details multimemory]
 * [http://exchange.munin-monitoring.org/plugins/apache_activity/details apache_activity]

== Setup Notes ==

Live site, directory and svn checkup:

{{{
mkdir -p /web/transitionnetwork.org/
cd /web/transitionnetwork.org/
svn co https://svn.webarch.net/transition/code/trunk
mv trunk www
cd /web/transitionnetwork.org/www/sites
mkdir cache
chown www-data:www-data cache
mkdir workspaces.transitionnetwork.org
rsync -av kiwi:/home/live/new-live-data/web/transitionnetwork.org/www/sites/workspaces.transitionnetwork.org/ workspaces.transitionnetwork.org/
chown -R www-data:www-data workspaces.transitionnetwork.org/files/
cd /web/transitionnetwork.org/www/sites/default
rsync -av kiwi:/home/live/new-live-data/web/transitionnetwork.org/www/sites/default/files/ files/
chown -R www-data:www-data files
}}}

Databases were imported using a script, /usr/local/bin/mysql-update-from-gaia the two settings.php files were edited to suite.

There was a problem with missing modules:

{{{
Warning: require_once(./sites/all/modules/cacherouter/cacherouter.inc) [function.require-once]: failed to open stream: No such file or directory in /web/transitionnetwork.org/www/includes/bootstrap.inc on line 1009

Fatal error: require_once() [function.require]: Failed opening required './sites/all/modules/cacherouter/cacherouter.inc' (include_path='.:/usr/share/php:/usr/share/pear') in /web/transitionnetwork.org/www/includes/bootstrap.inc on line 1009
}}}

This modules isn't in svn any more: https://svn.webarch.net/transition/code/trunk/sites/all/modules/cacherouter

So it was copied over manually:

{{{
mkdir /web/transitionnetwork.org/www/sites/all/modules/cacherouter/
rsync -av kiwi:/home/live/new-live-data/web/transitionnetwork.org/www/sites/all/modules/cacherouter/ /web/transitionnetwork.org/www/sites/all/modules/cacherouter/
}}}

The following Drupal errors:

{{{
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'ctools_export_ui_task_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'ctools_access_multiperm' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'ctools_export_ui_task_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'ctools_export_ui_task_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'views_import_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
# warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'nodereference_autocomplete_access' was given in /web/transitionnetwork.org/www/includes/menu.inc on line 452.
}}}

Were fixed by clearing the cache, using this page: https://live.quince.webarch.net/admin/settings/performance