quince.transitionnetwork.org / quince.webarch.net

This is the live server for ​www.transitionnetwork.org, ​wiki.transitionnetwork.org and ​static.transitionnetwork.org, a list of these sites is available on the server at ​quince.transitionnetwork.org.

This is a debian Xen virtual server with 1GB ram, 32GB HDD, single partition, 4 processors and one IP address, 81.95.52.88.

Munin starts for the server are available on ​the webarchitects monitoring server and on ​the transition network development server.

The notes about the old live server are here: LiveServer and the move to quince.transitionnetwork.org was done via ticket:147.

For admin related issues contact chris@….

TODO

1. Optimise and monitor also what php accelerator should we use? Filecache for the moment because of problems encountered with both memcache and apc. Tweak MySQL defaults?

2. Install ​http://awstats.sf.net/ for generating nice usage graphs from the apache logs and exim logs, see ticket:160

3. After testing on the dev server, install Varnish, see ticket:161

apache

The server is running the default debian apache2:

/usr/sbin/apache2 -v
  Server version: Apache/2.2.9 (Debian)
  Server built:   Apr 20 2010 15:40:17
/usr/sbin/apache2 -l 
  Compiled in modules:
    core.c
    mod_log_config.c
    mod_logio.c
    prefork.c
    http_core.c
    mod_so.c

The main configuration file is /etc/apache2/apache2.conf and the virtual hosts are sym linked from /etc/apache2/sites-enabled

After making any changes to the Apache configuration best do a configtest first to make sure the configuration is OK:

sudo /usr/sbin/apache2ctl configtest

And then to restart the apache server:

sudo /usr/sbin/apache2ctl restart

Redirects

Redirects for parked wiki:DomainNames are configured in /etc/apache2/sites-available/redirects.transitionnetwork.org.conf and there is a specific VirtualHost for transitiontowns.org so that specific pages that were on the old wiki site (see the ​2010 archive and ​2011 archive), the current redirects that are in place:

  RedirectMatch permanent ^/Bellingen(.*) http://www.transitionnetwork.org/cms/bellingen
  RedirectMatch permanent ^/Lewes(.*) http://www.transitiontownlewes.org/
  RedirectMatch permanent ^/Totnes(.*) http://www.transitiontowntotnes.org/
  RedirectMatch permanent ^/Brixton(.*) http://www.transitiontownbrixton.org/

HTTPS

The HTTPS VirtualHosts have the following directives:

SSLEngine on
SSLCipherSuite HIGH
SSLProtocol all -SSLv2
SSLCertificateFile      /etc/ssl/transitionnetwork.org/transitionnetwork.org.pem
SSLCertificateChainFile /etc/ssl/transitionnetwork.org/gandi.pem

The transitionnetwork.org.pem file contains both the certificate and the key (these are the files from gandi.net):

cat transitionnetwork.org.crt > transitionnetwork.org.pem
cat transitionnetwork.org.key >> transitionnetwork.org.pem

And the gandi.pem contains the cert and the chain of root certificates:

wget http://crt.gandi.net/GandiStandardSSLCA.crt
wget http://crt.usertrust.com/UTNAddTrustServer_CA.crt
wget http://crt.usertrust.com/AddTrustExternalCARoot.crt
openssl x509 -inform DER -in GandiStandardSSLCA.crt -out GandiStandardSSLCA.pem
openssl x509 -inform DER -in AddTrustExternalCARoot.crt -out AddTrustExternalCARoot.pem
openssl x509 -inform DER -in UTNAddTrustServer_CA.crt -out UTNAddTrustServer_CA.pem
cat transitionnetwork.org.crt > gandi.pem
cat GandiStandardSSLCA.pem >> gandi.pem
cat AddTrustExternalCARoot.pem >> gandi.pem
cat UTNAddTrustServer_CA.pem >> gandi.pem

The above was documented as a result of ticket:165, see also wiki:SecurityInfo.

To generate a new certificate, ​follow the gandi instructions (the only required field is the Common Name):

cd /etc/ssl/transitionnetwork.org/
mkdir 2011; cd 2011
openssl req -nodes -newkey rsa:2048 -keyout transitionnetwork.org.key -out transitionnetwork.org.csr
  Generating a 2048 bit RSA private key
  Country Name (2 letter code) [AU]:
  State or Province Name (full name) [Some-State]:
  Locality Name (eg, city) []:
  Organization Name (eg, company) [Internet Widgits Pty Ltd]:
  Organizational Unit Name (eg, section) []:
  Common Name (eg, YOUR name) []:*.transitionnetwork.org
  Email Address []:
  
  Please enter the following 'extra' attributes
  to be sent with your certificate request
  A challenge password []:
  An optional company name []:

apc

The php-apc package is installed and info about how it is preforming is at ​https://live.quince.webarch.net/info/apc.php it's protected using htauthentication, ask chris@… for the username / password if you need it.

The configuration is in /etc/php5/conf.d/apc.ini and the settings have been taken from here ​http://www.innovatingtomorrow.net/2008/01/17/improve-php-performance-apc

extension=apc.so
apc.enabled = 1
apc.shm_size = 128
apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

The wiki:NewLiveServer#mediawiki site is set to use APC via this setting in /web/wiki.transitionnetwork.org/www/LocalSettings.php

$wgMainCacheType = CACHE_ACCEL;

Drupal can be set to use it via /web/transitionnetwork.org/www/sites/default/settings.php but it doesn't appear to improve performance over the filecache and also it generates lots of errors in the Drupal logs like this:

unlink(/tmp/cache_views_lock) [<a href='function.unlink'>function.unlink</a>]: No such file or directory in /web/transitionnetwork.org/www/sites/all/modules/cacherouter/Cache.php on line 124.

See this thread for more on this problem: ​http://drupal.org/node/588820

mediawiki

The Mediawiki site at ​wiki.transitionnetwork.org is running on quince.webarch.net (see ticket:147 and ticket:148 for the move).

There is also a wiki:DevelopmentServer#Mediawiki version of this site at ​http://wiki.dev.transitionnetwork.org/ -- when upgrading Mediawiki please test the upgrade on the dev server first.

Mediawiki is installed in /web/wiki.transitionnetwork.org/www and the apache VirtualHost configuration is in /etc/apache2/sites-available/wiki.transitionnetwork.org.conf.

To upgrade the site to the latest version of Mediawiki, from ​http://www.mediawiki.org/wiki/Download you could follow the instructions from ​http://www.mediawiki.org/wiki/Upgrade or use the mediawiki-upgrade script which takes the latest version of Mediawiki as an argument on the command line and then does everything for you, including upgrading the ​installed extensions using subversion:

kiwi:~# mediawiki-upgrade 1.16.0

The main configuration file for Mediawiki is /web/wiki.transitionnetwork.org/www/LocalSettings.php and this are the things that have been changed from their default values:

$wgScript           = "/index.php";
$wgRedirectScript   = "/redirect.php";
$wgArticlePath      = "/$1";

$wgLogo             = "/images/wiki.png";

$wgEmergencyContact = "wiki@transitionnetwork.org";
$wgPasswordSender = "wiki@transitionnetwork.org";

$wgRightsPage = "Copyright"; # Set to the title of a wiki page that describes your license/copyright
$wgRightsUrl = "http://creativecommons.org/licenses/by-sa/2.0/uk/";
$wgRightsText = "Creative Commons Attribution-Share Alike 2.0 UK: England & Wales";
$wgRightsIcon = "/images/cc-by-sa.png";

# file types for uploads
$wgUploadSizeWarning = 6000 * 3000;
$wgMimeDetectorCommand = "file -bi";
$wgFileExtensions = array( 'avi', 'mp3', 'rm', 'mpg', 'mpeg', 'mp4', 'svg', 'png', 'gif', 'jpg', 'jpeg', 'pdf', 'rtf', 'doc', 'txt', 'ppt', 'odp', 'odc', 'odf', 'odg', 'odi', 'odif', 'odm', 'ods', 'odt', 'otc', 'otf', 'otg', 'oth', 'oti', 'otp', 'ots', 'ott', 'psd', 'ai', 'eps', 'tif');

# No anonymous editing allowed -
$wgGroupPermissions['*']['edit'] = false;

# allow users to be banned
$wgSysopUserBans = true;

# http://www.mediawiki.org/wiki/Extension:SyntaxHighlight_GeSHi
require_once("$IP/extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.php");

# http://www.mediawiki.org/wiki/Extension:SpamBlacklist
require_once( "$IP/extensions/SpamBlacklist/SpamBlacklist.php" );
$wgSpamBlacklistFiles = array(
   "$IP/extensions/SpamBlacklist/blacklist", // Combined blacklist 
);

# http://www.mediawiki.org/wiki/Extension:FCKeditor_%28Official%29
require_once( "$IP/extensions/FCKeditor/FCKeditor.php" );

cron

The cron job for the ​http://www.transitionnetwork.org/ site is set up for user chris and it contains:

# m h  dom mon dow   command
*/30 * * * * /usr/sbin/ab -n 1 http://www.transitionnetwork.org/cron.php >/dev/null 2>&1  
* */1 * * * /usr/sbin/ab -n 1 http://workspaces.transitionnetwork.org/cron.php >/dev/null 2>&1  

ab is ​apachebench.

backup2kiwi

To backup the Mysql database and the files for the web sites to the wiki:DevelopmentServer run the /usr/local/bin/backup2kiwi script, it puts the files in /home/live/quince on kiwi.webarch.net and these files are used by the scripts on kiwi to update the Drupal and Mediwiki sites with the latest data from the live sites.

A copy of this script is attached to this page: attachment:backup2kiwi​

mysql

/usr/bin/mysql_secure_installation has been run to secure the server:

In order to log into MySQL to secure it, we'll need the current
password for the root user.  If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): 
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

You already have a root password set, so you can safely answer 'n'.

Change the root password? [Y/n] n
 ... skipping.

By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y
 ... Success!

By default, MySQL comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] y
 - Dropping test database...
ERROR 1008 (HY000) at line 1: Can't drop database 'test'; database doesn't exist
 ... Failed!  Not critical, keep moving...
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!

mysql-backup

A MySQL Backup script from ​http://worldcommunitypress.com/opensource/mysql-backup is installed in /usr/local/bin and it's set to create backups in /var/backups/mysql/

It needed the libmime-lite-perl debian package to be installed.

To run it:

/usr/local/bin/mysql-backup

These lines have been changed from the original at ​http://worldcommunitypress.com/assets/files/opensource/utilities/mysql_backup.txt :

$admin_email_to              = 'chris@webarchitects.co.uk';
$admin_email_from            = 'root@quince.webarch.net';
$cnf_file                    = '/root/.my.cnf';
$site_name                   = 'quince.webarch.net';
$mysql_backup_dir            = '/var/backups/mysql';

backupninja

​Backupninja has been installed and set up -- it's set to backup files to another server in the same rack and then this backups up the data to a server in another colo. The main configuration file is /etc/backupninja.conf and the files containing the list of things to be backed up are in /etc/backup.d/. 60 days worth of backups are saved. It is set to backup MySQL and the following directories:

include = /var/spool/cron/crontabs
include = /var/backups
include = /etc
include = /root
include = /home
include = /usr/local/*bin
include = /var/lib/dpkg/status*
include = /web
exclude = /home/*/.gnupg
exclude = /home/*/.local/share/Trash
exclude = /home/*/.Trash
exclude = /home/*/.thumbnails
exclude = /home/*/.beagle
exclude = /home/*/.aMule
exclude = /home/*/gtk-gnutella-downloads

php

See ​https://quince.transitionnetwork.org/info/ for the php info, the php.ini file is /etc/php5/apache2/php.ini

PECL Uploadprogress was installed as suggested here: ​http://www.joergfelser.at/content/howto-install-pecl-uploadprogress-debian-50-lenny

aptitude install php5-dev
pecl install uploadprogress

And this was added to the php.ini file:

extension=uploadprogress.so

The, default php.ini files which had these changes:

expose_php = Off
memory_limit = 256M
extension=uploadprogress.so

Was moved to php.ini.dist.tweaked and then /usr/share/doc/php5-common/examples/php.ini-recommended was copied to /etc/php5/apache2/php.ini and a new /etc/php5/apache2/conf.d/uploadprogress.ini file was created with this in it:

extension=uploadprogress.so

And /etc/php5/apache2/php.ini was edited and these things were changed:

expose_php = Off
max_execution_time = 60     ; Maximum execution time of each script, in seconds
max_input_time = 120 ; Maximum amount of time each script may spend parsing request data
memory_limit = 256M      ; Maximum amount of memory a script may consume (128MB)
error_log = syslog
post_max_size = 12M
upload_max_filesize = 12M
display_errors = On
default_charset = "utf-8"

suhosin

Due to errors like this being sent out by logwatch:

Nov 29 20:16:54 quince suhosin[26422]: ALERT - configured POST variable limit exceeded - dropped variable '4[edit field_event_type]' (attacker 'XXX.XXX.XXX.XXX', file '/web/transitionnetwork.org/www/index.php')

Dec  3 15:03:17 quince suhosin[14383]: ALERT - configured request variable name length limit exceeded - dropped variable 'enabled_pattern*field_patterns_related_larger*pattern*field_patterns_related_smaller' (attacker 'XXX.XXX.XXX.XXX', file '/web/transitionnetwork.org/www/index.php')

Dec  7 10:08:56 quince suhosin[7269]: ALERT - configured POST variable name length limit exceeded - dropped variable '{"$":{"memLimit":2000,."autoFlush":true,."crossDomain":true,."includeProtos":false,."includeFunctions":false}}' (attacker 'XXX.XXX.XXX.XXX', file '/web/transitionnetwork.org/www/index.php')

These variables were changed in /etc/php5/conf.d/suhosin.ini as per ​this suggestion:

;suhosin.post.max_vars = 200
suhosin.post.max_vars = 10000

;suhosin.request.max_vars = 200
suhosin.request.max_vars = 10000

;suhosin.request.max_varname_length = 64
suhosin.request.max_varname_length = 256

;suhosin.post.max_name_length = 64
suhosin.post.max_name_length = 512

;suhosin.post.max_totalname_length = 256
suhosin.post.max_totalname_length = 2048

;suhosin.post.max_value_length = 65000
suhosin.post.max_value_length = 260000

phpmyadmin

This is available here: ​https://quince.transitionnetwork.org/phpmyadmin/ it's protected using htauthentication because there are a lot of attacks launched against phpmyadmin, ask chris@… for the username / password if you need it.

memcache

The memcache configuration file is /etc/memcached.conf the settings which have been changed from the default are:

# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 128

The use of memcache by Drupal is configured in /web/transitionnetwork.org/www/sites/default/settings.php:

$conf['cache_inc'] = './sites/all/modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
        'default' => array(
        'engine'  => 'memcache',
        'server'  => array('127.0.0.1:11211'),
        'shared'  => TRUE,
),
);

It's not clear if there is any gain from using memcache with one server, see this thread: ​http://groups.drupal.org/node/73513

munin

In addition to the plugins available by default these were installed:

• ​multimemory
• ​apache_activity

ftp

The server has ​vsftpd running for updating the ​static.transitionnetwork.org site, email mailto:chris@… if you need the username and password for the account to upload content.

vsftpd is configured via the /etc/vsftpd.conf file.