transitionnetwork.gaiahost.coop

The server is due to go offline in November 2010, the NewLiveServer took over at the start of November 2010.

The live server, transitionnetwork.gaiahost.coop has 67.217.100.83 as it's primary IP address and also 67.217.100.92 (the old live server is on 67.217.100.68 for now), it is provided by ​http://gaiahost.coop/

The sites running on 67.217.100.83 are ​https://www.transitiontoday.org/ | ​http://www.transitiontoday.org/ | ​http://atrium.transitiontoday.org/

And on 67.217.100.92 ​https://www.transitionnetwork.org/ | ​http://www.transitionnetwork.org/. There is a subdomain of the main site at ​http://workspaces.transitionnetwork.org/ | ​https://workspaces.transitionnetwork.org/. This sub-site shares the code base and some of the database tables for the live site, but additionally has its own database and files folders.

The server is a virtual server running FreeBSD 7.2-RELEASE #0, it has 4Gb of RAM (found via ​this script), 1Gb is available to the virtual server, burstable to 3 GB. The machine has two Xeon 3.2 GHz processors and the storage space available is 5Gb and the bandwidth allowed is 20Gb per month.

There are ​munin stats for the server.

The migration from the ​old live server to the new one is being tracked on ticket #97.

Backups

The gaia script for backing up the Mysql databases is /root/sbin/mysql_backup_daily.sh and it contains:

#!/bin/sh
BACKUP=/var/backups/mysql_backup.`date +%a`.sql.gz
/usr/local/bin/mysqldump -h localhost -u root -pXXX --all-databases | /usr/bin/gzip -c > $BACKUP

# backup to the development server, kiwi.webarch.net see ~/.ssh/config
scp $BACKUP kiwi:new-live-data/var/backups/

# file system backup
/usr/local/bin/rsync -qaz \
                     --exclude=transitionnetwork.org/tmp \
                     --exclude=transitionnetwork.org/log \
                     --exclude=transitionnetwork.org/www/sites/default/files/filecache \
                     --exclude=transitionnetwork.org/www/sites/default/files/css \
                     --exclude=transitionnetwork.org/www/sites/default/files/js \
                     --exclude=transitionnetwork.org/www/sites/workspaces.transitionnetwork.org/files/filecache \
                     --exclude=transitionnetwork.org/www/sites/workspaces.transitionnetwork.org/files/css \
                     --exclude=transitionnetwork.org/www/sites/workspaces.transitionnetwork.org/files/js \
                     /web/ kiwi:new-live-data/web/

# apache config backup
/usr/local/bin/rsync -qaz /usr/local/etc/apache22/ kiwi:new-live-data/usr/local/etc/apache22/

And /root/.ssh/config contains:

Host kiwi
  User live
  Hostname kiwi.webarch.net

The Mysql backups are written to /home/live/new-live-data/var/backups and the /web/ directory containing all the sites is rsynced to home/live/new-live-data/web/ on kiwi.webarch.net

Cron

The following crontab for the transnetwww user ensures that regular tasks are carried out, both on the main TN site and the Workspaces sub-site:

crontab -e -u transnetwww
# m h  dom mon dow   command
*/30 * * * * /usr/local/sbin/ab -v 1 -n 1 http://www.transitionnetwork.org/cron.php >/dev/null 2>&1
* */1 * * * /usr/local/sbin/ab -v 1 -n 1 http://workspaces.transitionnetwork.org/cron.php >/dev/null 2>&1

You can manually access the cron script to run it via ​https://transitionnetwork-live.gaiahost.coop/cron.php using a username / password, the http version is restricted to access from the local host only, see the configuration in /usr/local/etc/apache22/extra/httpd-ssl.conf and /usr/local/etc/apache22/Includes/vhost-transnetwww.conf

phpMyAdmin

phpMyAdmin is available at ​https://www.transitiontoday.org/phpmyadmin/

This is in /usr/local/www/phpMyAdmin211 (rather than /usr/local/www/phpMyAdmin as it was on the old live server) but the config file was at /usr/local/www/phpMyAdmin together with the .htpasswd file so the following sym links were created:

cd /usr/local/www/phpMyAdmin211
ln -s ../phpMyAdmin/config.inc.php
ln -s ../phpMyAdmin/.htpasswd 

One line needed changing in /usr/local/www/phpMyAdmin/config.inc.php:

/* $cfg['Servers'][$i]['host'] = 'db.transitionnetwork.gaiahost.coop'; */
$cfg['Servers'][$i]['host'] = 'localhost';

And apache was restarted.

Packages

The following packages have been installed.

screen

cd /usr/ports/sysutils/screen 
make config
  [ ] CJK           Treat CJK ambiguous characters as full width
  [X] INFO          Build and install info documentation
  [X] MAN           Build and install man pages
  [X] NETHACK       Enable nethack-style messages
  [X] XTERM_256     Enable support for 256 colour xterm
  [ ] HOSTINLOCKED  Print user@host in locked message
  [ ] SHOWENC       Show encoding on the status line
make install clean

wget

cd /usr/ports/ftp/wget 
make config
  [ ] GNUTLS   Support for SSL via GnuTLS
  [X] IPV6     Support for IPv6
  [X] NLS      Native Language Support with gettext
  [X] OPENSSL  Support for SSL via OpenSSL
make install clean

vim

cd /usr/ports/editors/vim
make config
  ===> No options to configure
make install clean

lynx

For apachectl status:

apachectl status
  lynx: not found

cd /usr/ports/www/lynx 
make config
  [X] SSL             SSL support
  [X] DEFAULT_COLORS  Colors support
  [ ] IPV6            IPv6 support
  [X] NLS             Native language support
make install clean

rsync

cd /usr/ports/net/rsync/
make config
  [ ] POPT_PORT  Use popt from devel/popt instead of bundled one
  [X] SSH        Use SSH instead of RSH
  [ ] ATIMES     Preserve access times, adds --atimes
  [X] ICONV      Add iconv support
make install clean

subversion

cd /usr/ports/devel/subversion/
make config
  [ ] MOD_DAV_SVN        mod_dav_svn module for Apache 2.X
  [ ] MOD_DONTDOTHAT     mod_dontdothat for Apache 2.X
  [X] NEON               WebDAV/Delta-V repo access module (neon)
  [ ] SERF               WebDAV/Delta-V repo access module (serf)
  [ ] SASL               SASL2 authorization support
  [X] BDB                db4 repository backend
  [ ] ASVN               Build and install Archive SVN (asvn)
  [ ] MAINTAINER_DEBUG   Build debug version
  [ ] SVNSERVE_WRAPPER   Enable svnserve wrapper
  [ ] SVNAUTHZ_VALIDATE  install svnauthz-validate
  [ ] STATIC             Build static version (no shared libs)
  [ ] GNOME_KEYRING      Build with GNOME Keyring auth support
  [ ] BOOK               Install the Subversion Book  
make install clean

p5-libwww

Installed for perl's LWP::UserAgent to fix this error in /var/log/munin/munin-node.log:

 Can't locate object method "new" via package "LWP::UserAgent" at /usr/local/etc/munin/plugins/apache_volume line 130.

cd /usr/ports/www/p5-libwww
make config
  ===> No options to configure
make install clean

munin-node

Munin was installed in order that server info could be ​generated on the dev server.

cd /usr/ports/sysutils/munin-node
make config
  ===> No options to configure
make install clean
  Would you like me to set up log rotation [y]? y
  
  ********************************************************************
  Unless this file already existed, a sample configuration file
  has been placed in /usr/local/etc/munin/munin-node.conf.
  
  Please edit it according to your needs.
  
  ********************************************************************
  
  The Munin client will *not* be started automatically. To allow it
  to start, put this line in /etc/rc.conf:
  
  munin_node_enable="YES"
  
  Then, it will be started on the next boot. If this line is already
  present, the client will be started now.  Otherwise, edit
  /etc/rc.conf and execute this command:
  
      /usr/local/etc/rc.d/munin-node start
  ********************************************************************

Then /usr/local/etc/munin/munin-node.conf was edited and the following was added:

allow ^81\.95\.52\.78$

And then the following was added to /etc/rc.conf:

munin_node_enable="YES"

And then it was started:

/usr/local/etc/rc.d/munin-node start

By default there was no apache plugins configured, so this was done:

cd /usr/local/etc/munin/plugins
ln -s /usr/local/share/munin/plugins/apache_accesses
ln -s /usr/local/share/munin/plugins/apache_processes
ln -s /usr/local/share/munin/plugins/apache_volume 
ln -s /usr/local/share/munin/plugins/df
ln -s /usr/local/share/munin/plugins/df_inode
ln -s /usr/local/share/munin/plugins/load
ln -s /usr/local/share/munin/plugins/memory
ln -s /usr/local/share/munin/plugins/multimemory
ln -s /usr/local/share/munin/plugins/mysql_bytes
ln -s /usr/local/share/munin/plugins/mysql_queries
ln -s /usr/local/share/munin/plugins/mysql_slowqueries
ln -s /usr/local/share/munin/plugins/mysql_threads
ln -s /usr/local/share/munin/plugins/netstat
ln -s /usr/local/share/munin/plugins/ntp_offset
ln -s /usr/local/share/munin/plugins/open_files
ln -s /usr/local/share/munin/plugins/processes
ln -s /usr/local/share/munin/plugins/sendmail_mailqueue
ln -s /usr/local/share/munin/plugins/sendmail_mailstats
ln -s /usr/local/share/munin/plugins/sendmail_mailtraffic
ln -s /usr/local/share/munin/plugins/swap
ln -s /usr/local/share/munin/plugins/vmstat

This error in /var/log/munin/munin-node.log

2010/07/30-11:46:26 [74194] Error output from sendmail_mailstats:
2010/07/30-11:46:26 [74194]     mailstats: /var/log/sendmail.st: Permission denied

Was fixed by adding the user and group for sendmail in /usr/local/etc/munin/plugin-conf.d/plugins.conf:

[sendmail*]
user root
group wheel
env.mspqueue /var/spool/clientmqueue
env.mailstats /usr/sbin/mailstats

The Mysql user 'munin' was carried over from the old live server, where it had been created using this command as the Mysql root user:

CREATE USER 'munin'@'localhost' IDENTIFIED BY '********';
GRANT USAGE ON * . * TO 'munin'@'localhost' IDENTIFIED BY '********' WITH MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;

And a a /root/.my.cnf file was created:

[client]
user = munin
password = ********
host = localhost

And the Mysql configuration in /usr/local/etc/munin/plugin-conf.d/plugins.conf was changed to:

[mysql*]
user root
group wheel
env.mysqlopts --defaults-extra-file=/root/.my.cnf
env.mysqladmin /usr/local/bin/mysqladmin
env.mysqlshow /usr/local/bin/mysqlshow

The following apache configuration was carried across from the old live server, in the default VirtualHost in /usr/local/etc/apache22/httpd.conf:

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from transitiontowns.gaiahost.coop localhost 127.0.0.1 67.217.100.68 67.217.100.83 
</Location>
 
ExtendedStatus On

And this was tested using wget:

wget http://127.0.0.1:80/server-status?auto -O server-status.txt

And munin-run:

munin-run apache_accesses
  accesses80.value 266
munin-run apache_processes
  busy80.value 2
  idle80.value 8
  free80.value 246
munin-run apache_volume
  volume80.value 342016

The p5-libwww was installed so munin could access the server-status and the following was added to /usr/local/etc/munin/plugin-conf.d/plugins.conf:

[apache_*]
user root
group wheel
env.url   http://127.0.0.1:%d/server-status?auto
env.ports 80 

The following apache config changes were also made, in /usr/local/etc/apache22/extra/httpd-info.conf and /usr/local/etc/apache22/httpd.conf

<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from .gaiahost.coop localhost 127.0.0.1 67.217.100.68 67.217.100.83
</Location>

The old, hacked, multimemory munin plugin (attached to this page) was copied from the old live server to /usr/local/etc/munin/plugins/multimemory.bak and a ​newer version which supports !FreeBSD by default was downloaded:

cd /usr/local/share/munin/plugins
wget http://exchange.munin-monitoring.org/plugins/multimemory/version/1/download -O multimemory
chmod 755 multimemory
cd /usr/local/etc/munin/plugins
ln -s /usr/local/share/munin/plugins/multimemory

And the following was added to /usr/local/etc/munin/plugin-conf.d/plugins.conf:

[multimemory]
user root
group wheel
env.os freebsd
env.names httpd mysqld fcgi sendmail

The ​apache_activity plugin was installed:

cd /usr/local/share/munin/plugins
wget "http://exchange.munin-monitoring.org/plugins/apache_activity/version/1/download" -O apache_activity
chown munin:munin apache_activity 
chmod 755 apache_activity 
cd /usr/local/etc/munin/plugins/
ln -s /usr/local/share/munin/plugins/apache_activity

pecl-uploadprogress

This message in the Drupal admin interface indicated that PECL uploadprogress needed to be installed:

Your server is capable of displaying file upload progress through APC, but it is not enabled. Add apc.rfc1867 = 1 to your php.ini configuration. Alternatively, it is recommended to use PECL uploadprogress, which supports more than one simultaneous upload.

So:

cd /usr/ports/devel/pecl-uploadprogress
make config
  ===> No options to configure
  ****************************************************************************
  
  The following line has been added to your /usr/local/etc/php/extensions.ini
  configuration file to automatically load the installed extension:
  
  extension=uploadprogress.so
  
  ****************************************************************************

mutt

For testing email:

cd /basejail/usr/ports/mail/mutt/
make config
  ===> No options to configure
make install clean
  ===>  Cleaning for sgmlformat-1.7_2
  ===>  Cleaning for mime-support-3.48.1
  ===>  Cleaning for jade-1.2.1_9
  ===>  Cleaning for linuxdoc-1.1_1
  ===>  Cleaning for docbook-4.1_4
  ===>  Cleaning for xmlcatmgr-2.2
  ===>  Cleaning for iso8879-1986_2
  ===>  Cleaning for unzip-6.0
  ===>  Cleaning for mutt-1.4.2.3_4