wiki:PuffinServer

Version 44 (modified by chris, 3 years ago) (diff)

--

Puffin

puffin.webarch.net is a 4GB RAM, 14 CPU core Debian Squeeze virtual server which replaced NewLiveServer and DevelopmentServer for running the Transition Network Drupal sites. It went live in early 2013.

This server is due to be upgraded from LennyToSqueeze on ticket:535 in May 2013.

It was agreed to call this server puffin at the ttech meeting on 22nd November 2012, see ticket:463. The install and initial configuration of this server was tracked on ticket:466, see also the other PuffinServer#migrationtickets. Other services from the old server were migrated to PenguinServer.

Munin Stats

There are munin stats for the server available here

See ticket:555#comment:13 for the notes regarding the installation of the mysql munin stats package.

Console Access

There is a Xen shell available for console access, see wiki:XenShell.

Barracuda Octopus Ageir

The server is using Octopus to manage Ageir and also the updates to the Transition Network Drupal site, this system is installed and upgraded using Barracuda, the Barracuda Octopus Aegir combination is documented on the BOA wiki.

The BOA install script output has been saved on ticket:466#comment:22

Upgrading BOA

The steps are documented in UPGRADE.txt, to upgrade everything run these commands, this process can take around 30 mins:

sudo -i
cd
wget -q -U iCab http://files.aegir.cc/BOA.sh.txt
bash BOA.sh.txt
barracuda up-stable
octopus up-stable all

To get the nginx and php-fpm munin stats working the following code starting with the comment needs adding to /var/aegir/config/server_master/nginx.conf in the nginx default server section:

#######################################################
###  nginx default server
#######################################################

server {
  limit_conn   gulag 32; # like mod_evasive - this allows max 32 simultaneous connections from one IP address
  listen       *:80;
  server_name  _;
  location / {
     root   /var/www/nginx-default;
     index  index.html index.htm;
  }
## chris
  location /nginx_status {
    stub_status on;
    access_log   off;
    allow 127.0.0.1;
    allow 81.95.52.103;
    deny all;
  }
  location ~ ^/(status|ping)$ {
    fastcgi_pass 127.0.0.1:9090;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_intercept_errors on;
    include fastcgi_params;
    access_log off;
    allow 127.0.0.1;
    deny all;
  }
}

And the following lines need uncommenting in /opt/local/etc/php53-fpm.conf:

pm.status_path = /status
ping.path = /ping

After the edits above have been made nginx and php-fpm need restarting:

/etc/init.d/php53-fpm reload
/etc/init.d/nginx restart

These fixes can be tested like this:

cd /etc/munin/plugins
munin-run phpfpm_connections
munin-run phpfpm_status
munin-run nginx_status 
munin-run nginx_request

To disable the clobbering of log files two shell scripts need editing and some lines commenting out (see ticket:555#comment:22):

vim /var/xdrago/graceful.sh /var/xdrago/clear.sh
:1,$s/echo rotate/# echo rotate/gc

Upgrade tickets:

System Updates

Don't use the regular debian tools for updating packages, do this:

barracuda up-stable system

After running the above command to update the system you also need to follow the steps documented above at PuffinServer#UpgradingBOA for php-fpm to get the Munin stats working again.

See also ticket:548#comment:33 for the steps that need to be followed after this to get BOA to work with the Session443 plugin.

CSF / LDF

BOA installs CSF / LDF and automatically blocks IP addresses after too many failed SSH login attempts, if someone is blocked who shouldn't be then they can be unblocked like this:

csf -dr 12.34.56.78

To check if a IP address is blocked:

csf -g 12.34.56.78

See this ticket for problems caused by CSF / LDF blocking the monitoring server: ticket:544

Backupninja

backupninja has been installed and configured to backup to another server in the Sheffield colo, three backup tasks have been configured in /etc/backup.d/, 10.sys which does backups of system settings, like all the packages installed, 20.mysql which dumps all the mysql databases into /var/backups/mysql and uses /etc/mysql/debian.cnf for authentication and finally 90.rdiff which is set to backup all these directories:

include = /var/spool/cron/crontabs
include = /var/backups
include = /var/aegir
include = /etc
include = /root
include = /home
include = /usr/local/
include = /var/lib/dpkg/status*
include = /opt
include = /srv
include = /data
exclude = /home/*/.gnupg
exclude = /home/*/.local/share/Trash
exclude = /home/*/.Trash
exclude = /home/*/.thumbnails
exclude = /home/*/.beagle
exclude = /home/*/.aMule
exclude = /home/*/gtk-gnutella-downloads
exclude = /var/cache/backupninja/duplicity

Postfix

Two changes were made the the default postfix install, it was set to send root emails out, see ticket:466#comment:23 and it was configured to use TLS with the transition network cert, see ticket:466#comment:25.

Nginx

The only changes made to the default nginx configuration was to move the key and cert it was using out of the way and symlink to the *.transitionnetwork.org ones, see ticket:466#comment:25.

Handy commands

There are some Bash aliases to quickly get around the system added by JK...

For root:

alias cdtn='cd /data/disk/tn/' # cd to tn directory
alias totn='su -s /bin/bash tn' # log into the tn user

# show file usages
alias duf='du -sk * | sort -n | perl -ne '\''($s,$f)=split(m{\t});for (qw(K M G)) {if($s<1024) {printf("%.1f",$s);print "$_\t$f"; last};$s=$s/1024}'\'

For tn

alias la='ls -Al --color=auto'
alias lc='ls -ltcr --color=auto'
alias lk='ls -lSr --color=auto'
alias ll='ls -la --group-directories-first --color=auto'
alias lr='ls -lR --color=auto'
alias ls='ls -hF --color=auto'
alias lt='ls -ltr --color=auto'
alias lu='ls -ltur --color=auto'
alias lx='ls -lXB --color=auto'

Vim config

To make vim the default editor for root the following was added to /root/.bashrc:

export EDITOR="vim"

To make config files nicer to read in vim the following was added to /root/.vimrc:

syntax on

And a /root/.vim/filetype.vim files was created with the following in it:

au BufRead,BufNewFile /etc/mysql/my.cnf, set ft=mycnf
autocmd BufRead,BufNewFile /etc/php5/fpm/* set syntax=dosini
autocmd BufRead,BufNewFile /opt/local/etc/php53-fpm.conf set syntax=dosini
au BufRead,BufNewFile /etc/nginx/*,/etc/nginx/conf.d/*,/var/aegir/config/server_master/nginx/*/* set ft=nginx
au BufRead,BufNewFile /data/disk/tn/config/server_master/nginx/vhost.d/* set ft=nginx

And a /root/.vim/syntax/ directory was created and mycnf.vim was created in it by downloading it from http://cvs.pld-linux.org/cgi-bin/cvsweb.cgi/packages/vim-syntax-mycnf/ and nginx.vim was downloaded from http://www.vim.org/scripts/script.php?script_id=1886

Migration Tickets

Tickets created during the migration of the http://www.transitionnetwork.org/ site from NewLiveServer to this server: