Ticket #466 (closed task: fixed)
Puffin install and configuration
| Reported by: | chris | Owned by: | chris |
|---|---|---|---|
| Priority: | major | Milestone: | PSE |
| Component: | Live server | Keywords: | |
| Cc: | ed, laura, jim, mark | Estimated Number of Hours: | 21.0 |
| Add Hours to Ticket: | 0 | Billable?: | yes |
| Total Hours: | 25.0 |
Description
This ticket is to track the install and configuration of the new wiki:PuffinServer which is to replace wiki:NewLiveServer and wiki:DevelopmentServer and is due to go live in early 2013.
Change History
comment:1 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 0.0 to 0.1
comment:2 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 2.8
- Total Hours changed from 0.1 to 2.9
The new server is up and running, I have done the following:
Set up the domain names puffin.webarch.net, puffin.transitionnetwork.org newdev.transitionnetwork.org and *.newdev.transitionnetwork.org to point to it's IP address, 81.95.52.103 and also asked for reverse dns to be set to puffin.transitionnetwork.org and edited /etc/hosts and /etc/hostname
Installed the follwing:
aptitude install vim screen sudo wget lynx pwgen rsync
Added the following to /root/.bashrc:
export EDITOR=vim
Created /root/.vimrc with the following in it:
syntax on
Generated a passphrase less root ssh key pair for rsyncing data off the quince and kiwi:
ssh-keygen -t rsa -b 2048
Added this public key to /root/.ssh/authorized_keys on kiwi and quince with this at the start of it to limit it's use to the puffin IP address:
from="81.95.52.103" ssh-rsa AAAA...
Created /root/.ssh/config with the following in it:
Host quince Hostname quince.transitionnetwork.org User root Host kiwi Hostname kiwi.transitionnetwork.org User root
Edited /etc/ssh/sshd_config and restarted ssh:
PermitRootLogin no AllowGroups sudo sshaccess
Edited /etc/sudoers:
#%sudo ALL=(ALL) ALL %sudo ALL=(ALL) NOPASSWD: ALL
Created user accounts for all the existing people who need it, matching the user and group ID's to the ones on quince, and copied across their ssh public keys and sent them email with their passwords, using the following:
export NEWUSER=username ssh quince grep $NEWUSER /etc/passwd # get the ID export NEWUSERID=123 addgroup --gid $NEWUSERID $NEWUSER adduser --gid $NEWUSERID --uid $NEWUSERID $NEWUSER adduser $NEWUSER sudo mkdir /home/$NEWUSER/.ssh chmod 700 /home/$NEWUSER/.ssh rsync -av quince:/home/$NEWUSER/.ssh/authorized_keys /home/$NEWUSER/.ssh/ chmod 600 /home/$NEWUSER/.ssh/authorized_keys chown -R $NEWUSER:$NEWUSER /home/$NEWUSER/.ssh
Installed a MTA and MUA:
aptitude install postfix mutt
Copied across the wild card ssl cert for *.transitionnetwork.org:
mkdir /etc/ssl/transitionnetwork.org/ chmod 700 /etc/ssl/transitionnetwork.org/ rsync -av quince:/etc/ssl/transitionnetwork.org/ /etc/ssl/transitionnetwork.org/
Edited the TLS settings in /etc/postfix/main.cf and restarted postfix:
# TLS parameters
smtpd_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt
smtpd_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key
smtpd_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
smtpd_tls_cipherlist = EDH:!EXP:!LOW
smtpd_tls_ask_ccert = yes
smtpd_tls_req_ccert = no
smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_enforce_tls = no
smtp_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt
smtp_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key
smtp_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem
smtp_tls_loglevel = 1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_session_cache_timeout = 3600s
myhostname = puffin.transitionnetwork.org
Edited /etc/aliases and added a destination for root emails and ran newaliases and sent a test email to root using mutt and then checked that TLS was used by looking at the mail headers.
Installed metche which will keep an eye on /root/Changelog and send system emails out.
Installed mysql and added the root password to /root/.my.cnf:
aptitude install mysql-server-5.1
Installed backupninja and created a user on the remote backup server and set up ssh:
adduser puffin mkdir /home/puffin/.ssh vi /home/puffin/.ssh/authorized_keys chmod 600 /home/puffin/.ssh/authorized_keys chmod 700 /home/puffin/.ssh chown -R puffin:puffin /home/puffin/.ssh
Ran ninjahelper and set up system, mysql and rdiff backups and tested that it was all working.
Installed git.
Installed https://drupal.org/project/barracuda:
cd
wget -q -U iCab http://files.aegir.cc/BOA.sh.txt
bash BOA.sh.txt
boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1
Barracuda [Fri Nov 30 21:25:25 GMT 2012] ==> BOA Skynet welcomes you aboard!
Barracuda [Fri Nov 30 21:25:29 GMT 2012] ==> INFO: NORMAL INIT
Barracuda [Fri Nov 30 21:25:30 GMT 2012] ==> INFO: Easy Public Setup Mode Active
Barracuda [Fri Nov 30 21:25:30 GMT 2012] ==> INFO: Creating your /root/.barracuda.cnf config file
Barracuda [Fri Nov 30 21:25:31 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait...
Barracuda [Fri Nov 30 21:25:46 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait...
Barracuda [Fri Nov 30 21:25:47 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install
Barracuda [Fri Nov 30 21:25:47 GMT 2012] ==> INFO: Downloading little helpers, please wait...
Barracuda [Fri Nov 30 21:25:50 GMT 2012] ==> INFO: Checking BARRACUDA version...
Barracuda [Fri Nov 30 21:25:50 GMT 2012] ==> INFO: Version test result: OK
Barracuda [Fri Nov 30 21:25:50 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait...
Barracuda [Fri Nov 30 21:26:06 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version...
Barracuda [Fri Nov 30 21:26:09 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4
Barracuda [Fri Nov 30 21:26:11 GMT 2012] ==> EXIT on error due to invalid DNS setup
* Your custom _MY_OWNIP is set to "81.95.52.103"
* Your custom _MY_HOSTN is set to "newdev.transitionnetwork.org"
* Your custom _MY_FRONT is set to "master.newdev.transitionnetwork.org"
* Your _MY_HOSTN and/or _MY_FRONT doesn't match your _MY_OWNIP,
or your hostname is not set properly yet.
* Please make sure that below command returns your FQDN hostname "newdev.transitionnetwork.org":
$ uname -n
* Your server has to have already working FQDN hostname matching your IP address.
This means you _have to_ configure the dns for your server IP/hostname _before_
trying to use this install script. Reverse dns is _not_ required.
* Your hostname appears to be newdev.transitionnetwork.org - are you sure it's a valid FQDN hostname?
* Are you sure your FQDN hostname matches your IP address: 81.95.52.103?
Barracuda [Fri Nov 30 21:26:11 GMT 2012] ==> EXIT on error due to invalid DNS setup
Octopus [Fri Nov 30 21:26:11 GMT 2012] ==> BOA Skynet welcomes you aboard!
Octopus [Fri Nov 30 21:26:14 GMT 2012] ==> INFO: Creating your /root/.o1.octopus.cnf config file
Octopus [Fri Nov 30 21:26:15 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait...
Octopus [Fri Nov 30 21:26:17 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install
Octopus [Fri Nov 30 21:26:17 GMT 2012] ==> ERROR: This script should be used only when the same version of BARRACUDA was used before
Octopus [Fri Nov 30 21:26:17 GMT 2012] ==> Your system has to be configured/upgraded by BARRACUDA version BOA-2.0.4 first
Octopus [Fri Nov 30 21:26:17 GMT 2012] ==> Bye
BOA in-stable completed
Bye
So the problems above were because the DNS for newdev.transitionnetwork.org hadn't yet updated, oops, I'll get some tea and carry on later...
Next steps:
- rerun the barracuda installer once the dns has updated
- Get the barracuda installed drupal to serve the transition network site.
- Copy across the live data and database, perhaps splitting it into seperate inodedb files in the process and write a script so that this can be easilly done again.
- Migrate all sites / applications we are keeping running off kiwi and quince.
comment:3 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 2.9 to 3.0
denyhosts was installed and it was set in /etc/denyhosts.conf to not send email, in the short time the server has been up two IP's got added to /etc/hosts.deny
/root/Changelog was created and a basic list of packages installed added to it:
2012-11-30 chris
* metche vim screen sudo wget lynx pwgen rsync : installed
* mysql-server-5.1 : installed
* backupninja : installed
* git : installed
* denyhosts : installed
* for further info see : https://tech.transitionnetwork.org/trac/ticket/466
comment:4 follow-up: ↓ 5 Changed 4 years ago by jim
Hi Chris, the main thing with Barracuda (BOA = Barracuda + Octopus) is that you need the system as 'out of the box' as possible. Don't install anything at this point that isn't needed, there's a chance BOA will remove it, or fail... This is true of things going forward too: Things might get removed during updates (you can put them back, obviously) if they relate to PHP, MySQL, Redis or NginX.
Also, don't just install the site under the base Barracuda package -- add Octopus too as it comes with a load of goodies we'll need and has loads of anti-bot, high performance Drupal tweaks. http://drupal.org/project/octopus
Really good info and links here on the BOA wiki: http://groups.drupal.org/node/163784
Finally you'll need to understand the Aegir Drupal hosting system which manages sites etc: http://community.aegirproject.org/notebook
I'm off to CA now, but will be on emails etc much of the time.
comment:5 in reply to: ↑ 4 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.15
- Total Hours changed from 3.0 to 3.15
Replying to jim:
Hi Chris, the main thing with Barracuda (BOA = Barracuda + Octopus) is that you need the system as 'out of the box' as possible. Don't install anything at this point that isn't needed
I know, I did read the README.txt, INSTALL.txt and SSL.txt and UPGRADE.txt before starting and decided that it should be OK to do the following before installing it:
- Configuring a MTA so that the server could securly send email out.
- Configuring backups, this necessitated the installation of mysql.
- Adding user accounts and configuring SSH.
Thanks for the links, I see the Aegir Open Atrium site is hosting spam, hope the trip goes well!
comment:6 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.43
- Total Hours changed from 3.15 to 3.58
Re-running the installer:
boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1 Please update installers on your system using BOA Meta Installer and try again. $ wget -q -U iCab http://files.aegir.cc/BOA.sh.txt $ bash BOA.sh.txt
So starting again:
wget -q -U iCab http://files.aegir.cc/BOA.sh.txt ; bash BOA.sh.txt BOA Meta Installer setup completed Please check docs/INSTALL.txt and docs/UPGRADE.txt for how-to Bye
boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1 Barracuda [Sat Dec 1 10:26:01 GMT 2012] ==> BOA Skynet welcomes you aboard! Barracuda [Sat Dec 1 10:26:05 GMT 2012] ==> INFO: NORMAL INIT Barracuda [Sat Dec 1 10:26:05 GMT 2012] ==> INFO: Easy Public Setup Mode Active Barracuda [Sat Dec 1 10:26:05 GMT 2012] ==> INFO: Reading your /root/.barracuda.cnf config file Barracuda [Sat Dec 1 10:26:06 GMT 2012] ==> NOTE! Please review all config options displayed below Barracuda [Sat Dec 1 10:26:06 GMT 2012] ==> NOTE! It will *override* all settings in the Barracuda script ### ### Configuration created on 121130-2125 ### with Barracuda version BOA-2.0.4 ### ### NOTE: the group of settings displayed bellow will *not* be overriden ### on upgrade by the Barracuda script nor by this configuration file. ### They can be defined only on initial Barracuda install. ### _HTTP_WILDCARD=YES _MY_OWNIP="81.95.52.103" _MY_HOSTN="newdev.transitionnetwork.org" _MY_FRONT="master.newdev.transitionnetwork.org" _THIS_DB_HOST=localhost _SMTP_RELAY_TEST=YES _SMTP_RELAY_HOST="" _LOCAL_NETWORK_IP="" _LOCAL_NETWORK_HN="" ### ### NOTE: the group of settings displayed bellow ### will *override* all listed settings in the Barracuda script, ### both on initial install and upgrade. ### _MY_EMAIL="chris@webarchitects.co.uk" _XTRAS_LIST="PDS CSF CHV FTP" _AUTOPILOT=YES _DEBUG_MODE=NO _DB_SERVER=MariaDB _SSH_PORT=22 _LOCAL_DEBIAN_MIRROR="ftp.debian.org" _LOCAL_UBUNTU_MIRROR="archive.ubuntu.com" _FORCE_GIT_MIRROR="" _DNS_SETUP_TEST=YES _NGINX_EXTRA_CONF="" _NGINX_WORKERS=AUTO _PHP_FPM_WORKERS=AUTO _BUILD_FROM_SRC=NO _PHP_MODERN_ONLY=YES _PHP_FPM_VERSION=5.3 _PHP_CLI_VERSION=5.3 _LOAD_LIMIT_ONE=1444 _LOAD_LIMIT_TWO=888 _CUSTOM_CONFIG_CSF=NO _CUSTOM_CONFIG_SQL=NO _CUSTOM_CONFIG_REDIS=NO _CUSTOM_CONFIG_PHP_5_2=NO _CUSTOM_CONFIG_PHP_5_3=NO _SPEED_VALID_MAX=3600 _NGINX_DOS_LIMIT=300 _SYSTEM_UPGRADE_ONLY=NO _USE_MEMCACHED=NO _NEWRELIC_KEY= _USE_STOCK=NO ### ### Configuration created on 121130-2125 ### with Barracuda version BOA-2.0.4 ### Barracuda [Sat Dec 1 10:26:08 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait... Barracuda [Sat Dec 1 10:26:14 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Barracuda [Sat Dec 1 10:26:16 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Barracuda [Sat Dec 1 10:26:16 GMT 2012] ==> INFO: Downloading little helpers, please wait... Barracuda [Sat Dec 1 10:26:19 GMT 2012] ==> INFO: Checking BARRACUDA version... Barracuda [Sat Dec 1 10:26:19 GMT 2012] ==> INFO: Version test result: OK Barracuda [Sat Dec 1 10:26:19 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait... Barracuda [Sat Dec 1 10:26:22 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version... Barracuda [Sat Dec 1 10:26:24 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4 Barracuda [Sat Dec 1 10:26:27 GMT 2012] ==> DNS test OK - your FQDN hostname is newdev.transitionnetwork.org and it matches your IP 81.95.52.103 Barracuda [Sat Dec 1 10:26:27 GMT 2012] ==> Please double check whether this is a valid result of my test Barracuda [Sat Dec 1 10:26:29 GMT 2012] ==> INSTALL START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your IP address appears to be 81.95.52.103 - are you sure it's a valid IP? * Your hostname appears to be newdev.transitionnetwork.org - are you sure it's a valid FQDN hostname? Barracuda [Sat Dec 1 10:26:29 GMT 2012] ==> INFO: Cleaning up temp files in /var/opt/ Barracuda [Sat Dec 1 10:26:31 GMT 2012] ==> INFO: Installing DNS cache pdnsd server, please wait... Barracuda [Sat Dec 1 10:26:51 GMT 2012] ==> INFO: DNS cache pdnsd server installation completed Barracuda [Sat Dec 1 10:26:53 GMT 2012] ==> INFO: Updating apt sources Barracuda [Sat Dec 1 10:26:56 GMT 2012] ==> INFO: We will use Debian mirror ftp.debian.org Barracuda [Sat Dec 1 10:27:00 GMT 2012] ==> INFO: Running aptitude update, please wait... Barracuda [Sat Dec 1 10:27:33 GMT 2012] ==> INFO: Installing required libraries and tools Barracuda [Sat Dec 1 10:27:33 GMT 2012] ==> NOTE! This step may take a few minutes, please wait... Barracuda [Sat Dec 1 10:36:04 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 10:36:06 GMT 2012] ==> INFO: Installed Nginx version nginx/1.2.5, upgrade required Barracuda [Sat Dec 1 10:36:08 GMT 2012] ==> INFO: Installing Nginx, please wait... Barracuda [Sat Dec 1 10:37:21 GMT 2012] ==> INFO: Installing MariaDB, please wait... Barracuda [Sat Dec 1 10:38:49 GMT 2012] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Sat Dec 1 10:39:05 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 10:39:07 GMT 2012] ==> INFO: Installed Nginx version nginx/1.3.8, no upgrade required Barracuda [Sat Dec 1 10:39:09 GMT 2012] ==> INFO: Installing /usr/bin/wkhtmltopdf x86_64 version, please wait... Barracuda [Sat Dec 1 10:39:14 GMT 2012] ==> INFO: Checking SMTP connections, please wait... Barracuda [Sat Dec 1 10:39:16 GMT 2012] ==> INFO: Installing VnStat monitor, please wait... Barracuda [Sat Dec 1 10:39:26 GMT 2012] ==> INFO: Installing a few more tools, please wait... Barracuda [Sat Dec 1 10:39:28 GMT 2012] ==> INFO: Installing IonCube x86_64 version for PHP-FPM, please wait... Barracuda [Sat Dec 1 10:39:32 GMT 2012] ==> INFO: Installing PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 10:39:55 GMT 2012] ==> INFO: Installing PhpRedis for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 10:40:26 GMT 2012] ==> INFO: Installing UploadProgress for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 10:40:48 GMT 2012] ==> INFO: Installing JSMin for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 10:41:16 GMT 2012] ==> INFO: Installing bzr, please wait... Barracuda [Sat Dec 1 10:41:48 GMT 2012] ==> INFO: Installing lshell, please wait... Barracuda [Sat Dec 1 10:41:56 GMT 2012] ==> INFO: Installing latest Pure-FTPd server, please wait... Barracuda [Sat Dec 1 10:43:24 GMT 2012] ==> INFO: Installing Redis update for Debian/squeeze, please wait... Barracuda [Sat Dec 1 10:44:22 GMT 2012] ==> INFO: Generating random password for Redis server Barracuda [Sat Dec 1 10:44:22 GMT 2012] ==> INFO: Updating init scripts Barracuda [Sat Dec 1 10:44:28 GMT 2012] ==> INFO: Restarting MariaDB Barracuda [Sat Dec 1 10:44:38 GMT 2012] ==> INFO: Starting Redis, PHP-FPM and Nginx Barracuda [Sat Dec 1 10:44:49 GMT 2012] ==> INFO: Generating random password for MariaDB Barracuda [Sat Dec 1 10:44:52 GMT 2012] ==> INFO: OS and services installation completed Barracuda [Sat Dec 1 10:44:54 GMT 2012] ==> INFO: Installing Aegir Master Instance, please wait... Barracuda [Sat Dec 1 10:45:08 GMT 2012] ==> INFO: Running hosting-dispatch (1/3), please wait... Barracuda [Sat Dec 1 10:45:14 GMT 2012] ==> INFO: Running hosting-dispatch (2/3), please wait... Barracuda [Sat Dec 1 10:45:20 GMT 2012] ==> INFO: Running hosting-dispatch (3/3), please wait... cat: /var/aegir/.drush/hostmaster.alias.drushrc.php: No such file or directory Barracuda [Sat Dec 1 10:45:25 GMT 2012] ==> INFO: Aegir Master Instance installation completed Barracuda [Sat Dec 1 10:45:28 GMT 2012] ==> INFO: Installing default SSL Wildcard Nginx Proxy, please wait... Barracuda [Sat Dec 1 10:45:33 GMT 2012] ==> INFO: Installing Chive MariaDB Manager, please wait... Barracuda [Sat Dec 1 10:45:37 GMT 2012] ==> INFO: Chive MariaDB Manager installation completed Barracuda [Sat Dec 1 10:45:39 GMT 2012] ==> INFO: MariaDB final setup Barracuda [Sat Dec 1 10:45:55 GMT 2012] ==> INFO: MariaDB setup completed Barracuda [Sat Dec 1 10:45:55 GMT 2012] ==> INFO: You can now log in as root by typing only 'mysql' on the command line Barracuda [Sat Dec 1 10:45:58 GMT 2012] ==> INFO: New entry added to /var/log/barracuda_log.txt Barracuda [Sat Dec 1 10:46:00 GMT 2012] ==> ALRT: Something went wrong Barracuda [Sat Dec 1 10:46:00 GMT 2012] ==> ALRT: Please check the install log for details: Barracuda [Sat Dec 1 10:46:00 GMT 2012] ==> ALRT: /var/aegir/install.log Barracuda [Sat Dec 1 10:46:12 GMT 2012] ==> INFO: Installing csf/lfd firewall, please wait... Barracuda [Sat Dec 1 10:46:18 GMT 2012] ==> INFO: csf/lfd firewall installation completed Barracuda [Sat Dec 1 10:46:20 GMT 2012] ==> CARD: Now charging your credit card for this automated install service... Barracuda [Sat Dec 1 10:46:26 GMT 2012] ==> JOKE: Just kidding! Enjoy your Aegir Hosting System :) Barracuda [Sat Dec 1 10:46:30 GMT 2012] ==> Final post-install cleaning, please wait a moment... Timeout, server not responding.
This is the point at which I regret not running the install in screen, do'h!
comment:7 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.56
- Total Hours changed from 3.58 to 4.14
So, let's try running the installer, in screen, again:
sudo -i screen cd boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1 Another BOA installer is running probably - /var/run/boa_run.pid exists
So I'll leave this a while and have a look around at what is installed and running...
- There is a "Under Construction" page up at http://newdev.transitionnetwork.org/ and also at https://newdev.transitionnetwork.org/ with an Ageir generated wildcard SSL certificate.
- A /home/redis/ directory and system accounts for messagebus, pdnsd, redis and aegir have been created with shells set to /bin/false
- nginx is up and running, but there are no sites configured in the /etc/nginx/sites-available/ directory.
- Other services running include, boa, pure-ftpd, php-fpm, mysqld, redis-server, nginx, pdnsd.
The PID file still exists and it hasn't been updated for 25 mins:
-rw-r--r-- 1 root root 0 Dec 1 10:46 /var/run/boa_run.pid
I haven't been sent a email, which I was expecting, from the installer, but I have got a SSH login alert email sent to root by lfd, which is a Perl script which has been installed at /usr/sbin/lfd
Lots of iptables rules have been set up:
root@newdev:/etc/nginx# iptables -L Chain INPUT (policy DROP) target prot opt source destination ACCEPT tcp -- dns1.webarchitects.co.uk anywhere tcp dpt:domain ACCEPT udp -- dns1.webarchitects.co.uk anywhere udp dpt:domain ACCEPT tcp -- dns1.webarchitects.co.uk anywhere tcp spt:domain ACCEPT udp -- dns1.webarchitects.co.uk anywhere udp spt:domain LOCALINPUT all -- anywhere anywhere ACCEPT all -- anywhere anywhere INVALID tcp -- anywhere anywhere ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp-data ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:smtp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:www ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:cvspserver ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:5280 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:git ACCEPT tcp -- anywhere anywhere state NEW tcp dpts:30000:50000 ACCEPT udp -- anywhere anywhere state NEW udp dpt:20 ACCEPT udp -- anywhere anywhere state NEW udp dpt:fsp ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain ACCEPT udp -- anywhere anywhere state NEW udp dpt:ntp ACCEPT udp -- anywhere anywhere state NEW udp dpt:snmp ACCEPT udp -- anywhere anywhere state NEW udp dpts:33434:33523 ACCEPT icmp -- anywhere anywhere icmp echo-request limit: avg 1/sec burst 5 ACCEPT icmp -- anywhere anywhere icmp echo-reply limit: avg 1/sec burst 5 ACCEPT icmp -- anywhere anywhere icmp time-exceeded ACCEPT icmp -- anywhere anywhere icmp destination-unreachable LOGDROPIN all -- anywhere anywhere Chain FORWARD (policy DROP) target prot opt source destination Chain OUTPUT (policy DROP) target prot opt source destination ACCEPT tcp -- anywhere dns1.webarchitects.co.uk tcp dpt:domain ACCEPT udp -- anywhere dns1.webarchitects.co.uk udp dpt:domain ACCEPT tcp -- anywhere dns1.webarchitects.co.uk tcp spt:domain ACCEPT udp -- anywhere dns1.webarchitects.co.uk udp spt:domain LOCALOUTPUT all -- anywhere anywhere ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp spt:domain ACCEPT udp -- anywhere anywhere udp spt:domain ACCEPT all -- anywhere anywhere INVALID tcp -- anywhere anywhere ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp-data ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ftp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:smtp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:www ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:pop3 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:imap2 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssmtp ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:submission ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:rsync ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:imaps ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:pop3s ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:cvspserver ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:mysql ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:5280 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:git ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:hkp ACCEPT tcp -- anywhere anywhere state NEW tcp dpts:30000:50000 ACCEPT udp -- anywhere anywhere state NEW udp dpt:20 ACCEPT udp -- anywhere anywhere state NEW udp dpt:fsp ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain ACCEPT udp -- anywhere anywhere state NEW udp dpt:113 ACCEPT udp -- anywhere anywhere state NEW udp dpt:ntp ACCEPT udp -- anywhere anywhere state NEW udp dpt:snmp ACCEPT udp -- anywhere anywhere state NEW udp dpts:33434:33523 ACCEPT icmp -- anywhere anywhere icmp echo-reply ACCEPT icmp -- anywhere anywhere icmp echo-request ACCEPT icmp -- anywhere anywhere icmp time-exceeded ACCEPT icmp -- anywhere anywhere icmp destination-unreachable LOGDROPOUT all -- anywhere anywhere Chain INVALID (2 references) target prot opt source destination INVDROP all -- anywhere anywhere state INVALID INVDROP tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,PSH,ACK,URG/NONE INVDROP tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,PSH,ACK,URG/FIN,SYN,RST,PSH,ACK,URG INVDROP tcp -- anywhere anywhere tcp flags:FIN,SYN/FIN,SYN INVDROP tcp -- anywhere anywhere tcp flags:SYN,RST/SYN,RST INVDROP tcp -- anywhere anywhere tcp flags:FIN,RST/FIN,RST INVDROP tcp -- anywhere anywhere tcp flags:FIN,ACK/FIN INVDROP tcp -- anywhere anywhere tcp flags:PSH,ACK/PSH INVDROP tcp -- anywhere anywhere tcp flags:ACK,URG/URG INVDROP tcp -- anywhere anywhere tcp flags:!FIN,SYN,RST,ACK/SYN state NEW Chain INVDROP (10 references) target prot opt source destination DROP all -- anywhere anywhere Chain LOCALINPUT (1 references) target prot opt source destination DROP tcp -- 222.37.135.2 anywhere tcp dpt:ssh DROP tcp -- 80-252-241-37.hosting.wildpark.net anywhere tcp dpt:ssh Chain LOCALOUTPUT (1 references) target prot opt source destination Chain LOGDROPIN (1 references) target prot opt source destination DROP tcp -- anywhere anywhere tcp dpt:bootps DROP udp -- anywhere anywhere udp dpt:bootps DROP tcp -- anywhere anywhere tcp dpt:bootpc DROP udp -- anywhere anywhere udp dpt:bootpc DROP tcp -- anywhere anywhere tcp dpt:sunrpc DROP udp -- anywhere anywhere udp dpt:sunrpc DROP tcp -- anywhere anywhere tcp dpt:auth DROP udp -- anywhere anywhere udp dpt:113 DROP tcp -- anywhere anywhere tcp dpts:loc-srv:netbios-ssn DROP udp -- anywhere anywhere udp dpts:loc-srv:netbios-ssn DROP tcp -- anywhere anywhere tcp dpt:microsoft-ds DROP udp -- anywhere anywhere udp dpt:microsoft-ds DROP tcp -- anywhere anywhere tcp dpt:isakmp DROP udp -- anywhere anywhere udp dpt:isakmp DROP tcp -- anywhere anywhere tcp dpt:login DROP udp -- anywhere anywhere udp dpt:who DROP tcp -- anywhere anywhere tcp dpt:520 DROP udp -- anywhere anywhere udp dpt:route LOG tcp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *TCP_IN Blocked* ' LOG udp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *UDP_IN Blocked* ' LOG icmp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *ICMP_IN Blocked* ' DROP all -- anywhere anywhere Chain LOGDROPOUT (1 references) target prot opt source destination LOG tcp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *TCP_OUT Blocked* ' LOG udp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *UDP_OUT Blocked* ' LOG icmp -- anywhere anywhere limit: avg 30/min burst 5 LOG level warning prefix `Firewall: *ICMP_OUT Blocked* ' DROP all -- anywhere anywhere
I'll get another cuppa and if the lock file still exists I think it'll time to delete it and re-run the installer...
comment:8 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.36
- Total Hours changed from 4.14 to 4.5
OK, the lock file is over an hour old, deleting it and re-running the installer:
rm /var/run/boa_run.pid boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1 Barracuda [Sat Dec 1 11:54:34 GMT 2012] ==> BOA Skynet welcomes you aboard! Barracuda [Sat Dec 1 11:54:38 GMT 2012] ==> INFO: FORCED INIT Barracuda [Sat Dec 1 11:54:38 GMT 2012] ==> INFO: Easy Public Setup Mode Active Barracuda [Sat Dec 1 11:54:38 GMT 2012] ==> INFO: Reading your /root/.barracuda.cnf config file Barracuda [Sat Dec 1 11:54:39 GMT 2012] ==> NOTE! Please review all config options displayed below Barracuda [Sat Dec 1 11:54:39 GMT 2012] ==> NOTE! It will *override* all settings in the Barracuda script ### ### Configuration created on 121130-2125 ### with Barracuda version BOA-2.0.4 ### ### NOTE: the group of settings displayed bellow will *not* be overriden ### on upgrade by the Barracuda script nor by this configuration file. ### They can be defined only on initial Barracuda install. ### _HTTP_WILDCARD=YES _MY_OWNIP="81.95.52.103" _MY_HOSTN="newdev.transitionnetwork.org" _MY_FRONT="master.newdev.transitionnetwork.org" _THIS_DB_HOST=localhost _SMTP_RELAY_TEST=YES _SMTP_RELAY_HOST="" _LOCAL_NETWORK_IP="" _LOCAL_NETWORK_HN="" ### ### NOTE: the group of settings displayed bellow ### will *override* all listed settings in the Barracuda script, ### both on initial install and upgrade. ### _MY_EMAIL="chris@webarchitects.co.uk" _XTRAS_LIST="PDS CSF CHV FTP" _AUTOPILOT=YES _DEBUG_MODE=NO _DB_SERVER=MariaDB _SSH_PORT=22 _LOCAL_DEBIAN_MIRROR="ftp.debian.org" _LOCAL_UBUNTU_MIRROR="archive.ubuntu.com" _FORCE_GIT_MIRROR="" _DNS_SETUP_TEST=YES _NGINX_EXTRA_CONF="" _NGINX_WORKERS=AUTO _PHP_FPM_WORKERS=AUTO _BUILD_FROM_SRC=NO _PHP_MODERN_ONLY=YES _PHP_FPM_VERSION=5.3 _PHP_CLI_VERSION=5.3 _LOAD_LIMIT_ONE=1444 _LOAD_LIMIT_TWO=888 _CUSTOM_CONFIG_CSF=NO _CUSTOM_CONFIG_SQL=NO _CUSTOM_CONFIG_REDIS=NO _CUSTOM_CONFIG_PHP_5_2=NO _CUSTOM_CONFIG_PHP_5_3=NO _SPEED_VALID_MAX=3600 _NGINX_DOS_LIMIT=300 _SYSTEM_UPGRADE_ONLY=NO _USE_MEMCACHED=NO _NEWRELIC_KEY= _USE_STOCK=NO ### ### Configuration created on 121130-2125 ### with Barracuda version BOA-2.0.4 ### Barracuda [Sat Dec 1 11:54:41 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait... Barracuda [Sat Dec 1 11:54:48 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Barracuda [Sat Dec 1 11:54:50 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Barracuda [Sat Dec 1 11:54:50 GMT 2012] ==> INFO: Downloading little helpers, please wait... Barracuda [Sat Dec 1 11:54:52 GMT 2012] ==> INFO: Checking BARRACUDA version... Barracuda [Sat Dec 1 11:54:52 GMT 2012] ==> INFO: Version test result: OK Barracuda [Sat Dec 1 11:54:52 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait... Barracuda [Sat Dec 1 11:54:55 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version... Barracuda [Sat Dec 1 11:54:57 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4 Barracuda [Sat Dec 1 11:55:00 GMT 2012] ==> DNS test OK - your FQDN hostname is newdev.transitionnetwork.org and it matches your IP 81.95.52.103 Barracuda [Sat Dec 1 11:55:00 GMT 2012] ==> Please double check whether this is a valid result of my test Barracuda [Sat Dec 1 11:55:02 GMT 2012] ==> INSTALL START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your IP address appears to be 81.95.52.103 - are you sure it's a valid IP? * Your hostname appears to be newdev.transitionnetwork.org - are you sure it's a valid FQDN hostname? Barracuda [Sat Dec 1 11:55:02 GMT 2012] ==> INFO: Cleaning up temp files in /var/opt/ Barracuda [Sat Dec 1 11:55:04 GMT 2012] ==> INFO: Updating apt sources Barracuda [Sat Dec 1 11:55:06 GMT 2012] ==> INFO: We will use Debian mirror ftp.debian.org Barracuda [Sat Dec 1 11:55:10 GMT 2012] ==> INFO: Running aptitude update, please wait... Barracuda [Sat Dec 1 11:55:28 GMT 2012] ==> INFO: Installing required libraries and tools Barracuda [Sat Dec 1 11:55:28 GMT 2012] ==> NOTE! This step may take a few minutes, please wait... Barracuda [Sat Dec 1 11:55:41 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 11:55:43 GMT 2012] ==> INFO: Installed Nginx version nginx/1.3.8, no upgrade required Barracuda [Sat Dec 1 11:55:45 GMT 2012] ==> INFO: Installing Nginx, please wait... Barracuda [Sat Dec 1 11:56:57 GMT 2012] ==> INFO: Installing MariaDB, please wait... Barracuda [Sat Dec 1 11:58:26 GMT 2012] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Sat Dec 1 11:59:11 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 11:59:13 GMT 2012] ==> INFO: Installed Nginx version nginx/1.3.8, no upgrade required Barracuda [Sat Dec 1 11:59:16 GMT 2012] ==> INFO: Installing Nginx, please wait... Barracuda [Sat Dec 1 12:00:30 GMT 2012] ==> INFO: Installing /usr/bin/wkhtmltopdf x86_64 version, please wait... Barracuda [Sat Dec 1 12:00:35 GMT 2012] ==> INFO: Checking SMTP connections, please wait... Barracuda [Sat Dec 1 12:00:37 GMT 2012] ==> INFO: Installing VnStat monitor, please wait... Barracuda [Sat Dec 1 12:00:47 GMT 2012] ==> INFO: Installing a few more tools, please wait... Barracuda [Sat Dec 1 12:00:49 GMT 2012] ==> INFO: Installed PHP version 5.3.19-1~dotdeb.0, no upgrade required Barracuda [Sat Dec 1 12:00:49 GMT 2012] ==> INFO: Installing IonCube x86_64 version for PHP-FPM, please wait... Barracuda [Sat Dec 1 12:00:53 GMT 2012] ==> INFO: Installing PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 12:00:57 GMT 2012] ==> INFO: Installing PhpRedis for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 12:01:30 GMT 2012] ==> INFO: Installing UploadProgress for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 12:01:52 GMT 2012] ==> INFO: Installing JSMin for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 12:02:20 GMT 2012] ==> INFO: Installing bzr, please wait... Barracuda [Sat Dec 1 12:02:50 GMT 2012] ==> INFO: Installing lshell, please wait... Barracuda [Sat Dec 1 12:02:55 GMT 2012] ==> INFO: Installing latest Pure-FTPd server, please wait... Barracuda [Sat Dec 1 12:04:23 GMT 2012] ==> INFO: Installed Redis version 2.6.4, no rebuild required Barracuda [Sat Dec 1 12:04:25 GMT 2012] ==> INFO: Installing Redis update for Debian/squeeze, please wait... Barracuda [Sat Dec 1 12:05:32 GMT 2012] ==> INFO: Updating init scripts Barracuda [Sat Dec 1 12:05:37 GMT 2012] ==> INFO: Restarting MariaDB Barracuda [Sat Dec 1 12:05:46 GMT 2012] ==> INFO: Starting Redis, PHP-FPM and Nginx Barracuda [Sat Dec 1 12:05:57 GMT 2012] ==> INFO: Generating random password for MariaDB Barracuda [Sat Dec 1 12:06:00 GMT 2012] ==> INFO: OS and services installation completed Barracuda [Sat Dec 1 12:06:02 GMT 2012] ==> INFO: Installing Aegir Master Instance, please wait... Barracuda [Sat Dec 1 12:06:16 GMT 2012] ==> INFO: Running hosting-dispatch (1/3), please wait... Barracuda [Sat Dec 1 12:06:22 GMT 2012] ==> INFO: Running hosting-dispatch (2/3), please wait... Barracuda [Sat Dec 1 12:06:28 GMT 2012] ==> INFO: Running hosting-dispatch (3/3), please wait... cat: /var/aegir/.drush/hostmaster.alias.drushrc.php: No such file or directory Barracuda [Sat Dec 1 12:06:33 GMT 2012] ==> INFO: Aegir Master Instance installation completed Barracuda [Sat Dec 1 12:06:36 GMT 2012] ==> INFO: Installing default SSL Wildcard Nginx Proxy, please wait... Barracuda [Sat Dec 1 12:06:41 GMT 2012] ==> INFO: MariaDB final setup Barracuda [Sat Dec 1 12:06:58 GMT 2012] ==> INFO: MariaDB setup completed Barracuda [Sat Dec 1 12:06:58 GMT 2012] ==> INFO: You can now log in as root by typing only 'mysql' on the command line Barracuda [Sat Dec 1 12:07:00 GMT 2012] ==> INFO: New entry added to /var/log/barracuda_log.txt Barracuda [Sat Dec 1 12:07:03 GMT 2012] ==> ALRT: Something went wrong Barracuda [Sat Dec 1 12:07:03 GMT 2012] ==> ALRT: Please check the install log for details: Barracuda [Sat Dec 1 12:07:03 GMT 2012] ==> ALRT: /var/aegir/install.log Barracuda [Sat Dec 1 12:07:13 GMT 2012] ==> CARD: Now charging your credit card for this automated install service... Barracuda [Sat Dec 1 12:07:19 GMT 2012] ==> JOKE: Just kidding! Enjoy your Aegir Hosting System :) Barracuda [Sat Dec 1 12:07:23 GMT 2012] ==> Final post-install cleaning, please wait a moment... Barracuda [Sat Dec 1 12:07:30 GMT 2012] ==> BYE! Octopus [Sat Dec 1 12:07:34 GMT 2012] ==> BOA Skynet welcomes you aboard! Octopus [Sat Dec 1 12:07:37 GMT 2012] ==> INFO: Reading your /root/.o1.octopus.cnf config file /var/backups/OCTOPUS.sh.txt: line 713: /data/disk/o1/log/fpm.txt: No such file or directory /var/backups/OCTOPUS.sh.txt: line 722: /data/disk/o1/log/cli.txt: No such file or directory Octopus [Sat Dec 1 12:07:38 GMT 2012] ==> NOTE! Please review all config options displayed below ### ### Configuration created on 121130-2126 with ### Octopus version BOA-2.0.4 ### ### NOTE: the group of settings displayed bellow ### will *override* all listed settings in the Octopus script. ### _USER="o1" _MY_EMAIL="chris@webarchitects.co.uk" _PLATFORMS_LIST="ALL" _ALLOW_UNSUPPORTED=NO _AUTOPILOT=NO _HM_ONLY=NO _O_CONTRIB_UP=NO _DEBUG_MODE=NO _MY_OWNIP= _FORCE_GIT_MIRROR="" _THIS_DB_HOST=localhost _DNS_SETUP_TEST=NO _HOT_SAUCE=NO _USE_CURRENT=YES _REMOTE_CACHE_IP=127.0.0.1 _LOCAL_NETWORK_IP= _PHP_FPM_VERSION=5.3 _PHP_CLI_VERSION=5.3 _USE_STOCK=NO ### ### NOTE: the group of settings displayed bellow will be *overriden* ### by config files stored in the /data/disk/o1/log/ directory, ### but only on upgrade. ### _DOMAIN="o1.newdev.transitionnetwork.org" _CLIENT_EMAIL="chris@webarchitects.co.uk" _CLIENT_OPTION="SSD" _CLIENT_SUBSCR="Y" _CLIENT_CORES="8" ### ### Configuration created on 121130-2126 with ### Octopus version BOA-2.0.4 ### Do you want to proceed with the install? [Y/n] Y Octopus [Sat Dec 1 12:10:08 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Octopus [Sat Dec 1 12:10:09 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Octopus [Sat Dec 1 12:10:10 GMT 2012] ==> NEW Aegir Satellite Instance setup in progress... Octopus [Sat Dec 1 12:10:10 GMT 2012] ==> START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your client e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your Aegir control panel for this instance will be available at https://o1.newdev.transitionnetwork.org * Your Aegir system user for this instance will be o1 * This Octopus will use PHP-CLI 5.3 for all sites * This Octopus will use PHP-FPM 5.3 both for D6 and D7 sites * This Octopus includes platforms: ALL / Unsupported: NO * This Octopus options are listed as SSD / Y / 8 C Do you want to proceed? [Y/n] Y Octopus [Sat Dec 1 12:10:48 GMT 2012] ==> 8s before we will continue... Octopus [Sat Dec 1 12:10:48 GMT 2012] ==> 8s before we will continue... Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INIT A: Aegir automated install script part A Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INFO A: Checking OCTOPUS version, please wait... Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INFO: Version test result: OK Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INIT A: INIT Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INIT A: Shared platforms code v.001 (hot new) will be created Octopus [Sat Dec 1 12:11:07 GMT 2012] ==> INIT A: Creating directories with correct permissions, please wait... Octopus [Sat Dec 1 12:11:09 GMT 2012] ==> INIT A: Adding user, please wait... ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES) /usr/bin/mysqladmin: connect to server at 'localhost' failed error: 'Access denied for user 'root'@'localhost' (using password: YES)' Octopus [Sat Dec 1 12:11:16 GMT 2012] ==> INIT A: Switching user and running AegirSetupB, please wait... Octopus [Sat Dec 1 12:11:18 GMT 2012] ==> INIT B: Aegir automated install script part B Octopus [Sat Dec 1 12:11:18 GMT 2012] ==> INIT B: Creating directories with correct permissions Octopus [Sat Dec 1 12:11:20 GMT 2012] ==> INIT B: Running standard installer Octopus [Sat Dec 1 12:11:22 GMT 2012] ==> INIT B: Downloading drush, please wait... Octopus [Sat Dec 1 12:11:25 GMT 2012] ==> INIT B: Drush seems to be functioning properly Octopus [Sat Dec 1 12:11:25 GMT 2012] ==> INIT B: Installing provision backend in /data/disk/o1/.drush Octopus [Sat Dec 1 12:11:26 GMT 2012] ==> INIT B: Downloading Drush and Provision extensions, please wait... Octopus [Sat Dec 1 12:11:32 GMT 2012] ==> INIT B: Running hostmaster-install, please wait... AegirSetupB.sh.txt: line 456: cd: /data/disk/o1/aegir/distro/001: No such file or directory Octopus [Sat Dec 1 12:11:36 GMT 2012] ==> INIT B: Running hosting-dispatch (1/3), please wait... Octopus [Sat Dec 1 12:11:42 GMT 2012] ==> INIT B: Running hosting-dispatch (2/3), please wait... Octopus [Sat Dec 1 12:11:48 GMT 2012] ==> INIT B: Running hosting-dispatch (3/3), please wait... Octopus [Sat Dec 1 12:11:49 GMT 2012] ==> INIT B: Simple check if Aegir install is successful Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> INIT B: FATAL ERROR: Required file /data/disk/o1/aegir/distro/001/sites/o1.newdev.transitionnetwork.org/settings.php does not exist Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> INIT B: FATAL ERROR: Aborting AegirSetupB installer NOW! Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> INIT A: FATAL ERROR: AegirSetupB installer failed Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> INIT A: FATAL ERROR: Aborting AegirSetupA installer NOW! Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> FATAL ERROR: AegirSetupA installer failed Octopus [Sat Dec 1 12:11:51 GMT 2012] ==> FATAL ERROR: Aborting Octopus installer NOW! BOA in-stable completed Bye
So, Jim was right, it looks like the fact that Mysql was installed before the installer was run has caused it to fail. I'll come back to this on Monday, it might be that it's easier to start from scratch with a fresh debian install rather than sort out this failed install. Sorry :-(
comment:9 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.7
- Total Hours changed from 4.5 to 5.2
I have just rebuilt puffin from scratch, it took an hour and a half as I did some debugging on our virtual server build scripts, but I'm not putting the time for this down here.
So, starting again...
ssh root@puffin.webarch.net aptitude install screen screen cd wget -q -U iCab http://files.aegir.cc/BOA.sh.txt bash BOA.sh.txt BOA Meta Installer setup completed Please check docs/INSTALL.txt and docs/UPGRADE.txt for how-to Bye boa in-stable public newdev.transitionnetwork.org chris@webarchitects.co.uk o1 Barracuda [Sat Dec 1 16:03:27 GMT 2012] ==> BOA Skynet welcomes you aboard! Barracuda [Sat Dec 1 16:03:31 GMT 2012] ==> INFO: NORMAL INIT Barracuda [Sat Dec 1 16:03:31 GMT 2012] ==> INFO: Easy Public Setup Mode Active Barracuda [Sat Dec 1 16:03:31 GMT 2012] ==> INFO: Creating your /root/.barracuda.cnf config file Barracuda [Sat Dec 1 16:03:32 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait... Barracuda [Sat Dec 1 16:04:07 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Barracuda [Sat Dec 1 16:04:11 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Barracuda [Sat Dec 1 16:04:11 GMT 2012] ==> INFO: Downloading little helpers, please wait... Barracuda [Sat Dec 1 16:04:17 GMT 2012] ==> INFO: Checking BARRACUDA version... Barracuda [Sat Dec 1 16:04:17 GMT 2012] ==> INFO: Version test result: OK Barracuda [Sat Dec 1 16:04:17 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait... Barracuda [Sat Dec 1 16:04:41 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version... Barracuda [Sat Dec 1 16:04:44 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4 Barracuda [Sat Dec 1 16:04:46 GMT 2012] ==> DNS test OK - your FQDN hostname is newdev.transitionnetwork.org and it matches your IP 81.95.52.103 Barracuda [Sat Dec 1 16:04:46 GMT 2012] ==> Please double check whether this is a valid result of my test Barracuda [Sat Dec 1 16:04:48 GMT 2012] ==> INSTALL START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your IP address appears to be 81.95.52.103 - are you sure it's a valid IP? * Your hostname appears to be newdev.transitionnetwork.org - are you sure it's a valid FQDN hostname? Barracuda [Sat Dec 1 16:04:48 GMT 2012] ==> INFO: Cleaning up temp files in /var/opt/ Barracuda [Sat Dec 1 16:04:50 GMT 2012] ==> INFO: Installing DNS cache pdnsd server, please wait... Barracuda [Sat Dec 1 16:05:09 GMT 2012] ==> INFO: DNS cache pdnsd server installation completed Barracuda [Sat Dec 1 16:05:11 GMT 2012] ==> INFO: Updating apt sources Barracuda [Sat Dec 1 16:05:13 GMT 2012] ==> INFO: We will use Debian mirror ftp.debian.org Barracuda [Sat Dec 1 16:05:17 GMT 2012] ==> INFO: Running aptitude update, please wait... Barracuda [Sat Dec 1 16:05:52 GMT 2012] ==> INFO: Installing required libraries and tools Barracuda [Sat Dec 1 16:05:52 GMT 2012] ==> NOTE! This step may take a few minutes, please wait... Barracuda [Sat Dec 1 16:13:32 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 16:13:34 GMT 2012] ==> INFO: Installed Nginx version nginx/1.2.5, upgrade required Barracuda [Sat Dec 1 16:13:36 GMT 2012] ==> INFO: Installing Nginx, please wait... Barracuda [Sat Dec 1 16:14:36 GMT 2012] ==> INFO: Installing MariaDB, please wait... Barracuda [Sat Dec 1 16:16:01 GMT 2012] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Sat Dec 1 16:16:15 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 1 16:16:18 GMT 2012] ==> INFO: Installed Nginx version nginx/1.3.8, no upgrade required Barracuda [Sat Dec 1 16:16:20 GMT 2012] ==> INFO: Installing /usr/bin/wkhtmltopdf x86_64 version, please wait... Barracuda [Sat Dec 1 16:16:25 GMT 2012] ==> INFO: Checking SMTP connections, please wait... Barracuda [Sat Dec 1 16:16:28 GMT 2012] ==> INFO: Installing VnStat monitor, please wait... Barracuda [Sat Dec 1 16:16:37 GMT 2012] ==> INFO: Installing a few more tools, please wait... Barracuda [Sat Dec 1 16:16:39 GMT 2012] ==> INFO: Installing IonCube x86_64 version for PHP-FPM, please wait... Barracuda [Sat Dec 1 16:16:44 GMT 2012] ==> INFO: Installing PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 16:17:04 GMT 2012] ==> INFO: Installing PhpRedis for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 16:17:27 GMT 2012] ==> INFO: Installing UploadProgress for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 16:17:41 GMT 2012] ==> INFO: Installing JSMin for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 1 16:18:01 GMT 2012] ==> INFO: Installing bzr, please wait... Barracuda [Sat Dec 1 16:18:33 GMT 2012] ==> INFO: Installing lshell, please wait... Barracuda [Sat Dec 1 16:18:39 GMT 2012] ==> INFO: Installing latest Pure-FTPd server, please wait... Barracuda [Sat Dec 1 16:19:35 GMT 2012] ==> INFO: Installing Redis update for Debian/squeeze, please wait... Barracuda [Sat Dec 1 16:20:25 GMT 2012] ==> INFO: Generating random password for Redis server Barracuda [Sat Dec 1 16:20:25 GMT 2012] ==> INFO: Updating init scripts Barracuda [Sat Dec 1 16:20:30 GMT 2012] ==> INFO: Restarting MariaDB Barracuda [Sat Dec 1 16:20:38 GMT 2012] ==> INFO: Starting Redis, PHP-FPM and Nginx Barracuda [Sat Dec 1 16:20:48 GMT 2012] ==> INFO: Generating random password for MariaDB Barracuda [Sat Dec 1 16:20:51 GMT 2012] ==> INFO: OS and services installation completed Barracuda [Sat Dec 1 16:20:53 GMT 2012] ==> INFO: Installing Aegir Master Instance, please wait... Barracuda [Sat Dec 1 16:21:40 GMT 2012] ==> INFO: Running hosting-dispatch (1/3), please wait... Barracuda [Sat Dec 1 16:21:46 GMT 2012] ==> INFO: Running hosting-dispatch (2/3), please wait... Barracuda [Sat Dec 1 16:21:52 GMT 2012] ==> INFO: Running hosting-dispatch (3/3), please wait... Barracuda [Sat Dec 1 16:21:57 GMT 2012] ==> INFO: Aegir Master Instance installation completed Barracuda [Sat Dec 1 16:22:00 GMT 2012] ==> INFO: Installing default SSL Wildcard Nginx Proxy, please wait... Barracuda [Sat Dec 1 16:22:05 GMT 2012] ==> INFO: Installing Chive MariaDB Manager, please wait... Barracuda [Sat Dec 1 16:22:09 GMT 2012] ==> INFO: Chive MariaDB Manager installation completed Barracuda [Sat Dec 1 16:22:11 GMT 2012] ==> INFO: MariaDB final setup Barracuda [Sat Dec 1 16:22:25 GMT 2012] ==> INFO: MariaDB setup completed Barracuda [Sat Dec 1 16:22:25 GMT 2012] ==> INFO: You can now log in as root by typing only 'mysql' on the command line Barracuda [Sat Dec 1 16:22:27 GMT 2012] ==> INFO: New entry added to /var/log/barracuda_log.txt Barracuda [Sat Dec 1 16:22:29 GMT 2012] ==> INFO: Congratulations, Aegir backend and frontend have been installed successfully Barracuda [Sat Dec 1 16:22:29 GMT 2012] ==> NOTE! Please wait 2 minutes before opening the following link in your web browser: Barracuda [Sat Dec 1 16:22:29 GMT 2012] ==> LINK: http://master.newdev.transitionnetwork.org/user/reset/1/XXXXX Barracuda [Sat Dec 1 16:22:41 GMT 2012] ==> INFO: Installing csf/lfd firewall, please wait... Barracuda [Sat Dec 1 16:22:47 GMT 2012] ==> INFO: csf/lfd firewall installation completed Barracuda [Sat Dec 1 16:22:49 GMT 2012] ==> CARD: Now charging your credit card for this automated install service... Barracuda [Sat Dec 1 16:22:55 GMT 2012] ==> JOKE: Just kidding! Enjoy your Aegir Hosting System :) Barracuda [Sat Dec 1 16:22:59 GMT 2012] ==> Final post-install cleaning, please wait a moment... Barracuda [Sat Dec 1 16:23:09 GMT 2012] ==> BYE! Octopus [Sat Dec 1 16:23:13 GMT 2012] ==> BOA Skynet welcomes you aboard! Octopus [Sat Dec 1 16:23:16 GMT 2012] ==> INFO: Creating your /root/.o1.octopus.cnf config file Octopus [Sat Dec 1 16:23:17 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Octopus [Sat Dec 1 16:23:19 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Octopus [Sat Dec 1 16:23:20 GMT 2012] ==> NEW Aegir Satellite Instance setup in progress... Octopus [Sat Dec 1 16:23:20 GMT 2012] ==> START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your client e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your Aegir control panel for this instance will be available at https://o1.newdev.transitionnetwork.org * Your Aegir system user for this instance will be o1 * This Octopus will use PHP-CLI 5.3 for all sites * This Octopus will use PHP-FPM 5.3 both for D6 and D7 sites * This Octopus includes platforms: ALL / Unsupported: NO * This Octopus options are listed as SSD / Y / 8 C Do you want to proceed? [Y/n] Y Octopus [Sat Dec 1 16:23:53 GMT 2012] ==> 8s before we will continue... Octopus [Sat Dec 1 16:24:07 GMT 2012] ==> INIT A: Aegir automated install script part A Octopus [Sat Dec 1 16:24:07 GMT 2012] ==> INFO A: Checking OCTOPUS version, please wait... Octopus [Sat Dec 1 16:24:07 GMT 2012] ==> INFO: Version test result: OK Octopus [Sat Dec 1 16:24:07 GMT 2012] ==> INIT A: INIT Octopus [Sat Dec 1 16:24:07 GMT 2012] ==> INIT A: Shared platforms code v.001 (hot new) will be created Octopus [Sat Dec 1 16:24:08 GMT 2012] ==> INIT A: Creating directories with correct permissions, please wait... Octopus [Sat Dec 1 16:24:10 GMT 2012] ==> INIT A: Adding user, please wait... Octopus [Sat Dec 1 16:24:16 GMT 2012] ==> INIT A: Switching user and running AegirSetupB, please wait... Octopus [Sat Dec 1 16:24:18 GMT 2012] ==> INIT B: Aegir automated install script part B Octopus [Sat Dec 1 16:24:18 GMT 2012] ==> INIT B: Creating directories with correct permissions Octopus [Sat Dec 1 16:24:21 GMT 2012] ==> INIT B: Running standard installer Octopus [Sat Dec 1 16:24:23 GMT 2012] ==> INIT B: Downloading drush, please wait... Octopus [Sat Dec 1 16:24:25 GMT 2012] ==> INIT B: Drush seems to be functioning properly Octopus [Sat Dec 1 16:24:25 GMT 2012] ==> INIT B: Installing provision backend in /data/disk/o1/.drush Octopus [Sat Dec 1 16:24:26 GMT 2012] ==> INIT B: Downloading Drush and Provision extensions, please wait... Octopus [Sat Dec 1 16:24:32 GMT 2012] ==> INIT B: Running hostmaster-install, please wait... Octopus [Sat Dec 1 16:25:14 GMT 2012] ==> INIT B: Running hosting-dispatch (1/3), please wait... Octopus [Sat Dec 1 16:25:21 GMT 2012] ==> INIT B: Running hosting-dispatch (2/3), please wait... Octopus [Sat Dec 1 16:25:28 GMT 2012] ==> INIT B: Running hosting-dispatch (3/3), please wait... Octopus [Sat Dec 1 16:25:30 GMT 2012] ==> INIT B: Simple check if Aegir install is successful Octopus [Sat Dec 1 16:25:32 GMT 2012] ==> INIT B: Aegir install test result: OK Octopus [Sat Dec 1 16:25:32 GMT 2012] ==> INIT B: Enhancing Aegir UI, please wait... Octopus [Sat Dec 1 16:26:16 GMT 2012] ==> INIT A: Aegir Satellite Instance installation completed Octopus [Sat Dec 1 16:26:20 GMT 2012] ==> INIT A: Creating shared directories, please wait... Octopus [Sat Dec 1 16:27:15 GMT 2012] ==> INIT A: Switching user and running Platforms build Octopus [Sat Dec 1 16:27:18 GMT 2012] ==> INIT C: Aegir automated install script part C Octopus [Sat Dec 1 16:27:18 GMT 2012] ==> INIT C: Shared platforms code v.001 (hot new) will be created Octopus [Sat Dec 1 16:27:18 GMT 2012] ==> INIT C: Preparing Pressflow +Extra 6.26.2 core, please wait... Acquia 6.26.2 P.001 - http://bit.ly/acquiadrupal [Y/n] Y Octopus [Sat Dec 1 16:27:52 GMT 2012] ==> DISTRO: Acquia 6.26.2 P.001 installation in progress... Octopus [Sat Dec 1 16:28:08 GMT 2012] ==> DISTRO: Acquia 6.26.2 P.001 installation completed CiviCRM 4.1.6 6.26.2 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 1 16:28:23 GMT 2012] ==> DISTRO: CiviCRM 4.1.6 6.26.2 P.001 installation skipped CiviCRM 4.2.6 7.17.1 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 1 16:28:42 GMT 2012] ==> DISTRO: CiviCRM 4.2.6 7.17.1 P.001 installation skipped CiviCRM 3.4.8 6.26.2 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 1 16:29:00 GMT 2012] ==> DISTRO: CiviCRM 3.4.8 6.26.2 P.001 installation skipped Conference 1.0-rc2 6.26.2 P.001 - http://usecod.com [Y/n] n Octopus [Sat Dec 1 16:29:06 GMT 2012] ==> DISTRO: Conference 1.0-rc2 6.26.2 P.001 installation skipped Commerce 1.11.1 7.17.1 P.001 - http://drupalcommerce.org [Y/n] n Octopus [Sat Dec 1 16:29:11 GMT 2012] ==> DISTRO: Commerce 1.11.1 7.17.1 P.001 installation skipped Commerce 2.0-rc4 7.17.1 P.001 - http://drupalcommerce.org [Y/n] n Octopus [Sat Dec 1 16:29:15 GMT 2012] ==> DISTRO: Commerce 2.0-rc4 7.17.1 P.001 installation skipped Commons 2.10 6.26.2 P.001 - http://acquia.com/drupalcommons [Y/n] n Octopus [Sat Dec 1 16:29:58 GMT 2012] ==> DISTRO: Commons 2.10 6.26.2 P.001 installation skipped Drupal 6.26.2 P.001 - http://pressflow.org [Y/n] y Octopus [Sat Dec 1 16:30:11 GMT 2012] ==> DISTRO: Drupal 6.26.2 D.001 installation in progress... Octopus [Sat Dec 1 16:30:13 GMT 2012] ==> DISTRO: Drupal 6.26.2 D.001 installation completed Octopus [Sat Dec 1 16:30:15 GMT 2012] ==> DISTRO: Drupal 6.26.2 S.001 installation in progress... Octopus [Sat Dec 1 16:30:17 GMT 2012] ==> DISTRO: Drupal 6.26.2 S.001 installation completed Octopus [Sat Dec 1 16:30:19 GMT 2012] ==> DISTRO: Drupal 6.26.2 P.001 installation in progress... Octopus [Sat Dec 1 16:30:21 GMT 2012] ==> DISTRO: Drupal 6.26.2 P.001 installation completed Drupal 7.17.1 P.001 - http://drupal.org/drupal-7.17 [Y/n] n 7.17.1 P.001 installation skipped Drupal 8.0-dev-120915 P.001 - http://drupal.org [Y/n] n Octopus [Sat Dec 1 16:31:04 GMT 2012] ==> DISTRO: Drupal 8.0-dev-120915 P.001 installation skipped ELMS 1.0-b1 6.26.2 P.001 - http://elms.psu.edu [Y/n] n Octopus [Sat Dec 1 16:31:25 GMT 2012] ==> DISTRO: ELMS 1.0-b1 6.26.2 P.001 installation skipped Feature Server 1.1 6.26.2 P.001 - http://bit.ly/fservermore [Y/n] n Octopus [Sat Dec 1 16:31:32 GMT 2012] ==> DISTRO: Feature Server 1.1 6.26.2 P.001 installation skipped MNews 1.2.2 6.26.2 P.001 - http://managingnews.com [Y/n] n Octopus [Sat Dec 1 16:31:43 GMT 2012] ==> DISTRO: MNews 1.2.2 6.26.2 P.001 installation skipped NodeStream 1.5.1 6.26.2 P.001 - http://nodestream.org [Y/n] n Octopus [Sat Dec 1 16:31:48 GMT 2012] ==> DISTRO: NodeStream 1.5.1 6.26.2 P.001 installation skipped NodeStream 2.0-b8 7.17.1 P.001 - http://nodestream.org [Y/n] n Octopus [Sat Dec 1 16:31:53 GMT 2012] ==> DISTRO: NodeStream 2.0-b8 7.17.1 P.001 installation skipped Open Atrium 1.6.1 6.26.2 P.001 - http://openatrium.com [Y/n] n Octopus [Sat Dec 1 16:31:58 GMT 2012] ==> DISTRO: Open Atrium 1.6.1 6.26.2 P.001 installation skipped OpenChurch 1.11-b5 7.17.1 P.001 - http://openchurchsite.com [Y/n] n Octopus [Sat Dec 1 16:32:02 GMT 2012] ==> DISTRO: OpenChurch 1.11-b5 7.17.1 P.001 installation skipped Open Deals 1.11 7.17.1 P.001 - http://opendealsapp.com [Y/n] n Octopus [Sat Dec 1 16:32:05 GMT 2012] ==> DISTRO: Open Deals 1.11 7.17.1 P.001 installation skipped Open Outreach 1.0-rc6 7.17.1 P.001 - http://openoutreach.org [Y/n] n Octopus [Sat Dec 1 16:32:09 GMT 2012] ==> DISTRO: Open Outreach 1.0-rc6 7.17.1 P.001 installation skipped OpenPublish 3.0-b7 7.17.1 P.001 - http://openpublishapp.com [Y/n] n Octopus [Sat Dec 1 16:32:13 GMT 2012] ==> DISTRO: OpenPublish 3.0-b7 7.17.1 P.001 installation skipped OpenScholar 2.0-rc1 6.26.2 P.001 - http://openscholar.harvard.edu [Y/n] n Octopus [Sat Dec 1 16:32:17 GMT 2012] ==> DISTRO: OpenScholar 2.0-rc1 6.26.2 P.001 installation skipped Panopoly 1.0-rc2 7.17.1 P.001 - http://drupal.org/project/panopoly [Y/n] n Octopus [Sat Dec 1 16:32:22 GMT 2012] ==> DISTRO: Panopoly 1.0-rc2 7.17.1 P.001 installation skipped Prosepoint 0.43 6.26.2 P.001 - http://prosepoint.org [Y/n] n Octopus [Sat Dec 1 16:32:26 GMT 2012] ==> DISTRO: Prosepoint 0.43 6.26.2 P.001 installation skipped Ubercart 2.10.1 6.26.2 P.001 - http://ubercart.org [Y/n] n Octopus [Sat Dec 1 16:32:30 GMT 2012] ==> DISTRO: Ubercart 2.10.1 6.26.2 P.001 installation skipped Ubercart 3.2.1 7.17.1 P.001 - http://ubercart.org [Y/n] n Octopus [Sat Dec 1 16:32:34 GMT 2012] ==> DISTRO: Ubercart 3.2.1 7.17.1 P.001 installation skipped Octopus [Sat Dec 1 16:32:36 GMT 2012] ==> INIT C: Removing some old core themes, please wait... Octopus [Sat Dec 1 16:32:36 GMT 2012] ==> INIT C: Running Platforms Save & Verify tasks, please wait... Octopus [Sat Dec 1 16:33:02 GMT 2012] ==> INIT A: Platforms installation completed Octopus [Sat Dec 1 16:33:02 GMT 2012] ==> INIT A: Cleaning up various dot files, please wait... Octopus [Sat Dec 1 16:33:06 GMT 2012] ==> INIT A: Adding ftps/lshell user Octopus [Sat Dec 1 16:33:09 GMT 2012] ==> INIT A: Adding symlink to the sites backups Octopus [Sat Dec 1 16:33:11 GMT 2012] ==> INIT A: Adding symlink to the system drush_make Octopus [Sat Dec 1 16:33:13 GMT 2012] ==> INIT A: Adding symlink to the system registry_rebuild Octopus [Sat Dec 1 16:33:15 GMT 2012] ==> INIT A: Adding symlink to the clients directory Octopus [Sat Dec 1 16:33:17 GMT 2012] ==> INIT A: Creating ftp symlinks Octopus [Sat Dec 1 16:33:19 GMT 2012] ==> INIT A: Preparing setupmail.txt Octopus [Sat Dec 1 16:33:21 GMT 2012] ==> INIT A: Sending setup e-mail on init, please wait... Octopus [Sat Dec 1 16:33:24 GMT 2012] ==> INIT A: New entry added to /data/disk/o1/log/octopus_log.txt Octopus [Sat Dec 1 16:33:24 GMT 2012] ==> INIT A: Final cleaning, please wait a moment... Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> INFO: Congratulations, Aegir backend and frontend have been installed successfully Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> NOTE! Please wait 1 minute before opening the following link in your web browser: Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> LINK: http://o1.newdev.transitionnetwork.org/user/reset/1/XXX Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> INIT A: Starting the cron now Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> INIT A: All done! Octopus [Sat Dec 1 16:33:26 GMT 2012] ==> BYE! BOA in-stable completed Bye
The only mistake I think I made above was to say Y to installing http://bit.ly/acquiadrupal when all we need is the D6 Pressflow AFAIK.
So, all good so far, TODO:
- Have a play with Ageir and work out what it can do with it.
- Make Ageir use HTTPS only (currently it works with both but allows HTTP),
- Redo the configuration done in ticket:466#comment:2 (user accounts, backups etc etc)
- Copy across the live data and database, perhaps splitting it into seperate inodedb files in the process and write a script so that this can be easilly done again.
- Migrate all sites / applications we are keeping running off kiwi and quince.
comment:10 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.35
- Total Hours changed from 5.2 to 5.55
So, taking items from the the TODO at the end of ticket:466#comment:9, starting with:
- Have a play with Ageir and work out what it can do with it.
I found on this page https://o1.newdev.transitionnetwork.org/admin/hosting/features the following option enabled and I have disabled it:
Sign up form Provides a simpler signup form that can be opened to anonymous users.
However it wouldn't accept that:
You cannot disable hosting_signup because aegir_custom_settings depends on it
More looking around is needed, but moving onto the next item in the TODO list:
- Make Ageir use HTTPS only (currently it works with both but allows HTTP)
On https://o1.newdev.transitionnetwork.org/hosting/platforms there are these options which are not enabled by default:
Nginx +SSL servers
Allow hostmaster to configure NGINX web servers with SSL support.
Depends on: Web Server (enabled), SSL support (disabled), Nginx servers (enabled), Hosting (enabled), Modal Frame API (enabled), jQuery UI (enabled)
SSL support
Allow hostmaster to configure web servers with SSL support
Depends on: Hosting (enabled), Web Server (enabled), Modal Frame API (enabled), jQuery UI (enabled)
Required by: Nginx +SSL servers (disabled)
So I have enabled them.
I have noted that the emails sent from the server didn't use TLS so, as expected, that needs sorting out.
comment:11 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.2
- Total Hours changed from 5.55 to 5.75
Testing the install of a site, calling it dev and once it's set up I'll see if I can copy dev data to it.
Clicked "Administration" -> "Add Site and Client" and filled in these values:
Domain name: dev.newdev.transitionnetwork.org
Installation profile: Pressflow
Platform: Drupal 6.26.2 P.001
Language: English
Database server: localhost
Domain aliases: dev.transitionnetwork.org
www.dev.transitionnetwork.org
Internal name: dev
Email address: chris@webarch.net
Client name: Transition Network Development Server
This generated:
User warning: Duplicate entry '3-user' for key 'uid_up_type' query: INSERT INTO userprotect (uid, up_roles, up_name, up_mail, up_pass, up_status, up_openid, up_delete, up_edit, up_type) VALUES (3, 1, 0, 0, 0, 0, 0, 0, 0, 'user') in _db_query() (line 169 of /data/disk/o1/aegir/distro/001/includes/database.mysqli.inc). transitionnetwor has been protected from the following editing operations: roles Task install was added to the queue. Next queue run is 18:53:06+0000, server time is 18:53:05+0000. Your site (dev.newdev.transitionnetwork.org) has been requested, and the moment it is ready you will receive a mail at chris@webarch.net with instructions on how to log into it.
So, that worked, there is a site here https://dev.newdev.transitionnetwork.org/ and I'll set up SSH so I can copy across the dev site from the dev server and while I'm at it set up some other system stuff.
comment:12 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.36
- Total Hours changed from 5.75 to 6.11
Some things to note if we start again with this process...
The "o1" used for the install results in system users being created like this:
ls -lah /home/ drwx------ 6 o1.ftp users 4.0K Dec 1 18:55 o1.ftp drwx------ 4 o1.transitionnetwor users 4.0K Dec 1 19:00 o1.transitionnetwor
With hindsight perhaps I should have used "tn" as a prefix.
Also note that the username, "o1.transitionnetwor" has been generated from the "Client name" in the add site stage, for this I should have used "Dev" and then we would have a "tn.dev" user.
In the /home/o1.transitionnetwor/ directory we have:
drwx------ 2 o1.transitionnetwor users 4.0K Dec 1 18:55 .drush drwx------ 2 o1.transitionnetwor users 4.0K Dec 1 18:55 .ssh lrwxrwxrwx 1 root root 38 Dec 1 19:05 sites -> /data/disk/o1/clients/transitionnetwor
And in the /data/disk/o1/clients/transitionnetwor directory we have another symlink:
lrwxrwxrwx 1 o1 users 85 Dec 1 18:53 dev.newdev.transitionnetwork.org -> /data/disk/o1/distro/001/pressflow-6.26.2-prod/sites/dev.newdev.transitionnetwork.org
And in /data/disk/o1/distro/001/pressflow-6.26.2-prod/sites/dev.newdev.transitionnetwork.org we have the actual site:
-r--r----- 1 o1 users 61K Dec 1 18:53 drushrc.php drwxrws--- 15 o1 www-data 4.0K Dec 1 18:53 files drwxrwsr-x 2 o1 users 4.0K Dec 1 18:53 libraries -r--r----- 1 o1 www-data 28 Dec 1 18:53 local.settings.php drwxrwsr-x 2 o1 users 4.0K Dec 1 18:53 modules drwxrws--- 5 o1 www-data 4.0K Dec 1 18:53 private -r--r----- 1 o1 www-data 3.2K Dec 1 18:53 settings.php drwxrwsr-x 2 o1 users 4.0K Dec 1 18:53 themes
So, I'm happy installing the mysql data and the files from the live dev site but what isn't clear to me is how we will install the modules and theme while still using Ageir, at the top of settings.php we have:
/** * @file Drupal's settings.php file * * This file was automatically generated by Aegir 6.x-2.x * on Sat, 01 Dec 2012 18:53:10 +0000. * * If it is still managed by Aegir, changes to this file may be * lost. If it is not managed by aegir, you should remove this header * to avoid further confusion. */
comment:13 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 6.11 to 6.21
This looks like the way to import the TN sites to Ageir:
http://community.aegirproject.org/content/importing-complete-drupal-platform
Jim - does that make sense to you?
comment:14 Changed 4 years ago by jim
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 6.21 to 6.46
That link is good BUT do not add any sites under the 'base' Aegir install -- all sites should be done via the o1 account and placed in /data/disk/o1/static... See the BOA specific from Omega8cc (the people who maintain it and run it commercially for their hosting):
- http://omega8.cc/ <-- see the various links in the 'library' section on the left, lots of handy stuff here.
All interaction with the system for web stuff needs to be via the o1 account (I think its su /bin/bash - o1 to switch user accounts from memory).
Putting stuff in /var/www or /var/aegir will lose some of the good stuff Octopus brings, plus mean we need root or high access accounts for developers when the ability to switch to o1 from our own accounts should be all that's needed on a regular basis.
Finally all that o1.* stuff is also quite neat as it creates limited user accounts for each 'client' in the system... Each with SFTP access, LShell SSH and quotas etc. Less useful for us, but again if we want to add a new site, or run a development environment on the server etc, we can with no risk to the other sites.
comment:15 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 6.46 to 7.46
Thanks Jim.
Following How to add custom platform properly? I have copied the files from Kiwi:
cd /data/disk/o1/static mkdir dev.transitionnetwork.org rsync -av rsync -av kiwi:/web/dev.transitionnetwork.org.webarch.net/www/ dev.transitionnetwork.org/ chown -R o1:users dev.transitionnetwork.org/ chmod 775 dev.transitionnetwork.org/
I then added it as a platform using the form here with these values:
Platform Name: Dev - Kiwi Import Platform Path: /data/disk/o1/static/dev.transitionnetwork.org Drush make option: Working copy - preserves SCM files Platform access control: Transition Network Development Server
This created a platform at https://o1.newdev.transitionnetwork.org/hosting/c/platform_DevKiwiImport
I couldn't see how to switch the site I set up at http://dev.newdev.transitionnetwork.org/ to use the new platform so I created a new site at http://newdev.newdev.transitionnetwork.org/ But it doesn't appear to be using the Transition Network templates and I'm not sure what the best way to import the database is, manually, using drush or using a web interface?
I have also created a Octopus admin account for Jim and emailed him the details.
I haven't set up backups and other things for puffin yet as I'm not convinced we won't be starting from scratch again...
comment:16 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.22
- Total Hours changed from 7.46 to 7.68
I have started to read around and document what the BOA stack is and how it works, see wiki:PuffinServer#BarracudaOctopusAgeir.
comment:17 follow-up: ↓ 18 Changed 4 years ago by chris
- Cc mark added
- Add Hours to Ticket changed from 0.0 to 1.76
- Total Hours changed from 7.68 to 9.44
I have just watched four "Drupal Tutorial - Aegir on Omega 8" videos, Introduction, Site Migration, Making a Platform and Configuring, I suggest that Laura and Mark watch these to get an idea about what we are getting into.
I have come to the conclusion that it's going to be Jim, Laura and Mark who are using the BOA web interface to backup, clone and migrate the TN drupal site after building platforms using drush and git.
I think we need to seriously consider having a seperate virtual server for BOA and one for everything else, I really think that mixing the BOA stack with a load of other things on the same virtual server is going to be asking for trouble, so I suggest:
- puffin.webarch.net 4GB RAM for BOA
- penguin.webarch.net 2GB RAM for Mediawiki, Piwik, Trac, static, archives and everything else
Done this sound like a plan?
comment:18 in reply to: ↑ 17 Changed 4 years ago by chris
Just to remind peole that I think this ticket is stalled at this point:
https://tech.transitionnetwork.org/trac/ticket/466#comment:17
comment:19 Changed 4 years ago by ed
As discussed today - Jim needs to contact Chris and set up/document how it works - also to discuss the puffin/penguin
comment:20 follow-up: ↓ 21 Changed 4 years ago by jim
Not forgotten about this, just not had a moment... Will get my thinking trousers on and reply properly, but the idea for the partition is probably be a good one.
comment:21 in reply to: ↑ 20 Changed 4 years ago by chris
Replying to jim:
the idea for the partition is probably be a good one.
I have just discussed this with Ed and in order that we can get on with things I'll split it into two seperate servers now.
comment:22 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.9
- Total Hours changed from 9.44 to 10.34
So, on a brand new 4GB RAM puffin:
ssh root@puffin.webarch.net
echo "puffin.webarch.net" > /etc/hostname # fqdn for outgoing email
aptitude install screen
screen
wget -q -U iCab http://files.aegir.cc/BOA.sh.txt
bash BOA.sh.txt
BOA Meta Installer setup completed
Please check docs/INSTALL.txt and docs/UPGRADE.txt for how-to
Bye
boa in-stable public puffin.webarch.net chris@webarchitects.co.uk tn
Barracuda [Sat Dec 15 15:45:57 GMT 2012] ==> BOA Skynet welcomes you aboard!
Barracuda [Sat Dec 15 15:46:01 GMT 2012] ==> INFO: NORMAL INIT
Barracuda [Sat Dec 15 15:46:02 GMT 2012] ==> INFO: Easy Public Setup Mode Active
Barracuda [Sat Dec 15 15:46:02 GMT 2012] ==> INFO: Creating your /root/.barracuda.cnf config file
Barracuda [Sat Dec 15 15:46:03 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait...
Barracuda [Sat Dec 15 15:46:48 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait...
Barracuda [Sat Dec 15 15:46:50 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install
Barracuda [Sat Dec 15 15:46:50 GMT 2012] ==> INFO: Downloading little helpers, please wait...
Barracuda [Sat Dec 15 15:46:53 GMT 2012] ==> INFO: Checking BARRACUDA version...
Barracuda [Sat Dec 15 15:46:53 GMT 2012] ==> INFO: Version test result: OK
Barracuda [Sat Dec 15 15:46:53 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait...
Barracuda [Sat Dec 15 15:47:19 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version...
Barracuda [Sat Dec 15 15:47:21 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4
Barracuda [Sat Dec 15 15:47:23 GMT 2012] ==> EXIT on error due to invalid DNS setup
* Your custom _MY_OWNIP is set to "81.95.52.103"
* Your custom _MY_HOSTN is set to "puffin.webarch.net"
* Your custom _MY_FRONT is set to "master.puffin.webarch.net"
* Your _MY_HOSTN and/or _MY_FRONT doesn't match your _MY_OWNIP,
or your hostname is not set properly yet.
* Please make sure that below command returns your FQDN hostname "puffin.webarch.net":
$ uname -n
* Your server has to have already working FQDN hostname matching your IP address.
This means you _have to_ configure the dns for your server IP/hostname _before_
trying to use this install script. Reverse dns is _not_ required.
* Your hostname appears to be puffin.webarch.net - are you sure it's a valid FQDN hostname?
* Are you sure your FQDN hostname matches your IP address: 81.95.52.103?
Barracuda [Sat Dec 15 15:47:24 GMT 2012] ==> EXIT on error due to invalid DNS setup
Octopus [Sat Dec 15 15:47:24 GMT 2012] ==> ALRT: Percona server not running!
Octopus [Sat Dec 15 15:47:24 GMT 2012] ==> EXIT: We can't proceed and will exit now
Octopus [Sat Dec 15 15:47:24 GMT 2012] ==> HINT: Please (re)start Percona server and then run Octopus installer again
Octopus [Sat Dec 15 15:47:24 GMT 2012] ==> Bye
BOA in-stable completed
Bye
The problem here was that I hadn't set up a wild card DNS entry for *.puffin.webarch.net, so that was set up and tested:
dig @dns0.webarchitects.co.uk transition.puffin.webarch.net ;; ANSWER SECTION: transition.puffin.webarch.net. 3600 IN A 81.95.52.103
And the script was run again:
boa in-stable public puffin.webarch.net chris@webarchitects.co.uk tn Barracuda [Sat Dec 15 15:55:15 GMT 2012] ==> BOA Skynet welcomes you aboard! Barracuda [Sat Dec 15 15:55:20 GMT 2012] ==> INFO: NORMAL INIT Barracuda [Sat Dec 15 15:55:20 GMT 2012] ==> INFO: Easy Public Setup Mode Active Barracuda [Sat Dec 15 15:55:20 GMT 2012] ==> INFO: Reading your /root/.barracuda.cnf config file Barracuda [Sat Dec 15 15:55:21 GMT 2012] ==> NOTE! Please review all config options displayed below Barracuda [Sat Dec 15 15:55:21 GMT 2012] ==> NOTE! It will *override* all settings in the Barracuda script ### ### Configuration created on 121215-1545 ### with Barracuda version BOA-2.0.4 ### ### NOTE: the group of settings displayed bellow will *not* be overriden ### on upgrade by the Barracuda script nor by this configuration file. ### They can be defined only on initial Barracuda install. ### _HTTP_WILDCARD=YES _MY_OWNIP="81.95.52.103" _MY_HOSTN="puffin.webarch.net" _MY_FRONT="master.puffin.webarch.net" _THIS_DB_HOST=localhost _SMTP_RELAY_TEST=YES _SMTP_RELAY_HOST="" _LOCAL_NETWORK_IP="" _LOCAL_NETWORK_HN="" ### ### NOTE: the group of settings displayed bellow ### will *override* all listed settings in the Barracuda script, ### both on initial install and upgrade. ### _MY_EMAIL="chris@webarchitects.co.uk" _XTRAS_LIST="PDS CSF CHV FTP" _AUTOPILOT=YES _DEBUG_MODE=NO _DB_SERVER=MariaDB _SSH_PORT=22 _LOCAL_DEBIAN_MIRROR="ftp.debian.org" _LOCAL_UBUNTU_MIRROR="archive.ubuntu.com" _FORCE_GIT_MIRROR="" _DNS_SETUP_TEST=YES _NGINX_EXTRA_CONF="" _NGINX_WORKERS=AUTO _PHP_FPM_WORKERS=AUTO _BUILD_FROM_SRC=NO _PHP_MODERN_ONLY=YES _PHP_FPM_VERSION=5.3 _PHP_CLI_VERSION=5.3 _LOAD_LIMIT_ONE=1444 _LOAD_LIMIT_TWO=888 _CUSTOM_CONFIG_CSF=NO _CUSTOM_CONFIG_SQL=NO _CUSTOM_CONFIG_REDIS=NO _CUSTOM_CONFIG_PHP_5_2=NO _CUSTOM_CONFIG_PHP_5_3=NO _SPEED_VALID_MAX=3600 _NGINX_DOS_LIMIT=300 _SYSTEM_UPGRADE_ONLY=NO _USE_MEMCACHED=NO _NEWRELIC_KEY= _USE_STOCK=NO ### ### Configuration created on 121215-1545 ### with Barracuda version BOA-2.0.4 ### Barracuda [Sat Dec 15 15:55:22 GMT 2012] ==> INFO: We need to install wget, axel, aptitude, netcat & git first, please wait... Barracuda [Sat Dec 15 15:55:29 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Barracuda [Sat Dec 15 15:55:31 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Barracuda [Sat Dec 15 15:55:31 GMT 2012] ==> INFO: Downloading little helpers, please wait... Barracuda [Sat Dec 15 15:55:33 GMT 2012] ==> INFO: Checking BARRACUDA version... Barracuda [Sat Dec 15 15:55:33 GMT 2012] ==> INFO: Version test result: OK Barracuda [Sat Dec 15 15:55:33 GMT 2012] ==> INFO: Installing lsb-release and dnsutils now, please wait... Barracuda [Sat Dec 15 15:55:36 GMT 2012] ==> INFO: Checking your Debian or Ubuntu version... Barracuda [Sat Dec 15 15:55:39 GMT 2012] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.4 Barracuda [Sat Dec 15 15:55:41 GMT 2012] ==> DNS test OK - your FQDN hostname is puffin.webarch.net and it matches your IP 81.95.52.103 Barracuda [Sat Dec 15 15:55:41 GMT 2012] ==> Please double check whether this is a valid result of my test Barracuda [Sat Dec 15 15:55:43 GMT 2012] ==> INSTALL START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your IP address appears to be 81.95.52.103 - are you sure it's a valid IP? * Your hostname appears to be puffin.webarch.net - are you sure it's a valid FQDN hostname? Barracuda [Sat Dec 15 15:55:43 GMT 2012] ==> INFO: Cleaning up temp files in /var/opt/ Barracuda [Sat Dec 15 15:55:45 GMT 2012] ==> INFO: Installing DNS cache pdnsd server, please wait... Barracuda [Sat Dec 15 15:56:05 GMT 2012] ==> INFO: DNS cache pdnsd server installation completed Barracuda [Sat Dec 15 15:56:07 GMT 2012] ==> INFO: Updating apt sources Barracuda [Sat Dec 15 15:56:09 GMT 2012] ==> INFO: We will use Debian mirror ftp.debian.org Barracuda [Sat Dec 15 15:56:13 GMT 2012] ==> INFO: Running aptitude update, please wait... Barracuda [Sat Dec 15 15:56:44 GMT 2012] ==> INFO: Installing required libraries and tools Barracuda [Sat Dec 15 15:56:44 GMT 2012] ==> NOTE! This step may take a few minutes, please wait... Barracuda [Sat Dec 15 16:05:55 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 15 16:05:57 GMT 2012] ==> INFO: Installed Nginx version nginx/1.2.6, upgrade required Barracuda [Sat Dec 15 16:05:59 GMT 2012] ==> INFO: Installing Nginx, please wait... Barracuda [Sat Dec 15 16:07:11 GMT 2012] ==> INFO: Installing MariaDB, please wait... Barracuda [Sat Dec 15 16:08:31 GMT 2012] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Sat Dec 15 16:08:46 GMT 2012] ==> INFO: Testing Nginx version... Barracuda [Sat Dec 15 16:08:48 GMT 2012] ==> INFO: Installed Nginx version nginx/1.3.8, no upgrade required Barracuda [Sat Dec 15 16:08:50 GMT 2012] ==> INFO: Installing /usr/bin/wkhtmltopdf x86_64 version, please wait... Barracuda [Sat Dec 15 16:08:59 GMT 2012] ==> INFO: Checking SMTP connections, please wait... Barracuda [Sat Dec 15 16:09:02 GMT 2012] ==> INFO: Installing VnStat monitor, please wait... Barracuda [Sat Dec 15 16:09:11 GMT 2012] ==> INFO: Installing a few more tools, please wait... Barracuda [Sat Dec 15 16:09:13 GMT 2012] ==> INFO: Installing IonCube x86_64 version for PHP-FPM, please wait... Barracuda [Sat Dec 15 16:09:27 GMT 2012] ==> INFO: Installing PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 15 16:09:50 GMT 2012] ==> INFO: Installing PhpRedis for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 15 16:10:21 GMT 2012] ==> INFO: Installing UploadProgress for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 15 16:10:42 GMT 2012] ==> INFO: Installing JSMin for PHP-FPM 5.3.18, please wait... Barracuda [Sat Dec 15 16:11:10 GMT 2012] ==> INFO: Installing bzr, please wait... Barracuda [Sat Dec 15 16:11:54 GMT 2012] ==> INFO: Installing lshell, please wait... Barracuda [Sat Dec 15 16:12:01 GMT 2012] ==> INFO: Installing latest Pure-FTPd server, please wait... Barracuda [Sat Dec 15 16:13:29 GMT 2012] ==> INFO: Installing Redis update for Debian/squeeze, please wait... Barracuda [Sat Dec 15 16:14:28 GMT 2012] ==> INFO: Generating random password for Redis server Barracuda [Sat Dec 15 16:14:29 GMT 2012] ==> INFO: Updating init scripts Barracuda [Sat Dec 15 16:14:34 GMT 2012] ==> INFO: Restarting MariaDB Barracuda [Sat Dec 15 16:14:44 GMT 2012] ==> INFO: Starting Redis, PHP-FPM and Nginx Barracuda [Sat Dec 15 16:14:55 GMT 2012] ==> INFO: Generating random password for MariaDB Barracuda [Sat Dec 15 16:14:58 GMT 2012] ==> INFO: OS and services installation completed Barracuda [Sat Dec 15 16:15:00 GMT 2012] ==> INFO: Installing Aegir Master Instance, please wait... Barracuda [Sat Dec 15 16:15:58 GMT 2012] ==> INFO: Running hosting-dispatch (1/3), please wait... Barracuda [Sat Dec 15 16:16:04 GMT 2012] ==> INFO: Running hosting-dispatch (2/3), please wait... Barracuda [Sat Dec 15 16:16:10 GMT 2012] ==> INFO: Running hosting-dispatch (3/3), please wait... Barracuda [Sat Dec 15 16:16:19 GMT 2012] ==> INFO: Aegir Master Instance installation completed Barracuda [Sat Dec 15 16:16:23 GMT 2012] ==> INFO: Installing default SSL Wildcard Nginx Proxy, please wait... Barracuda [Sat Dec 15 16:16:27 GMT 2012] ==> INFO: Installing Chive MariaDB Manager, please wait... Barracuda [Sat Dec 15 16:16:34 GMT 2012] ==> INFO: Chive MariaDB Manager installation completed Barracuda [Sat Dec 15 16:16:36 GMT 2012] ==> INFO: MariaDB final setup Barracuda [Sat Dec 15 16:16:53 GMT 2012] ==> INFO: MariaDB setup completed Barracuda [Sat Dec 15 16:16:53 GMT 2012] ==> INFO: You can now log in as root by typing only 'mysql' on the command line Barracuda [Sat Dec 15 16:16:56 GMT 2012] ==> INFO: New entry added to /var/log/barracuda_log.txt Barracuda [Sat Dec 15 16:16:58 GMT 2012] ==> INFO: Congratulations, Aegir backend and frontend have been installed successfully Barracuda [Sat Dec 15 16:16:58 GMT 2012] ==> NOTE! Please wait 2 minutes before opening the following link in your web browser: Barracuda [Sat Dec 15 16:16:58 GMT 2012] ==> LINK: http://master.puffin.webarch.net/user/reset/1/1355588148/db2776a99686789d1865b0225c659162 Barracuda [Sat Dec 15 16:17:10 GMT 2012] ==> INFO: Installing csf/lfd firewall, please wait... Barracuda [Sat Dec 15 16:17:16 GMT 2012] ==> INFO: csf/lfd firewall installation completed Barracuda [Sat Dec 15 16:17:18 GMT 2012] ==> CARD: Now charging your credit card for this automated install service... Barracuda [Sat Dec 15 16:17:25 GMT 2012] ==> JOKE: Just kidding! Enjoy your Aegir Hosting System :) Barracuda [Sat Dec 15 16:17:29 GMT 2012] ==> Final post-install cleaning, please wait a moment... Barracuda [Sat Dec 15 16:17:35 GMT 2012] ==> BYE! Octopus [Sat Dec 15 16:17:39 GMT 2012] ==> BOA Skynet welcomes you aboard! Octopus [Sat Dec 15 16:17:42 GMT 2012] ==> INFO: Creating your /root/.tn.octopus.cnf config file Octopus [Sat Dec 15 16:17:43 GMT 2012] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Octopus [Sat Dec 15 16:17:45 GMT 2012] ==> INFO: GitHub mirror repository will be used for this install Octopus [Sat Dec 15 16:17:46 GMT 2012] ==> NEW Aegir Satellite Instance setup in progress... Octopus [Sat Dec 15 16:17:46 GMT 2012] ==> START -> checkpoint: * Your e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your client e-mail address appears to be chris@webarchitects.co.uk - is that correct? * Your Aegir control panel for this instance will be available at https://tn.puffin.webarch.net * Your Aegir system user for this instance will be tn * This Octopus will use PHP-CLI 5.3 for all sites * This Octopus will use PHP-FPM 5.3 both for D6 and D7 sites * This Octopus includes platforms: ALL / Unsupported: NO * This Octopus options are listed as SSD / Y / 8 C Do you want to proceed? [Y/n] Y Octopus [Sat Dec 15 16:19:32 GMT 2012] ==> 8s before we will continue... Octopus [Sat Dec 15 16:19:32 GMT 2012] ==> 8s before we will continue... Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INIT A: Aegir automated install script part A Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INFO A: Checking OCTOPUS version, please wait... Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INFO: Version test result: OK Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INIT A: INIT Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INIT A: Shared platforms code v.001 (hot new) will be created Octopus [Sat Dec 15 16:19:51 GMT 2012] ==> INIT A: Creating directories with correct permissions, please wait... Octopus [Sat Dec 15 16:19:53 GMT 2012] ==> INIT A: Adding user, please wait... Octopus [Sat Dec 15 16:20:00 GMT 2012] ==> INIT A: Switching user and running AegirSetupB, please wait... Octopus [Sat Dec 15 16:20:02 GMT 2012] ==> INIT B: Aegir automated install script part B Octopus [Sat Dec 15 16:20:02 GMT 2012] ==> INIT B: Creating directories with correct permissions Octopus [Sat Dec 15 16:20:04 GMT 2012] ==> INIT B: Running standard installer Octopus [Sat Dec 15 16:20:06 GMT 2012] ==> INIT B: Downloading drush, please wait... Octopus [Sat Dec 15 16:20:09 GMT 2012] ==> INIT B: Drush seems to be functioning properly Octopus [Sat Dec 15 16:20:09 GMT 2012] ==> INIT B: Installing provision backend in /data/disk/tn/.drush Octopus [Sat Dec 15 16:20:10 GMT 2012] ==> INIT B: Downloading Drush and Provision extensions, please wait... Octopus [Sat Dec 15 16:20:16 GMT 2012] ==> INIT B: Running hostmaster-install, please wait... Octopus [Sat Dec 15 16:21:02 GMT 2012] ==> INIT B: Running hosting-dispatch (1/3), please wait... Octopus [Sat Dec 15 16:21:09 GMT 2012] ==> INIT B: Running hosting-dispatch (2/3), please wait... Octopus [Sat Dec 15 16:21:16 GMT 2012] ==> INIT B: Running hosting-dispatch (3/3), please wait... Octopus [Sat Dec 15 16:21:17 GMT 2012] ==> INIT B: Simple check if Aegir install is successful Octopus [Sat Dec 15 16:21:19 GMT 2012] ==> INIT B: Aegir install test result: OK Octopus [Sat Dec 15 16:21:19 GMT 2012] ==> INIT B: Enhancing Aegir UI, please wait... Octopus [Sat Dec 15 16:22:04 GMT 2012] ==> INIT A: Aegir Satellite Instance installation completed Octopus [Sat Dec 15 16:22:14 GMT 2012] ==> INIT A: Creating shared directories, please wait... Octopus [Sat Dec 15 16:24:13 GMT 2012] ==> INIT A: Switching user and running Platforms build Octopus [Sat Dec 15 16:24:15 GMT 2012] ==> INIT C: Aegir automated install script part C Octopus [Sat Dec 15 16:24:15 GMT 2012] ==> INIT C: Shared platforms code v.001 (hot new) will be created Octopus [Sat Dec 15 16:24:15 GMT 2012] ==> INIT C: Preparing Pressflow +Extra 6.26.2 core, please wait... Acquia 6.26.2 P.001 - http://bit.ly/acquiadrupal [Y/n] n Octopus [Sat Dec 15 16:24:50 GMT 2012] ==> DISTRO: Acquia 6.26.2 P.001 installation skipped CiviCRM 4.1.6 6.26.2 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 15 16:24:55 GMT 2012] ==> DISTRO: CiviCRM 4.1.6 6.26.2 P.001 installation skipped CiviCRM 4.2.6 7.17.1 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 15 16:24:59 GMT 2012] ==> DISTRO: CiviCRM 4.2.6 7.17.1 P.001 installation skipped CiviCRM 3.4.8 6.26.2 P.001 - http://civicrm.org [Y/n] n Octopus [Sat Dec 15 16:25:02 GMT 2012] ==> DISTRO: CiviCRM 3.4.8 6.26.2 P.001 installation skipped Conference 1.0-rc2 6.26.2 P.001 - http://usecod.com [Y/n] n Octopus [Sat Dec 15 16:25:07 GMT 2012] ==> DISTRO: Conference 1.0-rc2 6.26.2 P.001 installation skipped Commerce 1.11.1 7.17.1 P.001 - http://drupalcommerce.org [Y/n] n Octopus [Sat Dec 15 16:25:10 GMT 2012] ==> DISTRO: Commerce 1.11.1 7.17.1 P.001 installation skipped Commerce 2.0-rc4 7.17.1 P.001 - http://drupalcommerce.org [Y/n] n Octopus [Sat Dec 15 16:25:14 GMT 2012] ==> DISTRO: Commerce 2.0-rc4 7.17.1 P.001 installation skipped Commons 2.10 6.26.2 P.001 - http://acquia.com/drupalcommons [Y/n] n Octopus [Sat Dec 15 16:25:18 GMT 2012] ==> DISTRO: Commons 2.10 6.26.2 P.001 installation skipped Drupal 6.26.2 P.001 - http://pressflow.org [Y/n] y Octopus [Sat Dec 15 16:25:25 GMT 2012] ==> DISTRO: Drupal 6.26.2 D.001 installation in progress... Octopus [Sat Dec 15 16:25:27 GMT 2012] ==> DISTRO: Drupal 6.26.2 D.001 installation completed Octopus [Sat Dec 15 16:25:29 GMT 2012] ==> DISTRO: Drupal 6.26.2 S.001 installation in progress... Octopus [Sat Dec 15 16:25:32 GMT 2012] ==> DISTRO: Drupal 6.26.2 S.001 installation completed Octopus [Sat Dec 15 16:25:34 GMT 2012] ==> DISTRO: Drupal 6.26.2 P.001 installation in progress... Octopus [Sat Dec 15 16:25:36 GMT 2012] ==> DISTRO: Drupal 6.26.2 P.001 installation completed Drupal 7.17.1 P.001 - http://drupal.org/drupal-7.17 [Y/n] n Octopus [Sat Dec 15 16:25:43 GMT 2012] ==> DISTRO: Drupal 7.17.1 P.001 installation skipped Drupal 8.0-dev-120915 P.001 - http://drupal.org [Y/n] n Octopus [Sat Dec 15 16:25:48 GMT 2012] ==> DISTRO: Drupal 8.0-dev-120915 P.001 installation skipped ELMS 1.0-b1 6.26.2 P.001 - http://elms.psu.edu [Y/n] n Octopus [Sat Dec 15 16:25:52 GMT 2012] ==> DISTRO: ELMS 1.0-b1 6.26.2 P.001 installation skipped Feature Server 1.1 6.26.2 P.001 - http://bit.ly/fservermore [Y/n] n Octopus [Sat Dec 15 16:25:57 GMT 2012] ==> DISTRO: Feature Server 1.1 6.26.2 P.001 installation skipped MNews 1.2.2 6.26.2 P.001 - http://managingnews.com [Y/n] n Octopus [Sat Dec 15 16:26:01 GMT 2012] ==> DISTRO: MNews 1.2.2 6.26.2 P.001 installation skipped NodeStream 1.5.1 6.26.2 P.001 - http://nodestream.org [Y/n] n Octopus [Sat Dec 15 16:26:05 GMT 2012] ==> DISTRO: NodeStream 1.5.1 6.26.2 P.001 installation skipped NodeStream 2.0-b8 7.17.1 P.001 - http://nodestream.org [Y/n] n Octopus [Sat Dec 15 16:26:09 GMT 2012] ==> DISTRO: NodeStream 2.0-b8 7.17.1 P.001 installation skipped Open Atrium 1.6.1 6.26.2 P.001 - http://openatrium.com [Y/n] n Octopus [Sat Dec 15 16:26:17 GMT 2012] ==> DISTRO: Open Atrium 1.6.1 6.26.2 P.001 installation skipped OpenChurch 1.11-b5 7.17.1 P.001 - http://openchurchsite.com [Y/n] n Octopus [Sat Dec 15 16:26:43 GMT 2012] ==> DISTRO: OpenChurch 1.11-b5 7.17.1 P.001 installation skipped Open Deals 1.11 7.17.1 P.001 - http://opendealsapp.com [Y/n] n Octopus [Sat Dec 15 16:26:47 GMT 2012] ==> DISTRO: Open Deals 1.11 7.17.1 P.001 installation skipped Open Outreach 1.0-rc6 7.17.1 P.001 - http://openoutreach.org [Y/n] n Octopus [Sat Dec 15 16:26:50 GMT 2012] ==> DISTRO: Open Outreach 1.0-rc6 7.17.1 P.001 installation skipped OpenPublish 3.0-b7 7.17.1 P.001 - http://openpublishapp.com [Y/n] n Octopus [Sat Dec 15 16:26:54 GMT 2012] ==> DISTRO: OpenPublish 3.0-b7 7.17.1 P.001 installation skipped OpenScholar 2.0-rc1 6.26.2 P.001 - http://openscholar.harvard.edu [Y/n] n Octopus [Sat Dec 15 16:26:57 GMT 2012] ==> DISTRO: OpenScholar 2.0-rc1 6.26.2 P.001 installation skipped Panopoly 1.0-rc2 7.17.1 P.001 - http://drupal.org/project/panopoly [Y/n] n Octopus [Sat Dec 15 16:27:01 GMT 2012] ==> DISTRO: Panopoly 1.0-rc2 7.17.1 P.001 installation skipped Prosepoint 0.43 6.26.2 P.001 - http://prosepoint.org [Y/n] n Octopus [Sat Dec 15 16:27:04 GMT 2012] ==> DISTRO: Prosepoint 0.43 6.26.2 P.001 installation skipped Ubercart 2.10.1 6.26.2 P.001 - http://ubercart.org [Y/n] n Octopus [Sat Dec 15 16:27:08 GMT 2012] ==> DISTRO: Ubercart 2.10.1 6.26.2 P.001 installation skipped Ubercart 3.2.1 7.17.1 P.001 - http://ubercart.org [Y/n] n Octopus [Sat Dec 15 16:27:11 GMT 2012] ==> DISTRO: Ubercart 3.2.1 7.17.1 P.001 installation skipped Octopus [Sat Dec 15 16:27:13 GMT 2012] ==> INIT C: Removing some old core themes, please wait... Octopus [Sat Dec 15 16:27:13 GMT 2012] ==> INIT C: Running Platforms Save & Verify tasks, please wait... Octopus [Sat Dec 15 16:27:33 GMT 2012] ==> INIT A: Platforms installation completed Octopus [Sat Dec 15 16:27:33 GMT 2012] ==> INIT A: Cleaning up various dot files, please wait... Octopus [Sat Dec 15 16:27:38 GMT 2012] ==> INIT A: Adding ftps/lshell user Octopus [Sat Dec 15 16:27:40 GMT 2012] ==> INIT A: Adding symlink to the sites backups Octopus [Sat Dec 15 16:27:42 GMT 2012] ==> INIT A: Adding symlink to the system drush_make Octopus [Sat Dec 15 16:27:44 GMT 2012] ==> INIT A: Adding symlink to the system registry_rebuild Octopus [Sat Dec 15 16:27:47 GMT 2012] ==> INIT A: Adding symlink to the clients directory Octopus [Sat Dec 15 16:27:49 GMT 2012] ==> INIT A: Creating ftp symlinks Octopus [Sat Dec 15 16:27:51 GMT 2012] ==> INIT A: Preparing setupmail.txt Octopus [Sat Dec 15 16:27:53 GMT 2012] ==> INIT A: Sending setup e-mail on init, please wait... Octopus [Sat Dec 15 16:27:55 GMT 2012] ==> INIT A: New entry added to /data/disk/tn/log/octopus_log.txt Octopus [Sat Dec 15 16:27:55 GMT 2012] ==> INIT A: Final cleaning, please wait a moment... Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> INFO: Congratulations, Aegir backend and frontend have been installed successfully Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> NOTE! Please wait 1 minute before opening the following link in your web browser: Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> LINK: http://tn.puffin.webarch.net/user/reset/1/XXX Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> INIT A: Starting the cron now Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> INIT A: All done! Octopus [Sat Dec 15 16:27:58 GMT 2012] ==> BYE! BOA in-stable completed Bye
I have forwarded the one time login links for https://tn.puffin.webarch.net/ and https://master.puffin.webarch.net/ to Jim so he can sort out the BOA side of things.
Next I'll sort out the SSL certificate for the server, add ssh public keys to the root account for Jim and myself and also sort out backups.
Than I'll concentrate on PenguinServer ticket:470 for a while.
comment:23 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 10.34 to 10.84
Generate a ssh key and then add it to kiwi and quince:
cd /root/ ssh-keygen -t rsa -n 2048 Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub.
The key has been added to /root/authorized_keys on quince and kiwi with the IP address for extra security:
from="81.95.52.103" ssh-rsa AAAA...
And ~/.ssh/config was created with this in to make ssh'ing simpler:
Host kiwi Hostname kiwi.webarch.net User root Host quince Hostname quince.webarch.net User root
Fix vim syntax highlighting:
echo "syntax on" >> ~/.vimrc
Fix default editor:
echo "export EDITOR='vim'" >> ~/.bashrc
Install backupninja:
aptitude install backupninja
Created a /root/Changelog to document system updates:
2012-12-15 chris
* backupninja : installed
Install metche to keep track of the Changelog:
aptitude install metche
Installed a couple of scripts to /usr/local/bin for updating Changelog and running aptitude, a-up and logchanges.
Sorted out root email:
echo "root: chris@webarchitects.co.uk" >> /etc/aliases newaliases /etc/init.d/postfix restart
comment:24 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.15
- Total Hours changed from 10.84 to 10.99
I have set up backupninja and done a test run, the scripts are in /etc/backup.d/.
Next I'll sort the certificate and postfix TLS out.
comment:25 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.45
- Total Hours changed from 10.99 to 12.44
SSL cert copied across:
mkdir /etc/ssl/transitionnetwork.org rsync -av quince:/etc/ssl/transitionnetwork.org/ /etc/ssl/transitionnetwork.org/
Then the existing TLS configuration section in /etc/postfix/main.cf was replaced with:
smtpd_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt
smtpd_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key
smtpd_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem
smtpd_use_tls = yes
smtpd_enforce_tls = no
smtpd_tls_auth_only = yes
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
smtpd_tls_cipherlist = EDH:!EXP:!LOW
smtpd_tls_ask_ccert = yes
smtpd_tls_req_ccert = no
smtp_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtp_enforce_tls = no
smtp_tls_cert_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.crt
smtp_tls_key_file = /etc/ssl/transitionnetwork.org/transitionnetwork.org.key
smtp_tls_CAfile = /etc/ssl/transitionnetwork.org/gandi.pem
smtp_tls_loglevel = 1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_session_cache_timeout = 3600s
And mutt was used to test TLS.
The gandi.pem file was created like this:
wget http://crt.gandi.net/GandiStandardSSLCA.crt -O GandiStandardSSLCA.crt wget http://crt.usertrust.com/UTNAddTrustServer_CA.crt -O UTNAddTrustServer_CA.crt wget http://crt.usertrust.com/AddTrustExternalCARoot.crt -O AddTrustExternalCARoot.crt openssl x509 -inform DER -in GandiStandardSSLCA.crt -out GandiStandardSSLCA.pem openssl x509 -inform DER -in AddTrustExternalCARoot.crt -out AddTrustExternalCARoot.pem openssl x509 -inform DER -in UTNAddTrustServer_CA.crt -out UTNAddTrustServer_CA.pem cat GandiStandardSSLCA.pem > gandi.pem cat UTNAddTrustServer_CA.pem >> gandi.pem cat AddTrustExternalCARoot.pem >> gandi.pem
And for Aegir and nginx / ftp the answer was found here:
https://github.com/omega8cc/nginx-for-drupal/issues/8#issuecomment-422955
cd /etc/ssl/private/ mv nginx-wild-ssl.crt nginx-wild-ssl.crt.old mv nginx-wild-ssl.key nginx-wild-ssl.key.old mv pure-ftpd.pem pure-ftpd.pem.old ln -s ../transitionnetwork.org/transitionnetwork.org.key nginx-wild-ssl.key ln -s ../transitionnetwork.org/transitionnetwork.org.crt nginx-wild-ssl.crt ln -s ../transitionnetwork.org/transitionnetwork.org.pem pure-ftpd.pem
After nginx was restarted the gandi wild card cert was being used.
backupninja has been documented wiki:PuffinServer#Backupninja
postfix documentation: wiki:PuffinServer#Postfix
nginx documentation: wiki:PuffinServer#Nginx
I think that is me done on puffin, unless I have forgotten anything, as far as I'm aware the rest of the configuration and management of the sites on the server can be done via the web interface. So I'll next do some work on penguin, ticket:470
comment:26 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.6
- Total Hours changed from 12.44 to 13.04
Updating BOA, following http://drupalcode.org/project/barracuda.git/blob/HEAD:/docs/UPGRADE.txt
This has also been documented here wiki:PuffinServer#UpgradingBOA
screen wget -q -U iCab http://files.aegir.cc/BOA.sh.txt bash BOA.sh.txt BOA Meta Installer setup completed Please check INSTALL.txt and UPGRADE.txt at http://bit.ly/boa-docs for how-to Bye
Updating barracuda (output trimmed):
barracuda up-stable Barracuda [Tue Jan 8 12:32:42 GMT 2013] ==> BOA Skynet welcomes you aboard! Barracuda [Tue Jan 8 12:32:46 GMT 2013] ==> INFO: UPGRADE Barracuda [Tue Jan 8 12:32:46 GMT 2013] ==> INFO: Reading your /root/.barracuda.cnf config file Barracuda [Tue Jan 8 12:32:47 GMT 2013] ==> NOTE! Please review all config options displayed below Barracuda [Tue Jan 8 12:32:47 GMT 2013] ==> NOTE! It will *override* all settings in the Barracuda script Barracuda [Tue Jan 8 12:32:49 GMT 2013] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait... Barracuda [Tue Jan 8 12:32:51 GMT 2013] ==> INFO: GitHub mirror repository will be used for this install Barracuda [Tue Jan 8 12:32:51 GMT 2013] ==> INFO: Downloading little helpers, please wait... Barracuda [Tue Jan 8 12:32:53 GMT 2013] ==> INFO: Checking BARRACUDA version... Barracuda [Tue Jan 8 12:32:53 GMT 2013] ==> INFO: Version test result: OK Barracuda [Tue Jan 8 12:32:53 GMT 2013] ==> INFO: Checking your Debian or Ubuntu version... Barracuda [Tue Jan 8 12:32:56 GMT 2013] ==> Aegir with Nginx on Debian/squeeze - Skynet Agent v.BOA-2.0.5 Barracuda [Tue Jan 8 12:33:00 GMT 2013] ==> INFO: Cleaning up temp files in /var/opt/ Barracuda [Tue Jan 8 12:33:02 GMT 2013] ==> INFO: Updating apt sources Barracuda [Tue Jan 8 12:33:04 GMT 2013] ==> INFO: We will use Debian mirror ftp.debian.org Barracuda [Tue Jan 8 12:33:08 GMT 2013] ==> INFO: Running aptitude update, please wait... Barracuda [Tue Jan 8 12:33:17 GMT 2013] ==> INFO: Upgrading required libraries and tools Barracuda [Tue Jan 8 12:33:17 GMT 2013] ==> INFO: Now waiting 60 seconds for php-fpm pre-upgrade shutdown... Barracuda [Tue Jan 8 12:34:20 GMT 2013] ==> NOTE! This step may take a few minutes, please wait... Barracuda [Tue Jan 8 12:35:09 GMT 2013] ==> INFO: Testing Nginx version... Barracuda [Tue Jan 8 12:35:11 GMT 2013] ==> INFO: Upgrading Nginx, please wait... Barracuda [Tue Jan 8 12:36:31 GMT 2013] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Tue Jan 8 12:37:15 GMT 2013] ==> INFO: Testing Nginx version... Barracuda [Tue Jan 8 12:37:17 GMT 2013] ==> INFO: Installed Nginx version nginx/1.3.9, no upgrade required Barracuda [Tue Jan 8 12:37:19 GMT 2013] ==> INFO: Checking SMTP connections, please wait... Barracuda [Tue Jan 8 12:37:21 GMT 2013] ==> INFO: Upgrading a few more tools, please wait... Barracuda [Tue Jan 8 12:37:24 GMT 2013] ==> INFO: Checking if PHP upgrade is available Barracuda [Tue Jan 8 12:37:26 GMT 2013] ==> INFO: Installed PHP version 5.3.20-1~dotdeb.0, no upgrade required Barracuda [Tue Jan 8 12:37:26 GMT 2013] ==> INFO: Installing PhpRedis upgrade for PHP-FPM 5.3.20, please wait... Barracuda [Tue Jan 8 12:37:59 GMT 2013] ==> INFO: Installing JSMin upgrade for PHP-FPM 5.3.20, please wait... Barracuda [Tue Jan 8 12:38:26 GMT 2013] ==> INFO: Installing latest Pure-FTPd server, please wait... Barracuda [Tue Jan 8 12:39:58 GMT 2013] ==> INFO: Installed Redis version 2.6.4, upgrade/rebuild required Barracuda [Tue Jan 8 12:40:00 GMT 2013] ==> INFO: Installing Redis update for Debian/squeeze, please wait... Barracuda [Tue Jan 8 12:41:08 GMT 2013] ==> INFO: Restarting Redis and PHP-FPM, reloading Nginx Barracuda [Tue Jan 8 12:41:18 GMT 2013] ==> INFO: OS and services upgrade completed Barracuda [Tue Jan 8 12:41:20 GMT 2013] ==> INFO: Restarting MariaDB server, please wait... Barracuda [Tue Jan 8 12:42:35 GMT 2013] ==> INFO: Running Aegir Master Instance upgrade Barracuda [Tue Jan 8 12:42:37 GMT 2013] ==> INFO: Test OK, we can proceed with Hostmaster upgrade Barracuda [Tue Jan 8 12:42:37 GMT 2013] ==> INFO: Downloading drush, please wait... Barracuda [Tue Jan 8 12:42:41 GMT 2013] ==> INFO: Drush seems to be functioning properly Barracuda [Tue Jan 8 12:42:41 GMT 2013] ==> INFO: Moving old directories Barracuda [Tue Jan 8 12:42:41 GMT 2013] ==> INFO: Installing provision backend in /var/aegir/.drush Barracuda [Tue Jan 8 12:42:42 GMT 2013] ==> INFO: Downloading Drush and Provision extensions, please wait... Barracuda [Tue Jan 8 12:42:44 GMT 2013] ==> INFO: Running hostmaster-migrate, please wait... Barracuda [Tue Jan 8 12:43:47 GMT 2013] ==> INFO: Aegir Master Instance upgrade completed Barracuda [Tue Jan 8 12:43:48 GMT 2013] ==> INFO: New entry added to /var/log/barracuda_log.txt Barracuda [Tue Jan 8 12:43:48 GMT 2013] ==> CARD: Now charging your credit card for this automated upgrade service... Barracuda [Tue Jan 8 12:43:54 GMT 2013] ==> JOKE: Just kidding! Enjoy your Aegir Hosting System :) Barracuda [Tue Jan 8 12:43:58 GMT 2013] ==> Final post-upgrade cleaning, please wait a moment... Barracuda [Tue Jan 8 12:44:10 GMT 2013] ==> BYE! BARRACUDA upgrade completed Bye
Updating octopus (output trimmed):
octopus up-stable all
Octopus [Tue Jan 8 12:45:20 GMT 2013] ==> BOA Skynet welcomes you aboard!
Octopus [Tue Jan 8 12:45:23 GMT 2013] ==> INFO: Reading your /root/.tn.octopus.cnf config file
Octopus [Tue Jan 8 12:45:43 GMT 2013] ==> INFO: Testing GitHub, Drupal and Gitorious servers availability, please wait...
Octopus [Tue Jan 8 12:45:45 GMT 2013] ==> INFO: GitHub mirror repository will be used for this install
Octopus [Tue Jan 8 12:45:46 GMT 2013] ==> UPGRADE in progress...
Octopus [Tue Jan 8 12:45:46 GMT 2013] ==> START -> checkpoint:
* Your Aegir control panel for this instance is available at https://tn.puffin.webarch.net
* Your Aegir system user for this instance is tn
* This Octopus will use PHP-CLI 5.3 for all sites
* This Octopus will use PHP-FPM 5.3 both for D6 and D7 sites
* This Octopus includes platforms: ALL / Unsupported: NO
* This Octopus options are listed as SSD / Y / 8 C
Do you want to proceed? [Y/n] y
Octopus [Tue Jan 8 12:46:03 GMT 2013] ==> 8s before we will continue...
Octopus [Tue Jan 8 12:46:17 GMT 2013] ==> INIT A: Aegir automated install script part A
Octopus [Tue Jan 8 12:46:17 GMT 2013] ==> INFO A: Checking OCTOPUS version, please wait...
Octopus [Tue Jan 8 12:46:17 GMT 2013] ==> INFO: Version test result: OK
Octopus [Tue Jan 8 12:46:17 GMT 2013] ==> UPGRADE A: UPGRADE
Octopus [Tue Jan 8 12:46:20 GMT 2013] ==> UPGRADE A: Shared platforms code v.002 (hot new) will be created
Octopus [Tue Jan 8 12:46:20 GMT 2013] ==> UPGRADE A: Creating directories with correct permissions, please wait...
Do you want to upgrade this Aegir Satellite Instance? [Y/n] y
Octopus [Tue Jan 8 12:46:35 GMT 2013] ==> UPGRADE A: Switching user and running AegirSetupB, please wait...
Octopus [Tue Jan 8 12:46:38 GMT 2013] ==> INIT B: Aegir automated install script part B
Octopus [Tue Jan 8 12:46:38 GMT 2013] ==> UPGRADE B: Creating directories with correct permissions
Octopus [Tue Jan 8 12:46:40 GMT 2013] ==> UPGRADE B: Running standard installer
Octopus [Tue Jan 8 12:46:42 GMT 2013] ==> UPGRADE B: Downloading drush, please wait...
Octopus [Tue Jan 8 12:46:45 GMT 2013] ==> UPGRADE B: Drush seems to be functioning properly
Octopus [Tue Jan 8 12:46:45 GMT 2013] ==> UPGRADE B: Installing provision backend in /data/disk/tn/.drush
Octopus [Tue Jan 8 12:46:46 GMT 2013] ==> UPGRADE B: Downloading Drush and Provision extensions, please wait...
Octopus [Tue Jan 8 12:46:50 GMT 2013] ==> UPGRADE B: Hostmaster STATUS: upgrade start
Octopus [Tue Jan 8 12:46:52 GMT 2013] ==> UPGRADE B: Running hostmaster-migrate, please wait...
Octopus [Tue Jan 8 12:47:52 GMT 2013] ==> UPGRADE B: Hostmaster STATUS: upgrade completed
Octopus [Tue Jan 8 12:47:52 GMT 2013] ==> UPGRADE B: Simple check if Aegir upgrade is successful
Octopus [Tue Jan 8 12:47:54 GMT 2013] ==> UPGRADE B: Aegir upgrade test result: OK
Octopus [Tue Jan 8 12:47:54 GMT 2013] ==> UPGRADE B: Enhancing Aegir UI, please wait...
Octopus [Tue Jan 8 12:48:38 GMT 2013] ==> UPGRADE A: Aegir Satellite Instance upgrade completed
Do you want to install some latest, ready to use platforms? [Y/n] n
Octopus [Tue Jan 8 12:49:40 GMT 2013] ==> UPGRADE A: No new platforms added this time
Octopus [Tue Jan 8 12:49:42 GMT 2013] ==> UPGRADE A: Cleaning up various dot files, please wait...
/opt/tmp/nginx-for-drupal/aegir/scripts/AegirSetupA.sh.txt: line 1122: cd: /data/all/002: No such file or directory
touch: cannot touch `/data/all/002/dot-files-ctrl-BOA-2.0.5': No such file or directory
ln: creating symbolic link `/home/tn.ftp/platforms/002/keys': No such file or directory
touch: cannot touch `/data/all/002/javascript_aggregator.out.txt': No such file or directory
Octopus [Tue Jan 8 12:49:46 GMT 2013] ==> UPGRADE A: Preparing setupmail.txt
Octopus [Tue Jan 8 12:49:49 GMT 2013] ==> UPGRADE A: New entry added to /data/disk/tn/log/octopus_log.txt
Octopus [Tue Jan 8 12:49:49 GMT 2013] ==> UPGRADE A: Final cleaning, please wait a moment...
Octopus [Tue Jan 8 12:49:51 GMT 2013] ==> UPGRADE A: Adding missing strongarm module to old fserver platforms
Octopus [Tue Jan 8 12:49:51 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib/backup_migrate
Octopus [Tue Jan 8 12:49:51 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib/purge
Octopus [Tue Jan 8 12:49:51 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib/expire
Octopus [Tue Jan 8 12:49:55 GMT 2013] ==> UPGRADE A: New modules in /data/all/001/o_contrib: purge expire backup_migrate
Octopus [Tue Jan 8 12:49:55 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib_seven/purge
Octopus [Tue Jan 8 12:49:55 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib_seven/expire
Octopus [Tue Jan 8 12:49:55 GMT 2013] ==> UPGRADE A: Removed /data/all/001/o_contrib_seven/filefield_nginx_progress
Octopus [Tue Jan 8 12:49:59 GMT 2013] ==> UPGRADE A: New modules in /data/all/001/o_contrib_seven: filefield_nginx_progress purge expire
Octopus [Tue Jan 8 12:49:59 GMT 2013] ==> UPGRADE A: Starting the cron now
Octopus [Tue Jan 8 12:49:59 GMT 2013] ==> UPGRADE A: All done!
Octopus [Tue Jan 8 12:49:59 GMT 2013] ==> BYE!
Done for /data/disk/tn
OCTOPUS upgrade completed
Bye
comment:27 Changed 4 years ago by chris
A script has been created for rsyncing files from NewLiveServer to puffin, see ticket:472 and a link to this ticket has also been added to the wiki page at PuffinServer.
comment:28 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 13.04 to 13.29
Jim was accidentially blocked by the firewall on puffin, so I have unblocked him and documented how to do this here wiki:PuffinServer#CSFLDF
comment:29 Changed 4 years ago by jim
The server is plagued by segfaults, so I'm going to run the update again to see if that brings a better/patched/more stable version of MySQL or PHP-FPM... Following usual process as documented in above ticket.
puffin:~# tail /var/log/syslog Jan 23 21:41:01 puffin /USR/SBIN/CRON[1501]: (root) CMD (bash /var/xdrago/second.sh >/dev/null 2>&1) Jan 23 21:41:32 puffin mysqld: 130123 21:41:32 [Warning] Aborted connection 140990 to db: 'transitionnetwor' user: 'transitionnetwor' host: 'localhost' (Unknown error) Jan 23 21:41:52 puffin mysqld: 130123 21:41:52 [Warning] Aborted connection 141009 to db: 'transitionnetwor' user: 'transitionnetwor' host: 'localhost' (Unknown error) Jan 23 21:42:01 puffin /USR/SBIN/CRON[1935]: (root) CMD (bash /var/xdrago/runner.sh >/dev/null 2>&1) Jan 23 21:42:01 puffin /USR/SBIN/CRON[1936]: (aegir) CMD (/var/aegir/drush/drush.php '@hostmaster' hosting-dispatch ) Jan 23 21:42:01 puffin /USR/SBIN/CRON[1938]: (root) CMD (bash /var/xdrago/minute.sh >/dev/null 2>&1) Jan 23 21:42:01 puffin /USR/SBIN/CRON[1939]: (root) CMD (bash /var/xdrago/second.sh >/dev/null 2>&1) Jan 23 21:42:32 puffin postfix/pickup[28300]: 8AB4225C018: uid=0 from=<root> Jan 23 21:42:32 puffin postfix/cleanup[2226]: 8AB4225C018: message-id=<20130123214232.8AB4225C018@puffin.webarch.net> Jan 23 21:42:32 puffin postfix/qmgr[3817]: 8AB4225C018: from=<root@puffin.webarch.net>, size=622, nrcpt=1 (queue active)
Since the issue seems to be connection to MySQL, I've altered #~/.barracuda.cnf to remove the IP address (since this is the recommended setting), the hunch being it's trying to use the external IP for MySQL connections, rather than 127.0.0.1/localhost.
comment:30 follow-up: ↓ 34 Changed 4 years ago by jim
Have also added CGP Collectd monitor to the install by changing .barracuda.cnf:
_XTRAS_LIST="PDS CSF CHV FTP CGP"
(added 'CGP').
Also Chris, do we want FTP server running if we have SSH? I've personally disabled it on mine and feel that, unless we had multiple clients, it's adding nothing other than a possible attack vector... The solution is to remove 'FTP' from above and re-run the usual install.
Doing upgrade now with new settings.
comment:31 follow-up: ↓ 35 Changed 4 years ago by jim
Updated completed, Collectd installed, but no updates... relevant sections:
Testing Nginx version... Barracuda [Wed Jan 23 22:06:51 GMT 2013] ==> INFO: Installed Nginx version nginx/1.3.9, no upgrade required Barracuda [Wed Jan 23 22:06:53 GMT 2013] ==> INFO: Running aptitude full-upgrade again, please wait... Barracuda [Wed Jan 23 22:07:38 GMT 2013] ==> INFO: Testing Nginx version... Barracuda [Wed Jan 23 22:07:40 GMT 2013] ==> INFO: Installed Nginx version nginx/1.3.9, no upgrade required Barracuda [Wed Jan 23 22:07:42 GMT 2013] ==> INFO: Checking SMTP connections, please wait... Barracuda [Wed Jan 23 22:07:45 GMT 2013] ==> INFO: Upgrading a few more tools, please wait... Barracuda [Wed Jan 23 22:07:47 GMT 2013] ==> INFO: Checking if PHP upgrade is available Barracuda [Wed Jan 23 22:07:50 GMT 2013] ==> INFO: Installed PHP version 5.3.20-1~dotdeb.0, no upgrade required Barracuda [Wed Jan 23 22:07:54 GMT 2013] ==> INFO: Installed Redis version 2.6.7, no upgrade/rebuild required Barracuda [Wed Jan 23 22:07:59 GMT 2013] ==> INFO: Restarting Redis and PHP-FPM, reloading Nginx Barracuda [Wed Jan 23 22:08:10 GMT 2013] ==> INFO: OS and services upgrade completed Barracuda [Wed Jan 23 22:08:12 GMT 2013] ==> INFO: Restarting MariaDB server, please wait... Barracuda [Wed Jan 23 22:09:26 GMT 2013] ==> INFO: Running Aegir Master Instance upgrade Barracuda [Wed Jan 23 22:09:29 GMT 2013] ==> INFO: Testing previous install... Barracuda [Wed Jan 23 22:09:29 GMT 2013] ==> INFO: Test OK, we can proceed with Hostmaster upgrade Barracuda [Wed Jan 23 22:09:29 GMT 2013] ==> INFO: Downloading drush, please wait... Barracuda [Wed Jan 23 22:09:31 GMT 2013] ==> INFO: Drush seems to be functioning properly Barracuda [Wed Jan 23 22:09:31 GMT 2013] ==> INFO: Moving old directories Barracuda [Wed Jan 23 22:09:32 GMT 2013] ==> INFO: Installing provision backend in /var/aegir/.drush Barracuda [Wed Jan 23 22:09:33 GMT 2013] ==> INFO: Downloading Drush and Provision extensions, please wait... Barracuda [Wed Jan 23 22:09:37 GMT 2013] ==> INFO: Running hostmaster-migrate, please wait... Barracuda [Wed Jan 23 22:11:01 GMT 2013] ==> INFO: Aegir Master Instance upgrade completed Barracuda [Wed Jan 23 22:11:02 GMT 2013] ==> INFO: Installing Collectd Graph Panel, please wait... Barracuda [Wed Jan 23 22:12:19 GMT 2013] ==> INFO: Collectd Graph Panel installation completed Barracuda [Wed Jan 23 22:12:22 GMT 2013] ==> INFO: New entry added to /var/log/barracuda_log.txt
NOTE: CGP/Collectd is now available at: http://cgp.master.puffin.webarch.net/
I've documented this here https://tech.transitionnetwork.org/trac/wiki/PuffinServer#CGPCollectdsystemmonitor
So there's no change, and I still get repeated 502 'Bad Gateway' errors when trying to use the server. Syslog shows:
Jan 23 22:14:38 puffin mysqld: 130123 22:14:38 [Warning] Aborted connection 134 to db: 'v01newdevtransit' user: 'v01newdevtransit' host: 'localhost' (Unknown error)
However, a restart of mysql seems to have fixed it -- as it did on Sunday and Monday, so the result it temporary... So we have a buggy MySQL or PHP-FPM setup somehow.
I'll trawl the logs, but it might be worth re-running the install with one of these options set to 'YES' in the .barracuda.cnf -- the latter being the 'nuclear' option.
_NGX_FORCE_REINSTALL
_PHP_FORCE_REINSTALL
_SQL_FORCE_REINSTALL
_FULL_FORCE_REINSTALL
For now I'll look at the logs and raise another ticket for this if this continues to happen.
comment:32 Changed 4 years ago by jim
More syslog
Jan 23 22:30:49 puffin kernel: [3395287.943609] php-fpm[12862]: segfault at 4a9f2120 ip 00000000006d697d sp 00007fff1e8855b0 error 4 in php5-fpm[400000+826000]
And I'm seeing a lot of errors in tail -n10000 /var/syslog | grep drupal
Jan 23 06:59:10 puffin drupal: http://v01.newdev.transitionnetwork.org|1358924350|cron|127.0.0.1|http://v01.newdev.transitionnetwork.org/index.php||0||Cron has been running for more than an hour and is most likely stuck. Jan 23 07:24:12 puffin drupal: http://transitionnetwork.org|1358925852|cron|127.0.0.1|http://transitionnetwork.org/index.php||0||Cron run completed. Jan 23 07:59:18 puffin drupal: http://v01.newdev.transitionnetwork.org|1358927958|aggregator|127.0.0.1|http://v01.newdev.transitionnetwork.org/index.php||0||The feed from REconomy Project Feed seems to be broken, due to an error "Invalid document end" on line 1. Jan 23 07:59:55 puffin drupal: http://v01.newdev.transitionnetwork.org|1358927995|php|66.249.76.194|http://v01.newdev.transitionnetwork.org/||0||Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && Jan 23 07:59:55 puffin drupal: http://v01.newdev.transitionnetwork.org|1358927995|php|66.249.76.194|http://v01.newdev.transitionnetwork.org/||0||Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && Jan 23 07:59:55 puffin drupal: http://v01.newdev.transitionnetwork.org|1358927995|php|66.249.76.194|http://v01.newdev.transitionnetwork.org/||0||Warning: preg_match(): Compilation failed: disallowed Unicode code point (>= 0xd800 && ...
So I now wonder if there's an issue with the TN website somehow, the above v01.newdev.transitionnetwork.org was my first (broken) attempt. It seems to be to do with some regexp compilation issue. Will look on Drupal.org for this.
It's possible that this is something that runs on cron, and somehow messes up things. In the distant past I remember seeing a similar issue, possibly even on TN.org...
I'll keep prodding, but in the mean time delete v01.newdev.transitionnetwork.org since it's dead anyway.
comment:33 Changed 4 years ago by jim
Quick updates:
- The preg_match() error is fixed (was a Drupal CTools bug) - ignore that!
- I reckon the php-fpm segfault is a red-herring related to mysql restart - ignore that too!
- I've patched other Drupal modules and the site is nearly working as expected... Lets see if these fixes stop the database connection issue...
Chris, I saw a bunch of emails being sent to you when I was watching syslog... Apologies for these, but I hope to be fixing them slowly but surely.
More tomorrow, sleep time.
comment:34 in reply to: ↑ 30 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 13.29 to 13.54
Replying to jim:
do we want FTP server running if we have SSH? I've personally disabled it on mine and feel that, unless we had multiple clients, it's adding nothing other than a possible attack vector... The solution is to remove 'FTP' from above and re-run the usual install.
No, we don't need a FTP server, I have removed FTP from /root/.barracuda.cnf and re-run the upgrade steps as documented on the wiki, wiki:PuffinServer#UpgradingBOA
I don't know why MariaDB has been causing problems, I've not used it before, given the way Oracle has been behaving I would expect most Linux distros will switch from MySQL to MariaDB in time.
comment:35 in reply to: ↑ 31 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 13.54 to 13.64
Replying to jim:
NOTE: CGP/Collectd is now available at: http://cgp.master.puffin.webarch.net/
I've documented this here https://tech.transitionnetwork.org/trac/wiki/PuffinServer#CGPCollectdsystemmonitor
Thanks, I had been assuming that we would run a collectd server on penguin and clients on penguin and puffin, but perhaps if it doesn't use the same system resources as munin we can run either run and client and server on all machines or use puffin as the main server?
comment:36 follow-up: ↓ 37 Changed 4 years ago by jim
The crashes plaguing me a few days back seem to be gone -- looks like they were caused by one or more of the issues raised in the above 6 comments.
NOTE in syslog I just saw LOTS (thousands) of:
Jan 26 13:19:01 puffin pure-ftpd: (?@190.58.249.69) [INFO] New connection from 190.58.249.69 Jan 26 13:19:01 puffin pure-ftpd: (?@190.58.249.69) [DEBUG] Command [user] [Administrator] Jan 26 13:19:01 puffin pure-ftpd: (?@190.58.249.69) [WARNING] Sorry, cleartext sessions are not accepted on this server.#012Please reconnect using SSL/TLS security mechanisms.
So I ran a killall pure-ftpd to ensure no instances of the FTP server Chris removed were running. Then I edited /etc/csf.conf and removed access to ports 20 and 21, then csf -r to restart the firewall and LFD.
The above change will be overwritten on next BOA update unless we set _CUSTOM_CSF in #~/.barracuda.cnf to 'YES'. I'll leave this for now but we should close all ports not being used, and consider moving SSH port elsewhere too if possible. Thoughts Chris?
Anyway, since the DB/PHP issue seems to be gone, I'll proceed with the import of the TN site on ticket #478 Import TN.org site from Quince to Puffin.
comment:37 in reply to: ↑ 36 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 13.64 to 13.74
Replying to jim:
I ran a killall pure-ftpd to ensure no instances of the FTP server Chris removed were running.
Thanks, sorry I forgot to check if it was still running.
Then I edited /etc/csf.conf and removed access to ports 20 and 21, then csf -r to restart the firewall and LFD.
The above change will be overwritten on next BOA update unless we set _CUSTOM_CSF in #~/.barracuda.cnf to 'YES'.
Are there any disadvantages to doing that?
I'll leave this for now but we should close all ports not being used, and consider moving SSH port elsewhere too if possible. Thoughts Chris?
We could move SSH to a different port if you want, LDF is blocking several IP's that are trying to get i each dayn, but this is not unusual. We could also disable passwords so it's SSH keys only.
comment:38 Changed 4 years ago by jim
@Chris:
- Are there any disadvantages to doing that? (set custom CSF flag: well we wouldn't get any BOA-provided CSF tweaks (rare), but we'd be able to run our own ports at will. I do that on my VPS.
- SSH: On my box I moved the SSH port and saw a drop to almost 0 for drive-bys in the CSF/LFD logs. I also disabled password and logging in as root, forcing login to my account via SSH only, then sudo -i and a password to go further. Your call as to what level of security you want to set on SSH.
NOTE DB/502 issues have reappeared today (see https://tech.transitionnetwork.org/trac/ticket/466#comment:29) - I have therefore reinstalled Barracuda with _PHP_FORCE_REINSTALL=YES, and used barracuda up-head to bring latest updates and fixes.
Problems fixed for now but the last time this happened they were back again. Hopefully fixed for good, else we need to switch _DEBUG_MODE=YES to get some meatier logs.
SSL -- Chris, your comments please
- SSL was enabled by turning on 'SSL' and 'NginX SSL' in the black admin menu -> Hosting -> Features at master.puffin.webarch.net
- Edited the puffin.webarch.net server node via Content Management -> Content, set server to 'nginx_ssl' from 'nginx'.
- Sites and server was auto-verified... Green lights on these tasks
- Edited 'transitionnetwork.org' site and set Encryption to 'Enabled'. This leave question of encryption key, and per Aegir: Using SSL - Notes for Nginx users (foot of page), I left it set to transitionnetwork.org hopeing this will pick up the one Chris added, or allow us to replace this auto-generated one. I set the SSL domain to 'transitionnetwork.org' -- is this correct?
- Site reverified itself, now says "This site is using the SSL certificate and SSL key located at config/ssl.d/transitionnetwork.org/openssl.crt and config/ssl.d/transitionnetwork.org/openssl.key." -- Chris, is this where you put the keys? Can you put them there?
More on this tomorrow.
comment:39 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.8
- Total Hours changed from 13.74 to 14.54
I have symlinked to the cert and key in /etc/ssl/:
cd /data/disk/tn/config/ssl.d/transitionnetwork.org mkdir bak mv *.* bak/ ln -s /etc/ssl/transitionnetwork.org/transitionnetwork.org.key openssl.key ln -s /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem openssl.crt
Regarding the SSL domain, I guess it should be set to 'www.transitionnetwork.org'.
When we do the DNS switch I think we should also edit the server_name in the files in /var/aegir/config/server_master/nginx/vhost.d/, but this might be best done via the web interface?
When requesting the front page at http://newlive.puffin.webarch.net/ these files are 404's:
- http://www.transitionnetwork.org/sites/transitionnetwork.org/files/uploaded/u7/rp.jpg
- http://www.transitionnetwork.org/sites/transitionnetwork.org/files/uploaded/u7/tc.jpg
- http://www.transitionnetwork.org/sites/transitionnetwork.org/files/uploaded/u7/tnmov.jpg
- http://www.transitionnetwork.org/sites/transitionnetwork.org/files/uploaded/u7/tt.jpg
We either need to make them appear at their existing (old) address' or add a redirect from /sites/default/ to /sites/transitionnetwork.org/, I'm not sure where in the Nginx configuration to add this without it being overwritten, perhaps we should do it with a symlink?
- http://www.transitionnetwork.org/sites/default/files/uploaded/u7/rp.jpg
- http://www.transitionnetwork.org/sites/default/files/uploaded/u7/tc.jpg
- http://www.transitionnetwork.org/sites/default/files/uploaded/u7/tnmov.jpg
- http://www.transitionnetwork.org/sites/default/files/uploaded/u7/tt.jpg
Next I'll restrict SSH access as flagged up by Jim and then I want to revisit the wiki:LoadTimes page -- testing the new server vs the old one it doesn't currently seem quicker via pingdom:
- http://tools.pingdom.com/fpt/#!/SqZPGMH7/http://www.transitionnetwork.org -- No, of reqs: 35, Total size: 1.1 MB, Total time: 656 ms
- http://tools.pingdom.com/fpt/#!/cOgL7T7O9/http://newlive.puffin.webarch.net -- No, of reqs: 27, Total size: 1.6 MB, Total time: 4.49 s
comment:40 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.4
- Total Hours changed from 14.54 to 14.94
I have added new users accounts for myself and Jim on puffin, copied ssh public keys from quince, added us to the sudo group and I have changed these lines in /etc/ssh/sshd_config:
PermitRootLogin no AllowGroups sudo
So now only users in the sudo group can login via SSH.
I have also edited /etc/sudoers and set it so that members of the sudo group don't need to type a password to use sudo:
%sudo ALL = NOPASSWD: ALL
I haven't changed the SSH port (if Jim is keen on this it could be changed) and I haven't restarted ssh -- these changes will only take effect after a ssh restart -- Jim can you check that you can ssh in and sudo as user jim first?
comment:41 Changed 4 years ago by chris
I have created a separate ticket for the 502 bad gateway errors, ticket:483 -- I think this issue is probably a blocker.
comment:42 follow-up: ↓ 44 Changed 4 years ago by jim
Chris said: "When requesting the front page at http://newlive.puffin.webarch.net/ these files are 404's"
Some paths are absolute, so they will fail until the DNS switch happens... The actual path (not including the domain part) is correct on the new server, but not on the old one.
e.g. Changing http://www.transitionnetwork.org/sites/default/files/uploaded/u7/rp.jpg to
http://newlive.puffin.webarch.net/sites/default/files/uploaded/u7/rp.jpg works fine.
Hence this issue will not be fixed now, but will go away after the DNS switch.
comment:43 Changed 4 years ago by jim
(to be clear: no redirects needed)
comment:44 in reply to: ↑ 42 Changed 4 years ago by chris
Replying to jim:
his issue will not be fixed now, but will go away after the DNS switch.
Hopefully that's true for all the internal links but what about external ones -- if anyone had linked to a file on the site from another Transition site (plus search engines etc) -- I think a Nginx redirect will still be needed, I'm just not sure where to put it so that BOA doesn't overwrote it.
comment:45 follow-up: ↓ 46 Changed 4 years ago by jim
It is true for internal links, honest!.
As for external, your and Ed's call... People shouldn't be linking directly to files IMHO, but the page that contains them, since they are likely to be updated at the drop of a hat anyway. I say: "tough titties to cross-linkers" (but that's because I'm acting all badass).
And if the redirect is needed, you should check documentation first -- I'm not sure about setting up a redirect in the 'BOA way' properly, though there are plenty of places it can be added safely -- just need to make sure it won't confuse Aegir.
comment:46 in reply to: ↑ 45 Changed 4 years ago by chris
comment:47 Changed 4 years ago by jim
Why not?
Because! It makes this job harder ;-) Seriously though, if you/Ed feel the need go for it, just research the best way first to ensure Aegir isn't confused.
comment:48 follow-up: ↓ 49 Changed 4 years ago by ed
What type of links are we talking about here?
- Incoming links from hundreds of sites to TN.org
- Specific file links (e.g. resources)
- Internal links within TN.org
?
comment:49 in reply to: ↑ 48 Changed 4 years ago by chris
Replying to ed:
What type of links are we talking about here?
- Incoming links from hundreds of sites to TN.org
- Specific file links (e.g. resources)
- Internal links within TN.org
?
Links to any uploaded files from places other than the Transition Network site itself.
comment:50 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 14.94 to 15.19
Using apache bench to test download time of the front page BOA on wiki:PuffinServer is almost as fast as varnish on wiki:NewLiveServer, see wiki:LoadTimes#a2013-01-28
comment:51 follow-ups: ↓ 52 ↓ 55 Changed 4 years ago by jim
Varnish vs Nginx speedcache comparison good!
However, Puffin appears to be slower for Drupal operations: My measly 1.5Gb Linode (Babylon) is actually faster in side-by-side comparison than Puffin too, which is strange given that Puffin is a quieter VPS with much more horsepower. Database operations (like postcode search on /nearby) are 1.5-2x as fast on Babylon compared to Puffin. Odd...
Perhaps this implies the issues on Puffin are deeper than thought, or that normal operation is slowed because of the SSL and other issue per #483? Compare these as needed:
- http://tn.i-jk.co.uk/ (Babylon)
- http://newlive.puffin.webarch.net/ (Puffin)
I realise Babylon is not transferring images, but there's not that many on the site and they'd be cached by the second request, so I doubt they're the cause...
I'll now fix SSL per ticket:478#comment:12 as this misconfiguration could be slowing things.
comment:52 in reply to: ↑ 51 ; follow-up: ↓ 53 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 15.19 to 15.29
Replying to jim:
Varnish vs Nginx speedcache comparison good!
It seems to show that Nginx on a VPS with 14 CPUs and 4GB of RAM is almost as fast as Varnish on a VPS with 4 CPUs and 3GB of RAM... I was hoping that Puffin/Nginx would be faster than Quince/Varnish...
However, Puffin appears to be slower for Drupal operations: My measly 1.5Gb Linode (Babylon) is actually faster in side-by-side comparison than Puffin too
How did you measure that?
Compare these as needed:
- http://tn.i-jk.co.uk/ (Babylon)
- http://newlive.puffin.webarch.net/ (Puffin)
I have added some results for you server, see wiki:LoadTimes#a2013-01-28 but it's not a fair comparison as the test I did were from another server in the same rack.
comment:53 in reply to: ↑ 52 ; follow-up: ↓ 54 Changed 4 years ago by jim
Replying to chris:
I was hoping that Puffin/Nginx would be faster than Quince/Varnish...
It will be for 'normal' Drupal page flinging ops... Testing raw throughput isn't so important (as long as it's fast enough).
The Nginx speed cache is ~20% slower than Varnish, but takes no appreciable extra memory/CPU. It can be made faster by moving the cached pages from spinning disk to SSD, or even to Redis, though that's outside my knowledge by a _long_ way.
How did you measure that?
I was testing Drupal, not the server... so I went to both sites and went to same pages. My browser (on a v fast PC) showed the page generally faster on Babylon... The main 'DB' test was http://tn.i-jk.co.uk/nearby and http://newlive.puffin.webarch.net/nearby, where I entered a couple of postcodes then hit return quickly (within 1/2 second of eachother) on Puffin then Babylon's pages. Babylon returned before puffin by about a 1/2 to 1 second.
Not super scientific, but more 'Drupally' than a simple ab I reckon. Would be good to sort out some proper Drupal benchmarks, but that's for another day.
I have added some results for you server, see wiki:LoadTimes#a2013-01-28 but it's not a fair comparison as the test I did were from another server in the same rack.
Then my Linode holds up OK! Interesting, thanks. I reckon Puffin will kick ass when we're done fiddling.
SSL fixed now.
comment:54 in reply to: ↑ 53 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 15.29 to 15.39
Replying to jim:
I was testing Drupal, not the server... so I went to both sites and went to same pages. My browser (on a v fast PC) showed the page generally faster on Babylon... The main 'DB' test was http://tn.i-jk.co.uk/nearby and http://newlive.puffin.webarch.net/nearby, where I entered a couple of postcodes then hit return quickly (within 1/2 second of eachother) on Puffin then Babylon's pages. Babylon returned before puffin by about a 1/2 to 1 second.
Thanks, I'll write a script to test with lots of different post codes.
SSL fixed now.
Thanks.
comment:55 in reply to: ↑ 51 ; follow-up: ↓ 56 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 15.39 to 16.39
Replying to jim:
Database operations (like postcode search on /nearby) are 1.5-2x as fast on Babylon compared to Puffin. Odd...
There is a big difference between quince and puffin -- I have written the following script to test the nearby search facility I incremented the PO_NUMBER to ensure that no two identical requests were made each time the script was run (don't want cached pages), these are the results from the second run:
newlive.puffin.webarch.net start: Thu Jan 31 21:00:46 GMT 2013 newlive.puffin.webarch.net end: Thu Jan 31 21:02:48 GMT 2013 www.transitionnetwork.org start: Thu Jan 31 21:02:48 GMT 2013 www.transitionnetwork.org end: Thu Jan 31 21:04:08 GMT 2013
Puffin took 2 min 8 seconds and quince took 1 min 20 seconds.
I ran it for a third time and triggered a 502 on puffin, it took 2 min 3 seconds on puffin and 1 min 20 seconds on quince:
newlive.puffin.webarch.net start: Thu Jan 31 21:08:43 GMT 2013 Error code response from server newlive.puffin.webarch.net end: Thu Jan 31 21:10:47 GMT 2013 www.transitionnetwork.org start: Thu Jan 31 21:10:47 GMT 2013 www.transitionnetwork.org end: Thu Jan 31 21:12:07 GMT 2013
I then ran it for a 4th time:
newlive.puffin.webarch.net start: Thu Jan 31 21:22:25 GMT 2013 newlive.puffin.webarch.net end: Thu Jan 31 21:24:28 GMT 2013 www.transitionnetwork.org start: Thu Jan 31 21:24:28 GMT 2013 www.transitionnetwork.org end: Thu Jan 31 21:25:52 GMT 2013
Puffin took 2 mins and 2 seconds and quince took 1 min 24 seconds.
From this test I conclude that the way puffin is currently configured (BOA / Nginx) it is slower than quince (Varnish / Apache).
This is the script if anyone else wants to try it (change the PO_NUMBER variable, I have used 1, 2, 3 and 4 so far):
#!/bin/bash
DOMAINS="newlive.puffin.webarch.net www.transitionnetwork.org"
for DOMAIN in $DOMAINS ; do
echo "$DOMAIN start: $(date)"
PO_LETTERS="AB AL B BA BB BD BH BL BN BR BS BT CA CB CF CH CM CO CR CT CV CW DA DD DE DG DH DL DN DT DY E EC EH EN EX FK FY G GL GU HA HD HG HP HR HS HU HX IG IP IV KA KT KW KY L LA LD LE LL LN LS LU M ME MK ML N NE NG NN NP NR NW OL OX PA PE PH PL PO PR RG RH RM S SA SE SG SK SL SM SN SO SP SR SS ST SW SY TA TD TF TN TQ TR TS TW UB W WA WC WD WF WN WR WS WV YO ZE"
for PO_LETTER in $PO_LETTERS ; do
PO_NUMBER="3"
PO="$PO_LETTER$PO_NUMBER"
wget -q --user-agent="Mozilla/5.0 (TN Speed Test)" "http://$DOMAIN/nearby?distance[postal_code]=$PO&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile" -O /dev/null
# catch http errors
STATUS=$?
if [ $STATUS == "8" ] ; then
echo "Error code response from server"
fi
done
echo "$DOMAIN end: $(date)"
done
I'll follow up about the 502 on ticket:483
comment:56 in reply to: ↑ 55 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 16.39 to 16.64
Replying to chris:
From this test I conclude that the way puffin is currently configured (BOA / Nginx) it is slower than quince (Varnish / Apache).
I expect (this is a essentially a guess) that the reason for this is that on quince we have a lot more memory allocated to serving each request -- if there was a huge traffic spike quince would really struggle where as puffin would probably just take it in it stride.
If I'm correct then it suggests we should consider doing one of three things:
- Keep BOA the way it's set up out of the box and reduce the resources puffin has (as it's not going to make good use of them), drop it from 4GB RAM to 2GB and use the 2GB for another virtual machine doing something else (Wordpress farm?).
- Keep BOA and look at how settings on puffin can be tweaked to increase it's speed by making more use of the resources it has.
- Consider keeping the Apache / Varnish setup -- we could do a more accurate comparison of the quince setup vs. the puffin setup by copying the quince virtual machine over to the same hardware that puffin is on, setting up both virtual machines to have the same amount of RAM and then testing them side by side.
I'd be happy with any of the three approaches above, Jim what do you think, do you have any other suggestions?
Some work has been done on the wiki page:
comment:57 follow-up: ↓ 58 Changed 4 years ago by jim
Personally, I think 2GB of memory for the server would be plenty for now. However, I'd resist dropping Aegir because that give TN so much flexibility to run LOTS of things from Drupal land... And the talk of Community tools, an online store, event management and other things that can be installed in a minute or so with a Makefile, and managed through a life cycle. And the instant Dev/Stage/Test? as needed.
But I don't think we've quite got to the place where we know it's BOA causing every problem. It seems to me we have 2 things happening:
- 502s over on #483
- Slow performance.
I think 2 is the biggest chin scratcher... and I think it might point to a misconigured VM... The reason is I ran your postcode script locally on Babylon, changing DOMAINS="tn.i-jk.co.uk" -- the results are shocking:
babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:08:35 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:09:59 GMT 2013
run 1 = 1m 24s
babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:12:09 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:12:13 GMT 2013 babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:13:29 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:13:32 GMT 2013
runs two and three: 3-4 seconds! Redis + MySQL + Nginx + PHP-FPM is FAST...
I couldn't believe it, so here's a section of the access log from Babylon as proof the requests were coming in.:
babylon:~# tail -n100 /var/log/nginx/access.log "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:30 +0000] "GET /nearby?distance[postal_code]=E3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 48708 226 49415 "-" "Mozilla/5.0 (TN Speed Test)" 0.027 "-" "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:30 +0000] "GET /nearby?distance[postal_code]=EC3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 47936 227 48644 "-" "Mozilla/5.0 (TN Speed Test)" 0.026 "-" "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:30 +0000] "GET /nearby?distance[postal_code]=EH3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 45342 227 46050 "-" "Mozilla/5.0 (TN Speed Test)" 0.025 "-" "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:30 +0000] "GET /nearby? ... snip ... "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:32 +0000] "GET /nearby?distance[postal_code]=WV3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 47199 227 47907 "-" "Mozilla/5.0 (TN Speed Test)" 0.031 "-" "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:32 +0000] "GET /nearby?distance[postal_code]=YO3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 51965 227 52673 "-" "Mozilla/5.0 (TN Speed Test)" 0.026 "-" "178.79.169.201" tn.i-jk.co.uk [01/Feb/2013:10:13:32 +0000] "GET /nearby?distance[postal_code]=ZE3&distance[country]=uk&distance[search_distance]=50&distance[search_units]=mile HTTP/1.0" 200 51965 227 52673 "-" "Mozilla/5.0 (TN Speed Test)" 0.026 "-"
I ran it again on my local machine to add in the internet latency:
jimk@monolith:~$ ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:20:28 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:21:01 GMT 2013 jimk@monolith:~$ ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:22:59 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:23:38 GMT 2013
33 seconds and 39 seconds, cache still warm from running on Babylon directly.
Conclusion:
- Puffin is NOT configured properly, either at a software, hardware or VM level. What's the IO like on it? The memory throughput? Are there any good benchmarks we can run here?
- We should it reboot to ensure all the things are bedded in and running as expected -- Redis and Speed Cache are what making my Babylon server so fast, and they should be running on Puffin. I've done little to the BOA side on mine, mostly mucking around with mail stuff.
- After reboot, perhaps you fancy trying the BOA performance tuning script 'BOND.sh'?: http://drupalcode.org/project/barracuda.git/blob/HEAD:/aegir/tools/BOND.sh.txt -- It will tune the BOA system for available memory and CPU -- might be worth running to see what it recommends, and comparing that to what you've seen so far.
I just can't believe my Linode, whose parent hardware is running a bunch of busy VMs, plus the VM has a 15 of its own Drupal sites, and is on mediocre-to-good Xeon platform (see http://forum.linode.com/viewtopic.php?t=4326) that's a probably couple of years old at least is outperforming both Quince and Puffin.
There must be something not quite right on Puffin, don't you think Chris? I wouldn't have recommended BOA if I didn't think it was fast, but on Puffin, it's just not. Odd.
comment:58 in reply to: ↑ 57 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 16.64 to 17.14
Replying to jim:
Personally, I think 2GB of memory for the server would be plenty for now.
For a BOA server I agree.
However, I'd resist dropping Aegir because that give TN so much flexibility to run LOTS of things from Drupal land... And the talk of Community tools, an online store, event management and other things that can be installed in a minute or so with a Makefile, and managed through a life cycle. And the instant Dev/Stage/Test? as needed.
Fair enough.
But I don't think we've quite got to the place where we know it's BOA causing every problem. It seems to me we have 2 things happening:
- 502s over on #483
- Slow performance.
I think 2 is the biggest chin scratcher... and I think it might point to a misconigured VM... The reason is I ran your postcode script locally on Babylon, changing DOMAINS="tn.i-jk.co.uk" -- the results are shocking:
babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:08:35 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:09:59 GMT 2013run 1 = 1m 24s
That is the same speed as quince. (1m 20s and 1m 24s see ticket:466#comment:55).
babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:12:09 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:12:13 GMT 2013 babylon:~# ./postcode.sh tn.i-jk.co.uk start: Fri Feb 1 10:13:29 GMT 2013 tn.i-jk.co.uk end: Fri Feb 1 10:13:32 GMT 2013runs two and three: 3-4 seconds! Redis + MySQL + Nginx + PHP-FPM is FAST...
That is because you didn't increment the PO_NUMBER variable? If you didn't increment this number then it will have just been testing the cache rather than page generation.
Conclusion:
- Puffin is NOT configured properly, either at a software, hardware or VM level. What's the IO like on it? The memory throughput? Are there any good benchmarks we can run here?
There are some munin stats from the server the VM is on:
https://ecodissident.net/munin/webarch.net/red.webarch.net/index.html
I agree that testing IO and memory throughput is a good idea.
This is the output of iostat -x:
Linux 2.6.32-5-xen-amd64 (puffin.webarch.net) 02/01/13 _x86_64_ (14 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.22 0.00 0.28 0.07 0.01 99.42
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
xvdap2 0.00 5.29 0.19 9.44 8.97 189.97 20.67 0.48 49.78 1.42 1.37
xvdap1 0.00 0.00 0.00 0.00 0.00 0.00 9.35 0.00 8.34 1.29 0.00
The output on quince:
Linux 2.6.32-5-xen-amd64 (quince.webarch.net) 01/02/13 _x86_64_ (4 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
6.87 0.04 1.94 4.95 6.37 79.82
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
xvdap2 0.01 57.52 5.36 51.59 171.43 872.98 18.34 0.10 4.33 0.20 1.12
xvdap1 0.00 0.02 0.10 0.05 0.82 0.55 9.31 0.01 63.39 13.10 0.19
xvdap3 0.00 21.12 2.65 19.16 62.08 322.28 17.62 0.06 2.79 4.13 9.00
And hdparm -t /dev/xvda2 on puffin, which has 8 SAS disks running in RAID 10:
/dev/xvda2: Timing buffered disk reads: 184 MB in 3.04 seconds = 60.56 MB/sec
Quince has a root partition on a pair of mirrored SATA disks:
/dev/xvda2: Timing buffered disk reads: 196 MB in 3.02 seconds = 65.00 MB/sec
And the database is on a pair of mirrored SCSI disks:
/dev/xvda3: Timing buffered disk reads: 312 MB in 3.01 seconds = 103.58 MB/sec
I'll look for some other tests we can run.
- We should it reboot to ensure all the things are bedded in and running as expected -- Redis and Speed Cache are what making my Babylon server so fast, and they should be running on Puffin. I've done little to the BOA side on mine, mostly mucking around with mail stuff.
OK, it's been rebooted.
- After reboot, perhaps you fancy trying the BOA performance tuning script 'BOND.sh'?: http://drupalcode.org/project/barracuda.git/blob/HEAD:/aegir/tools/BOND.sh.txt -- It will tune the BOA system for available memory and CPU -- might be worth running to see what it recommends, and comparing that to what you've seen so far.
OK.
I just can't believe my Linode, whose parent hardware is running a bunch of busy VMs, plus the VM has a 15 of its own Drupal sites, and is on mediocre-to-good Xeon platform (see http://forum.linode.com/viewtopic.php?t=4326) that's a probably couple of years old at least is outperforming both Quince and Puffin.
It's matching quince speed wise.
There must be something not quite right on Puffin, don't you think Chris? I wouldn't have recommended BOA if I didn't think it was fast, but on Puffin, it's just not. Odd.
Agreed. More testing needed.
comment:59 Changed 4 years ago by jim
OK I've done some more research into this...
Running the postcode.sh Chris made locally on Quince returns in 3-4 seconds. So the performance is good. However I think the cause of slowness might be a networking issue... I did some pings to compare things:
On Puffin, Ping localhost
puffin:~# ping localhost PING localhost (127.0.0.1) 56(84) bytes of data. 64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.107 ms 64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.097 ms 64 bytes from localhost (127.0.0.1): icmp_req=3 ttl=64 time=0.100 ms 64 bytes from localhost (127.0.0.1): icmp_req=4 ttl=64 time=0.088 ms 64 bytes from localhost (127.0.0.1): icmp_req=5 ttl=64 time=0.102 ms 64 bytes from localhost (127.0.0.1): icmp_req=6 ttl=64 time=0.105 ms 64 bytes from localhost (127.0.0.1): icmp_req=7 ttl=64 time=0.106 ms ^C --- localhost ping statistics --- 7 packets transmitted, 7 received, 0% packet loss, time 5996ms rtt min/avg/max/mdev = 0.088/0.100/0.107/0.013 ms
... 0.1ms for a localhost ping is slow. Comparison:
On Quince, ping localhost
jim@quince:~$ ping localhost PING localhost (127.0.0.1) 56(84) bytes of data. [snip] --- localhost ping statistics --- 7 packets transmitted, 7 received, 0% packet loss, time 5998ms rtt min/avg/max/mdev = 0.032/0.048/0.055/0.008 ms
... Only 0.048ms average. And on my machine: rtt min/avg/max/mdev = 0.037/0.050/0.054/0.007 ms = 0.05ms.
Indeed, pinging:
- Quince ping puffin.webarch.net = 1.283 ms average
- Puffin ping quince.webarch.net = 0.584 ms average
Now if MySQL or NginX are using localhost, 127.0.0.1 or puffin.webarch.net, all of which are over twice as slow on Puffin vs Quince, I wonder if they latencies are adding up? Haven't run a longer test, but maybe some packets are being lost?
Anyway this seems like a good place to investigate, doesn't it Chris? Could the VM or Hypervisor have a wonky network setup? Or maybe there's a network hardware issue with the host Puffin lives on?
Would be great to run some actual benchmarks on each subsystem to establish any slowness:
- Network (I've kinda done a simple version of this)
- Disk IO
- Memory IO
- MySQL throughput
- anything else?
comment:60 Changed 4 years ago by jim
Ignore the last bit! You posted at the same time as me...
Apols for missing the cache thing -- I was expecting caching to make a huge difference, but didn't realise the PO_NUMBER was being incremented... Now I re-read your post I see what I missed.
But for me the network latency on Puffin could explain a few things. I'm in for another hour tonight Chris, shout if you need me to investigate/test/do anything.
comment:61 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 17.14 to 18.14
Interesting, I have been using sockets rather than the loopback address (the change for php-fpm / nginx is recorded in ticket:357#comment:16), so on wiki:PenguinServer php-fpm listens for connections using a socket, we have this in /etc/php5/fpm/pool.d/www.conf:
;listen = 127.0.0.1:9000 listen = /var/run/php5-fpm/phpfpm.sock
And in the Penguin Nginx config:
fastcgi_pass unix:/var/run/php5-fpm/phpfpm.sock;
Where as on wiki:PuffinServer we have this in /opt//local/etc/php53-fpm.conf:
listen = 127.0.0.1:9090
And I can't work out which Nginx config file the site on Puffin uses, but there are a couple of values for fastcgi_pass:
fastcgi_pass 127.0.0.1:9090; fastcgi_pass phpcgi;
I theory sockets should be quicker as they don't have the TCP overhead but after searching for some good article on this tonight I haven't actually found any.
We could try switching Puffin to use a socket?
I don't understand why pinging the loopback address is slower on puffin than quince, especially given that there is far more processing power on puffin (14 AMD 2000Mhz cores on puffin vs 4 Intel 2000Mhz cores on quince).
comment:62 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.0
- Total Hours changed from 18.14 to 19.14
On puffin the results from iperf look like this:
iperf -c localhost ------------------------------------------------------------ Client connecting to localhost, TCP port 5001 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 3] local 127.0.0.1 port 3165 connected with 127.0.0.1 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 7.8 sec 1.84 GBytes 2.02 Gbits/sec
Comparing puffin and quince with 100000 pings, first puffin:
ping -f localhost -c 100000 PING localhost (127.0.0.1) 56(84) bytes of data. --- localhost ping statistics --- 100000 packets transmitted, 100000 received, 0% packet loss, time 8198ms rtt min/avg/max/mdev = 0.012/0.013/0.680/0.006 ms, ipg/ewma 0.081/0.014 ms
And then quince:
ping -f localhost -c 100000 PING localhost (127.0.0.1) 56(84) bytes of data. --- localhost ping statistics --- 100000 packets transmitted, 100000 received, 0% packet loss, time 8837ms rtt min/avg/max/mdev = 0.009/0.011/26.778/0.120 ms, pipe 2, ipg/ewma 0.088/0.009 ms
Testing disk speed using iozone, quince:
iozone -s 2g -r 64 -i 0 -i 1 -t 1
Iozone: Performance Test of File I/O
Version $Revision: 3.308 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
Run began: Sun Feb 3 22:49:03 2013
File size set to 2097152 KB
Record Size 64 KB
Command line used: iozone -s 2g -r 64 -i 0 -i 1 -t 1
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 2097152 Kbyte file in 64 Kbyte records
Children see throughput for 1 initial writers = 12439.18 KB/sec
Parent sees throughput for 1 initial writers = 8739.77 KB/sec
Min throughput per process = 12439.18 KB/sec
Max throughput per process = 12439.18 KB/sec
Avg throughput per process = 12439.18 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 rewriters = 7506.82 KB/sec
Parent sees throughput for 1 rewriters = 6485.36 KB/sec
Min throughput per process = 7506.82 KB/sec
Max throughput per process = 7506.82 KB/sec
Avg throughput per process = 7506.82 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 readers = 35895.11 KB/sec
Parent sees throughput for 1 readers = 35892.10 KB/sec
Min throughput per process = 35895.11 KB/sec
Max throughput per process = 35895.11 KB/sec
Avg throughput per process = 35895.11 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 re-readers = 20054.65 KB/sec
Parent sees throughput for 1 re-readers = 20054.07 KB/sec
Min throughput per process = 20054.65 KB/sec
Max throughput per process = 20054.65 KB/sec
Avg throughput per process = 20054.65 KB/sec
Min xfer = 2097152.00 KB
iozone test complete.
And on puffin:
iozone -s 2g -r 64 -i 0 -i 1 -t 1
Iozone: Performance Test of File I/O
Version $Revision: 3.308 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
Run began: Sun Feb 3 22:54:31 2013
File size set to 2097152 KB
Record Size 64 KB
Command line used: iozone -s 2g -r 64 -i 0 -i 1 -t 1
Output is in Kbytes/sec
Time Resolution = 0.000002 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 2097152 Kbyte file in 64 Kbyte records
Children see throughput for 1 initial writers = 21821.23 KB/sec
Parent sees throughput for 1 initial writers = 16522.62 KB/sec
Min throughput per process = 21821.23 KB/sec
Max throughput per process = 21821.23 KB/sec
Avg throughput per process = 21821.23 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 rewriters = 64297.46 KB/sec
Parent sees throughput for 1 rewriters = 44621.53 KB/sec
Min throughput per process = 64297.46 KB/sec
Max throughput per process = 64297.46 KB/sec
Avg throughput per process = 64297.46 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 readers = 341842.59 KB/sec
Parent sees throughput for 1 readers = 341666.67 KB/sec
Min throughput per process = 341842.59 KB/sec
Max throughput per process = 341842.59 KB/sec
Avg throughput per process = 341842.59 KB/sec
Min xfer = 2097152.00 KB
Children see throughput for 1 re-readers = 815984.56 KB/sec
Parent sees throughput for 1 re-readers = 815463.20 KB/sec
Min throughput per process = 815984.56 KB/sec
Max throughput per process = 815984.56 KB/sec
Avg throughput per process = 815984.56 KB/sec
Min xfer = 2097152.00 KB
iozone test complete.
I really don't know where to go with this.
comment:63 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 19.14 to 19.64
Time recorded on this comment is for emails on the ttech list and wiki page updates.
comment:64 follow-up: ↓ 75 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.7
- Total Hours changed from 19.64 to 20.34
Installing a munin client on puffin.
Add the server IP to /etc/munin/munin-node.conf:
# https://ecodissident.net/munin/ allow ^93\.95\.226\.170$
Enable some plugins, nginx:
cd /etc/munin/plugins ln -s /usr/share/munin/plugins/nginx_status ln -s /usr/share/munin/plugins/nginx_request
Some additional configuration is needed for ngins as the munin plugin isn't getting at stats at http://localhost/status -- these plugins were tested like this:
munin-run nginx_request request.value U munin-run nginx_status total.value U reading.value U writing.value U waiting.value U
For php-fpm:
cd /usr/share/munin/plugins git clone git://github.com/tjstein/php5-fpm-munin-plugins.git chmod +x php5-fpm-munin-plugins/phpfpm_* cd /etc/munin/plugins
Again these plugins need some work:
munin-run phpfpm_average php_average.value awk: fatal: division by zero attempted munin-run phpfpm_connections accepted.value U munin-run phpfpm_memory ram.value Integer munin-run phpfpm_processes php_processes.value 0 munin-run phpfpm_status idle.value U active.value U total.value U
Mysql:
cd /etc/munin/plugins ln -s /usr/share/munin/plugins/mysql_bytes ln -s /usr/share/munin/plugins/mysql_queries ln -s /usr/share/munin/plugins/mysql_slowqueries ln -s /usr/share/munin/plugins/mysql_threads
All of these work by default.
Memory usage:
ln -s /usr/share/munin/plugins/multips_memory
And this one need this config in /etc/munin/plugin-conf.d/munin-node:
[multips_memory] env.names nginx php-fpm mysqld redis-server munin-node
The virtual machines never generate stats for this plugin for some reason:
cd /etc/munin/plugins rm iostat
However no stats were geing generated on the munin server, so in case it was a firewall issue the following was added to /etc/csf/csf.allow:
tcp:in:d=4949:s=93.95.226.170 # ecodissident.net for munin
And also this was run:
iptables -A INPUT -p tcp -s 93.95.226.170 --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPT
Based on suggestions here.
This appears to have done the trick, stats are being generated here:
So my tomorrow we should have some interesting graphs, TODO, sort out nginx and php-fpm stats.
Looking at the running processes I noticed that pure-ftpd is running -- Jim I though this was disabled?
comment:65 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.54
- Total Hours changed from 20.34 to 20.88
This comment is to cover the time just spent on the phone to Jim and Ed discussing the server migration.
comment:66 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.5
- Total Hours changed from 20.88 to 21.38
Picking up on the Munin work done last night on ticket:466#comment:64
Enabling Nginx status for localhost requests, after some trial and error this has been done by creating /etc/nginx/conf.d/localhost.conf with the following in it:
server {
listen 80;
server_name localhost;
# for nginx status
# chris
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
This comes before the BOA config in /etc/nginx/conf.d/aegir.conf however I need to check that there is other BOA config that need copying into /etc/nginx/conf.d/localhost.conf.
comment:67 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 1.25
- Total Hours changed from 21.38 to 22.63
We now have some nginx stats:
The only other reference to localhost I can find in the BOA nginx configuration is in /var/aegir/config/server_master/nginx/pre.d/nginx_wild_ssl.conf and it seem that this configuration isn't loaded due to the new config file, and this was causing a default "Welcome to nginx!" page to be served here:
However not at the IP address:
Chive, http://www.chive-project.com/ was being served.
I'm not sure if this will have any unintended consequences but in /etc/nginx/conf.d the following two files were created, with file names to ensure they are loaded before the aegir.conf symlink, 00_puffin.conf to serve the default page for the server, containing:
server {
listen 80 default_server;
server_name puffin.transitionnetwork.org puffin.webarch.net 81.95.52.111;
access_log /var/log/nginx/puffin.access.log;
error_log /var/log/nginx/puffin.error.log crit;
root /var/www/puffin;
index index.html;
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
location ~ .~$ {
return 403;
}
}
server {
listen 443 default_server;
server_name puffin.transitionnetwork.org puffin.webarch.net 81.95.52.111;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/puffin_ssl.access.log;
error_log /var/log/nginx/puffin_ssl.error.log crit;
root /var/www/puffin;
index index.html;
}
And for the requests for nginx status and php-fpm status from munin plugins, 01_localhost.conf:
server {
listen 127.0.0.1:80;
server_name localhost;
# for nginx status
# chris
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
# for munin, see
# http://tjstein.com/2010/09/php-fpm-plugin-suite-for-munin/
location ~ ^/(status|ping)$ {
fastcgi_pass 127.0.0.1:9090;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
include fastcgi_params;
access_log off;
allow 127.0.0.1;
deny all;
}
}
To enable the php-fpm status the following was uncommented in /opt/local/etc/php53-fpm.conf:
pm.status_path = /status ping.path = /ping
The munin php-fpm plugins needed editing as for penguin to hard code the process name, see ticket:470#comment:41
We should now start to get some stats for nginx and php-fpm here:
https://ecodissident.net/munin/webarch.net/puffin.webarch.net/
My TODO:
- Look at what other Munin plugins should be enabled
- Touch the file to ensure the changes to /opt/local/etc/php53-fpm.conf are not overwritten
- Check what tasks in the /var/xdrago/clear.sh cron job are safe to enable
- Do a final load test to see if the 502 issue has been totally solved, and if it has close trac:ticket/483
comment:68 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.8
- Total Hours changed from 22.63 to 23.43
The following needed adding to /etc/nginx/conf.d/00_puffin.conf to enable the nginx_request munin plugin:
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow 81.95.52.103;
deny all;
}
I think the munin stats are now adequate, to get some more interesting graphs and to double check that the 502 issue is solved a recursive wget is being run from penguin against puffin.
https://ecodissident.net/munin/webarch.net/puffin.webarch.net/
Some updates have been done on the wiki:PuffinServer page.
Following this advice:
Note: Barracuda will *not* overwrite your changes made to php.ini or php-fpm.conf on the next upgrade if you will touch the empty control file:
$ touch /opt/etc/custom.php.ini
This file was touched.
Looking at /var/xdrago/clear.sh it really doesn't do anything we need, it does things like clobbering log files but we are not so short of space that this is an issue so I don't think the root crontab needs /var/xdrago/clear.sh uncommenting, however after the next BOA upgrade we should check to see that it's not been reinstated.
I think my work on puffin is just about done:
- ticket:483 - has been downgraded to trivial rather than a blocker, all that is left is for Jim to raise a ticket upstream about the issue
There are these outstanding tickets of Jim's:
- ticket:479 Transfer live transitionnetwork.org site to puffin
- ticket:480 Transfer news.transitionnetwork.org to puffin
The migration date looks like it'll be either Monday 11th or Monday 18th Feb 2013.
comment:69 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 23.43 to 23.53
One thing that has surprised me from the munin stats is that despite a lot of tweaks to the quince mysql settings over the years it is only using around 450M of RAM, see:
The mysql on puffin is using 1.2GB of RAM:
comment:70 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.6
- Total Hours changed from 23.53 to 24.13
The problem with Munin stats for nginx and php-fpm raised at ticket:466#comment:64 and then "solved" on ticket:466#comment:66 ticket:466#comment:67 and ticket:466#comment:68 by creating /etc/nginx/conf.d/00_puffin.conf containing:
server {
listen 80 default_server;
server_name puffin.transitionnetwork.org puffin.webarch.net 81.95.52.111;
access_log /var/log/nginx/puffin.access.log;
error_log /var/log/nginx/puffin.error.log crit;
root /var/www/puffin;
index index.html;
# for nginx status
# chris
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow 81.95.52.103;
deny all;
}
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
location ~ .~$ {
return 403;
}
}
server {
listen 443 default_server;
server_name puffin.transitionnetwork.org puffin.webarch.net 81.95.52.111;
ssl on;
ssl_certificate /etc/ssl/transitionnetwork.org/transitionnetwork.org.chained.pem;
ssl_certificate_key /etc/ssl/transitionnetwork.org/transitionnetwork.org.key;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/puffin_ssl.access.log;
error_log /var/log/nginx/puffin_ssl.error.log crit;
root /var/www/puffin;
index index.html;
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
location ~ .~$ {
return 403;
}
}
and /etc/nginx/conf.d/01_localhost.conf containing:
server {
listen 127.0.0.1:80;
server_name localhost;
# for nginx status
# chris
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
# for munin, see
# http://tjstein.com/2010/09/php-fpm-plugin-suite-for-munin/
location ~ ^/(status|ping)$ {
fastcgi_pass 127.0.0.1:9090;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
include fastcgi_params;
access_log off;
allow 127.0.0.1;
deny all;
}
}
Broke the BOA configuration, sorry :-(
Jim fixed BOA by moving 00_puffin.conf and 01_localhost.conf into /root/ and restarting Nginx but we now have the issue of incomplete Nginx and php-fpm munin stats:
- https://ecodissident.net/munin/webarch.net/puffin.webarch.net/index.html#nginx
- https://ecodissident.net/munin/webarch.net/puffin.webarch.net/index.html#php
There is this suggestion from Jim, however the nginx config file that he suggested should contain the code for /nginx_status does already contain it, /var/aegir/config/server_master/nginx/vhost.d/cgp.master.puffin.webarch.net:
location /nginx_status {
stub_status on;
access_log off;
allow 81.95.52.103;
allow 127.0.0.1;
deny all;
}
I think the problem probably is that this isn't the default Nginx server, perhaps this one is in /var/aegir/config/server_master/nginx.conf:
server {
limit_conn gulag 32; # like mod_evasive - this allows max 32 simultaneous connections from one IP address
listen *:80;
server_name _;
location / {
root /var/www/nginx-default;
index index.html index.htm;
}
}
So the following was added to that file:
## chris
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow 81.95.52.103;
deny all;
}
location ~ ^/(status|ping)$ {
fastcgi_pass 127.0.0.1:9090;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_intercept_errors on;
include fastcgi_params;
access_log off;
allow 127.0.0.1;
deny all;
}
And the munin plugins were tested:
munin-run nginx_request request.value 2 munin-run nginx_status total.value 2 reading.value 0 writing.value 1 waiting.value 1 munin-run phpfpm_average php_average.value 107542528 munin-run phpfpm_connections accepted.value 203 munin-run phpfpm_memory ram.value 222187520 munin-run phpfpm_processes php_processes.value 3
So as long as the next upgrade of BOA doesn't clobber /var/aegir/config/server_master/nginx.conf this issue is solved. If that file does get clobbered then perhaps this is an issue that need raising as a ticket elsewhere (on which projects ticketing system, Barracuda, Octopus, Aegir or BOA?).
comment:71 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 24.13 to 24.23
DNS update, this is the old zone file:
* 3600 IN A 81.95.52.88 *.newdev 3600 IN A 81.95.52.103 2010.archive 3600 IN A 81.95.52.111 2011.archive 3600 IN A 81.95.52.111 @ 3600 IN A 81.95.52.88 lists 3600 IN A 212.113.133.235 mail 3600 IN A 212.113.133.235 newdev 3600 IN A 81.95.52.103 penguin 3600 IN A 81.95.52.111 puffin 3600 IN A 81.95.52.103 quince 3600 IN A 81.95.52.88 redirects 3600 IN A 81.95.52.111 static 3600 IN A 81.95.52.111 stats 3600 IN A 81.95.52.111 tech 3600 IN A 81.95.52.111 trac 3600 IN A 81.95.52.111 wiki 3600 IN A 81.95.52.111 www 3600 IN A 81.95.52.88 www.wiki 3600 IN A 81.95.52.111 9c086db58fc42046983c43dcd2e6dea0 10800 IN CNAME 8d511978467880c55b9a90e56ff8eb75b8eb813a.comodoca.com. @ 3600 IN MX 0 mx1.spamfiltering.com. @ 3600 IN MX 5 mx2.spamfiltering.com.
And this is the new one, the comodo CNAME has been omitted as it was only needed for the SSL cert generation.
* 3600 IN A 81.95.52.103 *.newdev 3600 IN A 81.95.52.103 2010.archive 3600 IN A 81.95.52.111 2011.archive 3600 IN A 81.95.52.111 @ 3600 IN A 81.95.52.103 lists 3600 IN A 212.113.133.235 mail 3600 IN A 212.113.133.235 newdev 3600 IN A 81.95.52.103 penguin 3600 IN A 81.95.52.111 puffin 3600 IN A 81.95.52.103 quince 3600 IN A 81.95.52.88 redirects 3600 IN A 81.95.52.111 static 3600 IN A 81.95.52.111 stats 3600 IN A 81.95.52.111 tech 3600 IN A 81.95.52.111 trac 3600 IN A 81.95.52.111 wiki 3600 IN A 81.95.52.111 www 3600 IN A 81.95.52.103 www.wiki 3600 IN A 81.95.52.111 @ 3600 IN MX 0 mx1.spamfiltering.com. @ 3600 IN MX 5 mx2.spamfiltering.com.
I think that *.newdev can be omitted but I have left this in for now. If Jim is happy with the above I can make it live!
comment:72 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 24.23 to 24.33
OK, confirmed with Jim in irc, the version above has been made live and the DNS servers should update within 15 mins, you can check them like this:
dig @A.DNS.GANDI.NET www.transitionnetwork.org +short dig @B.DNS.GANDI.NET www.transitionnetwork.org +short dig @C.DNS.GANDI.NET www.transitionnetwork.org +short
When 81.95.52.103 is returned rather than 81.95.52.88 then they have updated.
If you want to see the update before the DNS has updated then edit your /etc/hosts file and add this:
81.95.52.103 www.transitionnetwork.org news.transitionnetwork.org
comment:73 Changed 4 years ago by ed
- WYSIWYG editor not showing its various options
- Production bar still down the side
?
comment:74 Changed 4 years ago by jim
- now fixed, makefile updated
- As discussed, this will stay for all Developer role users.. Currently for site admins too as it's a good reminder where a user is in the world of multiple copies of the same site.
comment:75 in reply to: ↑ 64 ; follow-up: ↓ 76 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.25
- Total Hours changed from 24.33 to 24.58
Replying to chris:
However no stats were geing generated on the munin server, so in case it was a firewall issue the following was added to /etc/csf/csf.allow:
tcp:in:d=4949:s=93.95.226.170 # ecodissident.net for muninAnd also this was run:
iptables -A INPUT -p tcp -s 93.95.226.170 --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPTBased on suggestions here.
We have a new Munin server in Sheffield and the above steps were done for the additional IP address, I'll email the list with the details.
comment:76 in reply to: ↑ 75 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 24.58 to 24.68
Replying to chris:
Replying to chris:
However no stats were geing generated on the munin server, so in case it was a firewall issue the following was added to /etc/csf/csf.allow:
tcp:in:d=4949:s=93.95.226.170 # ecodissident.net for muninAnd also this was run:
iptables -A INPUT -p tcp -s 93.95.226.170 --dport 4949 -m state --state NEW,ESTABLISHED -j ACCEPTBased on suggestions here.
We have a new Munin server in Sheffield and the above steps were done for the additional IP address, I'll email the list with the details.
The above didn't d the trick, still no stats from puffin.webarch.net, I have now tried:
csf -r
comment:77 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.22
- Status changed from new to closed
- Resolution set to fixed
- Total Hours changed from 24.68 to 24.9
I have updated the wiki:PuffinServer documentation wiki:PuffinServer?action=diff&version=29&old_version=27 and moved the list of migration tickets, see wiki:PuffinServer#MigrationTickets, to the end of that page -- I'm closing this ticket, the only outstanding ticket on that list is ticket:483 Nginx 502 Bad Gateway Errors with BOA see the summary of that issue on ticket:483#comment:46.
comment:78 Changed 4 years ago by chris
- Add Hours to Ticket changed from 0.0 to 0.1
- Total Hours changed from 24.9 to 25.0
I just noticed that the server clock was out and it didn't have a network time protocol client running so I installed one:
2013-03-29 chris
* libopts25{a} ntp rdate : installed
