Upgrade Piwiki to 1.7

[chris]

A new version is available: ​http://piwik.org/blog/2012/02/new-features-in-piwik-1-7-screenshots/

[chris]

This upgrade was done at the same time as ticket:394 and the time spent on this task is recorded on that ticket.

Following the notes for the manual upgrade, ​http://piwik.org/update/ and doing it initially on kiwi, ​https://piwik.transitionnetwork.org/

cd /web/piwik.transitionnetwork.org
cp piwik/config/config.ini.php config.ini.php.2012-03-12.bak
wget http://piwik.org/latest.zip -O piwik.1.7.1.zip
mv piwik piwik.bak
unzip piwik.1.7.1.zip 
cp config.ini.php.2012-03-12.bak piwik/config/config.ini.php
chown -R www-data:www-data /web/piwik.transitionnetwork.org/piwik/themes
chown -R www-data:www-data /web/piwik.transitionnetwork.org/piwik/tmp
chown -R www-data:www-data /web/piwik.transitionnetwork.org/piwik/config
php /web/piwik.transitionnetwork.org/piwik/index.php -- "module=CoreUpdater"

On the live server the additional step to disable Piwik Tracking, ​http://piwik.org/faq/how-to/#faq_111 should be done and also the new version should be extracted to /tmp and set up before it replaces the existing install.

The live server update is still to be done but the dev site has been upgraded without issues.

[chris]

Upgrade on quince, first edit /web/stats.transitionnetwork.org/piwik/config/config.ini.php and add the following to the [Tracker] section, as per ​http://piwik.org/faq/how-to/#faq_111

record_statistics = 0

Then:

cd /web/stats.transitionnetwork.org
mkdir piwik.1.7.1
cd piwik.1.7.1
wget http://piwik.org/latest.zip -O piwik.1.7.1.zip
unzip piwik.1.7.1.zip
cd piwik
cp /web/stats.transitionnetwork.org/piwik/config/config.ini.php config/
chown -R www-data:www-data themes
chown -R www-data:www-data tmp
chown -R www-data:www-data config
cd /web/stats.transitionnetwork.org/
mv piwik piwik.bak ; mv piwik.1.7.1/piwik .
php /web/stats.transitionnetwork.org/piwik/index.php -- "module=CoreUpdater"
  PHP Fatal error:  Exception thrown without a stack frame in Unknown on line 0
 
  Fatal error: Exception thrown without a stack frame in Unknown on line 0
cd piwik
chown -R www-data:www-data tmp

Then this error in the web interface:

Unable to load plugin 'DoNotTrack' because '/web/stats.transitionnetwork.org/piwik/plugins/DoNotTrack/DoNotTrack.php' couldn't be found.

Fixed by:

cd /web/stats.transitionnetwork.org/
rsync -av piwik.bak/plugins/DoNotTrack/ piwik/plugins/
php /web/stats.transitionnetwork.org/piwik/index.php -- "module=CoreUpdater"

Then the web interface has the following error:

Fatal error: Class 'Piwik_Db_Schema_Myisam' not found in /web/stats.transitionnetwork.org/piwik/core/Db/Schema.php on line 139 

The only reference I could find to this error message doesn't help, ​http://forum.piwik.org/read.php?2,85650 -- perhaps !InnoDB isn't supported, see ​http://dev.piwik.org/trac/ticket/2593 and ticket:396

So switch the piwik database to Myisam:

mysql -e "show tables in stats;" | tail --lines=+2 | xargs -i echo "ALTER TABLE \`{}\` ENGINE=MYISAM;" > stats_
cat stats_alter.sql | mysql stats

Check that all tables are converted:

echo "SHOW TABLE STATUS;" | mysql stats | grep -i INNODB

Run the cronjob:

su - www-data
/web/stats.transitionnetwork.org/piwik/misc/cron/archive.sh 

There is still the same error in the admin interface, not sure what to try next, I haven't been able to raise a ticket on the piwik trac but I have posted to their forum:

​http://forum.piwik.org/read.php?2,85650,page=1#msg-87181

[chris]

Matt suggested uploading one php file again, ​http://forum.piwik.org/read.php?2,85650,page=1#msg-85657 I tried this:

cd /web/stats.transitionnetwork.org/piwik/core/Db
mv Schema Schema.bak
svn co http://dev.piwik.org/svn/trunk/core/Db/Schema/
  A    Schema/Myisam.php
  Checked out revision 6040.

It didn't solve the problem.

[chris]

I cleared the APC opcode Cache and restarted apache and tried running the cron job again, this time it generated an error:

su - www-data
/web/stats.transitionnetwork.org/piwik/misc/cron/archive.sh 

Starting Piwik reports archiving...


Archiving period = day for idsite = 1...
PHP Fatal error:  Call to undefined method Piwik_Config::getInstance() in /web/stats.transitionnetwork.org/piwi
/Schema/Myisam.php on line 437

Fatal error: Call to undefined method Piwik_Config::getInstance() in /web/stats.transitionnetwork.org/piwik/cor
ma/Myisam.php on line 437

I switched back to the old Schema/Myisam.php and the cron job runs without errors.

[chris]

The suggestion on the forum, which seemed crazy to me:

I had to remove APC completely and reinstall it. After that, everything was okay.

​http://forum.piwik.org/read.php?2,85650,page=1#msg-87189

Worked!

cd /etc/php5/conf.d/
cp apc.ini apc.ini.bak
dpkg -r php-apc
aptitude install php-apc
cp apc.ini.bak apc.ini
apache2ctl restart

So /web/stats.transitionnetwork.org/piwik/config/config.ini.php was edited to remove record_statistics = 0 and docs updated wiki:NewLiveServer#piwik to contain a link to this ticket which will now be closed.

Sorry that almost a days worth of stats have been lost during this upgrade.

[chris]

Sorry I have had to reopen this, it's not solved after all.

Following it up at the piwik trac, bug #3048 ​http://dev.piwik.org/trac/ticket/3048

[chris]

The ticket was closed on the piwik trac: ​http://dev.piwik.org/trac/ticket/3049#comment:1

I don't have permissions to reopen it, I guess if needs be I can open a new ticket to ask that the first ticket be reopened ​http://dev.piwik.org/trac/ticket/3049#comment:2

[chris]

Updated suggestion from the developers ​http://dev.piwik.org/trac/ticket/3049#comment:3

cd /web/stats.transitionnetwork.org/piwik/core/Db/Schema
rm Myisam.php
wget http://dev.piwik.org/trac/export/5924/trunk/core/Db/Schema/Myisam.php
apache2ctl restart

This didn't help, I have added a comment to the ticket on the piwik trac: ​http://dev.piwik.org/trac/ticket/3049#comment:4

[chris]

Posted another reply to the ticket on the piwik forum ​http://dev.piwik.org/trac/ticket/3049#comment:6

[chris]

Following the suggestion here ​http://dev.piwik.org/trac/ticket/3049#comment:9 to do a reinstall, first edit the [Tracker] section of /web/stats.transitionnetwork.org/piwik/config/config.ini.php to turn tracking off:

record_statistics = 0

Create a new directory and download the latest version:

cd /web/stats.transitionnetwork.org
mkdir piwik.1.7.1
cd piwik.1.7.1
wget http://piwik.org/latest.zip
unzip latest.zip
rm How\ to\ install\ Piwik.html
rm latest.zip
mv piwik/* .

Set up apache to serve this new install at ​https://newstats.transitionnetwork.org.uk/ config file /etc/apache2/sites-available/newstats.transitionnetwork.org.conf.

Piwik suggests changing some ownerships:

chown -R www-data:www-data /web/stats.transitionnetwork.org/piwik.1.7.1
chmod -R 0755 /web/stats.transitionnetwork.org/piwik.1.7.1/tmp
chmod -R 0755 /web/stats.transitionnetwork.org/piwik.1.7.1/tmp/templates_c/
chmod -R 0755 /web/stats.transitionnetwork.org/piwik.1.7.1/tmp/cache/
chmod -R 0755 /web/stats.transitionnetwork.org/piwik.1.7.1/tmp/assets/
chmod -R 0755 /web/stats.transitionnetwork.org/piwik.1.7.1/tmp/tcpdf/

Create a new database:

mysql> CREATE USER 'newstats'@'localhost' IDENTIFIED BY 'xxxxxx';
mysql> CREATE DATABASE IF NOT EXISTS `newstats`;
mysql> GRANT ALL PRIVILEGES ON `newstats`.* TO 'newstats'@'localhost';
mysql> FLUSH PRIVILEGES;

Complete the install using the web interface and the newsite has the same error:

Fatal error: Class 'Piwik_Db_Schema_Myisam' not found in /web/stats.transitionnetwork.org/piwik.1.7.1/core/Db/Schema.php on line 139 

So perhaps the problem is with APC?

[chris]

Replying to chris:

So perhaps the problem is with APC?

Actually I really don't know, the class can't be found in the file, this grep produces no results:

grep "Piwik_Db_Schema_Myisam" /web/stats.transitionnetwork.org/piwik.1.7.1/core/Db/Schema.php

I think the problem must be with Piwik.

Another comment has been added to the other ticket: ​http://dev.piwik.org/trac/ticket/3049#comment:10

[chris]

Following the suggestions here ​http://dev.piwik.org/trac/ticket/3049#comment:11 but these didn't solve the problem, see ​http://dev.piwik.org/trac/ticket/3049#comment:12

[chris]

Debugging suggestion here ​http://dev.piwik.org/trac/ticket/3049#comment:13 tried, results here: ​http://dev.piwik.org/trac/ticket/3049#comment:14

[chris]

Piwik developers have asked for ssh access to help solve this problem, ​http://dev.piwik.org/trac/ticket/3049#comment:16

I have emailed the list to check if this is OK.

[chris]

The Piwik developers haven't been in touch, I have asked them for either a ssh or gpg public key so I can set them up with access to the server.

I have added another comment to the ticket on the piwik trac site ​http://dev.piwik.org/trac/ticket/3049#comment:19

[chris]

Still not heard from them, have left another comment ​http://dev.piwik.org/trac/ticket/3049#comment:20

[chris]

I have shutdown the test site I installed for the Piwki developers and posted another comment asking for the bug to be reopened: ​http://dev.piwik.org/trac/ticket/3049#comment:21

[chris]

I have opened a new ticket asking that the first ticked be reopened:

​http://dev.piwik.org/trac/ticket/3092

[chris]

This is now fixed!

Read on for details...

One on the piwik developers found the problem was with these APC settings in /etc/php5/apache2/conf.d/apc.ini:

apc.include_once_override = 1
apc.mmap_file_mask = /tmp/apc.XXXXXX

When these were changed to:

apc.include_once_override = 0
apc.mmap_file_mask = /dev/zero

The problem went away, the piwik ticket, ​http://dev.piwik.org/trac/ticket/3049#comment:27 and forum thread have been updated, ​http://forum.piwik.org/read.php?2,85650,page=1#msg-88217

The documentation has been updated here wiki:NewLiveServer#apc

Before the Piwik developer came to the rescue I tried the suggestions here for clearing the APC cache, but none helped:

​http://stackoverflow.com/questions/911158/how-to-clear-apc-cache-entries

There is a password protected cache clearing script here:

​https://quince.transitionnetwork.org/info/apc-clear-cache.php

And you can see the results here:

​https://quince.transitionnetwork.org/info/apc.php

I'm not convinced that this has been deleting all the files from the cache as this graph didn't drop to zero:

​https://kiwi.transitionnetwork.org/munin/webarch.net/quince.webarch.net/php_apc_files.html