Pwiki 2014 annual report

[chris]

Ed would like a some PDF reports of Transition Network site usage for 2014 from Piwik.

[chris]

Penguin Memory 2014

[chris]

Penguin Swap 2014

[chris]

Pwiki is struggling with the lack of RAM on the server which hosts it, PenguinServer, see ​the Munin stats:

See the "committed" line in the graph above.

The server has 3GB of RAM, I'm going to dramatically increase this, which will require a reboot, and then generate the Piwik report and then put it back to 3GB and do another reboot.

[chris]

The server is now running with 15 rather than 8 CPUs and 20GB rather than 3GB of RAM.

[chris]

TN Piwik Stats 2014 Version 1

[chris]

Transition Network Pwiki Stats 2014

[chris]

I edited /etc/php5/fpm/pool.d/www.conf:

php_admin_value[memory_limit] = 16G

And /etc/php5/fpm/php.ini:

;max_execution_time = 30
max_execution_time = 3600

;max_input_time = 60
max_input_time = 3600

#memory_limit = 128M
memory_limit = 18G

And restarted php-fpm.

And then edited /etc/nginx/nginx.conf and edited:

        #keepalive_timeout 240;
        keepalive_timeout 3600;

And edited /etc/nginx/stats-shared :

        fastcgi_read_timeout 3600;

And restarted Nginx.

Things to note from watching top, each php-fpm process uses one processor, memory usage per php-fpm process reached 780M.

Dispite changing all these settings (and the values above are the final values not all the ones tried) I was still getting "504 Gateway Time-out" from Nginx when trying to generate a PDF via the browser, however the "send report via email" function, which I expect uses the cli version of php, does appear to work... However there are some stats from the imported GA data which are no longer being collected and also some graphs which go from 2005 to 2009 -- this appears to be a Piwik bug, see:

• attachment:ticket:823:Report_Transition_Network_-_2014.pdf​

So I generated another report without the buggy reports:

• attachment:ticket:823:Report_Transition_Network_-_2014_v2.pdf​

However I don't know if this has enough data?

A better job could be done manually generating graphs using the web interface and exporting tables etc, but that would be more time consuming -- Ed is the last PDF you have been sent adequate?

[chris]

The 2005 to 2009 graphs deserve a ticket with Piwik -- if these graphs are wanted from the PDF -- I'm happy to follow this up if you think it deserves the time.

[ed]

That's great Chris thanks. Closing ticket.

[chris]

OK, undoing the changes / setting sane values, /etc/php5/fpm/pool.d/www.conf:

php_admin_value[memory_limit] = 256M

In /etc/php5/fpm/php.ini:

max_execution_time = 60

max_input_time = 60

memory_limit = 256M

In /etc/nginx/nginx.conf:

keepalive_timeout 240;

In /etc/nginx/stats-shared:

fastcgi_read_timeout 240; 

And I'm now rebooting the server to reset the RAM and CPUs.

[chris]

It's worth noting that the MySQL dump of the Piwik database is 3.3G in size -- it is a lot of data...