APC Tuning on Parrot and Penguin

[chris]

As part of the upgrade from Squeeze to Wheezy, see ticket:535, Munin graphs for APC were added:

• ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/parrot.transitionnetwork.org/index.html#php-apc
• ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/index.html#php-apc

There is also more APC info here:

• ​https://parrot.transitionnetwork.org/apc.php
• ​https://penguin.transitionnetwork.org/info/apc.php

The documentation for the variables which can be set in /etc/php5/mods-available/apc.ini can be found here:

• ​http://php.net/manual/en/apc.configuration.php

This monitoring is generating quite a high volume of warnings about fragmentation and purges and this ticket has been created to try to sort this issue out.

[chris]

The number of alerts I'm getting from APC on wiki:PenguinServer is getting a bit annoying, every time the purge rate goes over 10% I get sent a email every 5mins:

• ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/php_apc_purge.html

The APC reference docs are here:

• ​http://php.net/manual/en/apc.configuration.php

[chris]

Quick fix for the warning emails for now, have edited /usr/share/munin/plugins/php_apc_ on penguin and changed this line:

purge_rate.warning 70

It was set to 10, see ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/php_apc_purge.html

I have also tweaked the cache sizes.

APC graphs for both servers can be viewed here:

• ​https://penguin.transitionnetwork.org/munin/php-apc-day.html

More fine tuning can wait till another month.

[chris]

Penguin APC Purge Rate 15th Dec 2013

[chris]

I have started getting email alerts every 5 mins from Munin due to the APC PHP purge rate hitting 70%:

So I have edited /usr/share/munin/plugins/php_apc_ and changed this:

purge_rate.warning 90

And restarted munin-node.

[chris]

Some warning thresholds and cache sizes were tweaked while the Piwik upgrade was running, ticket:660.

[chris]

Tewaked settings in /etc/php5/mods-available/apc.ini on wiki:PenguinServer:

apc.shm_size="96M"
apc.ttl=0
apc.user_ttl=7200
apc.gc_ttl=600
apc.max_file_size="2M"

I have also added a link to the APC stats from ​https://penguin.transitionnetwork.org/

These are the settings we have for /etc/php5/mods-available/apc.ini on wiki:ParrotServer:

apc.shm_size="180M"
apc.ttl=0
apc.user_ttl=7200
apc.gc_ttl = 600
apc.max_file_size="2M"

I have added a apache rule to make the APC info only available via HTTPS, in /etc/apache2/conf.d/webarch.conf:

RewriteRule ^/apc.php(.*) https://%{SERVER_NAME}/apc.php$1 [R,L]

And I have copies the Trac .htpasswd file to parrot so it can be used to protect access to ​https://parrot.webarch.net/apc.php and there is a now link to this from the front page for the server ​https://parrot.webarch.net/

The warning thresholds for the graphs here have also been tweaked:

• ​https://penguin.transitionnetwork.org/munin/php-apc-day.html

[chris]

wiki:ParrotServer APC settings in /etc/php5/mods-available/apc.ini were tewaked again based on the stats:

apc.shm_size="256M"
apc.ttl=0
apc.user_ttl=7200
apc.gc_ttl = 600
apc.max_file_size="3M"

[chris]

I'm getting an alert every 5 mins because the wiki:PenguinServer ​APC purge rate has hit 90%:

I'm not sure what should be done to best address this, for now I have edited /usr/share/munin/plugins/php_apc_ to increase the warning threshold to 99%:

purge_rate.warning 99

[chris]

Looing at the APC info at ​https://penguin.transitionnetwork.org/info/apc.php the memory usage is quite high:

• Free: 14.4 MBytes (15.0%)
• Used: 81.6 MBytes (85.0%)

So I have increased the available memory from 96MB to 128MB by editing /etc/php5/mods-available/apc.ini

apc.shm_size="128M"

And restarting php-fpm.

I have also added a link to this ticket from wiki:PenguinServer#APCStatsandPHPinfo

[chris]

wiki:PenguinServer purge rate is at 100% again and I'm getting alerts every 5 mins, I have done some research and looks at the stats and my best guess is that changing apc.ttl to 24 hours might help (it was at 0), so this is the updated /etc/php5/mods-available/apc.ini:

extension=apc.so
apc.shm_size="128M"
apc.ttl=86400
apc.user_ttl=7200
apc.gc_ttl=600
apc.max_file_size="3M"

Restart:

/etc/init.d/php5-fpm restart

[chris]

On wiki:PenguinServer the purge rate still tends to 100%, this isn't resolved, see:

• ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/penguin.transitionnetwork.org/php_apc_purge.html