Force HTTP for anonymous, HTTPS for logged in users

[jim]

To further reduce load and leverage the caching I've changed the Session 443 settings to force anon users to HTTP and logged in to HTTPS. The benefit of allowing a handful of users to choose is tiny compared the downsides of outages, 503s and higher load.

The ​"User state" setting on the config page is now " Redirect authenticated users to HTTPS and redirect anonymous users to HTTP (with the exception of login/registration pages).", was "Redirect authenticated users to HTTPS and redirect anonymous users on login/registration pages to HTTPS. Anonymous users visiting other pages may use HTTP or HTTPS."

I've also set user and site-wide contact forms, plus the mailchimp subs page force secure-only per the "Additional pages to make secure" setting.

We can see if this makes a difference, and this ticket is to track comments and see if it results in any improvement in performance/stability.

[jim]

Note this settings change happened at around 12.50 on Tuesday 16 July for monitoring purposes.

[jim]

OK so after a couple of days it's clear this was a good improvement. See attached image for evidence that disk service time has dropped and throughput increased despite the last few days being very busy.

This change will have better used the Nginx speed cache, and Drupal's page cache (via Redis) because and client browser caching, and minimised the risks from editors/users posting HTTPS.

Closing, but re-open if any comments or issues occur. Adding time for R&D & implementation.

[jim]

The cyan line in the attached image marks the point I changed the setting in the OP.

Closing.

[jim]

One last point: if it turns out we have low-grade DoS or a bunch of badly behaved bots hitting the site then we'd be in the strange situation of finding that any increase in throughput (to a point) would be absorbed by these agents simply being able to hit us faster, rather than waiting. I hope that's not the case...

Oh, and the HTTPS -> HTTP for non-logged in helps SEO too, I'd argue.

Over and out.

[jim]

Adding hours that didn't add...

[chris]

Replying to jim:

OK so after a couple of days it's clear this was a good improvement. See attached image for evidence that disk service time has dropped and throughput increased despite the last few days being very busy.

I don't see exactly why the disk service time should be related to this to be honest, I'd guess it is down to the fact that over the last few days we have migrated some of the virtual machines on red to use an external zfs disk array, rather than the disks in red. If you look at the annual IO service time it's heading back to where it was some months ago:

• ​https://penguin.transitionnetwork.org/munin/transitionnetwork.org/puffin.transitionnetwork.org/iostat_ios.html

I also don't understand why the number of connections through the firewall is directly related to this change?

This change will have better used the Nginx speed cache, and Drupal's page cache (via Redis) because and client browser caching, and minimised the risks from editors/users posting HTTPS.

I don't disagree that performance wise SSL has a overhead but I still think that in the age of GCHQ and the NSA slurping up all the traffic we should generally be aiming at using HTTPS for everything and as a minimum we should allow people to choose to use HTTPS if they don't want what they are reading to be transparent to the spooks.

[chris]

I should have mentioned this earlier but users of Firefox with the EFF HTTPS Everywhere plugin, ​https://www.eff.org/https-everywhere who don't login now simply get a redirect loop when they try to access ​http://transitionnetwork.org/ -- they can't access any content on the site due the the changes made to redirect non-authenticated users to the HTTP version of the site from the HTTPS version.

As I have said before I think we should allow non-authenticated users to access the site using HTTPS if they wish to do so for privacy reasons.

In light of ticket:574 I'm re-opening this ticket.

[chris]

Replying to chris:

As I have said before I think we should allow non-authenticated users to access the site using HTTPS if they wish to do so for privacy reasons.

For example:

I think we should allow non authenticated users to use HTTPS if they wish so I think "Force HTTP for anonymous users:" should be disabled

ticket:224#comment:13

[jim]

I'm 100% happy to roll this tweak back. The logic was simple: better use caching by enforcing a single HTTP context, rather than spread things over HTTPS too for anonymous users, plus reduce load on the server as some assets, pages and session-related stuff is not cached as long/all over HTTPS.

I still maintain this will have a positive effect on the server's load, though clearly it's not very significant. That the change I did coincided with Chris tweaking the host server meant I jumped to a wrong conclusion that the server was having less disk work to as a result of better caching things, and therefore spitting out pages faster. In my defence when one flicks a switch and something good happens, one tends to assume the switch was good (not that there was a wizard in the behind the curtain twiddling much heftier knobs!)

So I've put the setting in the OP back as it was, "Redirect authenticated users to HTTPS and redirect anonymous users on login/registration pages to HTTPS. Anonymous users visiting other pages may use HTTP or HTTPS" at 9.10pm 31 July in case we see any visible change in Munin.

[jim]

I took out some of my time as a goodwill gesture too.

Closing, will re-open if something else 'special' happens.

[ed]

Good work chaps