[[PageOutline(2-5, Table of Contents, floated)]] = Site Load Times = This page is to document the time the site takes to load. == 2014-01-29 == For reference, the following was run a few times on wiki:PuffinServer: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ }}} And the results were compared and this is a slow one: {{{ Completed 200 requests Finished 200 requests Server Software: nginx Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 13252 bytes Concurrency Level: 10 Time taken for tests: 0.859 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2758044 bytes HTML transferred: 2650400 bytes Requests per second: 232.74 [#/sec] (mean) Time per request: 42.966 [ms] (mean) Time per request: 4.297 [ms] (mean, across all concurrent requests) Transfer rate: 3134.35 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 1 1.5 1 8 Processing: 4 11 6.6 9 47 Waiting: 3 10 6.5 8 46 Total: 4 13 6.9 11 54 Percentage of the requests served within a certain time (ms) 50% 11 66% 15 75% 17 80% 17 90% 22 95% 24 98% 31 99% 36 100% 54 (longest request) }}} And this is a fast one: {{{ Completed 200 requests Finished 200 requests Server Software: nginx Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 13248 bytes Concurrency Level: 10 Time taken for tests: 0.200 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2757424 bytes HTML transferred: 2649600 bytes Requests per second: 998.54 [#/sec] (mean) Time per request: 10.015 [ms] (mean) Time per request: 1.001 [ms] (mean, across all concurrent requests) Transfer rate: 13444.29 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 2 Processing: 3 9 7.0 8 55 Waiting: 2 9 6.9 7 55 Total: 3 10 7.0 8 55 Percentage of the requests served within a certain time (ms) 50% 8 66% 10 75% 13 80% 14 90% 19 95% 23 98% 28 99% 38 100% 55 (longest request) }}} == 2013-10-20 == See LoadTimesOctThirteen. == 2013-05-29 == The slowness documented yesterday, LoadTimes#a2013-05-28 was due to redis being unavailable due to the password missing from the {{{/data/conf/global.inc}}} file, this was fixed on ticket:554#comment:5, following the fix the speed of the site is back to normal: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ LOG: Response code = 200 Completed 200 requests Finished 200 requests Server Software: nginx Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 10049 bytes Concurrency Level: 10 Time taken for tests: 0.197 seconds Complete requests: 200 Failed requests: 128 (Connect: 0, Receive: 0, Length: 128, Exceptions: 0) Write errors: 0 Total transferred: 2143788 bytes HTML transferred: 2007624 bytes Requests per second: 1016.38 [#/sec] (mean) Time per request: 9.839 [ms] (mean) Time per request: 0.984 [ms] (mean, across all concurrent requests) Transfer rate: 10639.16 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 1 Processing: 2 9 7.4 7 79 Waiting: 2 8 7.4 6 78 Total: 2 9 7.4 7 80 Percentage of the requests served within a certain time (ms) 50% 7 66% 9 75% 11 80% 12 90% 17 95% 22 98% 26 99% 28 100% 80 (longest request) }}} == 2013-05-28 == The front page index.php now takes around 5 seconds to generate, for example, following is a test done from ParrotServer, see ticket:554 for more on this issue: {{{ ab -n 1 -v 4 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking www.transitionnetwork.org (be patient)...INFO: POST header == --- GET / HTTP/1.0 Accept-Encoding: gzip Host: www.transitionnetwork.org User-Agent: ApacheBench/2.3 Accept: */* --- LOG: header received: HTTP/1.1 200 OK Server: nginx Date: Tue, 28 May 2013 18:08:04 GMT Content-Type: text/html; charset=utf-8 Connection: close Vary: Accept-Encoding X-Backend: C X-Allow-Redis: YES X-Purge-Level: 6 X-Cookie-Domain: .transitionnetwork.org X-Redis-Prefix: www.transitionnetwork.org_ X-Accel-Expires: Cache-Control: public, max-age=0 Last-Modified: Tue, 28 May 2013 18:07:58 +0000 Expires: Sun, 11 Mar 1984 12:00:00 GMT Vary: Cookie X-Engine: Octopus 1.0 ET X-Device: normal X-Speed-Cache: EXPIRED X-Speed-Cache-Key: / X-NoCache: Cache X-GeoIP-Country-Code: GB X-GeoIP-Country-Name: United Kingdom X-Server-Name: www.transitionnetwork.org Content-Encoding: gzip � LOG: Response code = 200 ..done Server Software: nginx Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 10052 bytes Concurrency Level: 1 Time taken for tests: 5.696 seconds Complete requests: 1 Failed requests: 0 Write errors: 0 Total transferred: 10735 bytes HTML transferred: 10052 bytes Requests per second: 0.18 [#/sec] (mean) Time per request: 5695.665 [ms] (mean) Time per request: 5695.665 [ms] (mean, across all concurrent requests) Transfer rate: 1.84 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 5695 5695 0.0 5695 5695 Waiting: 5695 5695 0.0 5695 5695 Total: 5696 5696 0.0 5696 5696 }}} For comparison with older tests documented on this page: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ Server Software: nginx Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 9768 bytes Concurrency Level: 10 Time taken for tests: 5.539 seconds Complete requests: 200 Failed requests: 1 (Connect: 0, Receive: 0, Length: 1, Exceptions: 0) Write errors: 0 Total transferred: 2090586 bytes HTML transferred: 1953787 bytes Requests per second: 36.11 [#/sec] (mean) Time per request: 276.929 [ms] (mean) Time per request: 27.693 [ms] (mean, across all concurrent requests) Transfer rate: 368.61 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 1 Processing: 2 35 391.0 6 5537 Waiting: 2 35 391.0 5 5537 Total: 2 36 391.1 6 5538 Percentage of the requests served within a certain time (ms) 50% 6 66% 9 75% 11 80% 13 90% 15 95% 21 98% 29 99% 44 100% 5538 (longest request) }}} Requesting the front page 200 times running 10 concurrent processes over time -- the site is now roughly back to the speed it had when it was running with apache without varnish, two years ago: * LoadTimes#a2013-05-28 5.539 seconds wiki:PuffinServer * LoadTimes#a2013-01-28 0.253 seconds wiki:NewLiveServer * LoadTimes#a2013-01-28 0.262 seconds wiki:PuffinServer * LoadTimes#a2011-07-13 3.417 seconds wiki:NewLiveServer without varnish * LoadTimes#a2011-07-05 5.131 seconds wiki:NewLiveServer without varnish == 2013-01-31 == A script was written to test the nearby post code search, see ticket:466#comment:55 == 2013-01-28 == Comparing the existing live server, wiki:NewLiveServer with the new wiki:PuffinServer. === apache bench === {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ Server Hostname: www.transitionnetwork.org Server Port: 80 Document Path: / Document Length: 10272 bytes Concurrency Level: 10 Time taken for tests: 0.253 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2150344 bytes HTML transferred: 2058286 bytes Requests per second: 790.39 [#/sec] (mean) Time per request: 12.652 [ms] (mean) Time per request: 1.265 [ms] (mean, across all concurrent requests) Transfer rate: 8298.90 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 1 3 1.0 3 5 Processing: 4 9 9.3 6 48 Waiting: 1 5 7.9 3 44 Total: 4 12 9.3 9 52 Percentage of the requests served within a certain time (ms) 50% 9 66% 10 75% 10 80% 11 90% 15 95% 38 98% 46 99% 47 100% 52 (longest request) }}} {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://newlive.puffin.webarch.net/ Server Hostname: newlive.puffin.webarch.net Server Port: 80 Document Path: / Document Length: 9248 bytes Concurrency Level: 10 Time taken for tests: 0.262 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 1973433 bytes HTML transferred: 1849600 bytes Requests per second: 762.21 [#/sec] (mean) Time per request: 13.120 [ms] (mean) Time per request: 1.312 [ms] (mean, across all concurrent requests) Transfer rate: 7344.61 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 0.5 1 3 Processing: 4 12 13.2 8 144 Waiting: 2 10 13.3 6 143 Total: 5 13 13.1 9 145 Percentage of the requests served within a certain time (ms) 50% 9 66% 12 75% 15 80% 16 90% 24 95% 33 98% 40 99% 86 100% 145 (longest request) }}} {{{ Server Hostname: tn.i-jk.co.uk Server Port: 80 Document Path: / Document Length: 10881 bytes Concurrency Level: 10 Time taken for tests: 0.569 seconds Complete requests: 200 Failed requests: 0 Write errors: 0 Total transferred: 2319227 bytes HTML transferred: 2196972 bytes Requests per second: 351.57 [#/sec] (mean) Time per request: 28.444 [ms] (mean) Time per request: 2.844 [ms] (mean, across all concurrent requests) Transfer rate: 3981.31 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 7 8 0.3 8 9 Processing: 18 20 3.1 19 50 Waiting: 10 11 3.2 10 42 Total: 26 28 3.2 27 58 Percentage of the requests served within a certain time (ms) 50% 27 66% 27 75% 27 80% 28 90% 28 95% 32 98% 40 99% 44 100% 58 (longest request) }}} == 2011-07-13 == The varnish settings for the dev server and the live server are exactly the same. However the dev server is serving pages via varnish: {{{ lynx -head -dump http://dev.transitionnetwork.org/ X-Varnish-Cache: HIT }}} And the live server is not: {{{ lynx -head -dump http://www.transitionnetwork.org/ X-Drupal-Cache: HIT X-Varnish-Cache: MISS }}} Note the '''X-Varnish-Cache: HIT''' header from the dev server and the '''X-Drupal-Cache: HIT''' from the live server. === live server === ==== apache bench ==== {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ Time taken for tests: 3.417 seconds Requests per second: 58.53 [#/sec] (mean) Time per request: 170.863 [ms] (mean) Percentage of the requests served within a certain time (ms) 50% 135 66% 179 75% 214 80% 247 90% 325 95% 407 98% 638 99% 705 100% 795 (longest request) }}} ==== curl ==== {{{ curl -o /dev/null https://www.transitionnetwork.org/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 30668 0 30668 0 0 449k 0 --:--:-- --:--:-- --:--:-- 14.6M }}} == 2011-07-08 == === dev server === HTTPS tests only. ==== pingdom ==== Times out after loading just a few objects. ==== curl ==== Varied between 1 and 7 seconds, for example: {{{ curl -o /dev/null https://dev.transitionnetwork.org/ Dload Upload Total Spent Left Speed 100 30808 0 30808 0 0 16579 0 --:--:-- 0:00:01 --:--:-- 38413 }}} == 2011-07-07 == The following results were generated with https redirecting to http so they are invalid for https. === Dev Server === Running Varnish on port 80 and apache on 443. ==== apache bench ==== On port 80, this is with the first hit being passed to apache: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://dev.transitionnetwork.org/ Time taken for tests: 4.302 seconds Percentage of the requests served within a certain time (ms) 50% 7 66% 8 75% 9 80% 55 90% 753 95% 2988 98% 2991 99% 2991 100% 2993 (longest request) }}} This is for subsequent requests which don't hit apache: {{{ Time taken for tests: 0.183 seconds Percentage of the requests served within a certain time (ms) 50% 7 66% 7 75% 8 80% 8 90% 9 95% 19 98% 40 99% 41 100% 41 (longest request) }}} ==== pingdom ==== * http://tools.pingdom.com/?url=http://dev.transitionnetwork.org/ * Total loading time: 2 seconds Wow, compare this with the last test! * http://tools.pingdom.com/?url=https://dev.transitionnetwork.org/ * Total loading time: 3.2 seconds ==== Load Impact ==== HTTP: http://loadimpact.com/result/dev.transitionnetwork.org/-d25171ed06575a89080ba08b0ab371bc {{{ Clients 10 20 30 40 50 Delay (ms) 449 495 443 433 465 }}} It won't run the HTTPS test today without paying them. ==== curl ==== For port 80 we consistently get around 30 mili seconds via Varnish: {{{ curl -o /dev/null http://dev.transitionnetwork.org/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 30856 100 30856 0 0 2327k 0 --:--:-- --:--:-- --:--:-- 28.4M }}} On 3 different machines this ism't working, I don't understand why: {{{ curl -o /dev/null https://dev.transitionnetwork.org/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 }}} == 2011-07-05 == Summary for the live server: * The front page, HTML only of the live site, not running varnish takes around 0.1 seconds to load * The front page of the live site, plus css / js / images etc takes over 30 seconds via pingdom Issues for the live server: * ETags and one CSS expression Summary for the dev server: * Very fast on port 80 via Varnish, less than two seconds for the page and all objects * Very slow for port 443, 7 seconds just for the HTML Issues for the dev server: * Speed up port 443 accesses! * Etag and gzipping of js files Detailed info follows: === Live Server === This is before Varnish has been deployed and also tests have only be done using http as https requests get a 302 redirect back to http. ==== apache bench ==== Requests were done from another machine in the same rack, 200 requests for the front page, 10 at a time, using http, this was done several times and the results below are for an average result, some were faster and some were slower, the first request took about 4 seconds and almost all the rest were less that 0.1 second: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://www.transitionnetwork.org/ LOG: Response code = 200 LOG: header received: HTTP/1.1 200 OK Date: Tue, 05 Jul 2011 12:02:58 GMT Server: Apache X-Drupal-Cache: HIT Etag: "1309867375-0" Cache-Control: public, max-age=0 Last-Modified: Tue, 05 Jul 2011 12:02:55 +0000 Expires: Sun, 11 Mar 1984 12:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip Content-Length: 7727 Connection: close Content-Type: text/html; charset=utf-8 Time taken for tests: 5.131 seconds Complete requests: 200 Percentage of the requests served within a certain time (ms) 50% 57 66% 66 75% 74 80% 77 90% 100 95% 4014 98% 4019 99% 4039 100% 4050 (longest request) }}} ==== YSlow ==== * The page has a total of 29 HTTP requests and a total weight of 421.8K bytes with empty cache * This page has 3 external stylesheets. Try combining them into one. * This page has 17 external background images. Try combining them with CSS sprites. * There are a total of 1 CSS expression: * http://www.transitionnetwork.org/sites/default/files/css/css_9bf5bdd80b33ef2e38c9b89845c474c9.css * CSS expressions (supported in IE beginning with Version 5) are a powerful, and dangerous, way to dynamically set CSS properties. These expressions are evaluated frequently: when the page is rendered and resized, when the page is scrolled, and even when the user moves the mouse over the page. These frequent evaluations degrade the user experience. * There are 27 components with misconfigured ETags (all css js and images) ==== pingdom ==== * http://tools.pingdom.com/?url=http://www.transitionnetwork.org/ The test here will run for 30 secs max and it generally takes longer than that to load the front page, in part because (as far as I can see) pingdom loads all images referenced in the CSS even if the pages doesn't use them. Some stats: * Total objects: 76 (677.5 KB) * (X)HTML: 1 (32.8KB) * CSS: 3 (92.7KB) * Scripts: 1 (150.7KB) * Images: 71 (401.2KB) === Dev Server === Running Varnish on port 80 and apache on 443. ==== apache bench ==== On port 80, this is with the first hit being passed to apache: {{{ ab -v 4 -n 200 -c 10 -H "Accept-Encoding: gzip" http://dev.transitionnetwork.org/ LOG: Response code = 200 LOG: header received: HTTP/1.1 200 OK Server: Apache Cache-Control: public, max-age=3600 Last-Modified: Tue, 05 Jul 2011 12:51:05 +0000 Expires: Sun, 11 Mar 1984 12:00:00 GMT Vary: Cookie,Accept-Encoding ETag: "1309870265" Content-Encoding: gzip Content-Type: text/html; charset=utf-8 Content-Length: 7353 Date: Tue, 05 Jul 2011 12:51:07 GMT X-Varnish: 1553439536 1553439337 Age: 0 Via: 1.1 varnish Connection: close X-Varnish-Cache: HIT Time taken for tests: 1.616 seconds Complete requests: 200 Percentage of the requests served within a certain time (ms) 50% 3 66% 3 75% 3 80% 3 90% 4 95% 4 98% 5 99% 1412 100% 1412 (longest request) }}} This is for subsequent requests which don't hit apache: {{{ Time taken for tests: 0.149 seconds Complete requests: 200 Percentage of the requests served within a certain time (ms) 50% 7 66% 7 75% 8 80% 8 90% 9 95% 9 98% 10 99% 11 100% 12 (longest request) }}} ==== YSlow ==== * Make fewer HTTP requests * This page has 3 external stylesheets. Try combining them into one. * This page has 14 external background images. Try combining them with CSS sprites. * There are 2 plain text components that should be sent compressed * https://piwik.transitionnetwork.org/piwik.js * http://dev.transitionnetwork.org/sites/default/files/js/js_cb10d5814b295fb1648af27eb7a3f74b.js * There is 1 component with misconfigured ETags * https://piwik.transitionnetwork.org/piwik.js ==== pingdom ==== * http://tools.pingdom.com/?url=http://dev.transitionnetwork.org/ Load time between 1 and 1.5 seconds *total*! * Total objects: 72 (809.7 KB) * (X)HTML: 1 (31.4KB) * CSS: 3 (95.8KB) * Scripts: 1 (113.2KB) * Images: 61 (364.7KB) * Other: 6 (204.7KB) However for HTTPS it times out before loading many objects: * http://tools.pingdom.com/?url=https://dev.transitionnetwork.org/ ==== curl ==== For port 80 we consistently get around 30 mili seconds via Varnish: {{{ curl -o /dev/null http://dev.transitionnetwork.org/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 32177 100 32177 0 0 2714k 0 --:--:-- --:--:-- --:--:-- 29.7M }}} Unlike ab we can use curl for port 443, just loading the front page HTML file takes around 7 seconds: {{{ curl -o /dev/null https://dev.transitionnetwork.org/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 32076 0 32076 0 0 4545 0 --:--:-- 0:00:07 --:--:-- 8172 }}}