Want to subscribe to topics you're interested in?
Become a Member

WebPerf PageSpeed How to use webpagetest.org for page load speed testing

Discussion in 'Web Development & Web Performance' started by eva2000, Jan 28, 2018.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    My favourite page speed tool for web page page load speed testing is webpagetest.org. It's one of the many tools I use. It has dozens of test geographical locations and web browser and mobile/tablet testing profiles so you can do detailed page load speed tests. WPT has recently updated all their Amazon AWS EC2 testing locations to using Linux which has improved the testing servers performance as well. You can also read an overview at WebPerf - Measuring Web Performance – Analyzing What Matters Most.

    You can register for a Webpagetest user account which is tied to their forums here so you get access to Webpagetest API, API key and forums and can set private test runs.
    Below is a brief run down on how to use webpagetest.org to test your web sites' pages.

    When you visit webpagetest.org, the first page you're greeted with is the Advanced testing tab where you enter your web site url, your choice of testing location, your choice of web browser to simulate page loads from. Further below that is Test settings page where
    • you select the connection speed profile - which usually I use 3G mobile or Cable 5Mbps as they are the averages worldwide
    • you select the number of tests to run - always set to odd number as the results are the mean of the number of runs for more accuracy so I usually select 3, 5 or 7 test runs
    • repeat view - I usually just select first view but you can select first and repeat view
    • capture video - check the box so you get filmstrip profile of how your page loads
    • keep test private - if you register for WPT account, you can keep your test results private from public. You'd need to uncheck the box if you want to share the result url link. Update: seems this option no longer exists all tests are private in so far as only if the person knows the test id url result page link, will be able to view the test.
    • label field to properly label your tests so if you have a registered WPT account, you can go to test result history and lookup past tests and compare past tests against each other.
    upload_2018-1-28_14-25-3.png

    The other most common settings you will probably use are for Script tab which has further configurations you can test outlined here.

    upload_2018-1-28_14-32-11.png

    Above screenshot example of setting the web browser test' custom viewport to a higher resolution than default IIRC is 1280x1080, i.e. 1920x1080 use setViewportSize and navigate. You will need to start the test from Script tab page for it to take effect.

    The Webpagetest result page will return the mean average of the number of test runs in summary table with a link in brackets to the relevant result. In this case test run 1 was the mean average of 3 run test. Clicking (Run 1) link will jump to the actual test run to inspect/view.

    Note with Scripted runs you have yellow Script text dropdown which can reveal the scripted test runs parameters.

    upload_2018-1-28_14-34-4.png

    upload_2018-1-28_14-35-33.png

    Run 1 has the waterfall, screenshot and filmstrip view results which you can click through to see the details.

    The waterfall

    upload_2018-1-28_14-37-1.png

    On waterfall page each request listed is clickable to reveal further info and stats. Example clicking on index page request 1

    the details tab. Note the summary table's reported TTFB 0.477s is actually made up of times of DNS look up 25ms + Initial Connection 31ms + SSL Negotiation 43ms and Time to First Byte 362ms to total around 462ms or 0.462s or there abouts so when you see WPT reported TTFB in summary table know that DNS, SSL negotiation response time is factored into it.

    upload_2018-1-28_14-50-25.png

    The response header tab

    upload_2018-1-28_14-51-8.png

    The screenshot

    upload_2018-1-28_14-37-34.png

    The filmstrip

    Notice thumbnail size and interval parameters you can change size of screenshots and interval of display which defaults to 0.5 seconds, i usually change to 0.1 seconds and small/medium sized thumbnails. The WPT Speedindex metric in milliseconds of 963 in Run 1 results closely correlates to filmstrip's perceived visual start render time.

    upload_2018-1-28_14-39-20.png

    If you registered a Webpagetest user account, you can lookup test result history to compare results. As you can see there is a Label column which would show your custom labels if you had configured one


    upload_2018-1-28_14-43-55.png

    Summary

    I hope this overview of Webpagetest.org page speed tool's usage is useful to folks. I always use this before and after optimisation of a site and server to measure performance improvements and regressions over time. I also use this before and after moving web servers/web hosts. You can also use this for your sites on non-Centmin Mod LEMP servers before migrating sites to Centmin Mod LEMP based servers to measure improvements or regressions as well. Pair Webpagetest with load testing tools like wrk, will allow you to measure before and after optimisation/migration improvements to your web sites :D
     
    Last edited: Jan 28, 2018
  2. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+

    Using Google Lighthouse With Webpagetest.org



    Update July 2018: Looks like WPT Google Lighthouse tests now are hard coded to use 3G Fast speed profile.

    Webpagetest.org also supports Google Lighthouse tool for Chrome browser tests. It supports web apps too but for general web site performance, we would be just focused on the sub test results for Performance, Accesibility, Best Practices and SEO. Note results will be different from Webpagetest.org as Google Lighthouse tool.

    For Webpagetest.org go to the Chrome tab in Advanced settings and check the box to Capture Lighthouse Report (Chrome only) and run the test from this tab.

    upload_2018-2-19_15-54-14.png

    On the results page top right corner, you will see a new performance indicator result button for Google Lighthouse with the score for Web App subtest result. You can ignore that score as we are focused on subtest results for Performance, Accessibility, Best Practices and SEO.

    lighthouse-cmm-calforinia-01.png
    Click the Lighthouse score button on top right (shows 55 in above screenshot) to reveal the Google Lighthouse report. Focus should be on the results for First meaningful paint, First Interactive, Perceptual Speed Index metrics.

    The Google Lighthouse Tool runtime environment configuration

    lighthouse-cmm-calforinia-00.png

    The Google Lighthouse Tool results.

    lighthouse-cmm-calforinia-03.png

    SEO results

    lighthouse-cmm-calforinia-04.png

    Webpagetest Emulate Mobile Browser



    You can also emulate mobile web browsers like Google Pixel 2, Samsung Galaxy S8 and iPhone 8 with a 3G connection speed profile. Seems only the Webpagetest tests use the emulate browser configuration while Google Lighthouse uses Nexus 5X emulation instead (see below runtime environment).

    lighthouse-cmm-calforinia-s8-00.png

    lighthouse-cmm-calforinia-pixel2-3g-01.png

    lighthouse-cmm-calforinia-pixel2-3g-02.png

    lighthouse-cmm-calforinia-pixel2-3g-03.png
     
  3. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Update, added a new script gitools.sh which can run Webpagetest, GTMetrix and Google PageSpeed Insight tests via their respective API. You can run gitools.sh via cronjob to schedule test runs and send results to your own Slack channels. Full details at WebPerf - PageSpeed - PageSpeed Testing Via APIs: Google PageSpeed Insights, GTMetrix & WebpageTest.org

    Example for Webpagetest API run with Google Lighthouse report enabled centminmod/google-insights-api-tools

    Code (Text):
    ./gitools.sh wpt https://community.centminmod.com dulles
    
    --------------------------------------------------------------------------------
    Dulles:Chrome.Cable WPT Results
    --------------------------------------------------------------------------------
    Test ID: 180705_SK_ARW
    https://www.webpagetest.org/result/180705_SK_ARW/
    https://www.webpagetest.org/lighthouse.php?test=180705_SK_ARW
    /home/wptresults/wpt-dulles.chrome.cable.050718-044704.log
    Test Started 27 seconds ago
    Test Started 27 seconds ago (100)
    waiting on results...
    Test Complete
    Test Complete (200)
    waiting on results...
    Test Complete
    Test Complete (200)
    ----
    firstPaint 1235 
    loadTime 1511 
    domInteractive 1459 
    fullyLoaded 3112 
    requests 25 
    TTFB 496 
    domElements 1979 
    visualComplete 2300 
    render 1300 
    SpeedIndex 1326 
    visualComplete85 1300 
    visualComplete90 1300 
    visualComplete95 1500 
    visualComplete99 1500 
    chromeUserTiming.firstPaint 1235 
    chromeUserTiming.firstContentfulPaint 1235 
    chromeUserTiming.domInteractive 1459 
    chromeUserTiming.firstMeaningfulPaintCandidate 1479 
    chromeUserTiming.firstMeaningfulPaint 1479 
    chromeUserTiming.domComplete 1511 
    lighthouse.Performance.first-contentful-paint 1814 
    lighthouse.Performance.estimated-input-latency 38 
    lighthouse.Performance.speed-index 2068 
    lighthouse.Performance.first-meaningful-paint 2176 
    lighthouse.Performance.first-cpu-idle 4164 
    https://www.webpagetest.org/results/18/07/05/SK/ARW/1_waterfall.png
    ok----
    --------------------------------------------------------------------------------
    


    wpt-dulles-slack-waterfall-01.png
     
  4. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Update July 2018: Looks like WPT Google Lighthouse tests now are hard coded to use 3G Fast speed profile.

    Capture Lighthouse Report = uses 3G Fast connection profile

    upload_2018-7-9_16-57-5.png

    Testing run using gitools.sh newly added WPT 3gfast profile test for both Webpagetest + Google Lighthouse report
    Code (Text):
    ./gitools.sh wpt https://community.centminmod.com california 3gfast
    

    Pretty close numbers for WPT speedindex = 1319ms vs Lighthouse speedindex = 1519 ms and WPT first paint = 1299.3 ms vs Lighthouse first contentful paint = 1264.3 ms
    Code (Text):
    ./gitools.sh wpt https://community.centminmod.com california 3gfast
    
    --------------------------------------------------------------------------------
    ec2-us-west-1:Chrome.3GFast WPT Results
    --------------------------------------------------------------------------------
    Test ID: 180709_H0_GXT
    https://www.webpagetest.org/result/180709_H0_GXT/
    https://www.webpagetest.org/lighthouse.php?test=180709_H0_GXT
    https://www.webpagetest.org/testlog.php?days=1&filter=community.centminmod.com&all=on&video=on
    /home/wptresults/wpt-california.ec2-us-west-1.chrome.3gfast.090718-070148.log
    Waiting behind 4 other tests...
    Waiting behind 4 other tests... (101)
    waiting on results...
    Waiting behind 2 other tests...
    Waiting behind 2 other tests... (101)
    waiting on results...
    Waiting at the front of the queue...
    Waiting at the front of the queue... (101)
    waiting on results...
    Test Started 23 seconds ago
    Test Started 23 seconds ago (100)
    waiting on results...
    Test Complete
    Test Complete (200)
    waiting on results...
    Test Complete
    Test Complete (200)
    ----
    bytesIn-fully-loaded 445586
    bytesInDoc-document-complete 299395
    chromeUserTiming.domComplete-time 2083
    chromeUserTiming.firstContentfulPaint 1299
    chromeUserTiming.firstMeaningfulPaint 1791
    chromeUserTiming.firstMeaningfulPaintCandidate 1791
    domComplete-time 2084
    domContentLoadedEventEnd 2077
    domContentLoadedEventStart 2077
    domElements 1981
    domInteractive 847
    firstPaint 1299.2999999988
    fullyLoaded-time 4189
    lighthouse.Accessibility 0.57
    lighthouse.BestPractices 0.71
    lighthouse.Performance 0.97
    lighthouse.Performance.estimated-input-latency 20.2873181818
    lighthouse.Performance.first-contentful-paint 1264.285
    lighthouse.Performance.first-cpu-idle 3151.611
    lighthouse.Performance.first-meaningful-paint 1780.913
    lighthouse.Performance.speed-index 1519
    lighthouse.ProgressiveWebApp 0.65
    lighthouse.SEO 1
    loadTime 2083
    start-render 1300
    requestsDoc 12
    requestsFull 25
    score_cache 75
    score_cdn 100
    score_compress 100
    score_gzip 100
    score_keep-alive 100
    SpeedIndex 1319
    TTFB 667
    visualComplete 3200
    https://www.webpagetest.org/results/18/07/09/H0/GXT/1_waterfall.png
    ok----
    --------------------------------------------------------------------------------
    


    gitools.sh webpagetest + lighthouse results sent to my custom Slack channel :)

    gitools-3g-fast-california-01.png
     
    Last edited: Jul 9, 2018
  5. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  6. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  7. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Webpagetest has now switched to Total Blocking Time metric

    Example testing US East Coast

    wpt-total-blocking-time-cmmofumr-01.png

    Example testing US West Coast

    wpt-total-blocking-time-cmmofumr-02.png
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Webpagetest.org includes Google Lighthouse tests if you choose to enable them as outlined here. Which is same as Google PageSpeed Insights.

    Note Google Lighthouse v6 is out now and scoring metrics have changed with removal of First CPU Idle and First Meaningful Paint and addition of Total Blocking Time, Largest Contentful Paint and Cumulative Layout Shift and also changes to the score weighting to further emphasize the impact of excessive Javascript. See Lighthouse performance scoring

    Lighthouse 5 Audit Weight
    Lighthouse 6 Audit Weight
    As you can see previously Time To Interactive was weighted at 33% of the total score in Lighthouse v5 so even if you had alot of Javascript slowing Time To Interactive, if you had a good First Contentful Paint (20%) and Speed Index (27%) which totaled 47% of the score, you could still get a relatively high Lighthouse score.

    However, with Lighthouse v6 the weighting has changed. They have lowered the weighting on First Contentful Paint (15%) and Speed Index (15%) which totaled 30% of the score. And increased weighting on metrics that Javascript impact such as Time To Interactive (15%) and Total Blocking Time (25%) and Cumulative Layout Shift (5%) which have a combined 45% weighting to the overall score.

    This means if your web site is Javascript heavy, you'd expect lower Google Lighthouse v6 scores in Webpagetest and Google PageSpeed Insights v6.
     
  9. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  10. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Google Search Console's Speed Report has now switched to using Core Web Vital Metrics as well and tracks both Largest Contentful Paint (LCP) and First Input Delay (FID) for real world visitors to your web site via Google Chrome User Experience Report (CRuX) data. You can measure your lab and field/origin metrics via Google PageSpeed Insights v6 & Google Lighthouse v6 metrics.

    The Largest Contentful Paint metric is an interesting one to optimise for. For my forums, majority of the threads are picking up the title bar's H1 tag as the LCP item. So thought I'd use Element Timings API to measure when my title bar's H1 tag actually renders for me in the real world. FYI, Google Search Console's Core Web Vital metrics reported my forum index having a LCP of 1.5s for 75% of visits which would of been made up of both guests + logged in members.

    For forum index page measured H1 title bar rendered times for me in Brisbane, Australia hitting my forums in US West Coast behind Cloudflare
    • Logged in member/visitor no CF full HTML page cache between 2000-2500ms so ~2-2.5 seconds
    • Logged in member/visitor no CF full HTML page cache but watching a video on my computer blew out the H1 tag render time to 19790ms ! Yes 19.79 seconds !
    • Guest logged out visitor with CF full HTML page cache between 900-1300ms with random variation spikes of 2400-4900ms (could be factors local to my PC/apps in use)
    Guess next steps is to optimise your forum page speed for logged in members. But goes to show in real world, your visitors maybe also multi-tasking on their computers which may impact their browsing page load experience and thus impact your real world Google Search Console Speed LCP/FID metrics and Google Page Speed Insight's reported field/origin metrics ;)

    Below is Opera browser dev tools console reported Elements Timing API render times for my forum index's title bar H1 tag.
    • left = logged in member = 2096ms render time
    • middle = logged in member while watching a video at same time = 19790ms render time
    • right = guest logged out visitor = 1032ms render time
    cmm-forum-index-element-tming-api-cache-nocache-watching-video-19790ms.png
     
  11. eva2000

    eva2000 Administrator Staff Member

    54,510
    12,208
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,777
    Local Time:
    10:12 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    Update by adding gitools_v6.sh to support Google Lighthouse v6 API metrics and can run Webpagetest, GTMetrix and Google PageSpeed Insight tests via their respective API. You can run gitools_v6.sh via cronjob to schedule test runs and send results to your own Slack channels. Full details at WebPerf - PageSpeed - PageSpeed Testing Via APIs: Google PageSpeed Insights, GTMetrix & WebpageTest.org

    Example for Webpagetest API run with Google Lighthouse report enabled centminmod/google-insights-api-tools

    Code (Text):
    ./gitools_v6.sh mobile https://community.centminmod.com
    
    --------------------------------------------------------------------------------
    mobile CrUX Rating: AVERAGE
    Test url: https://community.centminmod.com
    FCP: 2052ms (AVERAGE) LCP: 2439ms (FAST) FID: 23ms (FAST)
    29.30% pages fast FCP (<1000ms)
    63.10% pages average FCP (<3000ms)
    7.60% pages slow FCP (>3000ms)
    95.80% pages fast FID (<100ms)
    3.10% pages average FID (<300ms)
    1.00% pages slow FID (>300ms)
    91.70% pages fast CLS (<0.10)
    3.70% pages average CLS (<0.25)
    4.60% pages slow CLS (>0.25)
    76.50% pages fast LCP (<2500ms)
    13.10% pages average LCP (<4000ms)
    10.40% pages slow LCP (>4000ms)
    
    PageSpeed Insights v6 Score (mobile): 82 (average)
    https://community.centminmod.com
    Lighthouse Version: 6.0.0
    Cumulative-Layout-Shift: 0.00
    Time-to-Interactive: 5717
    Speed-Index: 2844
    Largest-Contentful-Paint: 3060
    Total-Blocking-Time: 355
    Total-Page-Size: 434 KB
    First-Contentful-Paint: 2472
    First-Meaningful-Paint: 3269
    First-CPU-Idle: 5672
    Estimated-Input-Latency: 35
    Time-To-First-Byte: 120 ms

    Scheduled cronjob results sent to Slack channel

    gitools_v6-cmm-forums-mobile-01.png
     
Thread Status:
Not open for further replies.