Test Results

So you've created your test, verified it, run it, and watched the results come in real-time on a niftly little graph. What does it all mean? Let's break down the results page.

Summary Data #

This section shows you a basic summary of the test:

  • Response Times
    • Average - the average time it took to get a response
    • Min - the fastest response
    • Max - the slowest response
    • Note that if a test has multiple URLs, these times are the sum of average, min, and max times for each URL. Pro users can select a specific URL to view the times for that specific URL 
  • Response Counts
    • Success - Anything with a response code less than 400 (usually 200s, but also includes 201, 202, 301, 302, etc).
    • Errors - Response codes 400 and up are grouped into buckets of 4xx and 5xx.
    • Timeout - If a client does not receive any data for the configured timeout period (by default 10 seconds), it is counted as a timeout. Note that this may or may not correspond to a timeout on your server, as many servers have a default timeout of 30 or 60 seconds.
    • Network - Network errors mean that either DNS resolution or TCP connection failed. Your DNS records may not be updated, or your server may be down or listening on the wrong port
  • Bandwidth
    • Sent - Data sent by loader in requests
    • Received - Data received by loader in responses
  • Redirects
    • Valid - Redirects to a verified target host are followed by loader
    • Invalid - Redirects to a host that is not verified are not followed. This isn't an error unless you need that redirect to be followed. If you do, you need to register a new target host and verify it.

Trends

The average response time shown under "trends" is the average response time over all test runs.

Click on the trends to see more detail and a summary of each run:

Graphs #

There are three graphs (four for pro users!) that give more detailed information about what is going on during the test:

  • Times shows the average response times and the number of active clients
  • Details shows details about requests made, responses, and errors
  • Bandwidth shows how much bandwidth was used
  • Distribution (Pro only) shows histograms of responses times and bandwidth

On each graph, you can show or hide different parts of the chart by clicking on the label in the key below the graph.

Times #

The response times graph shows the average response times (blue) and the number of active clients (green). For pro users, you can toggle the min/max response time range as well, with the average drawn inside of it.

Note that "active clients" is just the number of clients still waiting for a response at any point. It may or may not correlate with the number of clients you specified when creating the test, but if your server is responding quickly it usually will.

Details #

The details graph shows details about:

  • Total requests made
  • Total responses received
  • Success responses (response codes under 400) - ideally all responses fall in this category
  • 400-level errors (response codes 400-499) - often these indicate authentication problems, something is missing, or a number of other problems.
  • 500-level errors (response codes 500-599) - generally application errors. Check your logs for error details
  • Timeouts - no data received for the timeout period (10 seconds by default)
  • Network errors - DNS resolution or TCP connection problems

Details graphs can be rather boring if everything is looking good:

Or, when there are lots of errors they can look a little dangerous:

Bandwidth #

This graph just shows how much data loader sent in requests (green) and received from your app.

Distribution #

For pro users we display a histogram of response times and bytes received from your app. The histogram is generated by dynamically bucketing similar response times and sizes. The height of each bar on the histogram represents how many responses fit into that bucket. This shows you more detail than the average and min/max - the average is sometimes skewed, and min/max can sometimes be outliers. With histograms you see the spread of where all requests fall, soyou can see where the majority of your responses lie in relation to the minimum, average, and maximum response times.
For example in the histogram pictured here, the minimum response time was 131ms and the max was 363ms. The histogram shows that very few responses took longer than 120ms.