Testing code and packages that communicate with remote servers can be
painful. Dealing with authentication, bootstrapping server state,
cleaning up objects that may get created during the test run, network
flakiness, and other complications can make testing seem too costly to
bother with. But it doesn’t need to be that hard. The
httptest2 package lets you test R code that constructs API
requests and handles their responses, all without requiring access to
the remote service during the test run. This makes tests easy to write
and fast to run.
This vignette covers some of the core features of the
httptest2 package, focusing on how to mock HTTP responses,
how to make other assertions about requests, and how to record real
requests for future use as mocks. Note that httptest2
requires the testthat package, and it follows the testing
conventions and interfaces defined there, extending them with some
additional wrappers and expectations. If you’re not familiar with
testthat, see the “Testing” chapter chapter of
Hadley Wickham’s R Packages book.
httptest2 is designed for use with packages that rely on
the httr2 requests
library—it is a bridge between httr2 and
testthat. If you’re using httr, use httptest, on
which this package is based.
First, we need to add httptest2 to our package test
suite. Once you’ve configured your package to use testthat
(usethis::use_testthat() is one way), run
httptest2::use_httptest2(). This adds
httptest2 to your package DESCRIPTION and
makes sure it is loaded in your test setup code.
httptest2 provides several contexts, which are
“with”-style functions that you wrap around other code you want to
execute.
without_internet(): any httr2 request
performed will raise an informative errorwith_mock_api(): requests will look for an associated
mock file to return instead of making a real request; if none found, it
will raise an informative errorcapture_requests(): real requests will be saved to mock
files so that with_mock_api() can load themwith_mock_dir(): wrapper around
capture_requests() and with_mock_api(),
switching between them based on the existence of the given directory. If
it exists, with_mock_api() will use it; if not, real
requests will be made and responses will be recorded into it so that
subsequent runs will use them as mocks. To re-record real responses,
delete the directory.A good place to start is with a small number of end-to-end tests:
things that you can run against the real API to confirm that the whole
workflow functions correctly. To do this, we’ll use the
with_mock_dir() context.
Let’s suppose we’re making a package from the first example from the
httr2 vignette on wrapping
APIs, which wraps the faker
API:
faker_person <- function(gender = NULL, birthday_start = NULL, birthday_end = NULL, quantity = 1, locale = "en_US", seed = NULL) {
faker(
"persons",
gender = gender,
birthday_start = birthday_start,
birthday_end = birthday_end,
quantity = quantity,
locale = locale,
seed = seed
)
}
faker <- function(resource, ..., quantity = 1, locale = "en_US", seed = NULL) {
params <- list(
...,
quantity = quantity,
locale = locale,
seed = seed
)
names(params) <- paste0("_", names(params))
request("https://fakerapi.it/api/v1") %>%
req_url_path_append(resource) %>%
req_url_query(!!!params) %>%
req_user_agent("my_package_name (http://my.package.web.site)") %>%
req_perform() %>%
resp_body_json()
}We could add a test file at tests/testthat/test-person.R
in our package and put this test in it:
test_that("We can get people", {
expect_length(faker_person("female", quantity = 2), 2)
})If we run that test, it will make a live API call and return a response containing 2 fake people. Every time we run it, it will make an API request, and if there is no network connection, or the API happens to be down, the test will fail.
Instead, let’s wrap this test in with_mock_dir() so that
we can record that response the first time we run it and then load it as
a mock every time after.
with_mock_dir("person", {
test_that("We can get people", {
expect_length(faker_person("female", quantity = 2), 2)
})
})To illustrate how that works, let’s run that again outside of the test and with some verbose messages turned on.
options(httptest2.verbose = TRUE)
with_mock_dir("person", {
system.time(peeps <- faker_person("female", quantity = 2))
})
## Recording responses to tests/testthat/person
## Writing /Users/you/fakerpkg/tests/testthat/person/fakerapi.it/api/v1/persons-fdcc43.json
## user system elapsed
## 0.022 0.003 0.611Because tests/testthat/person does not exist, it makes a
real request, which takes around 600 milliseconds. And now if we look at
the file system, we see a new file
tests/testthat/person/fakerapi.it/api/v1/persons-fdcc43.jsonwhich contains the JSON response body from our request:
{
"status": "OK",
"code": 200,
"total": 2,
"data": [
{
"firstname": "Kyra",
"lastname": "Schuster",
"email": "mraz.jeffery@abernathy.com",
"phone": "+5531037042476",
"birthday": "1932-09-11",
"gender": "female",
...
},
...
]
}Note how the file path matches the request URL. Requests are
translated to mock file paths according to several rules that
incorporate the request method, URL, query parameters, and body. First,
the request protocol is removed from the URL. Second, if the request URL
contains a query string, it will be popped off, hashed by
digest::digest(), and the first six characters appended to
the file being read. Third, request bodies are similarly hashed and
appended. Finally, if a request method other than GET is used it will be
appended to the end of the end of the file name.
If you want to know what request was made, you can run the code
inside without_internet(), and the error message will tell
you:
without_internet({
faker_person("female", quantity = 2)
})
## Error: An unexpected request was made:
## GET https://fakerapi.it/api/v1/persons?_gender=female&_quantity=2&_locale=en_US
## Run `rlang::last_error()` to see where the error occurred.Back to with_mock_dir(): the next time we run the same
code, we’ll see that it loads the mock file.
with_mock_dir("person", {
system.time(peeps2 <- faker_person("female", quantity = 2))
})
## Using mocks found in tests/testthat/person
## Reading /Users/you/fakerpkg/tests/testthat/person/fakerapi.it/api/v1/persons-fdcc43.json
## user system elapsed
## 0.011 0.000 0.011Because the person directory exists now,
with_mock_dir() uses the previously recorded mocks, matches
our exact query with the one we did before, and returns it instantly,
without making a network connection. We can see that it is using the
previously recorded response because the result is identical—with the
faker API, you get different data back for every request.
identical(peeps, peeps2)
## [1] TRUENote that none of the test code inside with_mock_dir()
looks any different from how you’d write it if you were testing against
a live server using just testthat. The goal is to make your
tests just as natural to write as if you were using your package
normally.
Do your recorded responses contain sensitive information? Are they unnecessarily large? See
vignette("redacting", package = "httptest2")for how to trim and sanitize recorded mocks.
with_mock_dir() is convenient to use, but you probably
don’t want all of your tests to be based on real requests. Among the
reasons:
#rstats, the specific content will change
every time you record it, so your tests can’t say much about what is in
the response without having to rewrite them every time too.Moreover, we generally don’t need to test what the server does with our requests. We’re responsible for testing the R code on either side: (1) given some inputs, does my code make the correct HTTP request(s); and (2) does my code correctly handle the types of responses that the server can return? We often don’t need to make real requests to test that.
Sometimes it is more clear what you’re testing if you focus only on the requests. One case is when the response itself isn’t that interesting or doesn’t tell you that the request did the correct thing. For example, if you’re testing a POST request that alters the state of something on the server and returns 204 No Content status on success, nothing in the response itself (which would be stored in the mock file) tells you that the request you made was shaped correctly—the response has no content. A more transparent, readable test would just assert that the POST request was made to the right URL and had the expected request body.
Another case where you might want to focus just on the request,
without recording real API responses, is when you don’t actually want to
make the requests. Perhaps you’re testing a function that would call
DELETE on some resource. Or, to make it concrete, suppose
you were testing a function that sends a tweet: you probably don’t want
to send a real test tweet out into the world.
Here’s a function that would send a tweet using the Twitter API. Consulting the API docs, it looks like this is the request we’d make:
send_tweet <- function(text) {
resp <- request("https://api.twitter.com/2/tweets") %>%
req_method("POST") %>%
req_body_json(list(text = text)) %>%
req_perform()
resp_body_json(resp)$data
}Of course, this is overly simplified. According to the docs, there are lots more arguments you would want to add, and also your real function would need to apply your OAuth token. If you were really writing this function (or broader package), you’d solve that. For the purposes of illustration here, let’s assume that’s taken care of.
How do we test this without making a real request? First, let’s run
the function inside without_internet() so we can see what
request it makes:
without_internet({
send_tweet("Hello world!")
})
## Error: An unexpected request was made:
## POST https://api.twitter.com/2/tweets {"text":"Hello world!"}
## Run `rlang::last_error()` to see where the error occurred.That looks right according to the API documentation. We can now turn
this into a test that asserts we’re making the correct request, using
expect_POST().
without_internet({
test_that("We can send a tweet", {
expect_POST(
send_tweet("Hello world!"),
'https://api.twitter.com/2/tweets',
'{"text":"Hello world!"}'
)
})
})
## Test passed 🌈expect_POST() (and the other expect_VERB
functions) look for the request method, URL, and body, just as they are
printed in the error message if the request is not caught by
expect_POST().
If the test fails because the request that is made doesn’t match what we expected, it looks like this:
without_internet({
test_that("We can send a tweet", {
expect_POST(
send_tweet("Goodbye world!"),
'https://api.twitter.com/2/tweets',
'{"text":"Hello world!"}'
)
})
})
## ── Failure (Line 3): We can send a tweet ───────────────────────────────────────
## An unexpected request was made:
## Actual: POST https://api.twitter.com/2/tweets {"text":"Goodbye world!"}
## Expected: POST https://api.twitter.com/2/tweets {"text":"Hello world!"}Next, suppose we wanted to test how our code handles the API response. In our example, we aren’t doing much, just extracting the “data” element from the response, but in practice there could be much more involved code to turn API responses into objects that make intuitive sense to R users.
We can still set up this test without making a real request by
creating a mock based on the API documentation, which includes a sample
response. To start, let’s switch from without_internet() to
with_mock_api() and re-run the function:
with_mock_api({
send_tweet("Hello world!")
})
## Error: An unexpected request was made:
## POST https://api.twitter.com/2/tweets {"text":"Hello world!"}
## Expected mock file: api.twitter.com/2/tweets-8d6065-POST.*
## Run `rlang::last_error()` to see where the error occurred.We got the same error message before, but there is an extra line
telling us what mock file it was looking to load. If we take the JSON
example response from the API documentation and put it in that location,
with_mock_api() will load it.
dir.create("tests/testthat/api.twitter.com/2", recursive = TRUE)
cat('{
"data": {
"id": "1445880548472328192",
"text": "Hello world!"
}
}', file = "tests/testthat/api.twitter.com/2/tweets-8d6065-POST.json")
with_mock_api({
send_tweet("Hello world!")
})
## $id
## [1] "1445880548472328192"
##
## $text
## [1] "Hello world!"
##Now we can test that we can make that request and the mock response is loaded:
with_mock_api({
test_that("We can send a tweet", {
expect_equal(
send_tweet("Hello world!")$text,
"Hello world!"
)
})
})
## Test passed 🥇Because requests made in with_mock_api() that don’t find
a corresponding mock file raise the same error as
without_internet(), we can combine mock tests and tests
that assert that a request would be made:
with_mock_api({
test_that("We can send a tweet", {
# This one has a mock file now
expect_equal(
send_tweet("Hello world!")$text,
"Hello world!"
)
# This one does not
expect_POST(
send_tweet("Goodbye world!"),
'https://api.twitter.com/2/tweets',
'{"text":"Goodbye world!"}'
)
})
})
## Test passed 🌈For more examples of cases to test like this, see the httptest
vignette and this
blog post.