Four Key Reasons to Learn Markdown
Back-End Leveling UpWriting documentation is fun—really, really fun. I know some engineers may disagree with me, but as a technical writer, creating quality documentation that will...
In my previous post, we got request tests working in a Vapor 2.0 app. But we ran into a snag: the records created by the tests stuck around in our development database. This can cause two problems: your test data can get in the way of your development data, and your development data can can cause your tests to fail because of records the tests don’t expect.
Thankfully, there is a way to fix those problems. In this post, we’ll create a separate testing database and reset it after each test. Along the way, we’ll talk about what database to use and how to create records for your tests. Like last time, you can follow along with these steps, or get the completed project from GitHub.
The first decision to make is what kind of database to use for testing. Object-Relational Mapping libraries (ORMs) like Vapor’s Fluent allow you to change the database driver you’re using without making any changes to the rest of your code. This means you have the option to use different kinds of databases in development, testing and production. The early Rails community took advantage of this by using the simple Sqlite for local development work and a more robust multi-user database in production.
This should work in theory, if you aren’t using any database-specific queries. Eventually, though, the Rails community found that there are always implementation details that differ between databases: name length limits, data storage types and default sort orders, for example. You can work around individual differences, but you can never be sure when another difference will bite you. Although ORMs are beneficial for simplifying your database access code, they don’t really allow you to interchange databases with zero effort.
Because of these problems, the typical practice in the Rails community today is to keep your development and testing environments as similar to production as possible. We’ll follow that advice and use Postgres for our tests as well. We’ll just create a separate Postgres database.
Our development database is named nerdserver_development
; let’s call the test database nerdserver_test
. At the command line, run:
createdb nerdserver_test
When Vapor’s Postgres driver sets up a database connection, it checks your application’s configuration to get the database connection info. To change our tests to use our new database, we need to vary the database connection info that Vapor sees in different environments. Vapor’s built-in environments include development
for when you run the server locally, test
when you run your tests and production
which you can use for your live app.
Vapor allows us to vary the configuration across environments by creating different config files for each environment. If we add a folder under Config/
matching the name of the environment, any config files in there will override configs in the root Config/
folder. If you have a config file in secrets/
, that will override any other values. So to get a different database connection in the development and test environments, we’ll need to move our connection out of secrets/
and into development/
and test/
.
In your Config
folder, create a folder called development
and one called test
. Move your postgresql.json
file from Config/secrets/
to Config/development/
. Then copy postgresql.json
into Config/test/
. In that version, change the name of the database to nerdserver_test
:
{
"hostname": "127.0.0.1",
"port": 5432,
"user": "postgres",
"password": "",
- "database": "nerdserver_development"
+ "database": "nerdserver_test"
}
Now that you have a different postgresql.json
configuration for the development
and test
environments, your tests should be using a different server than your development server. Let’s confirm that this is the case. Connect to your development database using your SQL tool of choice, then execute DELETE FROM POSTS;
to clear out all your posts. Go to Xcode, and run your app using Product > Run. Now that the development database is empty and your server is running, add one post to the development database from Terminal.app:
# curl
--header 'Content-Type: application/json'
--data '{"content": "Hello, world!"}'
http://localhost:8080/posts
{"content":"Hello, world!","id":1}
# curl http://localhost:8080/posts
[{"content":"Hello, world!","id":1}]
Next we’ll make sure this record you added to your development database isn’t overwritten by the tests.
Run your test several times from Xcode’s Product > Test menu item to see that they succeed. Then launch your app again from Product > Run and lists all the posts in the development database using curl http://localhost:8080/posts
. It should still show only the “Hello, world!” post you created. This means your tests are running against a test database without affecting your development data!
Note that, unlike secrets/
, the environment folders you created aren’t ignored by Git by default, so your postgresql.json
files are available to commit to Git. It’s not really a security issue to commit them, but it may be an inconvenience to other developers if their local Postgres connection info is different from yours. So instead of committing them you may want to add **/postgresql.json
to your .gitignore
file.
Now that our test for creating a record is working, let’s add a test for the GET /posts
endpoint, which lists all the posts. This will demonstrate another challenge when using databases in testing.
First, let’s set up a few posts. We’ll use our Post
model class directly, so we need to import our App
target so we have access to it:
import Vapor
import XCTest
import Testing
+@testable import App
class PostRequestTests: TestCase {
What’s up with the @testable
annotation on import App
? Our Post
class is declared with the default internal
visibility, which means that it’s not accessible from within other targets like our testing target. We could get around this by declaring Post
and all its properties public
, but this is tedious because this isn’t library code where the public
access level is otherwise important. Instead, we use the @testable
annotation to allow internal
types like Post
to be used from our test target anyway.
Now that we have access to the Post
, let’s create some test data:
func testList() throws {
try Post(content: "List test 1").save()
try Post(content: "List test 2").save()
}
You may be wondering why we’re creating records with the Post
model instead of by sending a server request. There are testers who use either approach, and there are pros and cons to each. One view argues that you should set up test data using only publicly-available endpoints—that way you can’t end up with data in a state that’s impossible in the real app.
The other view is called direct model access. It says that populating data via endpoints creates coupling between tests and application code that’s unrelated to the test. If your post-creation feature breaks, it won’t just be the test for the post-creation feature that fails: lots of tests will fail. Using the Post
model directly is a way to keep your tests focused. It also makes the test more readable: in our case, you can easily see that we are creating two Post
s. Because of these advantages, direct model access is the approach I usually take in my tests.
Now that our data is set up, let’s send a request for /posts
.
func testList() throws {
try Post(content: "List test 1").save()
try Post(content: "List test 2").save()
+
+ let response = try droplet.testResponse(to: .get, at: "/posts")
}
Finally, we’ll confirm we get back the posts we expect. In Part 1, we used Vapor’s built-in assertion helper methods, but this time we need a bit more flexibility, so we’ll use basic XCTAssert
s:
let response = try droplet.testResponse(to: .get, at: "/posts")
+ XCTAssertEqual(response.status, .ok)
+
+ guard let json = response.json else {
+ XCTFail("Error getting JSON from response (response)")
+ return
+ }
+ guard let recordArray = json.array else {
+ XCTFail("Expected response json to be an array")
+ return
+ }
+ XCTAssertEqual(recordArray.count, 2)
+ XCTAssertEqual(recordArray.first?.object?["content"]?.string, "List test 1")
}
We confirm that:
.ok
."content"
field of the first entry is “List test 1”.We could also have checked the first entry’s ID field, as well as the ID and content fields of the second post. How many assertions to use in a test is a judgment call: you should add assertions that increase your confidence in your code, and avoid ones that are just repetition.
Run the test a few times. It fails! We’re asserting the recordArray.count
should be exactly two, but it goes up by two each time we run the test. It’s finding not just the records it inserted during the current test run, but also all the records it inserted in the past.
One way to fix this error would be to loosen our test criteria: instead of asserting the total number of posts, we could just test the last two posts to see if they are the ones we inserted. That would get our test passing for now, but it leaves a bigger issue unsolved. Tests should be independent from one another, so that running one has no effect on others.
Usually the best way to achieve independence for database tests is to use transactions to roll back database changes made during each test. But the way transactions work in Vapor 2.0 makes them difficult to use for this purpose. Instead, we’ll just delete all the records from the posts
table before each test. Because we’re using a test database that’s separate from our development database, there’s no risk of us deleting records we need.
Let’s clear the table in the setUp()
method, so all future tests we write in this class will start with an empty posts
table:
override func setUp() {
super.setUp()
Testing.onFail = XCTFail
+ try! Post.all().forEach { try $0.delete() }
}
Run your tests again, and now they should pass because /posts
only returns the two records you created.
Clearing out a single table is easy, but once you get a lot of related tables in your database, it can be tedious to delete records in an order that prevents foreign key errors. I’ll continue to investigate using transactions for testing instead.
In this two-part series we:
These practices will help you get to the sweet spot of testing, where the benefit outweighs the cost. You get protection against regressions without needing to spend a lot of effort maintaining your tests.
If you’re interested in getting this kind of test coverage for a new or existing web app, Big Nerd Ranch would love to talk with you. We can offer a unique combination of cross-platform back-end experience and deep Swift knowledge. Get in touch with us for more information on what we can do for you!
Writing documentation is fun—really, really fun. I know some engineers may disagree with me, but as a technical writer, creating quality documentation that will...
Humanity has come a long way in its technological journey. We have reached the cusp of an age in which the concepts we have...
Go 1.18 has finally landed, and with it comes its own flavor of generics. In a previous post, we went over the accepted proposal and dove...