Organize your tests in Xcode using Test Plans
How can we run our tests under different scenarios and configurations using Xcode Test Plans
Working on big projects has several challenges attached.
One of those is ensuring the quality of the code before shipping a new build to production. Having a solid test structure in place could help. With a solid structure, I mean not only having both Unit Tests & UI Tests but also testing our app under different configurations.
This way, we’ll cover as many scenarios as possible, and by default, we’ll improve our app’s quality.
Here’s when we can leverage Test Plans.
Let’s see how we can add them to our projects.
Introduction to Test Plans
A test plan is a document where we specify which tests we want to run and which configurations we should use.
That’s it.
All test plans have a shared configuration and as many custom configurations as we want. Having custom configurations is optional, but if the test plan has at least one, it will override the shared configuration.
Each test plan has one or more test bundles. It could be Unit Test Bundle or UI Test Bundle.
Configuring a Test Plan
Start by creating a new test plan. We can do this from the test navigator, from the menu by creating a new file, or by editing our scheme.
If you open the test plan file, you’ll see two tabs. One for setting the tests we want to include in the plan, and another for customizing the configurations.
Adding Tests to the Test Plan
We can add them from the navigator panel by right-clicking on the test bundle we want to include and selecting the option Add to TestPlan.
Also, we can add the bundles directly from the test plan file. Click on the + button in the bottom left corner and select the bundles you want to include.
For every bundle we add to the plan, we can set a couple of settings by clicking on the options button.
Configurations
If we go to the configurations tab inside the test plan file, we’ll see all the settings available.
Remember, the Shared Settings are the configurations that apply to all test plans, so be careful when changing things here.
If we want to set specific configurations for our test plan, we can add as many as we want. For example, we might want to run our tests with Address Sanitizer option, and another with the Thread Sanitizer option.
In case you’re interested or you don’t know:
- Address Sanitizer helps us to detect memory issues.
- Thread Sanitizer helps us to identify data races.If you want to learn more, I would recommend checking Apple’s documentation Diagnosing memory, thread, and crash issues early
Running Specific Test Plans
Once we have all our tests organized in test plans we could choose to run all the configurations at once or a specific one. This is also applicable for running individual test methods or individual test classes.
Just right-click the test you want to execute and choose between the options.
If we look at the test report, we'll see the status of each configuration used for the test plan.
In this case, I created two configurations for running my UI tests: one with Application Language = English and another one with Application Language = Spanish.
If our app starts to grow, so do our tests. Or at least it should be that way, it’s a direct relation if we want to keep a specific quality standard.
Test plans are helpful to keep our tests organized. In addition, creating custom configurations would help us group our test under different scenarios and conditions.
Although I don’t think every project we’re working on needs to use test plans, you should give it a try. Play around with it, test different configurations, and then evaluate if it would be a good fit to include it in your project.
This is my third article about testing in Swift. If you’re interested, you can check the previous ones:
