Before Xray 2.0, there was no specific Test Plan issue, though you could implement your “test plan” in two ways:
- just by creating Test Executions and consider them as your “test plan” for a given version
- or, by creating Test Sets; you could define one or more Test Sets for a given version using the Fix Version field
Either way, probably, you will have many test cycles or as we call – “Test Executions”. Each Test Execution has its own tests, depending on the requirements you want to validate, the available resources, the target revision, etc. Each one may be complementary, overlapping, be a subset/superset of another one or have exactly the same tests.
It’s quite normal that you end up having something similar to the following picture.
So, how can you have an overall status of your tests?
Introducing the “Test Plan”
The Test Plan purpose is to address effective test planning, namely by tracking relevant tests (i.e. the ones in the Test Plan) and their results.
With the new Test Plan issue, you have a consolidated overview of tests. You are also able to assign one, or more, Test Plans to a given release and track progress of those tests within each Test Plan.
Your Test Plan starts by specifying the planned begin and end dates, along with the Tests that will be part of it. Normally, you would also indicate the Fix Version, so that executions you create in the Test Plan context will inherit it.
As soon as you create a Test Plan, you are able to add more Tests, even from existing Test Sets. A progress bar shows the “Overall Execution Status”, grouping and couting tests by status.
Executing tests for a Test Plan is straightforward; you can easily create a Test Execution for all the tests in the Test Plan, or just for the ones that are still failing, or that are not yet ran, or for any other test run statuses you wish to.
If you already have Test Executions, you can make them part of an existing Test Plan by setting the “Test Plan” custom field in the Test Execution. As simple as that.
In the Test Plan screen you may see, and filter by, the tests that make your Test Plan.
For each Test, you may see its latest status and all related Test Executions.
As soon as you start executing tests in Test Executions that are associated with the Test Plan, you will see the results being reflected directly on the overall execution status. Notice the “Test Plan Status” custom field, which you may include in your screens or in your JIRA’s issues search results table, as a column.
Besides the traditional way of calculating requirement coverage for a given version, the requirement coverage may now be calculated taking into account tests from a given Test Plan, and related Test Executions. Therefore, you are able to analyze requirements based on a wider scope (i.e. the “version”) or on a more constrained way, just for the Test Plan.
Until now you had to organize your tests in Test Sets. And that makes all sense. But one thing is organizing the tests, by grouping them, and another thing is planning tests for a given release. Test Executions provided the basics for test planning but lacked a higher level and consolidated view of the status of a group of tests.
Xray’s “Test Plan” allows effective test planning and facilitates schedule test execution and tracking testing progress, for the tests that are most relevant to you.
Finally, before you ask…
- A “Test Plan” seems like a group of Test Executions or a group of Tests…
Yes, a bit of both in fact. A Test Plan contains a list of Tests. For calculating the status of each test, it takes into account test runs from Test Executions explicitly linked to the Test Plan, through the field “Test Plan”.
- Can I clone “Test Plans”?
Yes. The list of related Tests is copied; Tests are not cloned.
- Can I have multiple Test Plans for a version?
Yes, if you wish to track different groups of Tests (e.g. regression tests vs tests for new requirements).
- Do I have to use “Test Plans”?
You don’t but soon you will find great advantages if you start using them.