You're preparing for a release readiness meeting.
The Test Plan contains hundreds of Tests. Development continued until late yesterday, several Defects were resolved overnight, and only a few hours remain before stakeholders need an update. There is enough time to execute part of the Test suite, but not all of it.
The question isn't whether testing should continue. It's which Tests should be executed first. Every release forces QA teams to make prioritization decisions. The difference between a confident release and a risky one often comes down to whether the earliest Test results provide meaningful insight into the areas that matter most.
Xray's AI Test Prioritization helps teams answer that question with greater confidence. Powered by Sembi IQ, it analyzes historical testing data together with user-defined context to intelligently prioritize Tests within Test Plans and Test Executions, helping QA teams focus their efforts where they are most likely to reduce release risk.
Every release involves trade-offs
Software teams rarely work with unlimited testing time.
Some new functionality continues to be developed, Requirements evolve throughout the sprint, and release deadlines rarely move to accommodate additional testing. Even organizations with mature automation strategies eventually reach the point where execution time becomes the limiting factor.
When that happens, QA teams naturally begin making prioritization decisions. Some rely on experience. Others focus on recently modified functionality or business-critical features. While these approaches are valuable, they also depend heavily on individual knowledge and can vary between projects, teams, and release cycles.
The challenge is making those decisions consistently and using the testing intelligence already available across the project.
What makes one Test more important than another?
Some Tests consistently validate stable functionality, while others target areas that have historically generated Defects or experienced frequent failures. Certain Tests cover recently updated Requirements, while others validate functionality that hasn't changed for months.
AI Test Prioritization evaluates these differences by analyzing multiple signals already available in Xray, including:
- Historical Test Run results
- Associated Defects
- Recent and overall failure rates
- Test flakiness
- Test metadata, including Test type, labels, priority, and status
- Requirements and prerequisites
- User-defined prompts
- Optional Jira labels and configurable analysis timeframes
Rather than applying a fixed priority to every Test, the feature evaluates these signals together to recommend an execution order that reflects the current state of the project.
From recommendation to execution
Using AI Test Prioritization fits naturally into existing Jira workflows. Teams begin by opening a Test Plan or Test Execution and describing, in natural language, how they want Tests to be prioritized. They can further refine the analysis by selecting an analysis timeframe and specifying up to three Jira labels to provide additional context.
Sembi IQ analyzes the available testing data and automatically re-ranks Tests based on the selected criteria, allowing teams to begin execution with a prioritized Test list instead of manually reviewing hundreds of Test Cases.
The feature also introduces AI Priority Insights, a dedicated field that explains why each Test received its assigned priority. Every prioritization run is recorded in the Test Plan or Test Execution history, giving teams complete visibility into the strategy applied and making it easier to understand how execution decisions were made.
The value of AI Test Prioritization isn't simply that it changes the order in which Tests are executed. It changes the quality of the information available when release decisions need to be made.
When the highest-priority Tests are executed first, early Test results become significantly more valuable. QA Leads gain faster visibility into potential risks, Test Managers can adapt execution plans based on meaningful feedback instead of waiting for complete Test suite execution, and engineering teams receive earlier insight into areas that require attention before deployment.
Even when time constraints prevent every Test from being executed, teams can move forward with greater confidence because the functionality most likely to affect release quality has already been validated.
Building a more consistent testing process
Prioritization often depends on individual experience, making it difficult to apply a consistent execution strategy across multiple teams or projects.
AI Test Prioritization introduces a repeatable approach that can be applied regardless of team size or project complexity. Because prioritization decisions are supported by AI Priority Insights and recorded in execution history, teams gain transparency into how recommendations are generated while maintaining complete control over the final execution strategy.
This not only improves consistency but also simplifies collaboration between QA, engineering, and release stakeholders by providing a shared understanding of why certain Tests are executed first.
Part of Xray's latest AI capabilities
AI Test Prioritization is part of Xray's latest AI capabilities, supporting different stages of the testing lifecycle.
- AI Test Case Generation helps teams accelerate Test Case creation from Requirements.
- AI-powered Test Script Suggestions in Test Case Designer help teams generate data-driven Manual Scripts and Gherkin BDD Scripts using natural language descriptions.
- AI Test Model Generation (available in Xray Enterprise) supports model-based testing and structured coverage design.
- Xray's Rovo Test Plan Summarizer generates natural language summaries of Test Plans, helping teams quickly understand execution progress, coverage, and release readiness.
Together, these capabilities help teams design Tests more efficiently, prioritize execution more intelligently, and make faster, data-driven decisions throughout the software testing lifecycle.
Getting started with AI Test Prioritization
AI Test Prioritization is available across all Xray Cloud plans, with usage limits based on your subscription:
- Xray Standard: up to 75 Tests
- Xray Advanced: up to 150 Tests
- Xray Enterprise: up to 300 Tests
To use this feature, a Jira administrator must first enable the feature through Xray's AI Hub before making it available to all users or selected groups.
If you are using an older version of Xray, your Jira administrator will need to update the app through the Connected Apps section to access these capabilities.
FAQ: AI Test Prioritization
How does AI Test Prioritization help QA teams make better release decisions?
It helps QA teams execute the most relevant Tests first by analyzing historical Test Run data, associated Defects, Test metadata, Requirements, and user-defined context. This allows teams to identify potential risks earlier and make more informed release decisions, even when testing time is limited.
How does AI Test Prioritization work?
Teams provide a natural language prompt describing how they want Tests to be prioritized and can optionally refine the analysis using Jira labels and a configurable analysis timeframe. Sembi IQ analyzes multiple testing signals before automatically re-ranking Tests and providing AI Priority Insights that explain each assigned priority.
Which Xray plans include AI Test Prioritization?
AI Test Prioritization is available across all Xray Cloud plans:
- Xray Standard: up to 75 Tests
- Xray Advanced: up to 150 Tests
- Xray Enterprise: up to 300 Tests
The feature must first be enabled by a Jira administrator through Xray's AI Hub.
What information does AI Test Prioritization analyze?
AI Test Prioritization analyzes historical Test Runs, associated Defects, recent and overall failure rates, Test flakiness, Test metadata, Requirements, user-defined prompts, optional Jira labels, and configurable analysis timeframes to determine the most effective execution order.
What are AI Priority Insights?
AI Priority Insights are generated for every prioritized Test and explain why it received its assigned priority. This provides greater transparency into the prioritization process and helps teams understand the reasoning behind the recommended execution order.
How does AI Test Prioritization fit into Xray's latest AI capabilities?
AI Test Prioritization complements Xray's latest AI capabilities, including AI Test Case Generation, AI-powered Test Script Suggestions, AI Test Model Generation, and Xray's Rovo Test Plan Summarizer. Together, these capabilities support a connected workflow across Test design, execution planning, reporting, and release readiness.
