Using Atlassian Rovo and Xray - The future of AI-Powered Test Management

What is Rovo?

Atlassian Rovo is the latest evolution of AI-powered knowledge discovery and workflow assistance, built by Atlassian to help teams turn scattered information into meaningful actions. 

Rovo works on three core dimensions:

• Find: offers unified search across Atlassian tools (like Jira and Confluence) and third-party systems, delivering contextual results without switching tools
• Learn: uses AI to surface insights, provide knowledge cards, and enable conversational chat, helping users explore their company’s data in context. 
• Act: introduces AI-agents and automation: customizable virtual assistants that execute routine tasks, generate content, and integrate into workflows so teams can move faster and more decisively.

Rovo is available in all Atlassian products by default. Users have multiple methods to engage with the AI, each offering different levels of complexity and interaction:

• Search Functionality: users can access Rovo via the App Store and initiate basic searches. This interaction mode generally relies on syntax-based queries.
• Chat Functionality: the chat interaction can be utilized via a dedicated chat-only page or summoned wherever the user is working by asking Rovo directly. This mode is generally considered to be more understanding of context than the search functionality. It is better suited for complex natural language processing (NLP) queries, though users still need to frame requests as effectively as possible.

How does Rovo work within Jira?

Rovo operates as an advanced search engine, accessing general Jira issue data. Rovo excels at accessing and processing information that falls within standard Jira issue type fields, such as Title, Descriptions, Labels, Assignees, Components, etc. It can also access third-party systems and use that information to build on the results it delivers, such as the Xray Documentation. Finally, Rovo is also an excellent tool for transferring information between Atlassian products, such as Confluence, based on the retrieved data.

When exploring Rovo’s interaction with Xray, we identified a few areas for improvement. Notably, Rovo can only access data within Jira and cannot access any data within your Xray Cloud storage. Because of this, Rovo may occasionally have difficulty interpreting relationships between Xray artifacts and Jira fields. Similarly, as Xray maintains its own Cloud Storage, certain elements such as the Coverage Panel, Testing Board, Test Status, or the detailed content within a Test Plan are not currently accessible to Rovo.

It’s also worth noting that Rovo’s Natural Language Processing (NLP) capabilities are more responsive to precise queries. Clear and specific prompts (e.g., using “Xray test plans” instead of “tasks”) consistently yield better, more accurate results. A valuable tip to maximize the tool’s potential.

Overall, Rovo demonstrates strong performance in cross-system data retrieval and intelligent search, with room for deeper integration into Xray’s specialized data structure.

Using Rovo with Xray information

Despite the limitations, Rovo offers powerful search and creation capabilities that can immediately benefit Xray users, especially for quick visibility and documentation retrieval.

1. Finding Test Cases Based on Keywords

Rovo can be an efficient, conversational entry point for finding specific testing artifacts based on descriptive language, bypassing manual JQL construction.

• Scenario: A team member needs to quickly locate all existing manual tests relevant to a feature.
  
  
• Rovo Interaction: Use the Search feature (or Chat) and input descriptive terms, such as "manual test including [a keyword]". In this case, “Find all Tests in the XSP2 space that include Payments.
  
  
• Outcome: Rovo delivers an AI summary followed by a table listing work items (tests, test executions, etc.) where the keyword appears in the title or description of the work item. 

While caution is needed regarding filtering and accurate linking, this provides a highly visible list of relevant tests across the user-defined scope. 

Tip: By default, Rovo finds information across the whole instance. If you’re looking for information within a specific Space (formerly known as Projects), include the project key for more specific results.

2. Understanding Team Distribution of Test Plans (or Other Testing Artifacts) to Review Ownership/Workload

Team Leads or Project Managers can leverage Rovo to quickly assess assignee work distribution, relying on the AI's ability to fetch data based on assignees and Jira statuses.

• Scenario: A manager needs to understand how Test Plans are distributed among team members and gauge the current workload.
  
  
• Rovo Interaction: Query Rovo to "Based on assignee, how many different Test Plans I have assigned to each team member". More complex queries can ask Rovo to "split them per team member and status" (referring to the Jira status of the test plan, not the test status).
  
  
• Outcome: Rovo generates a list or summary showing which Test Plans are assigned to whom. This capability is helpful in fetching lists of assignees and providing project management visibility into employee workload. This functionality is particularly useful when managing multiple sprint boards simultaneously.

Tip: Try explicitly naming the projects or sprint boards you want visibility over. This will help with giving the exact results you’re looking for.

3. Creating Processes on Confluence Pages by Reading Through Xray Documentation

One of Rovo's strongest current capabilities is its ability to serve as a documentation agent, summarizing complex procedures and formatting the results for easy team consumption. This can help in establishing testing processes for your team or onboarding new employees.

• Scenario: A team needs a reference page within Confluence detailing the prerequisites and steps for Xray CI/CD integration.
  
  
• Rovo Interaction: Ask Rovo: “Create a page in Confluence on what are the steps to configure the CI/CD integration with Xray”.
  
  
• Outcome: Rovo fetches detailed information, often including prerequisites, steps (like exporting test cases and importing results), and relevant APIs (such as a cURL for Rest API endpoints) directly from official Xray documentation. Once this information is retrieved, Rovo can then be prompted to "build an Xray processes page" on Confluence based on this example, making it available for team training and reference. This essentially functions as an "Xray tutorial agent".

Tip: Define how you want to have the information specified in Confluence: in a step-by-step text guide, including a table for each step or any other format that is right for you.

Next Steps for Rovo and Xray

As Rovo can access Jira (Atlassian) data and other publicly available data, but not Xray data, there’s a gap which limits the interaction between both platforms. The good news is that work is already underway to build on top of Rovo’s capabilities and enable it to understand and access Xray data directly.

This foundational step will open the door to a new generation of Xray-Rovo Agents designed to process, interpret, and act on Xray-specific information, unlocking deeper insights and more intelligent, context-aware results.

In parallel, the Xray team is evaluating the most impactful Agent use cases to ensure the initial developments deliver real value to users. In the near future, teams will be able to rely on dedicated Agents that interact with Xray data in purposeful, task-oriented ways, enhancing productivity and helping teams get more out of both platforms.

If you are interested in contributing to the advancement of Xray-Rovo integration, feel free to leave your suggestions using the following survey.