The Company
The company is a leading financial institution in Latin America operating in one of the most highly regulated industries in the country. The organization provides a broad portfolio of digital banking services, including customer accounts, credit card products, and other financial solutions that require continuous delivery of reliable, secure, and compliant software.
Supporting these services requires collaboration across multiple business units, agile teams, and quality assurance teams. With more than 200 testers, maintaining consistency in testing practices and visibility into quality status had become increasingly important as the organization expanded its digital capabilities.
Software quality plays a particularly critical role in financial services. Every release must meet strict standards while also satisfying internal governance requirements and external audit expectations. Testing activities need to provide clear traceability, documented evidence, and visibility into risk exposure, ensuring stakeholders can make informed decisions before software reaches production.
At the same time, delivery teams were working at an increasingly fast pace. Some products followed weekly cloud release cycles, while others operated on bi-weekly release schedules. Maintaining confidence in every release while continuing to scale development required a more mature and unified approach to quality management.
The Challenge
Before implementing Xray, quality assurance processes were fragmented across the organization.
Three QA teams operated independently using different tools, methodologies, and reporting approaches. Two teams relied on TestLink for test management, while the largest team, consisting of approximately 160 testers, managed testing activities through Excel spreadsheets.
Although these approaches supported day-to-day testing activities, they created significant challenges as the organization grew and software delivery accelerated.
"Before Xray, our three QA teams — totaling over 200 testers — operated in silos with disconnected tools."
Cirley Salazar Flórez, QA Strategist
Limited visibility across the organization
One of the most significant challenges was the lack of centralized visibility.
Because quality data was distributed across multiple tools and spreadsheets, obtaining an accurate view of testing progress required manual effort from QA teams. Information had to be collected, consolidated, reviewed, and distributed before stakeholders could understand the status of testing activities.
This delayed decision-making and made it difficult for teams outside QA to gain a clear understanding of release readiness. Quality information was available, but it was not easily accessible.
"Before Xray, quality was a black box that opened at the end of a sprint."
Cirley Salazar Flórez, QA Strategist
Manual reporting and fragmented governance
Reporting was heavily dependent on manual processes.
Teams spent hours compiling data from different systems and spreadsheets to create coverage reports, execution summaries, and status updates. The effort required to maintain reporting increased alongside the growth of the organization and the complexity of its products.
At the same time, different teams maintained different testing practices, creating inconsistencies in how information was documented, tracked, and communicated. Without a centralized governance model, it became increasingly difficult to standardize quality practices across the organization.
Challenges with traceability and compliance
Operating within a highly regulated financial environment introduced additional complexity.
The organization regularly undergoes audits and must demonstrate traceability across requirements, test cases, executions, evidence, and defects. Maintaining this level of visibility across disconnected tools increased administrative overhead and made governance activities more difficult than necessary.
The team needed a solution capable of supporting both quality engineering and compliance requirements.
Automation and future scalability
Automation maturity also varied between teams. While automation was already part of the organization's strategy, frameworks and processes were not standardized. Some teams relied on Selenium, while others had limited automation resources available.
The organization wanted to modernize its automation practices while creating a foundation capable of supporting future innovation, including AI-assisted quality engineering initiatives.
"Xray's adoption drove the migration of all existing automations from Selenium to Serenity BDD, fully aligned with the Gherkin-based approach. CI/CD pipelines were migrated from Jenkins to GitHub Actions, meeting the security and licensing standards required in regulated financial environments."
Cirley Salazar Flórez, QA Strategist
At the time, test information lived across spreadsheets, disconnected repositories, and separate tools, limiting the ability to fully leverage emerging AI capabilities.
The Solution
The organization selected Xray as the foundation for a complete transformation of its quality management strategy.
Rather than simply replacing legacy tools, the objective was to establish a centralized quality platform capable of supporting more than 200 testers while improving visibility, traceability, collaboration, reporting, automation, and scalability.
Unifying quality management across three teams
The implementation brought together three previously disconnected QA organizations into a single testing ecosystem.
Supported by an official Atlassian Solution Partner, the rollout was executed in parallel across all teams. Three dedicated consultants supported implementation activities, helping the organization establish common processes and accelerate adoption.
By consolidating testing activities within Jira through Xray, the organization eliminated the need for multiple testing tools and created a single source of truth for quality information. This unified approach immediately improved collaboration between teams and provided a consistent framework for managing quality across the entire organization.
Building internal expertise through BDD and Gherkin
A major part of the implementation focused on standardizing testing practices. Approximately 160 testers were trained in BDD and Gherkin language, creating a common language that could be shared across QA, development, and business teams.
This standardization improved communication, simplified automation efforts, and helped ensure test cases could be understood beyond the QA function.
"Test cases became business-readable, reducing misalignment between requirements and validation."
Cirley Salazar Flórez, QA Strategist
The organization also adopted an internal champion model to support adoption. Experienced testers helped onboard colleagues, answer questions, and reinforce best practices across teams, accelerating the transition to the new platform.
Modernizing automation practices
The implementation of Xray became a catalyst for broader improvements across the organization's automation strategy.
Existing Selenium-based automation was migrated to Serenity BDD, aligning automated testing with the organization's Gherkin-based approach. CI/CD pipelines were also modernized through a migration from Jenkins to GitHub Actions, supporting both security requirements and long-term scalability objectives.
Today, automated regression testing supports both weekly cloud releases and bi-weekly traditional releases, helping teams deliver software with greater confidence and faster feedback cycles. The organization is currently evaluating Playwright as the next step in its automation evolution.
Transforming reporting and quality visibility
Beyond test management, Xray enabled the organization to rethink how quality information was communicated.
While team-level visibility improved through Jira and Xray, leadership required a more advanced reporting approach capable of translating testing activities into business insights. Using Xray's APIs and GraphQL capabilities, the organization integrated quality data into Power BI and Grafana.
These dashboards provide near real-time visibility into quality metrics, automation performance, coverage levels, execution progress, and risk exposure. Information is automatically refreshed continuously throughout the day, ensuring stakeholders always have access to current data.
The result was a shift from reactive reporting to proactive quality visibility.
"Quality data was always on, always traceable, and always connected to business decisions."
Cirley Salazar Flórez, QA Strategist
Supporting compliance through traceability and governance
Operating in a highly regulated financial environment requires more than effective test execution. Teams must be able to demonstrate how requirements are validated, provide evidence of testing activities, and maintain clear records that can be reviewed by auditors and stakeholders.
With Xray, the organization established end-to-end traceability between requirements, test cases, test executions, evidence, and defects. This provided a structured audit trail that made it easier to demonstrate compliance, investigate issues, and understand the quality impact of changes.
For Cirley's team, traceability also improved defect analysis. When issues were identified, teams could quickly review the associated test executions, evidence, environment details, and linked defects, making it easier to reproduce problems and assess business risk.
The organization also incorporated risk-based testing practices into its reporting process, helping teams communicate not only what was tested, but the potential impact and probability of issues on business operations.
Creating the foundation for AI-powered quality engineering
One of the most significant long-term benefits of implementing Xray was the creation of a structured quality data foundation.
By centralizing test cases, execution history, traceability links, coverage information, and testing evidence, Xray transformed quality information from disconnected records into structured, accessible data.
This foundation enabled the organization to begin exploring AI-powered quality engineering initiatives in ways that would not have been possible previously.
Leveraging Xray’s AI Test Case Generation
In addition to developing internal AI initiatives, the organization adopted Xray's AI Test Case Generation capability as part of its daily testing workflow. The feature quickly became a valuable tool for accelerating test design while maintaining full human oversight.
What impressed Cirley most was not simply the generated output, but the design of the workflow itself:
"The real value lies in the workflow orchestration, not just the output."
"You don't need a detailed prompt. Simply describing the type of test you want to design is enough. Xray generates multiple options that are closely aligned to the context of the user story being tested."
Cirley Salazar Flórez, QA Strategist
The feature allows users to define testing objectives, specify preconditions, select test types, and generate up to 60 proposed test cases. Rather than immediately creating tests, Xray provides a review stage where generated scenarios can be evaluated, edited, refined, organized, or discarded before final creation.
This approach enables testers to maintain control while significantly reducing manual effort.
Additional capabilities that stood out include:
- Automatic categorization of generated tests.
- Automatic creation of repository folders and subfolders.
- Gherkin-ready scenarios.
- Metadata enrichment during creation.
- Review and editing before final generation.
- Structured organization directly within the test repository.
The feature has proven particularly valuable for designing functional and negative testing scenarios.
"This feature alone [Xray’s AI Test Case Generation] has saved me up to 80% of my test design time. What used to take me two full days to produce — a complete test plan — now takes just a couple of hours."
Cirley Salazar Flórez, QA Strategist
The organization has also incorporated AI-assisted testing concepts into internal training programs, helping testers understand how AI can complement quality engineering practices while preserving human expertise and decision-making.
The Results
By implementing Xray, the organization transformed quality management from a fragmented, manual process into a centralized, scalable, and AI-enabled practice. What began as an effort to unify three disconnected QA teams evolved into a broader modernization initiative that improved visibility, strengthened governance, accelerated automation, and created the foundation for AI-powered quality engineering.
Today, quality data is accessible in real time, testing processes are standardized across the organization, and teams have the traceability needed to support both business stakeholders and regulatory requirements.
Key results achieved
- 200+ testers unified on a single platform.
- Three disconnected testing tools consolidated into a single quality ecosystem.
- Reporting effort reduced by approximately 70%.
- 160 testers trained in BDD and Gherkin.
- Standardized testing practices across multiple business units.
- Full traceability between requirements, tests, executions, evidence, and defects.
- Migration from Selenium to Serenity BDD.
- Migration from Jenkins to GitHub Actions.
- Real-time quality reporting through Power BI and Grafana integrations.
- Structured quality data foundation established for AI-powered quality engineering initiatives.
- Up to 80% reduction in test design time using AI Test Case Generation.
The impact extends beyond operational efficiency. By centralizing quality information within Xray, the organization gained continuous visibility into testing activities, risk exposure, and release readiness. Stakeholders no longer need to wait for end-of-sprint reports to understand quality status, while QA teams spend significantly less time on manual administration and more time focusing on quality strategy, automation, and risk management.
The structured testing foundation created through Xray has also enabled the organization to embrace AI-assisted quality engineering. From supporting the development of an AI agent independently designed by Cirley to accelerating test design through AI Test Case Generation, the team has successfully positioned itself to take advantage of emerging AI capabilities while maintaining governance, traceability, and human oversight.
As a result, the organization now operates with a quality practice that is more scalable, more collaborative, and better equipped to support the demands of modern software delivery in a highly regulated financial environment.
