In part 1of our software testing compliance blog we discussed the overall significance of compliance and how it can be facilitated at the strategic level for several popular universal standards. Now, let’s take a look at how day-to-day operations could be optimized with industry-specific examples of Xray’s applicability.
Do, Check, Act
The Do stage of Plan, Do, Check, Act (PDCA) is more about the tactical implementation of the Quality Management System (QMS). Check and Act stages detail how the quantified results are processed. The goals are to determine the effectiveness and efficiency of each process toward its objectives, to communicate these findings to the stakeholders, and to develop new best practices based on the audit.
The core compliance demands of these stages are relatively consistent across industries and standards, and we will focus on the four categories below (sharing specific examples for each).
Clear requirements definition
Define what a requirement is and its relevant metrics
The access controls and the data integrity measures mentioned previously improve efficiency and transparency at this step. Xray provides multiple levels of requirements coverage and different ways to define the hierarchical relationship (e.g., using issue links, sub-tasks). With Xray you can represent the compliance requirements in a way that reflects scope, dependencies, and progress.
Let's assume we are working with an Automated Driving System for this example. Here is how a few requirements for it could be arranged:
The coverage and linking from defect, to test, to story and epic are enabled with Xray and this hierarchical view example is achieved with the Structure app for Jira
Plus, capturing and tracking key pieces of information like industry-specific Components and compliance-related aspects like Automotive Safety Integrity Levels (ASILs) can be done through structured (custom fields) and unstructured (description, attachments, mockup links, etc.) formats.
Implement workflows to have explicit control over the process
Ensure issues get done by assigning them to users and tracking them using workflows. With Jira and Xray it is possible to:
- implement an approval mechanism, commonly having one approver
- implement quality gates based on field status
- make items "read-only" when transitioned to a certain workflow status
- restrict usage of Tests in a certain workflow status
- disallow executions for Test Executions in a specific status
Coverage and traceability
Guarantee requirements coverage
Epic and Story issues, common in Agile environments, can be covered by creating tests and tracking coverage directly from Jira using Xray. As a requirement example, by 2025, the MX format from ISO 20022 is expected to be the common language of the global financial industry:
Then, within the Test Coverage Report, you can have a birds-eye view of the quality status of your requirements based on the tests that cover them and their results. The information is updated in real-time and is multi-dimensional.
You can group stories by specific metrics, such as the standard or risk level, to ease your analysis. That way, you can have a better understanding of whether a higher-priority issue is ready to be released or not.
Enable audit with full traceability
For instance, PCI DSS Requirement 10 "ensures the presentation of an audit trail for all credit card-related processes." Xray helps address such compliance demands by providing full traceability between requirements, tests runs, and reported defects, including the relationship and status of each entity.
One of the primary artifacts, available right in Jira, is the Traceability Report. We can analyze our Epics, child Story issues, show the Tests that cover them, along with the reported results and, hopefully not many, defects that may have been found.
You can also handle reports focused on custom metrics. For example, one of the metrics that help address PCI DSS compliance is the percentage of known vulnerabilities patched or mitigated, which can be easily reported via Xporter.
Thorough testing techniques
For the testing side of the standards like ASPICE or PCI DSS, given the complexity of modern enterprises, all types of testing are necessary, with efficient coordination between them. And our suite of tools is flexible enough to do that.
Xray supports the full arsenal of techniques:
- Manual tests, including parameterized ones aligned with datasets
- Automated tests (Cucumber, including data-driven outline, and others)
- Exploratory testing using a desktop Xray Exploratory App
So far, we have talked about mitigating risk by evaluating the criticality of each requirement and guaranteeing its coverage with tests. But you can also "fortify" another angle - the coverage of important data interactions within your systems.
That can be achieved with the optimized scenario generation facilitated by Test Case Designer (part of Xray Enterprise). Its combinatorial, model-based approach is based on the research results about the causes of defects in production and deliver significant quality and speed improvements:
Visible execution
Once the test suite is established, you can launch the execution directly from Jira (for both manual and automated types). Regardless of the testing approach or execution type, Xray can provide visibility of testing results, including evidence, all in one place for faster feedback loops.
Whenever manually executing scripted test cases, we can report results at the step level and attach screenshots as evidence, accelerating the collaboration and defect triaging process. Similarly, Xray Exploratory App can integrate with Xray Test Management to bring the best of both worlds:
- Have exploratory testing evidence tracked in Jira
- Be accessible by the team
- Reflect on the related requirements
In Agile and DevOps teams, enabling the adoption of any test automation tool/library and CI/CD tool is especially important. With the new feature - Remote Jobs Triggering - you can launch the pipeline action directly from the Test Execution issue. Then, detailed results from the automation framework can be imported to Xray and tracked consistently with other tests. You can take it a step further and link test code to requirements.
For a more detailed breakdown, you can refer to our tutorials for automotive and financial compliance.
Achieving compliance with Xray
Organizations, especially those working in highly regulated environments (e.g., financial services, automotive, medical, security, aerospace, and defense), often should comply with specific standards to achieve excellence through quality and improve their market position.
Whether you are pursuing formal certification or want to improve performance, our suite has the versatility to support you in building a powerful quality management system across several stages of your product development & delivery.
- Single source of truth in Jira with strict access control
- Coverage and risk management are core to all activities
- Traceability, accountability, and collaboration
- Coordinated, multi-channel execution is straightforward
- Easy, customizable reporting for real-time compliance insights
Embark on your optimized compliance journey today with the free trials of our tools:
- Xray and Xray Enterprise
