I felt compelled to write the following article due to a division of opinion that has risen between several professionals within the quality assurance (QA) community and me on the importance of metrics in testing.
After 23 years in the QA industry, I concluded that measuring how efficient a team is in getting tasks done is crucial in driving cost calculations, team size allocations, future budget allocations and other important decisions. But many testing professionals see these factors as outgrowths of micromanagement rather than outcomes of output measurement. In fact, some voices in the testing community believe that putting productivity metrics around testing cases is useless altogether. In this piece, I’m going to dismantle this school of thought by debunking several myths aimed at discrediting the use of metrics in QA testing.
Myth 1: The following four activities carry greater value than metrics-based ones.
First, session-based testing management measures activities that testers perform in completed sessions. It enables testers to make orderly reports that third parties can understand without difficulty and organize their work without impeding the flexibility of exploratory testing. Each session is framed around a mission so testers know exactly what they’re testing and what types of bugs they’re looking for.
Let’s examine why supporting this idea is basically the same as using test cases: Session-based testing relies on charters that articulate the objective of each session and specify its agenda. In other words, its purpose is no different from that of an individual test case, which is to determine if features in a given application are performing as expected, uncover errors and defects in the platform and validate that the system satisfies relevant standards and requirements. In a nutshell, session-based testing is just another term for a test case with the same underlying objectives.
Second, a Net Promoter Score (NPS) gauges customer loyalty and satisfaction with questions on how likely your customers are to recommend your products and services to others on a scale of one to 10. The reality, however, is that the NPS metric isn’t underpinned by any scientific findings that are superior to other metrics. In fact, firms can fall into a trap of misallocating their resources on flawed strategies guided by NPS on firm performance, company value and shareholder wealth.
Third, continual service improvement (CSI) is a metrics-driven methodology based on past successes and failures that companies apply in order to spot improvement opportunities. The idea is that by upgrading their business processes, QA providers are able to give their customers more value via better design, strategy and operations. For the record, I think the CSI framework is a great way to improve service delivery to clients but there are some drawbacks. CSI doesn’t necessarily address specific benchmarks that are of value to clients. For example, a company can detect and fix a bottleneck in a given internal process using CSI—but does it invariably relate to a specific client concern? This makes it hard to capture ROI and justify its application.
Finally, sentiment analysis on user feedback determines whether the data is positive, negative or neutral. It helps brands monitor customer sentiment and gain an understanding of client needs. While this makes sense in absence of direct feedback, its conclusions are often ambiguous. The analysis reveals nothing on specific things such as test design, test execution and defect identification.
Myth 2: Metrics-driven testers are perpetuating harmful folklore and wasting customers’ time.
Some QA testers believe that, instead of ruminating on metrics, QAs should focus on the science called metrology, which they view as the essence of testing. Metrology establishes a common understanding of units in order to ensure that measurement meets specified degrees of precision and accuracy. As such, it’s a valid foundation to use. This is why, to measure productivity, all professional QAs use design metrics-driven evaluations that are made in the spirit of metrology.
As a reminder, testing metrics are defined as quantifiable measures that assess the progress, quality, productivity and validity of the software testing progress. Used correctly, metrics reveal important data so that enterprises can improve the efficiency and effectiveness of their operations and drive smarter decisions. That’s what makes these metric scores intellectually honest—they’re based on construct validity.
Myth 3: Focusing on metrics around test cases compromises quality.
A number of testers believe that getting wrapped up in “so called productivity metrics” makes testers lose their focus on quality. They argue that tracking the number of test cases that have passed or failed adds zero value to the end product. I understand that tracking test metrics results in an absolute value and that there are nuanced categories to consider, but it’s an important start. It helps us estimate how much work is left to be completed, while identifying red flags that require special attention.
I also came across testing professionals who believe that “quality is subjective and cannot be measured.” If quality were in the eyes of a beholder, there would be no need to set industry standards, such as application reliability, security, availability, performance, customer experience, etc. Standards reduce the potential for confusion and ensure important criteria are met. If quality is subjective, why was the global quality management software market valued at $8.25 billion in 2021 while expected to grow at a compound annual growth rate of 9.7% from 2021 to 2028?
Metrics evaluate the health of a testing effort, drive costs down and prevent defects.
The stance that QA professionals who favor metrics are polluting the QA industry with obsessive and nonsensical demands is controversial, to say nothing of misleading. It distorts the value of professional testing and assumes most people read metrics incorrectly. Without calculating KPIs to see how they compare to industry benchmarks, testers won’t know which areas to prioritize—areas that, for all we know, could be leaking money. Metrics shed light on the effectiveness of testing. Without them, QAs might as well be operating in the dark.
Looking to hire skilled software developers? Contact TP&P Technology - Leading Software Development Company in Vietnam Today
Không có nhận xét nào:
Đăng nhận xét
Lưu ý: Chỉ thành viên của blog này mới được đăng nhận xét.