Software Testing - discovering and exploring !

What is software testing like? Is it for folks who got rejected in development? Is it for those who are without a job and looking for a toehold? Or is it meant to be a stepping stone to perhaps, "greener" pastures in the world of development? These questions crop up every once in a while when interacting with software testers or potential testers.

The fact of the matter is that there will always be cross-functional movement. For example, there will be a few folks from software testing who will fantasize about software development and make attempts to move at an available opportunity. This isn't necessarily bad and there is no need to berate your selection process much. Of course, if folks are jumping ship in droves, your selection process for testers surely needs a close look. Once new folks come aboard the testing bandwagon, it is a matter of time before they figure out whether they are truly cut out to be testers or would rather "follow the herd" so to speak. I am not trying to indulge in berating development or show testing as a superior alternative to any other function.

Software testing, just like software development is a valuable piece of the whole that enables a great product to be produced. You cannot do without either of the functions. You could use the analogy of your own body to denote their significance. A hand is no more or less important than a leg or the brain more or less than the heart - there may be special situations where one may be used more; however, on the whole the entire body is complete and healthy when all its component parts work together.

Back to our original question - what is software testing like and what kind of folks would find testing interesting? Software testing is about discovery. Testing, more than any other function involved in producing software, is about discovery. Software professionals who love discovering and exploring are likely to feel at home in testing. Testers are constantly trying to discover what they do not know in the system. In the words of Philip Armour, "The challenge in testing systems is that testers are trying to develop a way to find out if they don’t know that they don’t know something. This is equivalent to a group of scientists trying to devise an experiment to reveal something they are not looking for. It is extremely difficult to do." He goes on to state that much of what we call as method or process in testing involves heuristic strategies.

We selectively test complex predicate logic, we create test cases that span the classes of inputs and outputs, we construct combinations of conditions, we press the system to its boundaries both internally and externally, we devise weird combinations of situations that might never occur in the real world, but which we think might expose a so-far unknown limitation in the system. None of these are guaranteed to throw a defect. In fact, nothing in testing is guaranteed, since we don’t really know what we are looking for. We are just looking for something that tells us we don’t know something. Often this is obvious, as when the system crashes; sometimes it is quite subtle.

If all this talk of discovery and exploration sounds too much, you might want to re-consider your choice of testing as a career and probably look at other options where things are clearly stated and probably involve such exciting tasks such as translating defined specs into a computer language following pre-defined coding standards and guidelines or regularly fixing issues in the code you or someone else wrote. Discovering is definitely not for everyone.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry

Image: Salvatore Vuono / FreeDigitalPhotos.net

Software Testing Then and Now: What Mature QA Has Learned

Looking back at older testing practices is useful when it helps us see which ideas still matter and which habits belong to a different delivery era. The tools have changed dramatically, but the core professional responsibility has not: testers create evidence about risk.

What has changed is where that evidence comes from. Modern QA cannot rely only on late manual execution, static test plans, and defect counts. Software now changes too quickly and runs in environments too complex for that model to be enough.

What still matters

The enduring skills are clear thinking, disciplined observation, risk analysis, test design, communication, and professional skepticism. A tester still needs to notice weak assumptions, ambiguous requirements, fragile workflows, and behavior that users will not trust.

Good defect reporting also remains valuable. A precise defect with context, impact, data, and diagnostic clues still saves engineering time.

What has changed

Testing is now part of a larger quality engineering system. Automated checks, CI/CD pipelines, service contracts, observability, feature flags, production telemetry, security scanning, and incident learning all contribute to release confidence.

The modern tester needs to understand architecture, APIs, data, logs, deployment models, and user behavior. That does not mean every QA professional must become a developer. It means technical fluency is now part of credibility.

The lesson for QA professionals

The profession is strongest when it keeps the old discipline and adopts the new evidence sources. The future does not belong to testers who only execute scripts, or to engineers who think automation replaces judgment. It belongs to quality professionals who can connect risk, evidence, and business decision-making.

How this shows up in QA leadership

A QA leader can use this idea to improve the quality conversation in a team. Instead of asking only whether testing is complete, ask what risk has been reduced, what evidence supports that claim, and what decision the team is now better able to make.

That is the difference between QA as activity tracking and QA as technical leadership. The strongest quality professionals make uncertainty visible in a way that helps people act.

Software Testing: Theory on Defect Detection

Theory: A defect can only be discovered in an environment that contains the defect. 

This seems very obvious! So why even bother to mention it, let alone post a blog entry about it?  The motivation for this entry comes from a familiar situation that i am sure most testers would have encountered. 

Software testers devise extensive sets of tests and execute them prior to a product's release. These tests are usually executed on setups that testers have prepared in their lab environment. Many organizations may have invested significant sums of money to have the lab infrastructure in place. However, what generally tends to happen is that when this product is released, the customer reports issues pretty quickly. So, what happened? What happened to all the man-hours spent on testing and the investment on expensive lab equipment? Why did our in-house testing efforts not show up these defects that a customer seemed to find "easily"? What are we doing wrong?

This brings us to our theory, i.e. a defect can only be discovered in a system or environment that contains the defect. A defect that might show up in a customer environment may not manifest itself in a sanitized lab environment. Often our lab environment is setup and controlled based on our view of how the product is likely to be used. Within the confines of the boundaries we have defined, we execute our battery of tests and feel confident when the tests run without reporting issues. However, a customer's environment suffers from no such boundary constraints and does not find it hard to expose a defect. Defects may not necessarily be in the product itself. It could arise from the interactions of the product with its operating environment, dependencies, usage, etc.

Therefore, unless we are able to replicate in sufficient detail the customer environment and the likely real-world usage scenarios after the product is released, we will likely continue to see an increasing trend of customer reported issues.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Email subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry
Image: Salvatore Vuono / FreeDigitalPhotos.net

Software Testing and the hammer and nail approach

If all you have is a hammer, every problem looks like a nail.

It is sometimes a similar situation with software testing and specifically with test automation. This may not be much of an issue for single-product companies but when your organization produces a range of products and has multiple teams of testers handling these different products then the likelihood of hitting the hammer and nail problem arises.

One might say that the problem is prevalent more in a centralized testing structure than in a decentralized structure. The issue I am alluding to is the mandate to use a common test tool, mostly for automation since that is an area that everyone would like to see standardized but poses most challenges to standardization.

I have come across many instances where there is the attempt to identify a common tool that can satisfy all the requirements for automating various products being tested by different test teams. In some cases, the products would be as different as chalk and cheese. For example, there was this situation where we had a suite of products addressing different customer needs. This suite has a thick client application written in Java, a Win 32 application, a complex Ajax based web application, a few server products that you interact with using CLI and APIs, some middle ware products and few mobile applications.

Trying to find a common tool to handle all of the varied requirements for automating the different products could lead us to three possible ways of solving them.

1. One, you talk to a few tool vendors who would naturally promise the moon and claim that whatever tool they are selling can automate every kind of application that ever existed and will come into existence in the undetermined future. You could take this option like many folks do and purchase expensive tools and licenses and then mandate your testing teams to go use these (and only these) to automate in a standardized manner. Simple solution using a brute-force approach.

2. The second approach may be to take the compromise route. Here, you realize that a single tool may not be able to handle all the unique requirements of automating each product and go ahead to procure a tool that probably results in being the lowest common denominator solution. The tool ends up being  a good enough solution which essentially means that it is neither good nor enough from a testing perspective. However, the organization gets a standardized jack-of-all-trades kind of tool that all groups could use with varying degrees of effectiveness.

3. A third approach could be to take the time to understand the specific requirements and needs for automating each product, perform due diligence in identifying & then evaluating the tools that best fit the specific automation requirements before deciding on what tools to procure. This may result in having to procure more than one tool. If your products have similar automation requirements you may end up with needing just one tool but in cases where you have products with differing needs similar to what I stated as an example earlier, you might realize that more than one tool is needed to perform effective test automation.

Effectiveness, is the key here. Ultimately you automate not for the sake of automating but to support your testing campaign and deliver value. Trying to force teams to use a specific tool that does not fully support their needs is akin to the hammer and nail solution. When all you have is a hammer, then every problem is dealt with as you would a nail. Sometimes it is the right thing to do and in many cases it may not be the optimal approach to follow. In testing, such an approach could lead to teams bending their testing and automation practices around what the tool is capable of while ignoring other possibly important areas of the product which are harder to automate using the tool. The automation tool should not dictate how & what you test. It should truly be a tool (amongst other tools and techniques) in your arsenal enabling you to tackle the challenges of testing your software.
***
Join my community of professional testers to receive free updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone. Subscriptions are managed by Google's FeedBurner service.

Share & Bookmark this blog entry

Image: Carlos Porto / FreeDigitalPhotos.net

Software testers, sharpen your saw

How would you list the day in the life of a software tester ? Get in to work and test software ? I realize this is an over-simplification, but i am sure most lists would place testing as the task that would take up most or all of a tester's time. After all isn't that why testers are paid? Software testers are normally expected to test and utilize their time optimally to find defects in the software. Test all day and find issues. We live in a society that values busyness and activity. It is easy to get caught up in the frenzy of running tests and trying to find issues. Before you think that i am trying to advocate testers to not do testing, let me clarify - testers must test ! that is their primary job responsibility. However, testing isn't all that a tester must do if he/she must remain relevant and valuable in the future.
At this point i would like to digress a bit to touch upon a concept that is mentioned in Stephen Covey's book, The 7 Habits of Highly Effective People. It is called "Sharpen the Saw". What this means is to preserve and enhance the greatest asset you have, which is … you

The book talks about having a balanced program for self-renewal in the four dimensions of your life: physical, social/emotional, mental and spiritual. Self-renewal enables you to create growth and change in your life. Sharpening the saw keeps you fresh so you can increase your capacity to produce and handle challenges. The book goes on to say that without this renewal, the body becomes weak, the mind mechanical, the emotions raw, the spirit insensitive, and the person selfish. 

This concept of sharpen the saw finds expression in another oft repeated tale about two wood-cutters who get down to cutting trees using their saws. One wood-cutter goes to work and relentlessly keeps at cutting wood. He spends a lot of time and effort to continuously cut wood. 

The other wood-cutter cuts some wood and then takes a little time off from cutting wood, to sharpen his saw. He then goes back to his task of cutting wood. He does this repeatedly. At the end of the day, it is observed that the second wood-cutter has cut more wood (increased productivity), is more relaxed (less stressed) and has both himself and his tools in good shape to handle another day's tasks. 

The saw of the first wood-cutter gradually grew blunt with increased use. The reaction of this wood-cutter was to increase his own effort at cutting wood hoping that the increased effort on his part would compensate for the reducing sharpness. Needless to state the obvious, the first wood-cutter ended up feeling burned-out and tired and probably surprised that all his efforts resulted in less  than optimal results. He did have a lot of busy-time but results did not match his level of activity.

Whats all this got to do with Software Testing ?  I am sure most of you would have figured out the connection and where we are heading. It is true that testers must test, but that isn't all that a tester should do. The best testers realize the principle of sharpening the saw. They must take the time to continually develop their skills and work on their creativity and thinking. These testers strive to constantly be abreast of developments in their area. Self-development need not be limited to areas that are directly connected to testing; do not hesitate to look at those areas that may not seem in any way related to testing. You never know where you might find ideas that can be implemented in your work.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.

Share & Bookmark this blog entry

Tester Intuition: Why Feelings Can Be Useful Quality Signals

Tester intuition is not a replacement for evidence, but it is often the first signal that the software violates an unstated user expectation.

Formal requirements are necessary, but they are never the whole product. Many important expectations are implied: the workflow should feel understandable, the system should respond quickly enough, errors should be clear, data should look trustworthy, and users should not feel trapped or confused.

When a tester feels irritation, confusion, hesitation, distrust, or surprise while using a product, that reaction is worth investigating. It may point to a usability problem, a missing requirement, a workflow gap, a performance issue, or a product decision that users will experience negatively.

Feelings Are Not Defects. They Are Signals.

A feeling by itself is not enough to file a high-quality defect. The tester's job is to convert that signal into evidence. What exactly felt confusing? Which step caused hesitation? How long did the system take? Which user goal was interrupted? What would a reasonable user expect instead?

This is where exploratory testing is valuable. It gives testers permission to follow observations, investigate inconsistencies, and study the product as a user would rather than only as a script executor.

Why Requirements Miss These Issues

Requirements often describe intended behavior, not experienced behavior. They may say that a user can submit a form, but not whether validation messages are understandable. They may say that a report is generated, but not whether users can trust the numbers. They may say that login failure is handled, but not whether the recovery path is clear.

Great testing looks for the gap between documented behavior and user experience.

How To Turn Intuition Into Strong Evidence

  • Describe the user goal that was disrupted.
  • Capture the exact workflow, data, role, browser, device, or environment.
  • Compare behavior with user expectations, competing products, standards, or internal design patterns.
  • Explain likely impact: confusion, delay, error, abandonment, support cost, or trust loss.
  • Suggest a clearer expected behavior without pretending product decisions are purely QA decisions.

When Teams Push Back

Issues based on user experience or implied expectations may be challenged. That is normal. QA should not rely on personal preference alone. Strengthen the case with evidence: user personas, analytics, accessibility guidance, support data, competitive examples, design-system rules, or customer feedback.

Testing is both analytical and human. The best testers use scripts, tools, data, and logic, but they also notice how the product feels to use. When handled professionally, tester intuition becomes a source of product insight.

Does Agile development need specialists ?

"Agile is to software development what poetry is to literature. 
It is 
very difficult to do well, very easy to do poorly, 
and most people 
don't understand 
why a good poem is good and a bad poem isn't...”
- from the web

Transition from a traditional development model to an agile methodology is often met with some degree of skepticism and doubts by testers. Books and programs on Agile development tend to emphasize the need for multi-skilled generalists who can take up different functional roles as needed. This tends to cause specialist testers to worry about losing their identity in an agile world. Is there a need for specialists in agile or is the agile world inhabited by generalists, the proverbial jack-of-all-trades ?

To answer this question, look no further than your favorite team sport. Agile software development is a team process involving members from the different functional groups coming together as part of one team to produce software. No longer are they members of distinct teams such as development, testing, technical writing, i18n, l10n, etc. Back to our earlier question on whether agile needs specialists or is the new world full of generalists ?

Like a team sporting activity, for a team to be successful it cannot be - 1) all members of one particular type or specialize in one activity e.g. development or testing alone 2) all members who are generalists and knowing part of all functions but not specialists of any function. How would you rate the chances of your favorite sports team if it were comprised of either types of members – a) all players who specialize in just one area e.g. of cricket which I follow: a team of just batsmen or just bowlers b) all generalists e.g. again of cricket: a team of just all-rounders – it might be better than the first choice with only players of one type but still not the best choice. An agile team takes a step towards success when it comprises specialists from the different functions coming together and working together collaboratively to produce software. Each member of the agile team brings to the table their unique set of skills that influence the software's development and success of the project.  However, an additional requirement for these specialists that are part of the agile team is to be able and willing to share in some of their non-core tasks. For example, a tester who can debug defects and make small fixes if need be, a developer who can also document their feature or do some testing, etc.

There is truth to the statement that agile needs generalists. However, it is better to have specialists who can go beyond their functional domains to help towards the release rather than specialists who just restrict their involvement to their functional area of specialization. Ultimately, agile development is a team effort and testers like others on the team must own the release from the beginning without merely trying to police it at the end.
***
Liked this entry? Join my community of professional testers to receive fresh updates by email. Use this link to add your email address to the community. Rest assured, I will neither spam nor share your email address with anyone else. Your email id will remain confidential. Subscriptions are handled by Google's FeedBurner service.

Share & Bookmark