Tester - skills needed for successful software testing (2)

We continue exploring the traits needed for testers to be successful in testing software. The previous post looked at traits 1-3. In this post, let us look at three more Software Tester traits.

Tester Trait 4: Perseverance

Testers must keep at testing, exploring and trying to unearth issues. Bugs may appear intermittently or under certain conditions. Testers should not ignore or give up, but instead try different scenarios to reproduce the issue.

Software Testers must also realize that all products have bugs. If a product looks to be free of bugs, it just needs more testing to find issues that current tests haven't looked at. Testers should always be in the pursuit of bugs and view every defect found by a customer as a slip or gap in their tests which must be addressed immediately.

Tester Trait 5: Creativity

Software Testing is an art. It is not enough to test for the obvious. Finding bugs requires creativity and out-of-the-box thinking in testing. Software Testing must be amongst the most creative of professions. Lets make a fairly simplistic comparison between testing software and software development which is considered to be a creative endeavor. Which of these disciplines needs more creativity ? Is it to "introduce" defects or find defects ? While the example is a bit crude, the idea is that it is harder to find defects when you do not know what and how many defects exist. It requires a high degree of creativity to discover defects in Software.

Tester Trait 6: "Flexible" Perfectionists

Software Testing requires the pursuit of perfection. However, the pursuit of perfection must be tempered with flexibility. There are times when perfection may not be attainable or even be feasible. Testers whilst seeking perfection, should adapt a certain degree of flexibility when perfection is not an ideal goal to seek. As an example, when testers report bugs, they must also pursue a fix for the bug. Now, a fix need not just mean fixing the software. It could be a statement in the release notes or other documentation that highlights the defect as a known and open issue; it could be a case of marketing toning down its spiel or enlightening customers about the potential issue - in the real world, it may not be possible to fix every defect that testers want fixed. Being able to prioritize and pick your battles appropriately, knowing when to give in and when to stick to your guns is important.

In the next upcoming post, we look at the remaining traits of a successful tester. Stay tuned.

http://www.techmanageronline.com

Tester - skills needed for successful software testing (1)

What are the qualities (a.k.a. attributes, traits, skills) that are required of Testers for testing software ?

In this post, let us look at three Software Tester traits. More to follow in the next post.

Software Tester Trait 1: Curiosity

Software Testing is about exploring and discovering. Software Testers should like exploring, be curious and inquisitive about everything and display keenness in understanding the why and how of products, their interactions, dependencies and ecosystem in totality. Testing software requires venturing beyond the realms of the tried and known, to discover what lies beyond. Installing new software builds, experimenting, seeking to better understand the software and breaking it are part of the skills needed in software testing.

Tester Trait 2: Detail oriented and thorough

Software Testing requires discipline and systematic approach. It is important for testers to be able to pay attention to details and be thorough. While testing involves exploration and experimentation, testing requires that testers not leave any gaps in test coverage. Ensuring that all requirements and areas are thoroughly tested is important. Having an eye for detail in testing would also mean looking out for oddities and incorrect behaviors in the software application being tested. What might seem like a small, insignificant or even irregular occurrence may be the harbinger of much larger issues. It pays to scrutinize each issue thoroughly.

Tester Trait 3: Trouble-shooter

Software Testing requires the ability to root-cause issues. Being good at finding out why something does not work is a useful attribute to possess in testing software. Testers should be able to narrow down the conditions that cause an issue and help identify or at least suggest causes for issues observed. A detailed bug report that lists the issue, narrowed-down steps to reproduce, as well as probable cause along with relevant details can help software development teams address issues faster and better. Also, being able to find out why something does not work can point to more issues lurking around or areas that may need more testing. A tester's job is not just about executing a standard set of software tests and reporting any failures.

We'll look at more traits in the next post.

http://www.techmanageronline.com

Build vs Buy For Testing Tools: A Quality Engineering Decision Framework

Build-vs-buy decisions for testing tools should be treated as engineering investment decisions, not as tool preference debates.

Testing tools shape how teams work. A test management system, automation framework, defect workflow, performance tool, reporting platform, or data generator does not merely support the process. It influences habits, feedback speed, visibility, ownership, and long-term maintenance cost.

The build-vs-buy question appears simple: should we create something internally or purchase an existing tool? In reality, the decision has long-term consequences for cost, capability, adoption, support, integration, and engineering focus.

When Building Looks Attractive

Internal tools are attractive because they promise control. The team can tailor workflows, integrate with internal systems, avoid licensing cost, and move quickly if the need appears narrow.

That argument is valid in some cases. Building may be the right choice when the problem is highly specific, the organization has strong engineering ownership, commercial tools do not fit the workflow, or the tool is strategically important to how the company delivers software.

But internal tools are rarely free. They require product management, architecture, coding, testing, documentation, support, enhancement, security review, onboarding, migration, and long-term ownership. If nobody owns the tool as a product, it becomes quality debt.

When Buying Looks Attractive

Buying can accelerate adoption. Commercial tools may provide mature features, vendor support, integrations, documentation, training material, and upgrade paths. For common problems, buying often avoids rebuilding commodity capability.

Buying is not maintenance-free. Teams still need configuration, workflow design, access management, migration, integration, training, administration, vendor management, and upgrade testing. A purchased tool can also push teams into a process model that does not fit how they actually deliver.

The Decision Criteria That Matter

• Strategic differentiation: does this tool create unique delivery capability or solve a commodity problem?
• Total cost: include licenses, engineering effort, support, migration, maintenance, training, and opportunity cost.
• Time to value: how quickly will teams get reliable benefit?
• Integration fit: does the tool work with source control, CI/CD, issue tracking, identity, environments, data, and reporting?
• Scalability: will the tool support more teams, products, data, and compliance needs later?
• Ownership: who will operate, improve, and support it?
• Exit cost: how painful will migration be if the decision is wrong?

A Common Failure Pattern

Many in-house tools begin as temporary solutions. The team needs something fast, so a small utility is built. It works well enough. More teams start using it. Workarounds accumulate. The original developer moves on. The tool becomes critical but underfunded. At that point, replacing it is expensive because the organization has built process around it.

The same pattern can happen with purchased tools. A team buys quickly, configures heavily, trains users, migrates data, and later discovers that the tool does not support the real workflow. Tool decisions create inertia.

Quality Engineering View

The right question is not "can we build it?" or "can we afford it?" The right question is: which option gives us the strongest quality signal, lowest sustainable friction, and best long-term adaptability?

For test automation frameworks, for example, building may make sense when the organization needs deep domain-specific abstractions. For performance testing, security scanning, accessibility analysis, or test management, buying or adopting mature open-source tools may be more sensible unless the internal need is truly distinct.

Tooling strategy is quality strategy. A senior quality leader evaluates build-vs-buy decisions through total cost, signal quality, team adoption, and long-term ownership. The wrong tool can slow delivery for years. The right tool can make quality evidence faster, clearer, and more trusted.

Costs involved in building vs buying tools

Continuing from our previous posts on the subject of build vs buy decision while choosing testing tools, listed here are the costs involved when deciding to either build or buy a tool in-house.

1. Costs to consider while deciding to build a tool in-house.

• Administrative costs involved in
• Defining the problem
• Collecting requirements
• Development environment and method selection
• Developing and administering training for users
• Managing the project and people involved
  

• Production costs involved in
• Planning the Tool development project
• Architecting and HLD (High Level Design)
• Detailed Design
• Coding / Implementation
• Client / Web
• Server / Back-end infrastructure
• Module integration
• Migration
• Software Testing
• Unit
• Integration
• System
• Performance
• Acceptance
• Debugging
• Documentation
• Deployment
  

• Support Costs involved in
• Ongoing maintenance activity
• Developing and Delivering upgrades / patches
• Defect tracking and reporting
• Ongoing support activities – such as running a help desk or similar mechanism

We must also consider the opportunity costs involved. The cost of time spent by employees (technical and business staff) along with any delays in schedules must be taken into account.

2. Costs to consider while deciding to buy a tool.

• Administrative costs involved in
• Defining the problem
• Collecting requirements
• Evaluation of various tools, Proof-of-Concept and Selection
• Managing Vendors
• User training
  

• Costs related to Procurement
• Cost of licenses
• Cost of Customizations
• Cost of tool deployment – setup, configuration, integration
• Tool testing – integration, compatibility, performance, acceptance
• Cost of migration 
  

• Support Costs involved in
• Annual Maintenance costs
• Costs involved to upgrade across versions
• Additional internal support costs

Also, as in the case of building a tool, the opportunity costs involved must be considered. The cost of time spent by employees (technical and business staff) in the above activities along with any delays in schedules must be taken into account.

Next up is a detailed article covering the popular myths and reasons behind making the build vs buy choice in testing tool selection. Stay tuned.

Image courtesy: Michelle Meiklejohn / FreeDigitalPhotos.net

Selecting a testing tool (6)

Having looked at some of the common reasons stated in favor of building tools in-house, let us now look at some of the reasons ascribed in favor of buying tools.

Reason – Buying will lead to faster deployment of the tool; a tool that is purchased will be faster to deploy.

Yes, when you consider that buying a tool cuts out the development costs involved in building it. Yet, the extent of quickness in deployment depends on your particular situation. 

There still needs to be due diligence towards selecting the right tool. Once that is done and a tool is procured, any needed customizations are to be performed. The tool then needs to be setup and configured, following which a detailed migration exercise has to be performed to move away from the existing tools and users need to be trained to work with the new tool. Time taken to accomplish the various tasks involved in deployment is also dependent on the extent of complexity. An important factor to consider would be the human acceptance of the tool. Just because a new tool is procured and setup, users & others associated with the roll-out will not automatically accept it. When a new tool replaces existing tools that are / were being used widely, you can expect some degree of resistance to the new tool. Comparisons are inevitable and in the initial days there are bound to be comments on the new tool vis-a-vis the existing tried and tested tools. Despite any shortcomings of an existing tool, users can be reluctant to adopting a new tool; more so, if they were involved / or had a stake in developing the existing tool. Customization, migration and acceptance are chief areas that can consume a lot of time.

I have come across teams that were reluctant (resistant would be more like it) to adopting a new tool. The new tool would technically help overcome drawbacks which the existing tool had – but folks had gotten used to the current tool, warts and all of it and were pushing back wider implementation of the new tool. They would find ways to delay implementation and after several deadlines for deployment had passed, the organization settled for a dual systems approach (use the old tool for some tasks and the newer one for some others). In some organizations, the transition to the new tool does happen but at a much later date than what was targeted.

Ultimately, the above elements will influence the amount of time it takes to deploy the tool you buy. Will it be faster than building a tool ? There is no definite answer. In some cases, it might be faster to build a tool while in other situations, buying a tool and deploying it will be quicker.

Image courtesy: Salvatore Vuono / FreeDigitalPhotos.net

Selecting a testing tool (5)

Here's the fifth part in this series of posts examining the buy vs build choice in test tool selection. Let us look at -

 

Reason 5 to build a tool in-house – Tools we build can be fully customized when compared to commercial software testing tools.

This is one argument that's hard to beat. It is true that tools you build will be more customizable than anything you can get externally.

However, that is just one part of the equation. We need to look at the emphasis on customization. Beyond a reasonable degree, the need for customization may be viewed as a factor emerging from an organization's fluid and yet-to-be-clearly-defined set of methods and processes that makes defining a set of requirements for a tool, hard. The preference in such as situation is to go with a tool that can theoretically be customized infinitely.

While the ability to customize is very good and useful, it must be viewed in the light of costs involved and time to deploy the tool. Here's a very simplistic example – let us assume that your software testing group wants a GUI automation tool. One possibility is to buy a commercial test tool such as SilkTest or WinRunner. The other option is to build a testing tool that is similar to these testing tools but can be customized to any degree you would like.

The points to consider would be - 1) the costs involved – you might end up spending huge sums of money (many times more than it would cost to buy licenses) to build such a test tool and much more to maintain it for as long as your organization needs to use it 2) The time to deploy – assuming (for some weird reason) your organization is willing to invest the money, the other important factor to consider would be the time to deploy the tool. Building a tool from scratch takes time, lots of it in fact. Factor this in while making your choice.

I must repeat that I am not taking sides here, nor am I suggesting you choose to either build or buy tools. The attempt here is to present various perspectives that can help you make an informed choice. There are situations where it makes better sense to build your own software testing tools, as there are situations where it makes sense to go out and buy a tool and if need be customize to your needs. In certain cases, it can be hard to find tools that meet your specific requirements. Also, with any commercial tool, you will have to adapt to the tool and the way it works. With a tool that you build, you can integrate it with your current way of working.

We continue our exploration of the build vs buy choice in upcoming posts. Stay tuned.

Image courtesy: Suat Eman / FreeDigitalPhotos.net

Selecting a testing tool (4)

Continuing from our earlier posts that looked at the common reasoning behind the question of whether to build or buy testing tools, let us now look at -

Reason 4 to build a testing tool in-house - We should build it since we just need a simple solution which our engineers should be able to quickly put together; our software testing requirements aren't too complex; we know what we need and figure it should not take much time or effort to build a testing tool

Organizations find it tempting to put together or build simple test tools to address a specific problem or area without considering, at the risk of making a cliched statement, the larger picture. If your test tool is not easily enhance-able / maintain-able and you are limited to the functionality you built-in initially, you can be sure that sooner than later, your needs will outgrow what your test tool offers - the test tool will soon hinder than aid your efforts. I have seen various groups create such “simple” tools to address specific pain points. Over time, various such “simple” tools were created and put to use by different groups. Since little thought was given to integrating the various tools or making them work together, the proliferation of these “simple & standalone” tools posed a significant handicap to the organization's software testing efforts.

From experience, such test tools tend to get discarded after some amount of usage and a suitable replacement is found. However, there are cases where these tools continue to be used – the organization has already invested significantly in the existing test tool and may not be willing to incur the additional costs associated with a migration to a new tool.  In either case, the organization has already incurred costs on the simple tool and will incur more costs to procure and move to a replacement tool. It would be better to consider the larger picture when making tool decisions. The test software (testing tools) should enable your organization to perform better software testing. Look beyond the confines of your group; examine (best) practices from other groups and tools they use; explore any possible changes to your current processes / additional requirements for the tool you intend to build and consider if the test tool can handle them.

It need not just be new or changed requirements that can make your tool redundant. Simple scalability issues can cause your simple tool to pose complex issues. If your tool is designed to handle the needs of your current group, consider if the same tool can handle the requirements of a larger group - will your tool scale or crawl ?

Does your tool consider all audiences who might potentially use / want / need to use it ? Some tools are created to solve pain points that a section of engineers may encounter. Their usage, output and reporting are often put together to provide the basic information that those engineers would need. However, when the same tool needs to provide information to a wider audience or someone in the business side of things, a different usage and reporting paradigm emerges. If the tool cannot meet the needs of the different audiences - current as well as potential future ones, it can again be a drag. What if the products you are testing changes significantly so that the tool also needs to be changed to handle newer operational / functional needs ?  Can your tool adapt to changes ?

We'll continue looking some more at the build versus buy choice for testing tools in this series of posts. Also, coming up is a comprehensive article on making testing tool decisions, listing some questions you need to ask and evaluate before making your decision of whether to build or buy a tool for your testing needs. Stay tuned. 

Image courtesy: Danilo Rizzuti / FreeDigitalPhotos.net

Selecting a Test Tool (3)

Continuing from our earlier posts that looked at the common reasons behind the question of whether to build or buy tools … lets now look at -

Reason 3 to build a tool in-house - Building a tool in-house saves a lot of money when compared to purchasing the tool externally. Since we are using our available resources, things would work out much cheaper than buying a tool

How easy is it to estimate costs accurately for a development project ? Try estimating the costs for your planned tool development project. Include the cost of resources involved in developing (gather requirements, analyze, design, implement, localize if needed, migrate data) the tool, documenting, testing (unit, functional, performance, etc), training users and performing on-going maintenance and enhancements of the tool. A common perception when developing in-house tools is that users of the tool will be more forgiving and adjusting of the tool's failings. Also, users might have reduced expectations from the tool. The fact is – users not only demand similar standards as external tools, they generally have more (rather than less) expectations from the in-house developed tool. After all, the developers are in-house and accessible. It is not hard to reach them and we can get them to add in a feature or customize something as we want it. Developer not willing to do it ? Lets go through the right channels / hierarchy and request for changes.

Cost calculations must also consider the opportunity cost involved with having both Developers & Management / Business staff working on the tool development project vs working on other revenue opportunities. We must not under-estimate the cost of tool maintenance. In many cases, on-going maintenance costs (defect tracking, regular updates / patch releases, support) can turn out to be a significant recurring investment similar to the licensing costs involved when buying a tool.

Basing tool decision purely on cost is not really a feasible option. What may seem to be a less expensive choice could turn out to be a less than feasible alternative. Focus on what provides maximum benefits and meets most requirements. 

CU3HFAVW6RFR

Image courtesy: graur razvan ionut / FreeDigitalPhotos.net
CU3HFAVW6RFR

Selecting a Test Tool (2)

Continuing our look at the factors contributing to the build vs buy decision making process, let us now look at -

Reason 2 to build a tool in-house – We have the talent (technical & functional) and understand our requirements / needs better than any external vendor. We can develop a tool that best matches our needs and better than what a commercial tool vendor may be able to provide.

Whom are we kidding ? Unless we are talking simple tools, the likelihood of internal groups being able to achieve the above and meeting requirements to a “satisfactory” extent is when the particular internal group that is expected to be building the tool – has tool building as it's main responsibility, has demonstrated expertise in successfully building tools, has the time / money / resources / skills to commit to building a tool that matches all requirements and can maintain the same on an on-going basis.

Why are these requirements needed of internal groups ? While it is easy to commit to developing tools, it is often hard to cater to all of the requirements and commit to sustain / maintain the tool for the longer term. Most commercial tool vendors have specialized groups dedicated to developing specific tools and have made large investments towards building tools. These tool vendors have built the expertise to address challenges and have the resources available to support the tool on an on-going basis.

Usually, when in-house tools are developed, it is the case that a set of engineers are assigned to work on the tool along with their regular tasks. Divided attention is hardly beneficial. If the tool developers have pressure to work on other tasks, they can easily lose focus and attempt to cobble up a tool that at best, barely meets requirements. When developers are assigned to work full time on creating tools, unless they have the aptitude and skills required – they can give up soon or even look at cutting / watering down requirements to complete their assignment and move on to other projects. Also, internal groups often face the challenge of maintaining and enhancing the tools that are developed. One other factor to consider while choosing to develop tools in house is testing. Like any product that is developed, the tool too must go through the same standard development processes and test efforts to be useful.

So, unless you are willing to commit the resources (time, money, effort) to develop, test and sustain a tool, it might not be a great idea to build it in-house. Now before you start thinking that i am trying to discourage any in-house development activity, let me state that the intent is to present some common reasoning from both sides of the fence (in-house vs external). I'll cover more reasons in subsequent posts. Stay tuned.

Image courtesy: Michal Marcol / FreeDigitalPhotos.net

Selecting a Test Tool

Test tool is a broad term that encompasses test automation tools and other tools used as part of testing. These could be defect tracking tools, reporting tools or test management tools. When an organization or a group within a larger organization decides that it needs a test tool, a likely question that can crop up would be – can we build it in-house or do we buy a commercial tool ?

For some groups, the decision's already made. There might be a centralized entity that has ordained that a particular tool or framework be used and all other groups simply toe the line. But what would you do if you are making a fresh choice ? Would you have to face the above question of build vs buy ?

Lets look at a few reasons why organizations might make either choices – build the tool in-house or buy the tool. First, why would organizations choose to build a tool.

Reason 1 – Short term solution

We just need something for the short term, a temporary solution, something quick for the time being. We are hard pressed for time and cannot spend time evaluating a lot of choices. Let us quickly put together something now and later explore more longer term solutions. Any of these sound familiar ? Looking for short term solutions ?

We must realize that when we decide to build something, even if it is supposed to be a temporary solution, we will be investing money, time and resources to create, use and maintain the tool.

The tools we employ to facilitate our testing, can soon mould our work habits around them. Sooner than later, the tool will become a part of your regular job and exercise a certain degree of inertia towards any change. When the tool falls short in certain areas, you will (have to) come up with workarounds to continue using the tool and over time it becomes harder to change established work patterns and tools learnt.

In most cases, once employed, a tool becomes hard to change. Lets take the example of an automation tool / harness that your in-house resources put together. Once you have reached a certain level of automation using the tool, how easy will it be to scrap the tool and opt for a new one ? If your tool is lacking in certain features, would you spend the time and effort to enhance the existing tool or would you ditch it to go for an external tool that offers those extra features ?

In my experience, I have used tools that were home-grown and met the basic requirements of our group. However, the tool paled in comparison to some of the externally available tools in terms of features and options available. Since we had invested enough time and resources on the tool, it became hard to let go or even migrate to another tool without incurring significant additional investments of money, time and resources – which we were reluctant to do.

In essence, you cannot afford to justify a decision based on it's perceived or assumed short-term need. Every decision carries a set of ramifications with it and it is rare that you can easily back out if things do not work out as expected. We'll explore more on the question of build vs buy in subsequent posts. Stay tuned and send in your feedback using the "Contact" link at the top of this blog.

Image courtesy: Simon Howden / FreeDigitalPhotos.net

Who automates your tests ?

Whom do you assign to work on automating your tests ?

Ideally, a person who is chosen to automate tests should have sound development and testing skills. Often times, people are chosen who might have expertise in either of these areas. This can be worked upon. If your test automation engineer is predominantly from a testing background and lacks development skills, you need to check if this person can pick up the required development skills and demonstrate the necessary rigor and discipline.

Generally, test engineers who desire to be programmers, jump at the opportunity to automate tests. This provides them an opportunity to develop their programming skills. While doing this, one must make sure that these engineers are well grounded on the automation requirements and focus on delivering them rather than getting carried away with the programming task. Sometimes, you may not need to automate all your tests given the available time and resourcing. Here, the ability to focus on those areas that can provide maximum return on investment is important.

A reason for requiring development skills is the need for focus on aspects such as maintainability, re-usability, reliability and importantly, the ability of the test automation suite to easily adapt to changes to the system under test. Test automation needs to be approached with the same rigor and commitment as a regular software development project.

Sometimes, developers get assigned to do test automation. In this case, an evaluation needs to be done to ascertain if this person can gain understanding of the testing discipline and develop a healthy respect for the function. When in-house engineers aren't available, it may be the case that contractors are employed to perform test automation. The advantage here is that you are likely to get engineers with matching skill-sets. However, one must be aware of & take appropriate steps to mitigate any maintenance challenges once the contractors have departed. Having a sound automation framework in place and reviewing conformance to stated standards and guidelines can be useful.

In some groups, the new college graduates, interns or junior programmers are assigned to work on automating tests. The argument may be that these folks have spent a lot of time learning about programming in college and would be able to apply their “fresh” learnings to automating tests which is a type of programming activity. Also, these new comers often are glad to have the chance to demonstrate their competence and creativity and show interest. In some cases, it may be that the management wants to keep these junior folks engaged on some relatively lower risk activity and hence assigns automation to these engineers. If what they develop, works – good and if it doesn't it can be shelved with low cost incurred. The new comers often are not well trained on the testing function and often end up using their programming skills to creating complex test automation suites that can be hard to maintain without meeting testing requirements.

Worser still, is the attempt to assign automation to the “lower-performers” from development or testing. These folks have poor skills, commitment and motivation in either area. The chances of success in this case is very low. Test automation requires skilled, committed and motivated folks who can work effectively with both development and testing teams.

Test Automation Pitfalls (3)

Continuing from the previous posts, we look at another factor contributing to less than effective test automation.

3. Stability of the system being tested and the ability of the automation suite to adapt to changes to the system

There are two parts to this – one, the system or application that is being tested needs to be in a relatively stable state before you embark on an extensive automation exercise. If you try to build your automation suite when the system on which these automation tests are based upon keeps changing frequently, you are guaranteed to encounter – false test results being reported by automation runs and ever increasing maintenance activities on your suite. We must however, realize that despite calling for “relative stability”, there will be some changes and any test automation exercise must accept and be open to some amount of change. Having a stable product can help minimize these changes.

This is where we look at part two, which is the ability of the test automation suite to adapt to changes to the product it is testing. When test automation is not based on a sound framework that can minimize the impact of changes to the system under test, extensive maintenance activities need to be performed. This can have the added effect of affecting stakeholder perception of both the reliability and usefulness of your automation suite.

From a process stand-point, when major changes are planned to be made to the system under test, these must be reviewed to evaluate their impact on existing test suites. Any resulting test automation activity must be planned and communicated appropriately.

Test Automation Pitfalls (2)

Continuing from the earlier post on the subject, lets look at another factor that leads to less than effective test automation.

2. Automation Framework and Process.

For many groups, test automation begins with the acquisition of a tool and from thereon it proceeds towards a race to automate as many tests as quickly as possible. Everyone's excited about the tool and its features, so much so that each one lets loose their creative energies on automating their set of tests quickly. During this time, some attributes of test automation such as – reusability, maintainability, scalability, separating data from the automated tests and such others are often ignored. These attributes unfortunately, are required for a robust and maintainable test automation suite.

An oft repeated aspect of test automation is that you must approach test automation just as you would a product that your organization produces. This implies that the same principles and practices need to apply to test automation; the same level of detailed planning, designing, architecting, process definitions, reviews, etc. need to apply. Test automation is software that you are developing and a reason that automation can fail is because most organizations do not treat the test software they develop with the same level of care and commitment as they do to the products they sell.

Dis-organized test automation or automation done with little attention to the required attributes, results in a suite that can be a maintenance nightmare. Groups soon realize that they need more resources to both maintain the automated suites as well as to automate newer tests, resulting in ever increasing costs and little added benefit.

It will be useful to remember that all automation needs to be maintained or abandoned. Make sure your automation is maintainable easily at low cost.

We'll continue examining additional factors in the next post.

Image courtesy: Francesco Marino / FreeDigitalPhotos.net

Test Automation Pitfalls (1)

Organizations make significant investments in Test Automation. However, few of them realize corresponding benefits from their investments.

Often, automation suites turn out to be expensive to create and subsequently maintain without being  effective in unearthing defects. The natural question that gets asked is – Why ?

There are various factors, some of which include the following.

1.    Quality of tests being automated.

"Garbage in, Garbage out” applies well to test automation. If your test automation is based on tests that are themselves of poor quality, then you can rest assured that your automation results will hardly be spectacular.

While analyzing issues with test automation, we must examine the underlying tests and their design. For most organizations, their test automation goal is to automate 100% of their existing manual tests. While it may be a challenge to reach this goal, especially in larger and complex projects, the usefulness of such a goal is debatable. Unless you have a good set of tests to automate, merely trying to convert all of your manual tests to automated ones, does not offer much benefit. The exercise could very likely incur significant investments of time and resources while yielding little effective returns.

Quoting Mark Fewster - “It doesn’t matter how clever you are at automating a test or how well you do it, if the test itself achieves nothing then all you end up with is a test that achieves nothing faster.”

We will look at more factors in upcoming posts. 

Image courtesy: Chris Sharp / FreeDigitalPhotos.net

Enterprise Application Development - changes needed

While reading a recent Forrester research report on - The Top Five Changes For Application Development, I was attracted towards the suggestion that enterprises re-invent themselves to think and behave like startups.

In the changed business environment following the recessionary times we've been through, it becomes even more important for enterprises to learn from startups. The need to be nimble, quick and able to satisfy shifting customer requirements, rapidly respond to competition, quickly change tracks when things are not working, slow down when business is slow but move swiftly when the “tipping point” arrives – all attributes that enterprises, should pickup from their startup brethren.

Other significant attributes of startups that enterprises would do well to learn include the following.

• For your developers to know the inner workings of your business and understanding what customers want, what competitors are doing, and what it will take to create a great user experience. Given that startups have very little resources, they have to be focussed on what will give the maximum return on investment, which in turn translates to doing exactly those activities that can be linked to their ability to generate revenue.
  

• Discard wasteful, redundant and non-value-add process steps from your Development process. Continuing from the earlier point about startups being business focussed, we also observe that startups do not adopt elaborate development processes or expensive tools. Their focus is on having just-enough processes and tooling to deliver solutions asap. Enterprises can pick up a lesson or two from startups here and challenge the status quo in their organizations by questioning the need for heavy-weight processes vis-a-vis their value proposition.

• Incorporate Agility into your DNA. Like startups who are glued to their customer requirements and competition, both of which have a common attribute – change, it is important for enterprises to align process, methods, resources and the organization itself towards agility. Being able to discard methods or even projects that no longer make business sense, being able to rapidly respond to changing customer requirements and market conditions while realizing that your organization's course is not set in stone, is key. It might make for sound business sense to explore Agile development practices for your organization.

• Do more with less. It is easy to focus on doing one of them – either more or less. The challenge is doing both; do more with less … also known as being “Lean and Mean”. This isn't a totally new attribute but rather an extension of the previously mentioned startup attributes – focus on what is important and smartly organize all your resources towards projects that have the most business impact, discard waste and be focussed on value generation, ensure your developers know the business thoroughly and understand what matters most in a project – technical genius is useless if the user experience of your product or service is bad.
  

The quote by Niccolò Machiavelli: "Never waste the opportunities created by a good crisis", is very apt. The recessionary times have reinforced the theory of survival of the fittest. Organizations will need to adopt practices that will help deliver better, quicker and cheaper – while meeting customer requirements, reducing costs, innovating and  responding to market opportunities quicker than the competition.

Image courtesy: djcodrin / FreeDigitalPhotos.net

Hashtags in Twitter

Today's blog entry is about the use of hashtags in twitter. Hashtags are reported to be amongst the most confusing aspects of twitter. It is however, a simple concept and definitely worth understanding.

Twitter does not yet, have a way to classify messages into categories. For example, there isn't a way to mark a bunch of messages as belonging to a specific category or event. To workaround this, twitter users use hashtags. A hashtag is basically a term prefixed with the # symbol. Example - #tech, #news, #anything …

Software professionals call the '#' symbol as a hash. When the hash symbol is combined with a term which is used to tag twitter messages, it is called a hashtag. When an user wants to classify related messages around an item, say news about apple, the user might come up with a hash tag as #apple and put it in their message. Then other users add the same hashtag to their messages about the similar subject. You can search messages based on hashtags.

While hashtags help identify events, they can also be used to denote messages from specific groups of people. People of a particular community might want to tag their messages with a common hashtag so it is easily identified and search-able. Some hashtags can be used to cross-post messages to both twitter and non-twitter applications such as Facebook (using hashtag “#fb”), Linkedin (using hashtag “#in”), etc.

To find messages classified with a particular hashtag, visit  http://search.twitter.com and query for the hashtag. Hashtags can be used for various purposes. A few tips are listed below.

• Use to collect ideas / crowd-sourcing – you can ask questions on twitter and give a hashtag that other twitter users can use to tag their answers to your question. That way you can quickly identify the responses

• Use to have a group discussion – you can have folks located anywhere on the planet, participate in discussions. Put out a hashtag that folks participating in the discussion can use. Formal discussions may be arranged in advance to occur at a specific time.  There are third party tools that allow you to collect messages that use the particular hashtag and aggregate them in the form of a conversation

• Use to share thoughts / experiences on any subject – unhappy about a product or service you received ? Use a hashtag along with your message. Generally, many people would be participating / using most common hashtags. You might want to check if a hashtag is being used on twitter search and also the conversations around those hashtags

Hopefully, this helps clarify the use of hashtags in Twitter.

Cloud computing - offline

With all the talk around cloud computing, a general assumption seems to be the availability of internet connectivity and uninterrupted availability of the cloud. Cut to the real world – where interrupted connectivity is a given and even the best of hosted clouds fall short of 100% availability. What would users of cloud based apps and services do in such situations ?

In my earlier post I was highlighting the usefulness of a cloud based suite of office productivity applications (such as Google Docs) and how they could either complement or in some cases replace traditional desktop software. There are many useful cloud based apps which can enable users to migrate from the traditional model of installing apps on their local machines and maintaining copies of their data at each location where they need to access it.

Any move to a cloud computing paradigm must consider the requirements for “offline” access to cloud resources – would users require offline access to their cloud applications and also understand the fact that not all applications lend themselves smoothly to offline access.

Looking back at the example of cloud based office applications (Google Docs) or even cloud based communications apps such as email and calendar, there are tools to enable offline access for these applications. Technologies such as Google Gears, facilitate this offline access. There are many real life use cases where such offline access is required – allowing users to, for example, download their emails & calendar events to a local cache while working offline and later uploading changes when connected back to the cloud.

However, there are applications where such offline access can pose significant challenges. Take the case of large distributed ERP applications that are hosted on a cloud wherein some users are working online (connected to the cloud) and some are working offline with a snapshot of the data downloaded to their devices. Issues around data integrity can quickly arise when folks who are offline make changes to data / create new entries and later try to sync back to the cloud while the folks who are already on the cloud have made other changes to the same set of data. Take a simple example of a salesman on the field who uses the offline data that shows stock available of a particular product and then commits to the customer to deliver it immediately. However, another sales person who is on the cloud has also made a sale of the same product to another customer and now there is no stock available. The sales person who is offline will later try to sync back to the cloud and realize that delivery as per commitment would not be possible. Again, looking at the same sales person who is offline and generates an invoice with the invoice number being auto generated based on the last invoice number. If folks on the cloud have also simultaneously been generating invoices, we can easily end up in a conflict situation where the same invoice number could be assigned to multiple customer orders. Such cases do not mean that we just say “No” to cloud based apps – there may be few cases when “No” might be a more plausible way to go – in most other cases, some amount of thought around re-design of applications to cater to the challenges of working on the cloud will be required.

Cloud computing - for everyone

Most of us produce and work with a variety of files (documents, spreadsheets and presentations) each day AND we do all of it using our traditional tried and tested Office software applications which we procure and install on each of our machines, be it at work, home or anywhere else. This post introduces web-based office applications such as the freely available Google Docs which could be leveraged in conjunction with with your existing tools to enhance productivity and collaboration.

I have been using the set of web-based applications that come bundled as part of Google Docs, in addition to the traditional office applications for a while now and find the web-based tools very useful in areas where i need more folks to collaborate (including in real time) on a document i am creating or to share a document to a wider audience and have the latest revision always be available to all of them without me having to resend a copy of the file to everyone or to have some of my files stored centrally and accessible from any where and continue working on them using just a simple web browser (no additional software required) or not having to worry if i have the latest version of a file or even about regularly backing up files! 

While there are very few data privacy concerns using Google Docs, i still retain confidential files on my local setup while using the web-based tool set to work on other documents that could benefit from being shared to folks i choose to view / edit them. This collaborative model may be used for a variety of applications, including tracking statuses, working jointly on a single document in real time and publishing reports that are automatically updated and available to everyone as when changes occur. You could find several other uses and scenarios where these tools may be applied to enhance productivity. If you haven't already been using these simple web-based tools, i suggest you give it a shot. You just might find them a nice complement or even in some cases a good-enough replacement for your present tools.