Test Automation Rant Part 4 (video) - Utopia Solutions
Full Service Software Testing Solutions. Our 25+ Years of Experience is Unparalleled Call us for more info: (630) 593-2074
Onsite – Onshore – Offshore request a discussion
Nov 03

Test Automation Rant Part 4 (video)

Why do words like tedious and time consuming keep coming up in these Rants?  Check out Lee’s latest verbal tirade on the “monumental importance” of the percentage of test cases automated.

Interested in knowing what your organization’s level of readiness is for implementing Test Automation? Click here to take Utopia’s Test Automation Assessment, our free online automation assessment will give each participant a personalized scorecard to gauge their test automation readiness and existing health.

Take our user-friendly functional test automation assessment to determine your automation potential. Whether you get top scores or have plenty of room to grow, Utopia Solutions will help you optimize the return from your investment in test automation.


-Check out more of Lee’s Rants-

Part 1 – Test Automation

Part 2 – Managing Expectations

Part 3 – Consultants and the Perfect Framework

Part 5 – Automation ROI



About The Author

Lee Barnes has over 20 years of experience in the software quality assurance and testing field. He has successfully implemented test automation and performance testing solutions in hundreds of environments across a wide array of industries. He is a recognized thought leader in his field and speaks regularly on related topics. As Founder and CTO of Utopia Solutions, Lee is responsible for the firm’s delivery of software quality solutions which include process improvement, performance management, test automation, and mobile quality. Lee holds a Bachelor’s Degree in Aeronautical and Astronautical Engineering from the University of Illinois.
  • dadams says:

    How about measuring # of validation points? We are looking to show this metric in an attempt to show that a test case can cover a number of functional elements. For example, you could have one test case that explicitly shows validation points. For example, in rspec (our test runner of choice, for now) has this:
    expect(message_invoice).to eq(invoice_number)

    There are multiple ‘expects’ in our test cases. UI automation (we use Selenium/Ruby) is expensive to run so we take advantage of inspections and add them throughout our tests. When that happens, could have 1 test with 10+ validations. People get nervous when there aren’t “enough” test cases.

    Myopic? Yes.

    I am hoping to change that perception at my shop and show the actual value automation is providing.

    We have the data we just need to advertise it better.

  • Lee Barnes says:

    Yes, validation points (and other relevant automation metrics) are a great thing to track. The point of this rant was that I see so many organizations start on their automation journey by blindly setting a goal of automating X% of their manual test cases. Whether you’re tracking validation points, test cases, or % of testing activity automated I believe the key is to set the goal intentionally using data that can support it.

    I like to track % of test activity (i.e. time spent on testing related tasks) as that gets people thinking about additional activities that can benefit from automation. What I don’t like specifically about % of test cases automated is that it often leads people to take their manual test cases and start automating them one-by-one verbatim. They completely skip the process of thinking about what should be automated, how to approach automation most efficiently, etc. and end up with a disaster they can’t maintain.

    Finally, I agree 100% on advertising your automation data. Your ahead of the game in that you HAVE data!

  • >