Jul 12

Issues with Image Matching/Analog Automation Approach

Be careful with this approach.  It is a form of analog testing and I would be hesitant on using an analog test automation approach these days.  If this is your only option then fine, however test automation tools have advanced enough that you can normally find one that will interface at the object level with your application under test if it is built with a commonly used development language/toolkit.  Otherwise this type of testing approach where it uses image matching technology versus interacting at the object level, tends to be easily broken.  Back in 1992 when I first got introduced to the Mercury tools that was their initial approach as test automation was still new; therefore I have first-hand experience with this approach.  Image Matching testing on a GUI application is very difficult and can be real frustrating.  It is like being in “analog mode” in QTP where all your actions are coordinate based (x,y coordinates off the screen) and then your synchronization and validations are all done by capturing images.  For synchronization and validation a new image is captured and the two images (expected and actual) are compared at the pixel level.  This is not very accurate since a lot of things can cause your test to fail when it is really not an issue.

Failures include:
– a slight adjustment in window or object location causes a mismatch on images
– change in font will cause a failure
– image resolution will cause failures
– different browsers can display the color palette slightly off.  the average person wouldn’t see the difference in hue but image comparison will fail
– computers with different graphic accelerators can cause images to appear slightly different, especially with colors.  Again the average person doesn’t really notice or care about the slight difference but the tool will see it as a difference and fail it
– since all actions are analog, any slight change or shift has the potential to cause your script to fail because the object it was going to click on might not be in the same spot
– keyboard actions can easily fail because you may not have the focus on the correct object/location due to the analog approach
– synchronization is very difficult in analog mode
– typically these kinds of scripts are difficult to maintain because they are coordinate based, so they are not as descriptive and so the code doesn’t always make sense and is not intuitive

As you can see there are a lot of factors that make automation in this kind of environment very difficult.  If it is your only choice to interact with an application, then you don’t have a choice.  When deciding on a tool, realize that QTP has an analog mode and a “low level recording” mode that will give you this capability when you are automating against a completely custom object/application or against an application that has been developed with a language that QTP does not support.  In that case you can switch to Analog mode in QTP.  QTP will allow you to switch back and forth between these modes if you want/need to, all in the same test script.  But if you have the choice between analog mode versus context sensitive (interacting directly with the objects) then you are always better off with the object interaction.  You can do more and have less failures that are misleading.

by sloporto | About the author:

Related Posts

  • No related posts found.

You must be logged in to post a comment.

Name (required)

Email (required)


XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Share your wisdom