ASI

The Tester Interview

I came across an interesting topic, it was a topic about what senior level QA Managers look for in hiring a software tester. I thought I would blog on this topic having numerous years of experience at the senior and executive levels (Director of QA, VP of QA, and President of ASI), having interviewed hundreds of candidates and probably hired close to 100 testers over the years. This is my perspective on what a solid tester should be and I totally agree with the statement in the article that says “Everyone has a different set of criteria for hiring testers”. Reason being is because every hiring manager has a different background, skill set and work ethic so every manager looks at this position in different views and has different expectations.

I want to begin with setting some ground work by giving an overview of my background which influences my views. I started my software career as a software developer/programmer. After many years I was introduced to test automation and due to my technical background as a developer I got pulled into implementing test automation solutions for QA teams and my ultimate transition into the QA arena. Having been a Captain in the Army I also enjoyed leadership roles and found myself quickly climbing the ranks in management. With over 23 years of experience now I’ve done everything from programming, testing, automation, load testing, release management, project management, technical training, consulting and production support. This well rounded software development background enables me to understand what is truly expected of the various job responsibilities.

So now the Testing Position: Over the years I have seen the testing position called several different titles but I personally tend to group them into three basic true job categories. Below I will address the three categories – what I think their title is, my opinion of what the job entails, the skill set required and the interview process.

The Manual Tester - I see this position called “Tester”, “Manual Tester”, “QA Tester”, “Software Tester” and some deviations. What it is NOT is a “QA/QC” title – that bugs me; if you see this then the company probably doesn’t really understand software testing or the position is not software related. “QA” stands for “Quality Assurance” which is what we do in software testing. “QC” stands for “Quality Control” and this really relates to the manufacturing industry which is a much different type of “quality/testing” position. Back to the Manual Tester – This person’s primary job is to manually test an application. They may use some tools to aid their job (like text comparison tools, SQL editors to query a database, Putty to access a Linux server, WinSCP to FTP files across servers, Excel, etc) but normally this person does not do any true test automation work. One reason is that this not considered a “technical” position because there is no test automation programming or script programming required. Although I don’t expect a manual tester to be able to program, I do expect this person to understand basic SQL in order to validate data in the database and the ability to navigate around on a server in order to access files and make modifications for testing purposes. This person is also responsible for all the positive testing, negative testing, boundary testing and data driven testing throughout the test phases of Functional, Regression and System/Integration. This person is meticulous, attention to detail, willing and able to open defects to track bugs found in the application, ability to learn new applications with ease, and able to follow instructions step by step without any deviations. I have always used the tool called the “Quality Center” which is the top of the line test management tool on the market, so therefore I look for candidates that possess the skill and experience of working with this product.

The Automation Tester – For me this position is a “Test Engineer” title but lots of companies refer to it with a manual testing title but then imply that you will implement test automation, but I think that is an injustice to the person since they are significantly more technically competent than a manual tester and desire the more appropriate and descriptive title of “Test Engineer”. I implement QuickTest Pro (QTP) for test automation solutions and expect this person to know the tool very well. This person needs to be able to take a manual test case and convert it into an automated test case. For QTP I expect the person to know VBScript since this is the programming language used by the tool and I expect them to be willing and able to make script enhancements using code to create advanced automation scripts. The ability to utilize FSO and ADO in the test scripts is also a given since these are basic testing needs needed in order to validate various functionality. I expect every tester (including my automation testers) to have a solid understanding and experience with manual testing as this is really the foundation for all testers. A Test Engineer is simply automating a manual test case in most situations; therefore they have to understand manual testing in order to develop meaningful automation test scripts. This person may do both manual testing and automated testing, however it is nice if you can have at least a couple of Test Engineers that can focus 100% of their time to just developing automation scripts and NOT be involved in any manual test efforts for projects because that can easily consume all your tester’s time to where they never get a chance to build an automation baseline of scripts.

The Load Tester – For me this position is a “Performance Engineer” title. This person needs to understand the differences and how to implement Load Testing, Performance Testing and Stress Testing (3 different types of load testing approaches) with a tool (I use LoadRunner). Therefore I expect the candidate to understand how to use VUGen, the Controller, the Analysis Tool and the various basic and advanced features of LoadRunner. This person needs to be able to program in “C” since this is the programming language LoadRunner uses. This person should also understand manual testing; however their focus is going to be much different and more technical in nature. In addition this person typically needs to understand a lot of the “behind the scenes” stuff – like infrastructures, load balancers, protocols, basic HTML “gets” and “posts”, the concept of correlation and how to interpret graphs for recommendations on hardware configuration changes to support load. This person normally ONLY works on load testing and does not do any other types of testing work. This is easily a full-time job/effort.

Interview Process:  This is my interviewing process with all candidates; I have a four step process on interviews -
1. First there is typically a phone screen that is intended to only last for about 15 minutes. It is a quick screen to make sure this person is really a good fit in the first place and not wasting everyone’s time. This is done by either the HR Recruiter or the hiring manager. There are a variety of things that can be quickly identified over the phone and may disqualify a candidate which can save the hiring team a lot of time from not having to waste time on the rest of the process.
2. After passing the phone screen, the candidate then comes in and takes a written test which asks basic testing questions about the Software Development Life Cycle (SDLC), basic test concepts, and some simple “challenge” questions to include writing a SQL statement and writing an actual test case. Since I prefer candidates with “Quality Center” experience, I then have some basic Quality Center questions to see how well they really know the tool. In addition if they are applying for an automation role, then the exam has technical questions about the tools they will be implementing (I use QuickTest Pro for test automation and LoadRunner for load testing so the questions will be specific to those tools).
3. In-Person Interview: after the written test I’ll review their answers and then meet the person to ask them more questions about their career, actual job experiences/implementations and some technical questions based on what job they are applying for. This also enables you to determine their personality and if they are a good fit for your team. Team chemistry is important so make sure you are hiring people that will work well with the rest of your team.
4. Hands-On Test: Now in the interview, I will have a laptop so that the candidate can demonstrate that what they claim to know, they really do know. I have found over the years that many candidates have “book knowledge”, meaning they really don’t have any real world experience implementing the tools but have done lots of reading. So they are the “academic” type that have book knowledge from reading the manuals and stuff, but have never really done the work for real so when you make them actually use the tools in front of you they can’t actually do it. For a manual tester I will have them create a manual test case in the Quality Center and do a few basic things in QC. For someone who will implement QuickTest Pro I will have them automate a very basic test case; and for a Performance Engineer who will conduct load testing I will have them create a basic load test scenario and then kick off a load test in the Controller for a few virtual users. What this hands-on test does is weed out the people who really don’t know what they are doing from the people with real experience. This last step I’ve found to be incredibly valuable in the interviewing process. It really gives you a feel for what this person is capable of and their basic knowledge. There is no fooling an interviewer on a hands-on test.

The one thing you can never truly know in an interview is the person’s integrity and work ethic. I have hired individuals that have passed the interview process successfully (so they demonstrated that they had the technical knowledge and experience for the position) but once they were hired they were lazy. No matter what you did to try and motivate them (even with counseling statements) they still did not get it and change their ways. They did not understand the concept of working hard and accomplishing tasks in a meaningful timeframe. They had opportunities to get things accomplished and learn but had no focus and commitment. Unfortunately this type of character you cannot really determine in an interview and will only know once they work for you. Hopefully you don’t get any of these people. They are not common, but they do exist. I want hard workers that are committed to doing the best job they can and work hard to meet and exceed expectations. These are the people I want on my team and they are the people that get rewarded.

Good Luck!