“Top 10 Leadership Characteristics”

This is a topic that you could write a book on and there are in fact many books on leadership.  The idea of this blog is to create a short but concise blog series on this important topic.  As a manager in my professional career (both military and civilian/corporate America) I have worked with many managers, some good ones and lots of bad ones.  Why are there so many bad managers/leaders out there?  One common issue I see is that the majority of them have never had the opportunity to be trained as a leader and how to manage people.  Having attended a Military College (Norwich University, the oldest private military college in the country) I had the benefit of being a part of an elite organization that was focused on developing the best leaders possible.  That involved taking many leadership courses in order to graduate with a commission in the Army and then the Army continued to provide more leadership training on a regular basis.  Unfortunately most people never get these same training opportunities and at some point in their career they get promoted into management and are just expected to know how to be a great leader and manager.  Sorry, but that doesn’t just come with the title.  Realizing that, I have traditionally run my own “Professional Development” classes to provide my staff with continued training.  For my leaders (team leads and managers) I would run this Leadership Class every two weeks to cover the variety of leadership and management traits and knowledge they need to be successful at their jobs.

So to begin I thought I would start the series with the topic of the “Top 10 Leadership Characteristics” that I feel are core to every great leader:

I don’t believe that being a great leader comes naturally but happens when an individual makes a conscious effort to go out of their way to lead.  Over time this may become more natural and second nature, but even then a great leader will continually analyze their leadership style & approach to identify areas they can improve upon.  The main reason why I believe leadership doesn’t come naturally is because leadership (like most things) requires effort!  There are many different leadership styles and characteristics, and I’ve listed my Top 10 below, but one key ingredient and fundamental quality to a great leader is that “You cannot demand respect – you earn it!”  There is a huge difference between someone doing something because “they have to” versus because “they want to”.  If your employees, team members, subordinates, players, etc. truly respect you then they will go out of their way to work hard and do things for you because they believe in you, want to follow you and truly feel like they are part of a meaningful team.  In the military there is a significant difference in salutes –one is because a subordinate respects you, and the other has no respect and he’s simply “saluting your rank due to protocol”.  A true leader will also know the difference as to whether the individual respects you or is “just saluting your rank” – whether he runs another lap because he knows he screwed up or because you are just  being a jerk trying to show who’s in charge – whether the team is willing to stay late and put in the extra effort to get a project done (and you are right there by there side) or whether they are complaining the whole time because they know how ridiculous the expectation is, due to poor leadership and planning and have no desire to be there.

Here are my main top 10 leadership characteristics that can make you a better leader.  These are high-level overview descriptions but if you want more you can Google any one of these topics and I guarantee you will find plenty of articles on them as they truly are core characteristics to great leadership.

1. Treat others the same way you expect to be treated – think about others; care about others.  This is an extremely simple concept but one that expects you to get off your throne and come down to earth.  It goes both ways – if you expect your boss to communicate to you about things, then you need to do the same to your staff.  If you would like to be cut a break on leaving early one day for a family need and you have put in plenty of late nights, then why wouldn’t you do the same for your own employees who put in extra efforts and stay late to get projects done.
2. Leadership by Example – this is huge!  This is something I constantly remind myself about.  If you expect others to follow you then you need to show that you are not only competent but are willing to “roll you sleeves up and get dirty”.  Everything you expect your staff to do you need to do the same.  If you expect them to be cranking out the work and not just hanging out talking with everyone, then you shouldn’t be.  If you expect your team to stay late or work over the weekend, then you better be there working with them too. 
3. Do not micro manage – entrust your team with responsibility.  You hired them because you felt they were competent to get the job done – so let them do it!  If you can’t trust them to do the job right, then get rid of them and hire someone who can.  But if you feel the need to do everything then there is a problem.  If you are not willing to replace an employee then the problem is probably you! 
4. Recognize and reward – very important.  In today’s tough economy you might have to get creative here.  There are many resources available for ideas on this and one I have that I’ve gotten some good ideas from is a book called “1001 Ways to Reward Employees”.  Financial rewards can come in many sizes and forms; if your company will allow financial rewards to be issued then these are great when people achieve some significant accomplishment that greatly benefits the company.  When money is tight this might not be possible but it doesn’t mean you still can’t reward people.  Three basic things you can do are:
   1. Verbal recognition in front of the team in a meeting/formal setting.  Individual recognition can go a long way for many people.
   2. Fun rewards that are very inexpensive – like when someone does something “sweet”, I would recognize them in front of the group and reward them with a 1lb chocolate bar as that was the “sweet reward”.  Believe it or not this actually went over well, the individual received individual recognition in front of the group, received something tangible, and the majority of people love chocolate so it was something fun to eat.  Always be concise about food rewards and who you are giving it to.  The whole reward process can be ruined if you give someone a reward they can’t eat for some reason (like if they are allergic to chocolate or are lactose intolerant).  If you did this then you will lose on two fronts – one the reward is no good to the person so it loses it value and impact, and #2 you lose credibility because it shows you are not connected with your staff enough to know them to realize they have this allergy/medical issue or possibly are on a diet and they are really trying to stay away from something like this.  Make sure you know your people!
   3. “Day Off Reward” – using postcard paper from the office store, I print off special “Day Off Reward” cards that I will issue people.  For many people time off can be as valuable as money.  The way it works is that the person is allowed to use this card to take a day off of their choosing (now or in the future) and when they use it they turn it back in with the day off written on it (this way it can’t be reused).  When the employee takes the day off, as a manager you simply record their time in the time tracking system as being present.  It keeps things simply, avoids paperwork and politics, but is also legit because you have the reward card in the person’s file for tracking purposes if there were ever any questions.  I’ve done this for years and never had a problem and was never questioned.
5. Continuously look for areas to improve with yourself – great leaders will never feel comfortable with themselves nor believe they are great.  Instead they will continuously evaluate what their weaknesses are and where they could improve.  Being honest with your self-evaluation is the only way to improve in this area.  This can be a significant ability to improve one’s self, if you can handle being brutally honest.
6. Don’t get comfortable – a lot of people do things well and then get comfortable thinking everything is good.  But life and business is very dynamic and constantly changing, therefore you need to constantly develop and adjust to the changes around you.  For instance in software development I’ll define and implement a great methodology in one environment and years later when I move on to another career opportunity I find the environment there is different and I have to modify my methodology.  I find that most of the time my “core” knowledge, methodologies, process & procedures, leadership & management style, expertise will continue to be my core foundation – the essence of my style; but I always have to be willing to adjust to the current environment I’m in.  Then once you refine your process and procedures, don’t get comfortable!  Look for areas of improvement and how you can take your team to the next level.  Just like CMM Levels (Capability Maturity Model), you are always looking for new ways to improve in order to move up to the next level.
7. Continued Professional Development – the best leaders and best workers are ones that continuously look to develop themselves.  This item comes in two forms – developing yourself and developing your staff.
   1. You need to constantly look at areas and ways of improving yourself with knowledge and skills.  Three main areas to constantly work on improving are 1) management skills, 2) your specific industry knowledge (if you are in the medical field learn that area of healthcare; if you are in the banking industry learn something new like lending models or trading or the secondary market and 3) actual hard skills – like if you are in software development learn a new technology or take a skill you know and go from the basics to very advanced.
   2. Your staff – Most people love to learn something new.  By investing in people you build a solid team.  This doesn’t mean you have to pay for them to go to some expensive training class or seminar; you can do a lot of this in-house by using the expertise you already have available to you.  For example, I regularly run a “Professional Development” class for my staff – either 1 hour a week or 1 hour every two weeks (the environment will dictate this).  One of my expertises is test automation, so a regular series in this class is me teaching my staff how to use QuickTest Pro and various tips, tricks and techniques.  This constantly improves your team’s skills and knowledge.  I also find that my staff really looks forward to the class and gets really disappointed if I have to cancel it for some reason.  Likewise it goes a long way for establishing your competency with the team and also connecting and bounding with the staff. 
8. Don’t be afraid to confront problems and deal with issues – this is huge!  I find a lot of leaders are afraid to deal with confrontation.  No one likes dealing with problems but this is one of your job responsibilities as a leader.  I see a lot of leaders don’t want to deal with issues and they think if they ignore it they will go away.  Guess what, most of the time they don’t and only get worse!  If you don’t deal with the issues, not only do they stick around but you will lose respect and your effectiveness as a leader.  Most people are not stupid and know more about what is going on than you realize.  The quicker you deal with the problem the quicker you can get back to normal.  If you have an issue with an employee that is just not fitting in, then the only way to ultimately get rid of them is by confronting the problem, conducting personal reviews to address the issue, writing the issue down in a counseling statement with goals on what they need to do in order to improve and conducting 30 day follow-ups.  I find that I will get one of two results: 1) the person improves and the problem goes away or 2) after 3 months you have enough documentation and probation time to out-process someone and it is easier to do because there are no surprises as the person had at least 90 days at this point to improve.  Normally you will have met all legal requirements with the time duration and documentation to let a problem person go.   Bottom line is if you deal with issues right away, you can normally avoid these kinds of long-term problems.
9. Surround yourself with other great people – greatness breeds greatness.  I learned this years ago when I was an officer in the Army; I was an S2 in an Infantry Battalion and LTC Clemens was the Battalion Commander (the best leader I’ve ever worked for in both my military and civilian careers) and we were having a Battalion Staff meeting and so all the Battalion leadership was present and he said “…what makes me so successful is surrounding myself with the best leaders”.  He said “…let me give you some advice, as you guys advance in your careers, in order  to be a great leader you need to surround yourself with the best staff you can where-ever you go.  Together you’ll be great.”
10. Know your people – most great leaders are known for connecting with their people.  If you don’t have a “real” relationship with your staff then they will not truly follow you, which will ultimately limit your abilities to be successful.  Get off your throne and get into the trenches with your people.  If you don’t know what day to day life is like, then how can you effectively lead them and make operational changes that will improve the process.  If you know your people and connect with your people, then they will follow you.  Likewise if a time comes where you need to stand up for your people – you better!  A true leader stands up and protects their people.  Believe me, your people will know whether you do this or not.  You can either gain a ton of credibility as a leader or lose a ton of credibility. Know your people!  Like with rewards, know what motivates a person, understand what their personal goals are, know the basics of what is going on in their life, and make time for them.  Don’t just say you have an “open door policy” but get out of your office or cube and go over to them and connect.  Know when their birthday is.  Every employee of mine fills out a basic Employee Profile that has contact info and stuff, along with their birthday.  I then put their birthday into my Outlook calendar so it reminds me of their birthday, when it is time I then get a birthday card for that person and leave it on their desk first thing in the morning.  Then we always do a cake in the afternoon.  If you have a large team, then do “monthly birthdays” so you can control expenses.  If they have a baby, make a big deal about it.  If their child accomplishes something big (like winning a game, or getting selected to represent the school’s math team, or getting accepted to college, etc) then take the time to know about it, follow up on it and take the time to talk about it.  Make them feel special.  I also try to stay connected with solid employees from the past – whether they left or you moved on.  Basic email is fine or connecting on LinkedIn.  Keep it professional but you never know when you may be able to hire them again, or they may have some connection you need in the future.  Recently I had two former employees that had their first child.  Each time I sent them a simple email saying congratulations.  That simple email goes a long way as they are usually surprised that you took the time to do this.  Most managers don’t – especially former managers.

Remember it takes work to be good.  Nothing comes easy in life!

For big projects I like to follow the process of time estimate refinement where you start with an “E0″ estimate which is +/- 100% (because this is based off of the initial business request (BRD) and lacks detail), then it is followed with an “E1″ which is typically +/- 50% because it should be based off detailed Functional Requirements and a Software Requirements Specification document (SRS). Once a Design Document and a technical approach discussed the estimate can be finalized with an “E2″ which is +/- 10% (some environments don’t bother with the E2 and require the E1 to be more specific because they will jump to the tech spec).

Some people talk about using “industry averages” in estimating because they don’t want to take the time to proper define the requirements, design and approach. They just want to move as quick as possible into the Coding phase; this is another discussion in itself, so back to the topic…I don’t like using “industry averages” as they tend to be too generic and quite often recommend less time than what is really needed. Applications today are more complex than ever and if a Test Engineer is to do a thorough in-depth test effort, it will take a lot of time to test. A lot of companies will make no effort to really figure out the true test effort and go with the industry average that QA will take 30-40% of the development time. However a developer could take 10 minutes to make a change in a critical part of the system (like the report engine of a financial app) and this would imply that QA time would only be 4 minutes which everyone would laugh at. Although they should laugh at the fact that the developer only spent 10 minutes developing before turning the project over to QA – what happened to the Unit Testing? So obviously industry averages are not very useful unless it is a big project and you are pressured to throw out an estimate for some meeting – then go with the 40% of development time. Otherwise if you have the ability to do your job correctly then put some meaningful effort into looking at past test efforts.

My opinion is that proper time estimation is best done by diligently tracking actual time for these elements in your current environment in a test management tool. I’ve always used the Quality Center and in that tool I have customized it to track various test execution times. The bulk of your “time estimation” consists of actual test execution time and if you have a good modular design for your application you can then draw on actual past experience to know how long a task like regression testing took in the past and the various functional testing tasks. Every application & environment is different so when managers change jobs they have to start this process over again to establish the duration times for the application at hand. Couple of pointers:
- Test Cases: if you use a test management tool, then you should create a modular structure that enables you to create and REUSE test cases. Therefore your test case development time estimate will change for each project. For enhancement and maintenance releases this time estimate should be minimal since you should be able to capitalize on existing test cases already created in the past and reuse them. For bigger projects that have lots of new functionality then you should refer back to similar projects in the past and see how much time was needed back then to create the variety of new test cases.
- Test Execution: In the Quality Center I have customized it to track overall project test effort times besides the individual test case execution times. This way I can once again refer back to actual past test efforts and see how long something took to test (especially when you have a project that modifies functionality that you have tested in the past). So during our process, when we write the Test Plan we include what testing will be done in each of the test phases. Once that has been established we then reference past test runs and see how long it took to test the same or similar functionality in the past and that becomes our basis for the current time estimates. This way our time estimates are not “feels like” times but timings that are based on real test efforts.
- Bug fixing: this effort is based on past experience with specific development teams. Every developer is different so you will need to apply past project experiences with specific developers to current efforts. Once again using a tool like the Quality Center with some customization in place, we can generate reports on specific project efforts and see how many defects were generated and come up with averages on what we expect with similar projects (based on functionality and developer). This helps with estimating the additional test cycles required during the bug fixing phase. By tracking defects generated by developer, you will have a good idea how “clean”/bug free the code will probably be based off past trends. This really helps you to estimate how many test cycle iterations you may have to execute.

Bottom line is that a test management tool that is customized to your environment can greatly help provide very useful time estimates; but it doesn’t do this on its own. The test team as a whole (testers, team leads and managers), have to all be diligent in documenting all the necessary details during all the various testing tasks. A little extra time and effort during the testing process greatly benefits the team down the road in planning future test efforts.

Now having more accurate time estimates will enable you to apply cost to the process much more realistically because you now know how long a project will take (“man-hours”) and you can then decide on the number of resources you want to apply to reduce the timeline (as long as they are available in the timeframe you are looking at).

So the basics of a proper software development environment are going to consist of a Dev Environment, QA Environment, Staging Environment and Production Environment, with a version control server supporting these environments that contains your code. The only exception to this is your Production environment – that environment may or may not have direct access to the version control server for security reasons. If it does not have access to the version control server then you will not be able to export your Trunk to the Production servers so you will need to have a build process that enables you to build install scripts that install the new version of code into Production (more on that later). Let’s look at an overview of each environment:

Dev Environment – this environment is owned by the software developers (programmers). This is an environment that enables them to work on new code and do anything necessary to the environment to support their development efforts. This can include re-configuration of the environment, dumping new data, changing file access and even crashing their environment. Normally this environment is configured “similar” to the Production environment, but scaled down significantly (like they may have 1 web server and 1 database server versus 5 web servers and 8 database servers like in the Production environment). Likewise the server(s) the programmers are developing on are actually a shared hardware environment with each developer having their own “area” to work in by virtualizing the server and providing each develop their own private slice to develop in (this is a great configuration as they can work independent of the other developers and not worry about bringing them down if they want to start and stop processes and likewise reduce server cost in reducing the number of servers needed to support the developers). This environment would be for developers only to work in and even though it may be virtualized, there still may be many development servers based on the size of the development team. The environment is considered dynamic and unstable due to the constant changes that occur in the process of developing code to support new functionality for an application. This environment needs access to the version control system to check in & out code.

QA Environment – this environment is owned by the software testers. This is an environment that enables the software testers to work in a “controlled” environment that is as close to a Production configuration as possible. Since a Production environment normally consists of numerous servers, databases and network gear it normally isn’t possible to duplicate Production due to cost reasons. Therefore the QA environment will be a scaled down instance of Production. So if the Production environment consists of 5 web servers that are load balanced and 8 databases servers that consist of replication and offline database instances used for reporting – you might want to create a minimal hardware environment without the load balancing or full replication in place. Therefore you might just have 1 web server and 1 database server that contains all the database and table instances with replication contained on that one server instead of across numerous servers. Although this reduces the similarities between QA and Production, it is a more cost effective approach when several QA environments are required. Normally the QA Department will identify that issue as a business risk and offset that risk with having a Staging environment that will be closer in configuration to the Production environment that QA can log into and perform Production load tests to address the performance of the application & hardware when under load testing and using load balancers.

Now the QA environment will normally consist of several duplicate environments. So if you have scaled down your Production environment to 1 web server and 1 database server for QA, then you will most likely want to have several of these environments so that you can be testing various projects concurrently that have different delivery dates without impacting the testing of another project. So if you look at this image we show a basic diagram of how you may have a QA1 and QA2 environment (there could be more). So why can’t I test multiple projects on one environment and reduce the hardware environments? You can, however you run into a coordination effort within the test team when it comes to shared processes and data. Maybe you have Project 1, 2, 3 and 4 that are going to be deployed to Production next week but Project 5 is a large project that will not be delivered to Production for another two weeks. For projects that make significant changes to some processes and/or database schemas, and that project is not being delivered to Production the same time as the other projects currently being tested, then you really need a separate environment to host that project. Otherwise you invalidate the testing of the other projects due to changes in the environment. Normally you want to have several QA environments and target projects to these servers based on functional changes and delivery dates. It is also very important that your database server be robust enough to hold a very large amount of data if not a full copy of the Production database. Numerous issues will always revolve around data and if you don’t have a robust database you may completely miss defects in your system. This environment needs access to the version control system to check out code only (changes to code should not be made in the QA environment, only in development – follow proper standards and processes). The developers should not have access to this environment to avoid any changes being made (security/code integrity issue). Access should be restricted to guest access for developers, but full access to the testers.

QA Tools – in regards to tools I specialize in and use the Quality Center, LoadRunner and QuickTest Pro so I will address environments based on these tools. This will give you a basis on configurations that you can apply to other tools:

Quality Center – This is a Test Management tool and is normally a single server meeting the specs specified by the installation guide. I normally have the Quality Center software and the Quality Center database (I typically use Microsoft SQL Server) installed on the same server. If you have a robust server you can avoid having to purchase multiple servers. This will reduce cost. The Quality Center then connects to all the various PCs of the users accessing the tool. Normally this server is behind the firewall.

LoadRunner – normally I like to have 3 servers for this setup – one main server hosting the full LoadRunner installation and where the Controller is run from. Then I like to have a minimum of two servers used by VUGen to off load the scenarios that are going to be executed. These servers take care of connecting to the farm of servers that are going to be load, stress and/or performance tested. Likewise it helps to keep the data processing being collected during the load tests, off the server that is running the Controller for more accurate monitoring of the load test. The server hosting the Controller is normally behind the firewall; however the Virtual User Generators may or may not be behind the firewall depending on the type of load testing being performed.

QuickTest Pro is installed on individual PCs so there is no special setup or server needed. If a Concurrent License Server is being used, I would just install that on the same server as the Quality Center to save money, because this “server software” is minimal in resources and a very small footprint on this server. Likewise QuickTest Pro needs to be able to connect to the Quality Center to save, open and run test scripts so having the Concurrent License Server software running on this server helps to eliminate any access issues with the Quality Center.

Staging Environment – this is an isolated environment that is as close to the Production environment as possible. This environment is used primarily for two purposes:
1.) User Acceptance Testing (UAT) where the business user(s) log in to verify the functionality before it goes live in Production.
2.) Performance Testing.
If the Production environment is too large to duplicate here due to cost reasons, then create a scaled down version that maintains the same type of hardware specs, database and network gear (like load balancers, firewalls, routers, switches etc) but yet still provides for a very similar configuration so that you can extrapolate your results correctly. So instead of having 6 web servers with 2 load balancers, you might have 2 web servers with 1 load balancer. At least this way you can do performance testing with a load balancer with the test scenario. Likewise the database should ideally be a Production copy. Normally you would want to use a copy of the backup snapshot here. As long as the hardware specs and hardware configurations are the same as Production (just scaled down in quantity) then you should be able to do a decent extrapolation of your load/stress/performance testing results to predict how Production will perform. If this environment is not the same hardware specs and setup, then it is impossible to do any kind of extrapolation of data, and you’ll only be able to use this as a UAT environment. This environment will have access to the version control system to check out the code base being deployed to Production and is intended to have restricted access to everyone using it – it should be just like it would be in Production, however there should be NO connection to Production. I ran into this once where one environment was doing Production database replication to the Staging environment that everyone forgot about. So when I ran a performance test with LoadRunner I directly impacted Production performance since the backup database was maintaining a live connection to the Production database. By slowing down the Staging server significantly with a performance test, the backup instance could not update from Production as expected and affected the end-user’s experience on the web application. Not good. This old design had been overlooked and we then looked at moving this database replication to another Production database.

Production Environment – this is your robust Production environment that can handle high volumes of load, maintain security and have limited server and database access by appropriate personnel only (SysAdmins, DBA’s, Release Manager, and potentially architects – every environment is different on Production access; the fewer people with access maintains the best security). This environment may not have access to the version control system. If it does then the Release Manager can deploy code directly to the server, but if it does not then a build tool needs to be used to bundle the code into an install script/build file/tar file/etc. This is a great approach if you can do it because you take security to a higher level and you avoid mistakes made by individuals executing their various tasks in the Production environment. In an environment where a build tool is not used, then you typically have at least 3 individuals in Production doing tasks for every release – a release engineer, SysAdmin and a DBA. With a build tool, all these tasks are scripted and it becomes an executable script that a SysAdmin will move to the Production environment and run. Simple and reduces the risk of error made by numerous people working directly on the Production servers.

Notice in the code block there are many variables used.  The concern about not declaring your variables up front is that when you access a variable later on in the code (like the variable “addrLength”), if you accidentally typed the variable name incorrectly you would end up with an incorrect result.  What adds to the problem is that it is difficult to figure out why the result is wrong because you might not catch the problem and waste a lot of time trying to figure out why.  So if I ended up typing this line within the code block instead:

I would end up with the wrong results all because of a typo.  The problem with implicit declarations is that VBScript says “hey this person wants to create a new variable” and just creates a new one called “addrLenght” and assigns the value of Null to it.  So now your count will be totally off for this formula.  But if you had declared your variables at the beginning of the code using the “Dim” statement, then you could avoid this issue.  As part of this approach you would want to also add the statement “Option Explicit” just before the declaration like this:

The Option Explicit requires that all variables be defined in the code prior to using them.  So although this approach doesn’t come out and say “hey you have a typo”, what it does say is that “addrLenght” was not declared and this should help you to identify that there is a typo because you would say “I did declare it, it’s right there in the Dim statement”.  When you then look at the variable name in the formula you would then notice the typo.  Without this, the formula just produces the wrong results with no error message or anything to help you out.

Defining variables up front is a good habit to get into because the majority of programming languages require you to define all variables at the beginning of your program before using them.  So in being consistent with most programming standards is a great practice to get into.

Note that in VBScript, Constants and Arrays must be declared before using them.  This topic only applies to scalar variables.

Variable Name Restrictions

• Must begin with an alphabetic character.
• Cannot contain an embedded period.
• Must not exceed 255 characters.
• Must be unique in the scope in which it is declared.

Lots of organizations think that “testing” is not difficult and that you could just grab someone off the street, sit them down in front of the computer and let them play with the application.  Some technology shops may do this (and I know some actually do) but if you want to do a professional job then there is significantly more work involved – therefore the reason why there are very experienced professional career testers.

The large the project, the more testing involved.  For small enhancement projects and Production Support fixes, the life cycles are shorter and the test efforts tend to not be as involved (however realize there is also some business risk in reducing the amount of testing performed).  What is listed below are the standard Test Phases that may or may not be performed.  For instance, some companies don’t do User Acceptance Testing and some don’t do Load Testing because they don’t have the software or hardware environments to conduct it in.  The only “test effort” not performed by the QA Test Team is “Unit Testing” which is the responsibility of the developers and should be performed before turning the software over to the test team.  Unfortunately this is either quickly skimmed over or completely ignored by the developers.

In an ideal test effort the following would occur:

Functional Testing: Functional testing exercises test cases that target each desirable and undesirable function of the software. The focus is normally on the new features/functionality added to the application.  Each new function of the software is targeted and tested in separation from other functions of the software.  This is normally performed manually as it can be challenging to develop automated scripts in a timely manner on brand new functionality that may or may not be buggy and still make the time-lines for the project (normally the new features are automated after the release and added to the regression test bed).
   1. Positive Testing – This involves running through test cases using valid data.
   2. Negative Testing – This involves running through test cases using invalid data; data intended to generate error messages.  Remember that when running negative test cases, when you get the error message displayed that you are looking for that this becomes “an expected result” not an error.
   3. Data Driven Testing – This involves running through the same functionality over and over again with data variations.  This is not a true load test, but a mini load test to make sure the application can support users doing the same thing over and over again – like submitting requests or ordering parts or doing searches.  You want to run through the functionality using different data (and normally this is all valid data) one iteration after another.  If your application is to support 1000 requests a day, then your application better be able to support that requirement.  Likewise the other benefit is to run through the functionality using various data combinations.  Simple test cases have a tendency to use similar data, so by running a data driven test you can see if the application has an issue with various formats or data combinations.  This is an extremely valuable type of testing, but very tedious and monotonous so using an automation tool for this is very important.
   4. Boundary Testing – This is the process of testing the “boundaries” of acceptable data – the min’s and max’s.  Once again in testing, lots of testers get caught in the process of using similar data and don’t change it up much.  So you may have a field that accepts two digit numbers in the range of 1-99 so you would want to not only test entering in “1″ and “99″ but do a few valid deviations in between (like “9″, “24″ and “75″) but also jump outside the boundary and test “0″, “01″, and “100″ and see if the application works as expected per the requirements.
   5. Single User Performance Testing – This test effort verifies that the user experience for an individual with no or minimal load on the system is acceptable per the requirements.  This is best done by running through many iterations to average the performance time and is best done using an automated test tool.
   6. SQL Penetration – This type of testing checks to see if a hacker can execute SQL commands in an edit field.  A trick hackers use is by entering SQL queries in an edit field to see if they can retrieve data from your database using a field that actually access the database in its normal design.  Make sure a hacker can’t penetrate your database this way.
   7. Security Testing – This is to verify that the web sites and applications are not vulnerable to security attacks and hacks.
   8. Exploratory Testing – is the random testing of the entire system done in creative and unusual ways to uncover defects found on the system’s edges – issues between modules, exploiting hidden dependencies, or unexpected state combinations.  This should be the last form of testing done as it is very unstructured and what is more like the idea of “playing with the application”.  It is useful but not structured.  Many times when a defect is uncovered the tester really has no idea what they did to create the defect since they are not tracking their actions so it becomes difficult to reproduce the defect because they were just “winging it”.

The next two test phases can either be executed separately or together.  For large scale projects they can easily be executed separately and typically are.  For small scale projects, feature enhancements or bugs this test phase is normally done together as one testing process -

Integration Testing:  Integration testing exercises several functional units that have been combined to form a module, subsystem, or system.  Integration testing focuses on the interfaces between units, to make sure the units work together.  The objective of this testing is to verify that the components of the system are working as defined per the approved requirements. 

System Testing:  In system testing, black box testing is performed to verify the system correctly functions per the specified requirements.  The test environment must be as close to production as possible, including factors such as hardware setup, database size and complexity.  By replicating the production environment, we can more accurately test ’softer’ system features (performance, security and fault-tolerance).  This is an end-to-end test.

Regression Testing: In this testing phase, the Test Team attempts to execute a mix of test cases involving different functions of the software to form a regression test.  This regression test needs to be executed each time a software code deployment is made to the QA testing environment.  Outside of an application redesign which is an overhaul of the application functionality and look & feel, the majority of development efforts are enhancement projects.  The norm is that about 75% of the application is not changing and only 25% or less is changing.  Therefore that means 75% of the application functionality is “regression testing”.  Most organization don’t have the time and resources to fully regression test a major application so the Test Team attempts to exercise as much of the existing application functionality to make sure the new features do not have any negative effect on the existing functionality that exists in production today.  What is identified for testing is normally based on business risk (risk analysis) and whatever automated test scripts have been already built to verify functionality.  Note that this is where you get the biggest “bang for the buck” when it comes to investing in test automation.  Regression testing is the best candidate for automation.  If properly approached and an automation test bed is created, then you can be running automated test scripts on one (or many) PCs to verify existing functionality (taking care of your regression testing) while you are conducting Functional testing of the new functionality that has just been built.

User Acceptance Testing (UAT):  The objective of User Acceptance Testing is to evaluate the product for its readiness in the marketplace and provide our customers with the confidence and assurance that the software is ready to be used.   The major focus of acceptance testing is on the user and proving that the system works.  This testing is performed by “Business” entities – typically the business owner of the project request or a manager/supervisor of the department that will be using the new features in Production.

Load/Stress/Performance Testing:  The Performance Engineer on the Test Team will put the application(s) under various load tests when requirements dictate.  If the requirements do not specify a load test is necessary, the Performance Engineer will attempt to perform a minimal load test as time permits.  Depending on technical constraints the testing may occur in the QA Environment or in the “UAT/Staging” environment.  See our other blog post on “Load vs Stress vs Performance Testing” for more information on the details of this test phase.

The first thing to note with any one of the load testing concepts is that we are not trying to find functional defects.  The expectation is that the code being tested is already stable; so at a minimum Functional and System/Integration testing should have already been performed to identify and eliminate the bugs that would prevent an end-to-end test from executing successfully.  With load testing we need to be able to execute end-to-end tests and if there are bugs breaking the process, then how can we simulate large volumes of users coming through the system using it?  What we are trying to identify with Load testing in general is “can the application and/or system handle high volumes of load without the system slowing down or crashing”. So with a “bug free” functioning application, we can begin executing the various types of load testing in order to “tune” the application in order to best handle volume with acceptable performance times.

Load Testing Concepts:

Load Testing – this type of test is where a heavy user load is simulated to determine at what point the software system’s response time degrades or fails. The purpose is to determine the capability limitation of the system.  This is typically performed with defined scenario groups executing a large number of virtual users with increased numbers added to the test over specified periods of time.

Stress Testing – this type of load test will immediately jump to large volumes of virtual users with the intent of seeing how the system handles a large stress on it.  This type of test is usually focused on finding out if servers crash, databases crash, applications freeze or performance just bottoms out.

Performance Testing – this type of load test is designed to see at what points of load on the system will create performance hits/impacts.  Normally the load test will begin with a minimal number of virtual users and then gradually increase the number of virtual users over a defined period of time.

Load Test Types:

Scalability Test – Perform repeated stress, overload, volume, and longevity tests with different server or network hardware configurations to determine the most cost-effective options for supporting the targeted load levels.
  1. Using “Vertical scaling” (up) adds additional RAM or CPUs within individual server boxes.
  2. Using “Horizontal scaling” (out) adds additional server boxes to a cluster of servers.

Longevity Test – Test that the system can sustain a consistent number of concurrent Vusers executing transactions using near-peak capacity, over a minimum 24-hour period (note we may not always be able to execute as long as 24 hours, but that is the ideal scenario). Longevity testing uses more disk space and other system resources and can be used to measure the build-up of obsolete logs, intermediate data structures, and statistical data that need to be periodically removed. Longer runs allow for the detection and measurement of the impact of occasional events (like log truncations) and anomalies that occur infrequently. These tests verify provisions for managing space, such as log truncation jobs that only usually occur in the middle of the night.

Volume Test – Test that the system can handle a massive data entry/load to the applications affected. During the test, the pattern of the application response time is measured while more data is added. Volume tests make sure there are sufficient system resources to handle the stream of data.

Overload Sustainability Test – Test how well the hardware allocated for the application can support the number of anticipated users. The goal is to quantify the degradation in response time and resource consumption at various levels of simultaneous users. This is done by gradually ramping up the number of Vusers until the system “chokes” at a breakpoint (when the number of connections flattens out, response time degrades or times out and errors appear). During tests, the resources used by each server are measured to make sure there is enough transient memory space and that the application correctly manages the system memory.

Stress Test – Test the maximum load that can be placed on the application to measure its upper performance boundaries and how it handles peak conditions. First it tests the load that the system was planned to withstand. Then additional load is added (more Vusers) that creates an overload condition on the system for a certain period of time. After that time, the load is reduced to the previous level and the operating system parameters are monitored. Response times are also measured at this stage.

I came across an interesting topic from uTest; it was a topic about what senior level QA Managers look for in hiring a software tester. I thought I would blog on this topic having numerous years of experience at the senior and executive levels (Director of QA, VP of QA, and President of ASI), having interviewed hundreds of candidates and probably hired close to 100 testers over the years. This is my perspective on what a solid tester should be and I totally agree with the statement in the uTest article that says “Everyone has a different set of criteria for hiring testers”. Reason being is because every hiring manager has a different background, skill set and work ethic so every manager looks at this position in different views and has different expectations.

I want to begin with setting some ground work by giving an overview of my background which influences my views. I started my software career as a software developer/programmer. After many years I was introduced to test automation and due to my technical background as a developer I got pulled into implementing test automation solutions for QA teams and my ultimate transition into the QA arena. Having been a Captain in the Army I also enjoyed leadership roles and found myself quickly climbing the ranks in management. With over 21 years of experience now I’ve done everything from programming, testing, automation, load testing, release management, project management, technical training, consulting and production support. This well rounded software development background enables me to understand what is truly expected of the various job responsibilities.

So now the Testing Position: Over the years I have seen the testing position called several different titles but I personally tend to group them into three basic true job categories. Below I will address the three categories – what I think their title is, my opinion of what the job entails, the skill set required and the interview process.

The Manual Tester - I see this position called “Tester”, “Manual Tester”, “QA Tester”, “Software Tester” and some deviations. What it is NOT is a “QA/QC” title – that bugs me; if you see this then the company probably doesn’t really understand software testing or the position is not software related. “QA” stands for “Quality Assurance” which is what we do in software testing. “QC” stands for “Quality Control” and this really relates to the manufacturing industry which is a much different type of “quality/testing” position. Back to the Manual Tester – This person’s primary job is to manually test an application. They may use some tools to aid their job (like text comparison tools, SQL editors to query a database, Putty to access a Linux server, WinSCP to FTP files across servers, Excel, etc) but normally this person does not do any true test automation work. One reason is that this not considered a “technical” position because there is no test automation programming or script programming required. Although I don’t expect a manual tester to be able to program, I do expect this person to understand basic SQL in order to validate data in the database and the ability to navigate around on a server in order to access files and make modifications for testing purposes. This person is also responsible for all the positive testing, negative testing, boundary testing and data driven testing throughout the test phases of Functional, Regression and System/Integration. This person is meticulous, attention to detail, willing and able to open defects to track bugs found in the application, ability to learn new applications with ease, and able to follow instructions step by step without any deviations. I have always used the tool called the “Quality Center” which is the top of the line test management tool on the market, so therefore I look for candidates that possess the skill and experience of working with this product.

The Automation Tester – For me this position is a “Test Engineer” title but lots of companies refer to it with a manual testing title but then imply that you will implement test automation, but I think that is an injustice to the person since they are significantly more technically competent than a manual tester and desire the more appropriate and descriptive title of “Test Engineer”. I implement QuickTest Pro (QTP) for test automation solutions and expect this person to know the tool very well. This person needs to be able to take a manual test case and convert it into an automated test case. For QTP I expect the person to know VBScript since this is the programming language used by the tool and I expect them to be willing and able to make script enhancements using code to create advanced automation scripts. The ability to utilize FSO and ADO in the test scripts is also a given since these are basic testing needs needed in order to validate various functionality. I expect every tester (including my automation testers) to have a solid understanding and experience with manual testing as this is really the foundation for all testers. A Test Engineer is simply automating a manual test case in most situations; therefore they have to understand manual testing in order to develop meaningful automation test scripts. This person may do both manual testing and automation testing, however it is nice if you can have at least a couple of Test Engineers that can focus 100% of their time to just developing automation scripts and NOT be involved in any manual test efforts for projects because that can easily consume all your tester’s time to where they never get a chance to build an automation baseline of scripts.

The Load Tester – For me this position is a “Performance Engineer” title. This person needs to understand the difference and how to implement Load Testing, Performance Testing and Stress Testing (3 different types of load testing approaches) with a tool (I use LoadRunner). Therefore I expect the candidate to understand how to use VUGen, the Controller, the Analysis Tool and the various basic and advanced features of LoadRunner. This person needs to be able to program in “C” since this is the programming language LoadRunner uses. This person should also understand manual testing; however their focus is going to be much different and more technical in nature. In addition this person typically needs to understand a lot of the “behind the scenes” stuff – like infrastructures, load balancers, protocols, basic HTML “gets” and “posts”, the concept of correlation and how to interpret graphs for recommendations on hardware configuration changes to support load. This person normally ONLY works on load testing and does not do any other types of testing work. This is easily a full-time job/effort.

Interview Process:  This is my interviewing process with all candidates; I have a four step process on interviews -
1. First there is typically a phone screen that is intended to only last for about 15 minutes. It is a quick screen to make sure this person is really a good fit in the first place and not wasting everyone’s time. This is done by either the HR Recruiter or the hiring manager. There are a variety of things that can be quickly identified over the phone and may disqualify a candidate which can save the hiring team a lot of time from not having to waste time on the rest of the process.
2. After passing the phone screen, the candidate then comes in and takes a written test which asks basic testing questions about the Software Development Life Cycle (SDLC), basic test concepts, and some simple “challenge” questions to include writing a SQL statement and writing an actual test case. Since I prefer candidates with “Quality Center” experience, I then have some basic Quality Center questions to see how well they really know the tool. In addition if they are applying for an automation role, then the exam has technical questions about the tools they will be implementing (I use QuickTest Pro for test automation and LoadRunner for load testing so the questions will be specific to those tools).
3. In-Person Interview: after the written test I’ll review their answers and then meet the person to ask them more questions about their career, actual job experiences/implementations and some technical questions based on what job they are applying for. This also enables you to determine their personality and if they are a good fit for your team. Team chemistry is important so make sure you are hiring people that will work well with the rest of your team.
4. Hands-On Test: Now in the interview, I will have a laptop so that the candidate can demonstrate that what they claim to know, they really do know. I have found over the years that many candidates have “book knowledge”, meaning they really don’t have any real world experience implementing the tools but have done lots of reading. So they are the “academic” type that have book knowledge from reading the manuals and stuff, but have never really done the work for real so when you make them actually use the tools in front of you they can’t actually do it. For a manual tester I will have them create a manual test case in the Quality Center and do a few basic things in QC. For someone who will implement QuickTest Pro I will have them automate a very basic test case; and for a Performance Engineer who will conduct load testing I will have them create a basic load test scenario and then kick off a load test in the Controller for a few virtual users. What this hands-on test does is weed out the people who really don’t know what they are doing from the people with real experience. This last step I’ve found to be incredibly valuable in the interviewing process. It really gives you a feel for what this person is capable of and their basic knowledge. There is no fooling an interviewer on a hands-on test.

The one thing you can never truly know in an interview is the person’s integrity and work ethic. I have hired individuals that have passed the interview process successfully (so they demonstrated that they had the technical knowledge and experience for the position) but once they were hired they were lazy. No matter what you did to try and motivate them (even with counseling statements) they still did not get it and change their ways. They did not understand the concept of working hard and accomplishing tasks in a meaningful timeframe. They had opportunities to get things accomplished and learn but had no focus and commitment. Unfortunately this type of character you cannot really determine in an interview and will only know once they work for you. Hopefully you don’t get any of these people. They are not common, but they do exist. I want hard workers that are committed to doing the best job they can and work hard to meet and exceed expectations. These are the people I want on my team and they are the people that get rewarded.

Good Luck!

A common problem and question that I see a lot from people is “How do I see my QuickTest Pro script in the Quality Center?”  This is normally a very simple issue to resolve, mainly because it was a process that was just not setup initially.  After the installation of the Quality Center and QuickTest Pro you still have an add-in to install and establish connection between the tools.  This description is based on the latest versions of QC & QTP which is currently 10.0 for both tools at the time of writing this article; however the process is the same for most of the older versions.

Common Issue #1:

After having installed both the Quality Center (server and client install) and QuickTest Pro locally, you will want to install the “QuickTest Professional Add-in” found in the section “Add-Ins Page > More HP Quality Center Add-ins > HP Tool Add-ins” found off the main Quality Center web page.   This page is the one found at the “qcbin” URL location (before getting to the login page); like “http://<server reference>/qcbin/”.   This is a local install to your desktop where you QC & QTP are already installed.  Normally you can accept all the defaults for the installation (it is a simple install).

Once the add-in is installed you will now be able to view QuickTest Pro test cases from within the Quality Center.  In the Test Plan the QTP test case will display the icon highlighted in the image below (without the icon there will be a question mark instead).  Likewise if you click on the “Test Script” tab for the test case the QTP script code will be displayed like in the image.

Common Issue #2:

From QTP, if you want to access the test scripts that have been saved directly in the Quality Center Project, then you must first establish a connection between the two tools.  In QTP select the following option from the menu bar “File > Quality Center Connection”.  Now fill in each section with the correct information to connect to your Quality Center Project (just like if you were logging in from the Quality Center login page).  Now you will be able to see the QC Test Plan tree from within QTP.

Common Issue #3:

Now when you want to be able to have the Quality Center kick off QTP scripts and execute the test scripts from QC, then you have to make sure you have the following setting turned on – in the QuickTest Pro tool you will want to invoke the Options Window by selecting “Tools > Options” and then click on the Run item on the left navigation bar.  The Run pane is displayed on the right side of the window and you need to make sure the option “Allow other HP products to run tests and components” is checked.  This is not checked by default so you need to make sure you set it so that the Quality Center can control QuickTest Pro (launch it, connect QTP to the QC Project, open the test case and run it).

This QuickTest Pro code sample shows how you can create a subroutine that will send an email using Microsoft Outlook.  The subroutine is defined and can either be placed at the top of your script (must be executed first before it is called in order for the script to know about the subroutine) or it can be placed in a Function Library and associated with the test script. 

The reason to use a subroutine is to isolate the process code that creates and sends the Outlook email to a common subroutine in order to take advantage of reuse.  This way every time a test script needs to send an email it just needs to have the bottom three lines of code shown below in the code image (lines 35 – 37).  It utilizes the “CreateObject” to create an instance of the Outlook application in order to create the actual email.  The data used in the email is passed into the subroutine using an array and then each element is referenced within the subroutine code.

Here is the result of sending the email, the email was received and it looks like this:

This is kind of a neat feature that you may find a need for.

In addition to the images above, here is a snippet of the code for easier readability and the ability to copy the code for your own use:

Good Luck.

Document Update – January 2010

January 2010 Update: Originally I wrote a document several years ago when QuickTest Professional was really hitting the market as a popular tool.  The document was written to help address the numerous questions received on what was a better solution for a company to go with – WinRunner versus QuickTest Pro.  I originally planned to update the document but then realized it was pointless; read on to understand why.

Background

WinRunner and QuickTest Professional were both created by Mercury Interactive.  QuickTest Pro was originally conceived in two unique and specific forms –

• AstraQuickTest: a functional testing tool for Web-based applications only.
• “QuickTest for SAP/R3”: a functional testing tool for SAP.

In both cases, the tool was very similar in UI design, features and method of implementation.  Due to the success and appeal it was determined to expand the capabilities to be more diversified instead of being so specific to an environment, Astra only worked for Web applications used through a browser and “QuickTest for SAP/R3” was only for SAP apps.  Why not be more like WinRunner and support the numerous other technologies?  So they did and developed QuickTest Professional.   Mercury Interactive then decided to eliminate WinRunner, because why should they have two tools that basically did the same thing.  I was part of the Mercury BizDev Team and I was the primary presenter that conducted the “Early Access Program” that introduced QuickTest Pro from a Marketing angle and Technical Training seminar to a select audience of Customers (50 companies were represented), select Business Partners and senior technical Mercury Engineers.  This 3 daylong seminar introduced the new QuickTest Pro application and Mercury’s strategy of replacing WinRunner (WR) within 1 year with QuickTest Pro (QTP).  Everyone loved QTP, its capabilities and implementation strategy – however there was some significant resistance from customers on the idea of eliminating WinRunner and its support after companies had already invested so much time and effort in its implementation.

Due to this resistance, Mercury decided to delay the “replacement” approach and keep the two tools on separate life-span tracks.  WinRunner would be kept alive due to the large quantity of existing customer test scripts created in the tool, and customers would have the choice of continuing to run and maintain them in WinRunner, or they could now call the WinRunner scripts from within a QuickTest Pro script.  At this point, for the most part WinRunner went into a “maintenance mode” where very few new features & improvements were made and all the significant development efforts were put into QuickTest Pro.

July 25, 2006 – HP formally announces acquisition of Mercury Interactive.

January 1, 2011 – HP has announced that January 1, 2011 will be the official “end of life” for WinRunner.  At that point WinRunner will no longer be supported and QuickTest Pro will be the only Functional Testing tool offered by HP (besides LoadRunner (load/stress/performance testing), SiteScope (server monitoring) and the Quality Center (test management)).

Comparison

At this point in the original document there was a series of sections that compared the differences between WR and QTP.  Today that section becomes irrelevant for the most part because QuickTest Pro has been improved to handle all the features that WinRunner has.  The table has now been turned to where QuickTest Pro can do everything WinRunner can do, plus more.  So as you can tell at this point in the document there is no need to conduct a comparison anymore because there is no reason to go with WinRunner.  So the comparison can be eliminated and the conclusion to go with QuickTest Pro based on two major points:

1. QuickTest Pro can actually do more than WinRunner today.
2. WinRunner has a defined “end of life” in one year as of the writing of this document – January 1, 2011.