Chapter 7: Automated Testing

(B) What are Good Candidates for Automation in Testing?
OR
(B) Does Automation Replace Manual Testing?

Automation is the integration of testing tools into the test environment in such a manner that the test execution, logging, and comparison of results are done with little human intervention. A testing tool is a software application which helps automate the testing process. But the testing tool is not the complete answer for automation. One of the huge mistakes done in testing automation is automating the wrong things during development. Many testers learn the hard way that everything cannot be automated. The best components to automate are repetitive tasks. So some companies first start with manual testing and then see which tests are the most repetitive ones and only those are then automated.

As a rule of thumb do not try to automate:

• Unstable software: If the software is still under development and undergoing many changes automation testing will not be that effective.

• Once in a blue moon test scripts: Do not automate test scripts which will be run once in a while.

• Code and document review: Do not try to automate code and document reviews; they will just cause trouble.

The following figure shows what should not be automated.

Figure 126: What should not be automated

All repetitive tasks which are frequently used should be automated. For instance, regression tests are prime candidates for automation because they’re typically executed many times. Smoke, load, and performance tests are other examples of repetitive tasks that are suitable for automation. White box testing can also be automated using various unit testing tools. Code coverage can also be a good candidate for automation. The following figure shows, in general, the type of tests which can be automated.

 

(I) Which Automation Tools Have You Worked with and Can You Explain Them Briefly?

Note

For this book we are using AutomatedQA as the tool for testing automation. So we will answer this question from the point of view of the AutomatedQA tool. You can install the AutomationQA tool and practice for yourself to see how it really works.

Figure 127: Candidates for automation

In this answer we will be testing a tool called “WindowsFileSearch.” This tool offers the following functionality:

• This tool basically searches files by name and internal content of the file.

• It also has a wildcard search as well as an extension search which means we can search the file by extension, for instance, *.doc, *.exe, etc.

Note

To answer this answer in detail we have used the FileSearch application. You can experiment with any other application installed on your system such as a messenger or office application.

Let’s go step by step to learn to use the AutomatedQA tool to automate our testing process. First, start the tool by clicking all programs à AutomatedQA à TestComplete 5. Once the tool is started you will get a screen as shown here. We first need to create a new project by using the New Project menu as shown in the following figure.

Figure 128: Create a new project

After clicking on the new project we will be prompted for what kind of testing we are looking at, i.e., load testing, general testing, etc. Currently, we will select only General-Purpose Test project. At this moment, you can also specify the project name, location, and the language for scripting (Select VBscript, currently).

Figure 129: Select the type of project

Once the project name and path are given you will then be prompted with a screen as shown here. These are project items which we need to be included in your project depending on the testing type. Because currently we are doing a Windows application test we need to select the project items as shown in the figure. Please note events have to be selected compulsorily.

Figure 130: Select project items

Once you have clicked finished you will get the Test Complete Project Explorer as shown here. The Test Complete Project Explorer is divided into three main parts: Events, Scripts, and TestedApps. Script is where all the programming logic is present. In TestedApps we add the applications that we want to test. So let’s first add the application in TestedApps.

Figure 131: Project explorer

In order to add the application EXE in TestedApps we need to right click on the TestedApps folder and click on New Item.

Figure 132: Add new applications to the project

You will then be prompted with a screen as shown here. Browse to your application EXE file and add it to the TestedApps folder.

Figure 133: Add the EXE to your TestedApps folder

Currently, we have added the WindowsFileSearch application. Now that your application is added we need to start recording our test. In order to start recording click on the button shown in the figure or push SHIFT + F1.

Figure 134: EXE has been added successfully

Once the recording toolbar is seen right click on the application added and run your test. In this scenario you can see the WindowsFileSearch application running. In this we have recorded a complete test in which we gave the folder name and keyword, and then tried to see if we were getting proper results. Your application being tested can be something different so your steps may vary.

Figure 135: Recording

Once the test is complete you can stop the recording using the button on the recording toolbar. Once you stop, the recording tool will generate script of all your actions done as shown in the figure. You can view the programming script as shown here.

Figure 136: Script generated for the recording

Once the script is recorded you can run the script by right clicking and running it. Once you run it the script tool will playback all the test steps which you recorded.

Figure 137: Running the recorded test

If everything goes right you can see the test log as shown here which signifies that your script has run successfully.

Figure 138: Successful execution of the scripts

(I) How Does Load Testing Work for Websites?

In order to understand this we need to first understand the concept of how websites work. Websites have software called a web server installed on the server. The user sends a request to the web server and receives a response. So, for instance, when you type http://www.questpond.com (that’s my official website) the web server senses it and sends you the home page as a response. This happens each time you click on a link, do a submit, etc. So if we want to do load testing you need to just multiply these requests and responses “N” times. This is what an automation tool does. It first captures the request and response and then just multiplies it by “N” times and sends it to the web server, which results in load simulation.

Figure 139: Concept of load testing

So once the tool captures the request and response, we just need to multiply the request and response with the virtual user. Virtual users are logical users which actually simulate the actual physical user by sending in the same request and response. If you want to do load testing with 10,000 users on an application it’s practically impossible. But by using the load testing tool you only need to create 1000 virtual users.

Figure 140: Load testing simulation by virtual user

(A) Can You Give An Example Showing Load Testing for Websites?

Note

As said previously we will be using the AutomatedQA tool for automation in this book. So let’s try to answer this question from the same perspective. You can get the tool from the CD provided with the book.

The first step is to open a new project using TestComplete.

Figure 141: Create a new project

After that select HTTP Load Testing from the project types.

Figure 142: Select HTTP load testing

Once you click “OK” you will get different project items which you need for the project. For load testing only select three, i.e., Events, HTTP Load Testing, and Script as shown here.

Figure 143: Select the three items in load testing

This project has the following items: Stations, Tasks, Tests, and Scripts. Stations basically define how many users the load testing will be performed for. Task has the request and response captured. Tests and Scripts have the Script which is generated when we record the automated test.

Figure 144: Load testing explorer

Figure 145: Project items

You need to specify the number of virtual users, tasks, and the browser type such as Internet Explorer, Opera, etc.

Figure 146: Assign the number of virtual users and the browser

As said previously the basic idea in load testing is the request and response which need to be recorded. That can be done by using the recording taskbar and clicking the icon shown.

Figure 147: Record HTTP task

Once you click on the icon you need to enter the task name for it.

Figure 148: Specify task name

In order to record the request and response the tool changes the proxy setting of the browser. So you can see from the screen here just click yes and let the next screen change the proxy settings.

Figure 149: Prompt to change proxy setting

Figure 150: Changing proxy settings

Once the setting is changed you can then start your browser and make some requests and responses. Once that is done click on the stop button to stop the recording.

Figure 151: Stop the task once done

The tool actually generates a script for the task recorded. You can see the script and the code generated in the following figure. To view the code you can double click on the Test2 script (here we have named it Test2 script).

Figure 152: Test2 created

If you double click the test you can see the code.

Figure 153: Code generated for test

Right click on the task and run it and you will see a summary report as shown in the figure.

Figure 154: Load test summary report

(I) Can You Explain Data-Driven Testing?

Normally an application has to be tested with multiple sets of data. For instance, a simple login screen, depending on the user type, will give different rights. For example, if the user is an admin he will have full rights, while a user will have limited rights and support if he only has read-only support rights. In this scenario the testing steps are the same but with different user ids and passwords. In data-driven testing, inputs to the system are read from data files such as Excel, CSV (comma separated values), ODBC, etc. So the values are read from these sources and then test steps are executed by automated testing.

Figure 153: Data-driven testing

(I) Can You Explain Table-Driven Testing?
OR
(I) How Can You Perform Data-Driven Testing Using Automated QA?

[This question is left to the user. Please install the tool and try for yourself.