Quality Assurance Technologies: TestNG

TestNG:

- most widely used testing framework in software industry

- provides lot of features over JUnit framework

- used in different type of testing like unit, functional, integration and so on

Advantages of TestNG over JUnit 4:

- Extra before and after annotations such as Before/After Suite and Before/After group

- Dependency test

- Grouping of test methods

- Multithreaded execution

- In built reporting framework

In testNG, suites and tests are configured or described mainly through XML files.

By default, name of the file is testng.xml.

TestNG allows users to do test configuration through XML files and allows them to include or exclude respective packages,classes and methods in their test suite.

Downloading TestNG:

- You need Java JDK5 or above.

Install JDK from http://www.oracle.com/technetwork/java/javase/downloads/index.html

- TestNG can be downloaded, installed and run in multiple ways:

Using command line

Eclipse Plugin

IntelliJ IDEA plugin

ANT

Maven

- Test NG Jar can be extracted from http://testng.org/testng-6.8.zip

- Get eclipse from http://www.eclipse.org/downloads/

TestNG in to eclipse:

- Open your eclipse application

- Click on add button next to the Work with text box

- Name: TestNG site, URL: http://beust.com/eclipse, ok

- With OK, TestNG update site will get added to Eclipse.

- Available software window will show tools available to download under the TestNG site.

- Select TestNG and click on Next

- Accept license information and click on finish.

This will start the download and installation of the TestNG plugin onto Eclipse.

- Eclipse needs to be restarted. Click on yes.

- Once eclipse is restarted, verify the TestNG plugin installation by going to Window>Preferences.

You can see TestNG section under preferences window

This will help us in executing our TestNG tests or suite using Eclipse.

Go ahead and write our first TestNG test.

Writing first TestNG test.

- Java project create in Eclipse to add our TestNG test classes.

File>New>Other>JavaProject>Next> FirstProject> Finish

Project> Properties>JavaBuildPath> Libraries tab> Add Library

Select TestNG on Add library window shown in screenshot and click Next

Finish

Creating first TestNG class:

File> New>Other>Add wizard>TestNG class>Next

Click on browse button and select Java project where you need to add your class.

Enter package name and test class name then Finish

Write following code to your newly created test class:

package test;

import org.testng.annotations.Test;

public class FirstTest{

@Test

public void testMethod(){

System.out.println("First TestNG test");

}

Now to run tests:

- Select Java project in the Eclipse and go to Run> Run configurations

- Select TestNG in given options and select New to create new configuration

- TestNG provides multiple options for running your test cases:

Class: using this option you can provide class name along with package to run only said specific test class.

Method: using this you can run only specific method in test class.

Groups: in case you would like to run specific test methods belonging to particular TestNG group, you can enter those here for executing them.

Package: If you want to executed all test inside package, specify these in this box

Suite: in case you have suite files in form of testng.xml files, you can select those here for execution.

Understanding testng.xml

- is configuration file for testNG.

- used to define test suites and tests in TestNG.

- gives different options to include packages, classes, and independent test methods in our test suite.

- it also allows us to configure multiple tests in single test suite and run them in multithreaded environment

- TestNG allows us to create tests with packages

create tests with classes

create tests using test methods

include/exclude particular package, class or test method use of regular expression while using include/exclude feature
strong parameter values for passing to test methods at runtime

configure multithreaded execution options

Creating a test suite:

Let us create our first TestNG test suite using testng.xml.

- select project and then right click on it and select New>File. Select project in file window.

- In file name enter testng.xml then finish

- Eclipse will add new file to your project and will open file in the editor.

- Add following snippet to newly created testng.xml file and save it.

</classes>

</test>

</suite>

Running testng.xml

- Open command prompt on your system

- Goto Test Java project folder where the new testng.xml is created.

- Type following line:

java -cp "/opt/testng-6.8.jar:bin" org.testng.TestNG testng.xml

After execution HTML report is generated by TestNG in the folder named test-output under same directory

where you had run the command.

Running using eclipse:

- Open eclipse

- Goto project where we created testng.xml file

- Select testng.xml file, right click on it, and select Run as> TestNG Suite

Configuring eclipse to run particular TestNG xml file

- Run>Run configurations

- Select TestNG from set of configurations and click on the New Launch Configuration icon

- On configuration window give name MyTest to the configuration

- Go to project section, click on browse and select project on project window

Now go to Suite section and click on Browse

Select the mytestng.xml configuration.

Click on apply and then click on Run

This will run selected testng xml configuration file.

Lets learn how to create multiple test sections inside test ng XML file.

Lets create testng XML file with multiple tests and run it:

- open eclipse and create new project with name Multitest having firstTestClass.java and SecondTestClass.java

- Open firstTestClass.java and add following code snippet to it:

package test.firstpackage;

import org.testng.annotations.Test;

public class FirstTestClass {

@Test

public void firstTest(){

System.out.println("First test method");

}

- Open SecondTestClass.java file and add following code snippet to it:

package test.firstpackage;

import org.testng.annotations.Test;

public class SecondTestClass{

@Test

public void secondTest(){

System.out.println("Second test method");

}

- Now open testng.xml file and add:

</classes>

</test>

</classes>

</test>

</suite>

Now run the testng.xml file using eclipse.

We successfully created testng XML configuration file with multiple test section and then ran it using TestNG.

Adding classes, packages and methods to test Sample project:

- before going ahead with creation of test suite with classes, packages and test methods, we will need a sample project in place for defining test suites for test execution.

Lets create sample project first:

- open eclipse and create new project with three packages, each package containing two classes. Also add testNG library to the project as mentioned.

Creating a test with classes:

- add new file TestNG configuration XML by name class-testng.xml to project with following content:

</classes>

</test>

</suite>

Creating a test with packages:

- add new testNG configuration XML by name package-testng.xml to the project with following content:

</packages>

</test>

</suite>

- In case we want to execute sub packages under package test.

</packages>

Creating a test with methods:

</methods>

</class>

</classes>

</test>

</suite>

Creating a test with packages, classes and methods

- Add a new file TestNG configuration XML by name combine-testng.xml to project with following content:

</packages>

</methods>

</class>

</classes>

</test>

</suite>

Test suite to include a particular package:

- Create testng XML file with name include-package-testng.xml in the project. Add following code to it:

</package>

</packages>

</test>

</suite>

Test suite to exclude particular package:

Create a testng XML file with name exclude-package-testng.xml in the project.

</package>

</packages>

</test>

</suite>

Test suite to exclude particular method:

<class>

</methods>

</class>

</classes>

</test>

</suite>

Using regular expressions to include/exclude:

- Create RegularExpressionProject

- Package: test.regularexppackage

- class: regularExpClass.java

In class:

package test.regularexppackage

import org.testng.annotations.Test;

public class RegularExpClass{

@Test

public void includeTestFirst(){

Sysem.out.println("First include test method");

}

@Test

public void includeTestSecond(){

Sysem.out.println("Second include test method");

}

@Test

public void excludeTestFirst(){

Sysem.out.println("First exclude test method");

}

@Test

public void excludeTestSecond(){

Sysem.out.println("Second exclude test method");

}

@Test

public void includeMethod(){

Sysem.out.println("Include method");

}

@Test

public void excludeMethod(){

Sysem.out.println("Exclude method");

}

Preceding code contains multiple test methods with different names and we will use these methods to learn about usage of regular expressions in the include and exclude tags in TestNG

Using regular expressions for test:

<class>

</methods>

</class>

</classes>

</test>

</suite>

Preceding testng XML suite is configured to consider only those test methods from particular class

whose name contains word Test in it.

- Regular expression is considered by use of .* before and after the text.

Annotations:

- Is used to add metadata to Java source code.

- Can be applied for classes, methods, variables and parameters.

- Annotation:

@BeforeSuite or @AfterSuite: executed before and after any tests

declared inside the TestNG suite.

@BeforeTest or @AfterTest: executed before and aftger each test section

declared inside a TestNG suite.

@BeforeGroups or @AfterGroups:

associated with groups feature in TestNG.

@BeforeClass or @AfterClass:

Beforeclass executed before any test method of test class

Afterclass executed after execution of every test methods

of test class are executed.

@BeforeMethod and @AfterMethod: these methods are executed before/after execution

of each test method

@DataProvider: marks a method as data providing method for test method.

@Factory: Marks a annotated method as factory that returns an array

of class objects. These class objects will then be used as test classes by TestNG. This is used to run set of test cases with different values.

@Listeners: Applied on test class. Helps in tracking execution status and logging purpose.

@Parameters: used to pass parameters to test method. these parameter values are provided using testng.xml

config file at runtime

@Test: Marks a class or method as test method. If used at class level, all public methods of class will be considered as test method.

Before and after annotations:

- @BeforeSuite/@AfterSuite

- @BeforeTest/@AfterTest

- @BeforeGroups/@AfterGroups

- @BeforeClass/@AfterClass

- @BeforeMethod/@AfterMethod

Running Before and After annotations

import org.testng.annotations.AfterClass;

import org.testng.annotations.AfterGroups;

import org.testng.annotations.AfterMethod;

import org.testng.annotations.AfterSuite;

import org.testng.annotations.AfterTest;

import org.testng.annotations.BeforeClass;

import org.testng.annotations.BeforeGroups;

import org.testng.annotations.BeforeMethod;

import org.testng.annotations.BeforeSuite;

import org.testng.annotations.BeforeTest;

import org.testng.annotations.Test;

public class TestClass {

/**

*Before suite method which is executed before

*starting of any of test in the suite.

@BeforeSuite

public void beforeSuite(){

System.out.println("Before Suite Method");

}

@BeforeTest

public void beforeTest(){

System.out.println("Before Test Method");

}

@BeforeClass

public void beforeClass(){

System.out.println("Before Class Method");

}

@BeforeGroups(groups={"testOne"})

public void beforeGroupOne(){

System.out.println("Before Group Test One Method");

}

@BeforeGroups(groups={"testTwo"})

public void beforeGroupTwo(){

System.out.println("Before Group Test Two Method");

}

@BeforeMethod

public void beforeMethod(){

System.out.println("Before Method");

}

@AfterSuite

public void afterSuite(){

System.out.println("After suite method");

}

@AfterTest

public void afterTest(){

System.out.println("After test method");

}

@AfterClass

public void afterClass(){

System.out.println("After suite method");

}

@AfterGroups(groups={"testOne"})

public void afterGroupOne(){

System.out.println("After Group Test One method");

}

@AfterGroups(groups={"testTwo"})

public void afterGroupTwo(){

System.out.println("After Group Test two method");

}

@AfterMethod

public void afterMethod(){

System.out.println("After method");

}

@Test(groups={"testOne"})

public void testOneMethod(){

System.out.println("Test one method");

}

@Test(groups={"testTwo"})

public void testTwoMethod(){

System.out.println("Test two method");

}

Create a testng.xml file to the project and add the following code into it:

</methods>

</class>

</classes>

</test>

</methods>

</class>

</classes>

</test>

</suite>

We executed all kinds of before and after annotations and executed it using testng.xml.

Test annotation:

One of basic annotation of TestNG is Test annotation.

Attributes:

- alwaysrun

- dataProvider

- dataProviderClass

- dependsOnGroups

- dependsOnMethods

- description

- enabled

- expectedExceptions

- groups

- timeOut

Disabling a test:

- can be achieved in TestNG by setting the enable attribute of the Test annotation to false.

Disabling test method:

import org.testng.annotations.Test;

public class DisableTestClass{

@Test(enabled=true)

public void testMethodOne(){

System.out.println("Test Method one");

}

@Test(enabled=false)

public void testMethodTwo(){

System.our.println("Test Method two");

}

@Test

public void testMethodThree(){

System.out.println("Test method three.");

}

Exception test:

TestNG provides feature to test scenarios by allowing user to specify type of exceptions that are expected

to be thrown by test method during execution.

Writing a exception test:

- Create a new class with name ExceptionTest and add following code to it:

package.test.exception;

import java.io.IOException;

import org.testng.annotations.Test;

public class ExceptionTest{

@Test(expectedExceptions={IOException.class})

public void exceptionTestOne() throws Exception {

throw new IOException();

}

@Test(expectedExceptions={IOException.class,NullPointerException.class})

public void exceptionTestTwo() throws Exception{

throw new Exception();

}

Writing exception test verifying message

- create new class with name ExceptionMessageTest inside the Java project created in earlier section.

Add following code to it:

import java.io.IOException;

import org.testng.annotations.Test;

public class ExceptionMessageTest{

/**

* Verifies exception message based on exact error message thrown

@Test(expectedExceptions = {IOException.class},expectedExceptionMessageRegExp="Pass message test")

public void exceptionMsgTestOne() throws Exception{

throw new IOException("Pass Message test");

}

//This test verifies that the exception message contains text

"Message" in it.

@Test(expectedExceptions={IOException.class},expectedExceptionMessageRegExp=".*Message.*");

public void exceptionMsgTestTwo() throws Exception{

throw new IOException("Pass Message test");

}

The preceding class contains two test methods each throwing same exception but with different error messages.

Time test as suite level:

- Add a sample test class with name TimeSuite and add following code to it:

package test.timetest;

import org.testng.annotations.Test;

public class TimeSuite{

@Test

public void timeTestOne() throws InterruptedException{

Thread.sleep(1000);

System.out.println("Time test method one");

@Test

public void timeTestTwo() throws InterruptedException{

Thread.sleep(400);

System.out.println("Time test method two");

}

Add testng.xml file to the project and put the following code to it:

</classes>

</test>

</suite>

Time test at test method level:

Add a new test class to project created in earlier section under timetest package with name TimeMethod:

Add following code to it:

package test.timetest;

import org.testng.annotations.Test;

public class TimeMethod{

@Test(timeOut=500)

public void timeTestOne() throws InterruptedException{

Thread.sleep(1000);

System.out.println("Time test method one");

}

@Test

public void timeTestTwo() throws InterruptedException{

Thread.sleep(400);

System.out.println("Time test method two");

}

Parameterization of test

One of the important features of TestNG is parameterization.

This feature allows user to pass parameter values to test methods as arguments.

This is supported by using Parameters and DataProvider annotations.

Two ways to provide data values to the test methods:

Through testng XML configuration file

Through DataProviders.

Parameterization through testng.xml

Open eclipse and create simple Java project called ParameterizationProject

Add new Java class file with name ParameterTest and copy following code to it:

package test.parameter;

import org.testng.annotations.Parameters;

import org.testng.annotations.Test;

public class ParameterTest{

@Parameters({"suite-param"})

@Test

public void parameterTestOne(String param) {

System.out.println("Test one suite param is:" +param);

}

@Parameters({"test-two-param"})

@Test

public void parameterTestTwo(String param) {

System.out.println("Test two param is:" +param);

}

@Parameters({"suite-param","test-three-param"})

@Test

public void parameterTestThree(String param, String paramTwo) {

System.out.println("Test three suite param is:" +param);

System.out.println("Test three param is:" +paramTwo);

}

Providing optional values

Create new class file with same OptionalTest inside the Java project created earlier.

Copy following code to Java file and save it

package test.parameter;

import org.testng.annotations.Optional;

import org.testng.annotations.Parameters;

import org.testng.annotations.Test;

public class OptionalTest{

@Parameters({"optional-value"})

@Test

public void optionTest(@Optional("optional value") String value){

System.out.println("This is: "+value);

}

Create new testng xml file with name optional-testng.xml and copy following code in it:

</classes>

</test>

</classes>

</test>

</suite>

Groups:

Grouping tests:

- user can group multiple test methods into a named group.

- you can also execute particular set of test methods belonging to group or multiple groups.

creating test that belong to a group

Create a new class with name TestGroup under test.groups package and replace following code in it:

package test.groups.

import org.testng.annotations.Test;

public class TestGroup{

@Test(groups={"test-group"})

public void testMethodOne(){

System.out.println("Test method one belonging to group");

}

@Test

public void testMethodTwo(){

System.out.println("Test method two not belonging to group");

}

@Test(groups={"tes-group"})

public void testMethodThree(){

System.out.println("Test method three belonging to group");

}

Running a TestNG group through Eclipse

- Open eclipse and go to Run>Run configurations

- Select TestNG from list of available config and click on new config icon.

- In config window give config name, for ex, GroupTest

Go to Project section and click on browse button.

Select previously created project that is GroupsProject

- Go to Groups section and click on Browse button.

Select group which you like to execute from list, say test-group

- Click apply button and then click on Run.

Running TestNG group using testng XML

<run>

</run>

</groups>

</classes>

</test>

</suite>

Creating test having multiple groups

Including and excluding groups using testng xml

Using regular expression in testng XML

<run>

</run>

</groups>

</classes>

</test>

</suite>

Dependency test:

Creating a test that depends on another test

Create new class named SimpleDependencyTest under the test.depends.method package and replace

the following code in it:

package test.depends.method;

import org.testng.annotations.Test;

public class SimpleDependencyTest{

@Test(dependsOnMethods={"testTwo"})

public void testOne(){

System.out.println("Test method one");

}

@Test

public void testTwo(){

System.out.println("Test method two");

}

Creating test that depends on multiple tests

Creating test that depends on inherited tests

Creating test that depends on group

XML based dependency configuration

Simple group dependency

- Using simple dependency in XML

- defining multigroup dependency in XML

- using regular expressions for dependency

Quality Assurance Technologies

Thursday, March 6, 2014

TestNG

No comments:

Post a Comment

Blog Archive