1

I have test automation framework with a page object model. All my test are located in separate classes in same package.

In testng.xml i have

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Smoke Test">
    <test name="SmokeTest">
        <classes>
            <class name="name.test1"/>
            <class name="name.test2"/>
            <class name="name.test3"/>
        </classes>
    </test>
</suite>

Problem is that after running TestNG.xml if the 1st test will fail, it will stop test execution. But i want to continue executing of all test cases.

I use Jenkins on my project and if one of the tests are failed it stops execution immediately.

Example of test 

public class LoginTestTest {
    public AndroidDriver<AndroidElement> driver;
    public AOWebClient aoWebClient;

    AOWebClient aoWeb;
    public LoginTestTest(AndroidDriver<AndroidElement> driver, AOWebClient aoWeb){
        this.driver = driver;
        this.aoWeb = aoWeb;
        PageFactory.initElements(new AppiumFieldDecorator(driver), this);
    }

    public LoginTestTest() {
    }

    SoftAssert softAssert = new SoftAssert();

    @BeforeClass
    public void setUp() throws Exception {

        driver = DesiredCapabilitiesSetup.startAppiumServer();
        aoWebClient = DesiredCapabilitiesSetup.getAOWeb();

        LogIn logIn = new LogIn(driver,aoWebClient);
        logIn.logIn();
    }

    @org.testng.annotations.Test
    public void goToSettings() throws InterruptedException {
        HeaderMenu header = new HeaderMenu(driver,aoWeb);
        HamburgerMenuList ham = new HamburgerMenuList(driver);

        header.clickHamburgerButton();
        header.clickHamburgerButton();

        header.editButtonClick();


        softAssert.assertAll();
    }   

    @AfterClass
    public void tearDown(ITestResult result) throws Exception {
            if (result.getStatus() == ITestResult.FAILURE) {
                TakeScreenshot screenshot = new TakeScreenshot();
                screenshot.TakeScreenshot("screenshots/");
            }
        LogOut logOut = new LogOut(driver,aoWeb);
        logOut.logOut();
    }
}

If my test will fail in @Test it will never continue to @AfterClass method. And I want that if @Test fail it will continue to @AfterClass method and After This Class continue executes test from other classes from testng.xml.

juherr
  • 5,640
  • 1
  • 21
  • 63
Anton Anton
  • 103
  • 2
  • 3
  • 8

3 Answers3

6

Your suite tag in your xml should include configfailurepolicy="continue". This tells testng that even though a config failed in one class, you still want to run other classes in that suite. See "configfailurepolicy" in the documentation.

So your xml would become:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Smoke Test" configfailurepolicy="continue">
    <test name="SmokeTest">
        <classes>
            <class name="name.test1"/>
            <class name="name.test2"/>
            <class name="name.test3"/>
        </classes>
    </test>
</suite>
Jerrod Anderson
  • 61
  • 1
  • 3
  • The official documentation says about -configfailurepolicy "Whether TestNG should continue to execute the remaining tests in the suite or skip them if an @Before* method fails. Default behavior is skip." (https://testng.org/doc/documentation-main.html). It seems the attribute depends only on the methods with @Before* annotations – AnnaL Jul 20 '21 at 15:28
2

From the documentation:

alwaysRun: For after methods (afterSuite, afterClass, ...): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.

Then, just replace your @AfterClass by @AfterClass(alwaysRun = true).

juherr
  • 5,640
  • 1
  • 21
  • 63
  • I found out that this method works when you run a package in project with some tests. But if you run Testng.xml It is still stops executing even having alwaysRun in tests. When first test fail, it`s king of freeze on execution. – Anton Anton Aug 11 '16 at 16:32
  • We also user Gradle, tasks.withType(Test) { scanForTestClasses = false include "**/*Test.class" } test { useTestNG { suites 'testng.xml' } } – Anton Anton Aug 11 '16 at 16:35
  • Another this which i found: When in testng.xml i point to run a package it Runs all Test Classes even if first one is failed. And after it show me only faild Tests. But if i point multiple classes instead of package in testng.xml, the first fail will stop execution. – Anton Anton Aug 11 '16 at 16:50
  • Sorry, but I don't understand what is the expected behavior and what is the observed one. – juherr Aug 11 '16 at 17:05
  • I thought that if i make a Test Suite (testng.xml) with a list of Classes one by one in Suite, it should continue run all of them one by one even of first one is fail. But the reality is , if 1 st one fail, it will stop executing the rest. And when instead of pointing to Test Classes in testng.xml but point to the whole Package , it will execute all package even if some of tests are fail. – Anton Anton Aug 11 '16 at 22:07
  • You are right. It is supposed to run all classes. Could you try the latest testng version ? – juherr Aug 12 '16 at 05:50
  • I just tested it with version `6.15.0-SNAPSHOT` and can confirm it still reproduced. – Itai Bar-Haim Apr 02 '18 at 07:12
  • @ItaiBar-Haim I opened an issue but we are not able to reproduce. Could you help us to fix it there? https://github.com/cbeust/testng/issues/1744 – juherr Apr 20 '18 at 07:51
  • I already found the correct way to do what I wanted using Listeners. I'm not sure this is not a bug, but having a way around it solved the problem for me. I'll try to reproduce it, though, and report, but it might take some time. – Itai Bar-Haim Apr 20 '18 at 17:10
0

Add below attribute in testng.xml file under Suite syntax

<suite thread-count="25" name="Smoke" parallel="classes" skipfailedinvocationcounts="true">

Work for me, hope it will work for you as well :)

n0noob
  • 889
  • 8
  • 23
Rahul
  • 1
  • 1