TestResult lost if failure creating RetryAnalyzer #3064
Description
TestNG Version: 7.9.0
Expected behavior
If the ITestListener already executed other methods for the same ITestResult, for example: onTestStart(), then the same instance should be provided for methods executed after, like: onTestFailure(). Or at least, all information added to the result should be available in every ITestListener method.
Actual behavior
Test starts running. Pre run listener hooks are (onTestStart()) are run correctly. However, a failure in the IRetryAnalyzer creation is not handled and bubbles up to the logic handling exceptions for "non-started" tests. Where in this case the test already executed.
This causes the ITestListener to receive a new instance of the ITestResult loosing all stored information.
Test case sample
ITestListener
package some.test;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class EvidenceListener implements ITestListener {
private static final String ATTRIBUTE_KEY = "attributeKey";
@Override
public void onTestStart(ITestResult result) {
System.out.println("OnTestStart is executed!");
result.setAttribute(ATTRIBUTE_KEY, "attributeValue");
}
@Override
public void onTestFailure(ITestResult result) {
System.out.println("Attribute value is: " + result.getAttribute(ATTRIBUTE_KEY));
}
}
IRetryAnalyzer
package some.test;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class EvidenceRetryAnalyzer implements IRetryAnalyzer {
public EvidenceRetryAnalyzer() {
throw new RuntimeException("Failed on purpose");
}
@Override
public boolean retry(ITestResult result) {
return false;
}
}
Evidence Test.
package some.test;
import static org.testng.AssertJUnit.fail;
import org.testng.ITestContext;
import org.testng.ITestNGMethod;
import org.testng.Reporter;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Test
@Listeners(EvidenceListener.class)
public class EvidenceTestOne {
@BeforeSuite(alwaysRun = true)
public void suiteSetup() {
ITestContext context = Reporter.getCurrentTestResult().getTestContext();
for (ITestNGMethod method : context.getAllTestMethods()) {
method.setRetryAnalyzerClass(EvidenceRetryAnalyzer.class);
}
}
@Test
public void testOne() {
fail();
}
}
Running the evidence test you will see that the console prints out:
OnTestStart is executed!
Attribute value is: null
Where the attribute value retrieved should be the one stored in the onTestStart() method.
Contribution guidelines
Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.