1

We are using Serenity BDD and Appium for automation testing of our Android app. Most of our high level testcases look as follows:

        observedToast = toastMessageStep.getToastMessage();

    log.info("Observed toast message: " + observedToast);

    if(!observedToast.isEmpty())
    {
        if(observedToast.equalsIgnoreCase(expectedToast))
        {
            log.info("Verification - Result: " + observedToast.equalsIgnoreCase(expectedToast));
            testCase.observation = String.format("CP message = %s", observedToast);
            resultWriter.appendAcceptanceCriteriaResultToCSV(testCase.testProtocolID, testCase.testID, testCase.acceptance_criteria, testCase.observation, 
                    TP397_Constants.acceptanceCriteriaResult_PASS);
            resultWriter.appendTestCaseResultToCSV(testCase.testProtocolID, testCase.testID, TP397_Constants.testCaseResult_PASS);
        }
        else if(!observedToast.equalsIgnoreCase(expectedToast)) 
        {
            log.info("Verification - Result: " + observedToast.equalsIgnoreCase(expectedToast));
            testCase.observation = String.format("CP message = %s", observedToast);
            resultWriter.appendAcceptanceCriteriaResultToCSV(testCase.testProtocolID, testCase.testID, testCase.acceptance_criteria, testCase.observation, 
                    TP397_Constants.acceptanceCriteriaResult_FAIL);
            resultWriter.appendTestCaseResultToCSV(testCase.testProtocolID, testCase.testID, TP397_Constants.testCaseResult_FAIL);
        }
    }
    else
    {
        //Check if OR Session got created and report failure
        try
        {
            String observation = createOR.viewORSessionStimulationMode();

            if(observation.contentEquals("Tonic"))
            {
                testCase.observation = "OR Session was created";
                resultWriter.appendAcceptanceCriteriaResultToCSV(testCase.testProtocolID, testCase.testID, testCase.acceptance_criteria, testCase.observation, 
                        TP397_Constants.acceptanceCriteriaResult_FAIL);
                resultWriter.appendTestCaseResultToCSV(testCase.testProtocolID, testCase.testID, TP397_Constants.testCaseResult_FAIL);
            }
        }
        catch(NoSuchElementException exp)
        {
            testCase.observation = "Invalid OR Session was created";
            resultWriter.appendAcceptanceCriteriaResultToCSV(testCase.testProtocolID, testCase.testID, testCase.acceptance_criteria, testCase.observation, 
                    TP397_Constants.acceptanceCriteriaResult_FAIL);
            resultWriter.appendTestCaseResultToCSV(testCase.testProtocolID, testCase.testID, TP397_Constants.testCaseResult_FAIL);
        }
    }   
}
catch(Exception e)
{
    e.getMessage();
    testCase.observation = "Unable to perform the required actions";
    resultWriter.appendAcceptanceCriteriaResultToCSV(testCase.testProtocolID, testCase.testID, testCase.acceptance_criteria, testCase.observation, 
            TP397_Constants.acceptanceCriteriaResult_NOT_TESTED);
    resultWriter.appendTestCaseResultToCSV(testCase.testProtocolID, testCase.testID, TP397_Constants.testCaseResult_INCOMPLETE);
}

resultWriter.close();



}

The reason for writing in the way above, is that we not just want our tests to show fail in the reports, but also log the tests to a separate Results file that we submit to the FDA to show testing done on our device. The problem is the high level tests look too busy. Is there another solution that others are using to do the same? I want to mark a fail if any of the acceptance criteria fails.

Some of our tests are as small as just pressing the up arrow on the screen and noticing if the range shown went up by the required amount. Next test is to press the down button and do the same, etc. We do not want to write separate testcases for such small tests where the application opens up before each test and closes after each test, since that seems like a very silly approach to smaller tests. So we grouped the tests and want to write the results to results file about whether the test passed or failed but do not want to fail the entire test script for one failure. We just want to move on to the next testcases in the test script.

Lee Jensen
  • 2,151
  • 1
  • 10
  • 23
Suhas
  • 48
  • 4

2 Answers2

1

In the past I wanted to assert a bunch of elements were on a page and not fail out on the first ElementNotFoundException. So what I did was create an exception array.

// Pseudo-code
var exceptions = [];
if (!element1.displayed) { exceptions.add("element1,") };
...
expect(exceptions.join).toEqual("");

If element1 is not displayed the exceptions array (as a string) has some length and so it prints out the array of elements not found (by printing out I expected X but saw Y).

kirbycope
  • 4,090
  • 1
  • 13
  • 30
0

https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html

There listener classes for the framework you use like JUnit, cucumber etc. when you use junit runner you can use both junit or cucumber listener classes.

The best way is to use cucumber event listner class

https://www.javadoc.io/doc/io.cucumber/cucumber-core/4.7.2/cucumber/api/event/EventListener.html

This will send information about the step and scenarios . Use this information to write to your custom report

PDHide
  • 11,065
  • 2
  • 14
  • 42