About the Pass/Fail Tab

Use the Pass/Fail tab in the Test Session window to define a set of criteria used to determine if a test passes or fails. The overall result for a test is the logical combination of all defined criteria, where zero criteria must be failed for the test to pass. You can set up criteria to check for specific measurement values, compare two measurements, or check for a test case or the test session reaching a specific state or step. You can also set up multiple Criteria in a test.

Pass/Fail tab components:

Send SNMP Trap Failed Severity Category

Select when to Send SNMP Trap. Traps are only sent if the TAS is enabled for SNMP. Use Server Settings to enable system level traps and view the MIB.

NOTE: Only 1 trap will be sent at the end of the test to indicate the overall Pass or Fail status for the test session. No traps will be sent if there are no Pass/Fail criteria defined.

Options:

OFF/NONE - Do not send an SNMP trap.

ALL Statuses - Always send an SNMP trap when using Pass/Fail criteria, regardless of the final Status, e.g. PASSED, FAILED, or NA.

Failed Only - Only send a Trap if the Test completes FAILED,.

Passed or Failed - Only send a Trap if the Test completes PASSED or FAILED.

Failed or NA Only - Only send a Trap if the Test completes FAILED or NA Status.

Passed Only - Only send a Trap if the Test completes PASSED.

NA Only - Only send a Trap if the Test completes NA Status.

Passed or NA Only - Only send a Trap if the Test completes PASSED or NA Status.

Select the Failed Severity. Available for All Statuses or Failed Only traps. Options:

Info - Passed and NA statuses will report Info.

Minor - Major failures only.

Major - Major failures only.

Critical - Critical failures only.

Enter a Category. Used to categorize groups of tests so that when a Trap is received it can be handled properly by the NMS.

Example: Categories by device - MME, PGW, PGW or region - NE, SE, NW or SW

The Pass/Fail criteria are defined in the Criterion Editor and displayed in a grid that identifies:
# — This is the identifier/index of the criterion.
Condition — The condition used in the criterion. The pass/fail criterion defined by clicking the Add button.
Status — Indicates the State/Status of individual criterion. The possible values are N/A, PENDING, PASSED, FAILED, or OCCURRED.
- N/A — The value shown before a test is started.
- PENDING — The initial value after the test has started and the criterion has expired or been evaluated.
- PASSED — The criterion has passed, either by expiring (Result Type: FAIL) or evaluating true (Result Type: PASS).
- FAILED — The criterion has failed, either by expiring (Result Type: PASS) or evaluating true (Result Type: FAIL).
- OCCURRED — A result type LOG criterion has evaluated true.

NOTE: When you run the test, each criterion will first go to the Pending state, then either to Passed/Failed accordingly.

The overall pass/fail status of the test is shown in parenthesis next to the Overall State on the Test Session window, e.g., Overall State: RUNNING (PENDING). The overall status is calculated based on the sum of all the Criteria as follows:

- PENDING if all Criteria are PENDING
- FAILED if any Criteria are FAILED
- PASSED if at least one criterion is PASSED and none are FAILED

Failures — Indicates the number of times the criterion has failed. Multiple failures can occur when a criterion is reset.

QuickList Control Buttons — See QuickList for details.
Criterion Control Buttons:

Add — Click the Add button to add a Pass/Fail criterion. See Criterion Editor (Add/View/Edit) for details.
Delete — To delete a Pass/Fail criterion from the list, select the criterion and click the Delete button.
Edit — To edit a Pass/Fail criterion from the list, select the criterion and click the Edit button or press the Alt+E keys. See Criterion Editor (Add/View/Edit) for details.
Move Up — To move Criteria one row higher in the list, select one or more Criteria and click Move Up.
Move Down — To move Criteria one row lower in the list, select one or more Criteria and click Move Down.

The CSV Criterion Editor supports the ability to assign Pass/Fail criteria of the individual Test Case CSV results files. Additional details in CSV Criterion Editor.

NOTE: The Add Criterion, Delete Criterion, Edit Criterion, Add to QuickList, and Open QuickList buttons are also available on the Right-Click menu.

Learn more about: