CSV Criterion Editor
The CSV Criterion Editor supports the ability to assign Pass/Fail criteria of the individual Test Case CSV results files.
The option “Include Instance Count” when enabled will include the number of instances that the condition occurred in the generated file vs only specifying the 1st instance of the condition. By default it is unselected for performance reasons (due to unknown size of the csv file that the condition(s) are being processed for). The CSV criteria are evaluated in the order of (1) the Test Case and (2) alphabetical order of the csv file (3) order in the table. There is limited validation of the user configured conditions – user is responsible for entering correct filename / column names.
You can view/add/edit a criterion on the View/Edit CSV Criterion Condition window by clicking the Add or Edit button from the Pass/Fail Tab . Limited to 300 CSV criteria per Test Case. A Criterion defines a condition to check. You can specify up to four conditions listed below (If Field ) and when to check the condition and what to indicate when the condition is true.
The "If Field" is the MAIN Condition that is checked and the three other fields are qualifiers. The qualifiers help to determine when the MAIN condition is checked.
The following are the main components of the View/Edit Criterion Condition window:
The information/parameters displayed on the View/Edit Criterion Condition depends on the options you select from the various dropdown lists.
Result Type
Each Criterion has a Result Type that determines which state should be indicated if the condition is true. The Result Types are PASS, FAIL and LOG, where the true states are PASSED, FAILED, and OCCURRED respectively and the false states are FAILED, PASSED and PENDING.
Similar to Info Criteria, all the statuses of each criteria of a Test Case are “ANDed” to determine the TC CSV status and subsequently all the TC CSV statuses are “ANDed” to determine the CSV overall status (which is used to overall Session State).
This Result Type Machine is:
- PASS Criteria are PENDING Evaluated if true => PASSED, if false => FAILED.
- CSV Status labels will show PENDING while test is running and will be updated after result files are created and processed when test is completed
- FAIL Criteria are PENDING Evaluated if true => FAILED, if false => PASSED.
- LOG Criteria are PENDING Evaluated if true => OCCURRED else PENDING.
- If any CSV criteria are configured, it will generate a file “…csv_criteria.csv”
Each criteria will have a row in the file that will contain the row/rows that matches the criteria or any possible issues with processing the criteria evaluation (missing source file / invalid column name).
Contents:
ts,tc,crit,file,condition,status,instances,additional info
0,0,1,23-11-16_16.59.29__RID-2__ts0_tc0_SrcEnhanced.csv,[ PASS if ((FIELD.DMF_Name == Amazon_http) AND (FIELD.trafCommand Invalid Messages == 0) AND (FIELD.trafCommand DNS Request Count > 0)) ],PASSED,1,rows(2)
If Field
The main condition is IF FIELD ( ==, !=, >, ≥, ≤, <, matches) VALUE, this correlates to a row in the info report file.
The "matches" operator checks for Regular Expression Match. Vs == which is an exact EQUAL.
Additional information about Regular Expression can be found on these websites.
The >, <, ≥, ≤ operators are mathematical in nature and will not work on String values thus you must use an Integer value for the comparison.
-
Measurement — FIELD to evaluate. Type in the CSV field name. . Must be a Value in the CSV File.
-
Operator — Select the Criterion Operator.
The operator can be one of the following: >, >=, ==, !=, <, or <=, matches
- Value — Enter the value that will be compared with the measurement.
Condition Preview
The Condition Preview displays the Criterion that is being generated. See example:
Tcl API and RESTful API:
REST API
Follows same pattern as existing ueInfoCriteria – in that you can only replace the whole TC configuration for csvCriteria and not individual criteria/conditions.
Expanded csvCriteria:
Swagger Schema:
TCL API
CsvCriterion is a child under the testcase and follows similar pattern established by UeInfoCriterion
#### Test Case Instance ts[0]::tc[0]
set tc_ [ls::retrieve testcase -libraryId -3 "IP Application Node"]
### [ PASS if ((FIELD.DMF_NAME == Amazon_http) AND (FIELD.trafCommand Invalid Messages == 0) AND (FIELD.trafCommand DNS Request Count > 0)) ]
set csvcrit_ [ls::create CsvCriterion -under $tc_ ]
ls::config $csvcrit_ -FileType SrcEnhanced.csv
ls::config $csvcrit_ -ResultType PASS
set cond_ [ls::create CsvCondition -under $csvcrit_ ]
ls::config $cond_ -Source FIELD -Field "DMF_NAME" -Operator EQ -Value "Amazon_http"
set cond_ [ls::create CsvCondition -under $csvcrit_ ]
ls::config $cond_ -Source FIELD -Field "trafCommand Invalid Messages" -Operator EQ -Value "0"
set cond_ [ls::create CsvCondition -under $csvcrit_ ]
ls::config $cond_ -Source FIELD -Field "trafCommand DNS Request Count" -Operator GT -Value "0"