Report Formats
Click here to see this page in full context

Landslide Performance Test System – Release 24.3 September 2024

Report Formats

When you capture a report while the test is running, you can specify the report content, value calculations, or file format depending on the report type. You can also customize the Auto-Saved report definitions for a test session in the same ways. This topic explains the different options and formats available with Test Reports and Per Session Reports. In addition, this topic also includes a description of the report buttons you can use to perform functions that assist with report analysis.

Test Reports

A Test Report is always saved on the TAS after a test session that has been successfully started and complete. By default, summary  measurements from the Summary view on the Reports tab, and the parameter settings in the test definition are exported to an Excel workbook (.xls) format and saved to the TAS in your test results directory. The file names are comprised of the test date and time, the test session name, and the test session's run ID. You can also capture a full or partial Test Report while the test is running or after it has completed.

In a Test Report, measurements are grouped on worksheet tabs just as they are grouped on the report tabs in the Test Session window. Worksheets are labeled by the report tab names: Favorites, Pass-fail History, Test Summary, for example. The measurements are arranged in columns with a row of measurements for each interval. The first row in the worksheet contains the measurement names as shown in the example below.

Parameter settings are grouped on worksheet tabs by test case parameter tab. These worksheets are labeled with "CFG-" followed by the parameter tab name: CFG-Network Devices contains the settings for all parameters on the Network Devices tab, as shown in the example. The first row of the worksheet identifies the test session, date and time, and TAS software version. The columns in these worksheets contain the test server and software version, the test case, the parameter name, and the parameter value. When a test session includes more than one test case, common configuration tabs contain a set of parameter settings for each test case.

A report tab can contain subtabs of both defined measurements and subtotal measurements. A report tab containing subtotals would have a first tab labeled "Totals" and separate subtabs for each subtotaled item, e.g.Session_0, Session_1, or DMF_0, DMF_1. Each subtotal tab contains the same set of measurements as the Totals Tab. A report tab containing subtabs of defined measurements would have custom subtab names and each subtab would have different sets of measurements in them. e.g. L5-7 Client has Basic and Advanced subtabs.  Furthermore, subtabs of defined measurements can themselves contain subtabs of subtotal measurements, L5-7 Client Basic could contain Totals, DMF_0, DMF_1, etc.. subtotal tabs.

In TCL API and other areas, defined subtabs are identified by "TAB|SUBTAB". Subtotal tabs are identified with a separate subtotal_tab or subtotal attribute.

NOTEs:
  • For example, to access a sub total tab, click the subtotal name reported by the TS, that is, DMF_0, Bearer_1, MN_3, Sess_2, etc. Also, sub tabs may report sub totals, e.g., the Basic data measurements for the second DMF in the TC:  L5-7|Basic, subtotal DMF_1.  When a sub total tab is reported, all the measurements display on a Totals sub tab in the GUI.
  • Use Report Options to turn on subtotal measurements, however, some test cases may report these values automatically.
  • DMF Subtotals are based solely on the Mainflows list plus additional DMFs from Volte/eMBMS, not the Instances on Assignments  tab. Starting with DMF_0 would be the first DMF in the Mainflows list. For example if you had 3 Mainflow DMFs plus running Gm Volte RTP, you could have DMF_0 thru DMF_2 from Mainflows, and DMF_3 would be the VoLTE RTP subtotal. If you add extra Instances of DMF Mainflows on the Assignment tab, those extra instances traffic are added together with all Instances of the same Mainflow.  The Subtotal is a Mainflow subtotal, not an Instances Subflow.

In the Excel and .csv reports, the  tab name represents the subtabs in the GUI, for example, L5-7|Basic-HTTP, where "|" denotes a defined subtab and "-" denotes a subtotal tab.

Test Reports are also available in a comma-separated value (.csv) file format. One csv file contains all of the measurement and configuration information included in the Excel format. The file is arranged with the worksheet tab name, followed on the next line by the column titles, and then the remaining worksheet rows are listed, each on a separate lined as shown in the example below. A blank line separates each tab section.  

Test Report (Excel) Analysis Buttons

The Excel reports generated by the TAS includes a tool bar with buttons to perform functions that assist with setting up charts/graphs and new worksheets with selected measurements for analysis.  

NOTES:

 

You can use these buttons to analyze and display data generated by the TAS.

NOTE: If you select any cells in the first three columns of the report, a message displays asking you to deselect any cells in the first three columns, as they cannot be charted.

Buttons

Functions
Create P-I Column Click to automatically create per interval column for every non-rate measurement column in the report.  As part of the function, all columns with zero values can be removed.
Hide All-0-Columns Click to hide all columns with only zero values.
Unhide All-0-Columns Click to display all columns with only zero values which were previous hidden.
Delete All-0-Columns Click to delete all columns with only zero values.
Create a Chart of Selected Columns

Select the columns you wish to chart and click the Create a Chart of Selected Columns button. You can choose to:

  • Create a new chart as a new worksheet or
  • Create a new chart embedded in an existing worksheet by selecting the worksheet.  
  • Add the columns to an existing embedded chart by selecting the worksheet and chart from the appropriate dropdown list
  • Add the columns to existing worksheet chart by selecting the chart.

Click the Create Chart button to create a chart from the selected columns in the appropriate worksheet.
Copy Selected Columns into a Worksheet Select columns and click the Copy Selected Columns into a Worksheet button to create and/or insert measurement into a worksheet. You may insert the select columns into:
  • A New worksheet, enter the new worksheet name, and also include Interval Column, Iteration Column, and Elapsed Time Column.
  • An existing worksheet by selecting the appropriate worksheet from the dropdown list
Click the Insert button to paste the selected columns in the appropriate worksheet.
Create Example Report 1 Click the Create Example Report 1 button to generate an example report. The example report creates two charts and a per-session collated table for L3 packet measurements. It requires the Test Summary, L3 Server, L3 Client, and L3 Client per session tabs in the Excel spread sheet.
NOTE:  While generating an example report, if a report does not contain L3 Client per Session measurements, the collated data is skipped and only the charts are generated.

 

TIP: Use the ExampleTemplate1 procedure contained  in the Example Visual Basic module as a template to create your own customized reports. See Report Functions - Core and Example Modules for a list of functions.

Test Report Options

You capture a Test Report according to the your specification on the Report Options, you can capture the report in an Excel format as displayed on the Reports tab — Summary or test case view with cumulative or per-interval values or a mix of both — and include all of the report intervals or only the intervals displayed. You can also select options to subtotal, save to TS Generated Report, and save to TAS Generated Report. The subtotalling affects both what you see in the GUI Reports and what you get in the TS Generated report.

Per Session Reports

Per Session Reports provide a snapshot of cumulative measurements at a lower granularity than that available in displayed reports. Rather than report intervals, measurements are recorded for every instance of an MN or network node emulator. Per Session Reports are always generated in a csv format, with a separate file for each applicable report tab in each test case. All transport, control, and bearer plane report tabs are included in Per Session Reports, as are node emulator report tabs.

One or more columns that contain unique identifiers for the emulator instance are added to Per Session Reports in addition to a sequential Instance column. Depending on the protocol, these identifiers may be IP addresses, user names, or any message attribute that uniquely identifies a mobile node. Network node emulators are identified by IP address, and when more than one of the same type of emulator is used in a test case, each one is reported as a separate Instance.

Per Session Reports are always saved to the TAS in your test results directory. The file names are comprised of the test session's run ID, the test case's log identifier, and the report name. For example, the name of the file shown below is RID-5_ts0_tc0_IpInstance.csv.

Data Traffic and Network Host Per Session Reports are split into multiple files:

In GGSN Nodal, HA Nodal, IP Application Node, MME Nodal, PGW Nodal, SGW Nodal, UMTS Nodal , WiFi Offload Gateway Nodal Test cases and DMFs for all test cases added indexing info, such as IMSI, Bearer IP Address and SRC/DST Port, to correlate the rows by UE.

Examples:

UE 1:1 Default_Bearer 1:1 DMF

Each UE has 1 default bearer, each default bearer has 1 DMF.

Sample:

2 UEs, each UE has 1 default bearer, and the default bearer is assigned 1 DMF.

 

1.       MN_Bearer_Context

2.       SrcBasic(L5-7)

3.       SrcBasic_socket(L4)

4.       IpInstance(L3)

UE 1:* Default_Bearer 1:1 DMF

Each UE has multiple default bearers, each default bearer has 1 DMF.

 

Sample:

2 UEs, each UE has 2 default bearers, and each default bearer is assigned 1 DMF.

 

1.       MN_Bearer_Context

UE-1 (123456789012345) : 2 Default Bearers (88.0.0.1, 88.0.0.3)

UE-2 (123456789012346) : 2 Default Bearers (88.0.0.2, 88.0.0.4)

2.       SrcBasic(L5-7)

3.       SrcBasic_socket(L4)

4.       IpInstance(L3)

UE 1:1 Default_Bearer 1:*DMF

Each UE has 1 default bearer, each default bearer is assigned multiple DMFs.

 

Sample:

2 UEs, each UE has 1 default bearer, and the default bearer is assigned 2 DMFs.

 

1.       MN_Bearer_Context

2.       SrcBasic(L5-7)

Each default bearer is assigned 2 DMFs

3.       SrcBasic_socket(L4)

4.       IpInstance(L3)

UE 1:* Default_Bearer 1:*DMF

Each UE has multiple default bearers, each default bearer is assigned multiple DMFs.

 

Sample:

2 UEs, each UE has 2 default bearers, and each default bearer is assigned 2 DMFs.

 

1.       MN_Bearer_Context

UE-1 (123456789012345) : 2 Default Bearers (88.0.0.1, 88.0.0.3)

UE-2 (123456789012346) : 2 Default Bearers (88.0.0.2, 88.0.0.4)

2.       SrcBasic(L5-7)

Each default bearer is assigned 2 DMFs

3.       SrcBasic_socket(L4)

4.       IpInstance(L3)

UE 1:* Default_Bearer 1:*DMF with * Network Hosts

Each UE has multiple default bearers, each default bearer is assigned multiple DMFs.

And the DMFs are assigned to different network hosts.

 

Sample:

2 UEs, each UE has 2 default bearers, and each default bearer is assigned 2 DMFs.

2 network hosts, the first 2 DMFs are assigned to node-0, and the last 2 DMFs are assigned to node-1.

In this case, the Dest_IpAddr in L3-7 reports is different and reflect the node assignment.

 

1.       MN_Bearer_Context

2.       SrcBasic(L5-7)

The Dst_IpAddr is different for different node assignment.

3.       SrcBasic_socket(L4)

4.       IpInstance(L3)

NOTES:

  • In an iterative test such as Session Loading, control protocol reports include cumulative measurements for all Session Loading runs but Data Traffic and Network Host measurements only include sessions that are active at the time the report is generated.

  • If the reports are generated after the Network Host has released the resources for an MN's data session, "invalid address" is displayed in both IP address columns. The resources for active sessions are released when the test is stopped, but resources can also be released due to inactivity or when the MN's control session has been deactivated.

Per Session Report Options

When you capture a Per Session Report, you can choose to include all MN sessions or a range of sessions.

Learn how to:

 

© 2024 Spirent Communications Inc. All Rights Reserved.