VisionWorks Solution

Customer Experience Management and Mobility Infrastructure business units have integrated VisionWorks Analytics and Landslide products under the VisionWorks® solution suite. VisionWorks Test Analytics joins Landslide’s Test Agent with the VisionWorks Analytics Engine and GUI. The Landslide Active Test Agents run continuous tests by simulating users/devices that consume live network services. The results of these tests are fed into the VisionWorks Analytics Engine, where insights are presented through an interactive user interface. This provides insights for Operators to pro-actively detect, isolate and troubleshoot network services before customers are impacted.

VisionWorks is a licensed feature.

Several Use cases are currently supported : AMF Nodal, AMF Node, UE Node, MME Nodal, PGW Nodal, SGW Nodal, IP Application Node, Wifi Offload Gateway Nodal

This is not a complete list of Use Cases - (see VisionWorks Test Analytics for complete list )

Landslide Interface	Landslide Menu Setting	Data	VoLTE	Messaging	Visual Voice Mail	Latency	INTERCARRIER	NB-IOT	Video
All	Test Server Configuration - UE Info Level	10	10	10	10	10	10	10	10
	Info Report Tab-> UE Info Reporting	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs	Enabled- All Subs
	Customizing Report Options - VisionWorks Results	Enabled	Enabled	Enabled	Enabled	Enabled	Enabled	Enabled	Enabled
	Test Session-Intent of Test	Data	VoLTE	SMS-MMS	VoLTE	Network-Latency	INTERCARRIER	NB-IOT	OTT-Video

UE Node	LTE U1 Modem - (External Apps - HTTP Client, Video test, Speedtest, TraceRoute, Ping
UE Node	UE Handset - (Ue Node App - Mobile Info, DNS Test, Speedtest.net, ,HTTP, Latency/TraceRoute, Iperf, Voice, Message, Voice Mail, Video, WhatsApp	UE Node App	UE Node App	UE Node App	UE Node App	UE Node App		NA	UE Node App

MME Nodal	External App						External App (MMS Client)	MME Nodal Test Cnfg (CIoT), S1-MME Interface - NAS Tab
	VoLTE
	DMF based HTTP
	S1-MME	Added ARP_LEVEL field for (S1MME) in the TEST.PROCS.file - PER.UE.BEARER and PER.DEDID (When S1-AP Version > = 10.5.0)
	NB-IOT							MME Nodal Test Cnfg (CIoT), S1-MME Interface - NAS Tab New TEST.PROCS fields : T3324_TIME, T3412_TIME, T3412_EXT_TIME, EDRX_PAGE_BUFFER_TIME, EDRX_REF_TIME, UE_HASH_ID_TYPE)
AMF Nodal	Data Traffic, Capacity Test, External Apps

SGW Nodal	External Apps


PGW Nodal	External Apps


	Enterprise-ping
IP App Node	External Apps

WiFi Offload Gateway Nodal	External Apps		VoIP

To create the data in Landslide that is used to feed the VisionWorks® solution suite you must begin by:

Setup Info/Monitoring via the Info Tab.

Choose Intent of Test for Spirent VisionWorks Analytics to use to determine displays and select VisionWorks Results to enable Landslide Test Analytics Reporting. - Customizing Report Options.

Intent of Test : Choose the Service Assurance Test Type that Spirent VisionWorks Analytics should use to determine displays. The Intent of Test can be set specifically for individual test cases. See additional details in About the Test Case Settings Window.

Enable the following checkboxes to ensure Per Session Data is included in the IpInstance.csv file:

The VisionWorks Analytics Parser script is used to convert the data (in the data files) based on the data model.

NOTE: For VisionWorks link to the PCAP file, you must enable Associate Captures with Test Session (ACTS).

VisionWorks Test Analytics can be deployed in either Test or Production networks. Use cases can be licensed individually or bundled.

Display types - This is not a complete list (see VisionWorks Test Analytics for complete list)

Enterprise Service Assurance	Enterprise customers provide high value recurring revenue to Operators. It is critical that the services provided are proactively monitored to detect issues before they become impactful to the enterprise customers. For example, proactively detect that performance is trending lower, and is isolated to an enterprise problem. The operator notifies the enterprise before they know about it. The added value service retains and attracts high value customers. Landslide test requires configuration in 3 areas: Reports, Info Tab and Test Server Configuration To ensure test is configured to report results: Go to Reports, select Options and configure the settings listed here: Customizing Report Options Intent of Test : Choose Enterprise Test Type for Spirent VisionWorks Analytics to use to determine displays and select VisionWorks Results to enable Landslide Test Analytics Reporting. On the Subtotals Pane: L4-7 - select DMF, L3 select None and enable the Per Session checkboxes for each On the Per Session and Per Mobile Node Options pane: Select All Sessions On the TS Generated Per MN/Session Reports (s) Pane: Select Enable (includes end-of-test report and ability to generate during test) Select Include Per Session Data Select Include L3 Rates To ensure Emulated UE is set up to report results go the Test Session and Setup Info/Monitoring via the Info Tab. Select Configure and configure the settings listed here: Select Enabled Box Select Monitor Total UEs to be >= number of Subscribers in each test case (Max 1000 per Test Server) To ensure Test Equipment will track emulated UE messages go the Test Server Administration and Configure the Test Server UE Info Level to 10. UE Info Level should be set to 10 on Test Server Configuration.
Data Service Assurance	Mobile data traffic consumption is increasing at extreme rates with the availability of smart devices, larger screens and rich video content. The operators are motivated to provide high quality data services to retain and attract new customers. For example, quickly isolate performance issue to the OTT partner or operator network for prompt resolution. Intent of Test : Choose Data Test Type for Spirent VisionWorks Analytics to use to determine displays and select VisionWorks Results to enable Landslide Test Analytics Reporting - Customizing Report Options.
VoLTE Service Assurance	Operators are continuing aggressive plans to roll out VoLTE service, and are seeking proactive monitoring tools help simplify the detection and isolation of issues before they become impactful to customers. For example, proactively discover VoLTE performance by region based on fixed test criteria. This will enable Operators to focus and prioritize resources. A Landslide active test agent will generate a comprehensive list of VoLTE KPIs for delivery to our VisionWorks Test Analytics system to provide a detailed analysis of VoLTE results for Active Service assurance applications. Intent of Test : Choose VoLTE Test Type for Spirent VisionWorks Analytics to use to determine displays and select VisionWorks Results to enable Landslide Test Analytics Reporting - Customizing Report Options.
Test Case Analytics	VisionWorks Test Analytics provides generic pass/fail analytics outcomes for test session, test case, and test criteria. This is especially useful when special tests are run with no pre-defined network service use case. For example, discover when a specific DNS failures exceeds a limit, then and prioritize actions to remedy.

When VisionWorks Results reporting is enabled, all of the Pass Fail Criteria and UE Info Criteria in the test session are included in the VisionWorks Analytics TEST.CRITS result file and additionally there are Test Case records summarizing each test case and a Test Session record summarizing the test session, as per the Data Model. The user can assign a SUT to a given Test Case or Test Session by putting the SUT name in the Test Case or Test Session name after ##, e.g. MyTestName##MySutName. If the SUT name is found in the TAS by name, the TAS will also populate the SUT_TYPE and SUT_IP with the associated values found in the TAS DB associated with that SUT.

TAS will populate the SUT_TYPE and SUT_IP with the associated values found in the TAS DB associated with that SUT.

Ensure the following steps are taken in order for SUT_TYPE and SUT_IP to be populated:

SUT Name and TYPE in SUT Admin are defined as shown : In this example, the SUT Name = Plano , SUT TYPE = MME
Test case and / or Session must use the identical name to reference SUT :
Test Session Name Reference appears on the summary line of the TEST.CRITS file ( Row 2 only) :
Test Case Name Reference appears in the TEST.CRITS file for the individual criteria defined for the tests ( Rows 3 - n ) :
Analytics - Test Case Analytics case will display SUT as shown:

NOTEs:

To enable all the features, must have UE Info Level = 10 on Test Server Configuration and VisionWorks Results enabled on Report Options. For USER PLANE, must add the correct DMFs and enable L4-7 Per Session End of Test results.
May get NULL Values for items that are not reported by your test case. When setting a UE Info Level X (1 to 10), that means that the UE Info component will include fields A, B, and C. But that does not mean the test will always report B or C, thus the UE Info report will show a column for B or C and it would be null. for example, if using level 9 and you do not do a POLQA on Default Bearer, you will get POLQA columns, but they will always be null.
In the VisionWorks Collector ensure that you add the TAS IP address.