Media Qos Settings
Click here to see this page in full context

Landslide Performance Test System – Release 24.3 September 2024

Media Qos Settings

In VoLTE / VOIP, IMS Node, CDMA Voice, CDMA MSCe and Network Host testing, the Media Qos Settings (QoM) tab allows you to activate POLQA, VMAF, PEVQ and IVR capabilities.

 

Voice   

Enable POLQA - Used to Enable Measurement of Speech Quality in the VoLTE environment by means of POLQA.

 
Video  (deprecated, replaced by VMAF)

Enable PEVQ - Used to Enable Measurement of Video Quality in the VoLTE environment by means of PEVQ.

 
Video  

Enable VMAF - Used to Enable Measurement of Video Quality in the VoLTE environment by means of VMAF.
Voice Recognition  

Enable IVR - Used to Enable Measurement of Voice Analysis in the VoLTE environment by means of IVR.

 

Voice

POLQA

Enable (POLQA)

Enable (POLQA) - Quality of Speech. Can only be enabled if at least one of selected DMFs is of RTP Voice type. When Selected the POLQA subframe is enabled. Additional information in About VoLTE POLQA Testing.

Tcl Parameter : PolqaEn
POLQA

POLQA Operational Mode - Support two different operational modes:

 

  • Narrowband (MOS-LQOn) - (speech in range of 300..3400Hz)
  • Superwideband (MOS-LQOsw) - (speech in range of 50Hz..14000Hz)

 

There are four POLQA Specific Speech Files in the TDF Library :

·         Narrowband (NB) 6 second for AMR

·         Narrowband (NB) 8 seconds for AMR

·         Wideband(WB)  6 second for AMR-WB

·         Wideband (WB) 8 seconds for AMR-WB

 

In the narrowband mode the score (MOS-LQOn) is within the range of 1-4.5.  MOS = 4.5 for transparent narrowband signal.

 

In the Superwideband mode the score (MOS-LQOsw) is within the range of 1-4.75. MOS = 4.75 for transparent Superwideband signal.

Tcl Parameter : PolqaOpMode
UEs Performing POLQA Measurements

Number of UEs - Default = 1, Range from 1 to 1440 (or <= #UEs, whichever is smaller)

There is one-to-one or one-to-two relation between number of UE in VoLTE test and number of RTP sessions carrying voice traffic. Some of the RTP sessions should be designated to perform POLQA measurements: POLQA Sessions. Maximum number of POLQA sessions available in test depends on reference speech file (X(t) signal) size, hardware platform, and number of cores for POLQA threads. If number of POLQA sessions configured in a test exceeds max number the test must stop. The test should also report how many POLQA sessions for the given X(t) signal and number of cores are available.

 

Tcl Parameter : PolqaNumUes

 

Selection Mode -There are two options to select UEs for POLQA measurements

 

  • Continuous subset starting at First UE. For example, a test with 20,000 UEs and 100 UEs performing POLQA measurements, if First UE = 10,001 the range is 10,001 to 10,100. (Enter FIRST UE)

  • Random sampling of UEs in the Test. For example, a test with 20,000 UEs and 5 UEs performing POLQA measurements, the set of POLQA UEs could be {3, 127, 12321, 15386, 16391}

Tcl Parameter : PolqaSelectMode

Tcl Parameter : PolqaFirstUe
Enable Pilot Tone Preceding rtpvoice DMFs

Enable Pilot Tone Preceding rtpvoice DMFs for Wave Form to be added at head of the wave file defined in rtpvoice DMF. 

Pilot detection feature requires that audio codec supports per-audio-frame decoding.

Supported codecs:

  • OPUS
  • G729AB
  • G729B
  • G722
  • EVRC-B (this codec does support per-audio-frame decoding but the codec itself is not suitable to detect the tone pattern the PILOT consists of)

POLQA is FR (Full Reference) quality model. It compares the original (reference) wav file to the degraded wav one, and finally it produces MOS Score. Refered as Ref-Signal and Deg-Signal here. If, in fact, Ref-Signal  packetized consists of 400 RTP packet (8 seconds as usual) then on the receive side, 400 packets are also expected.  It starts RTP packet collection from the very first packet. As soon as 400 packets are collected and decoded to Deg-Signal the POLQA algorithm will be accomplished. RTP packet 401 is the first packet of the second stream, so packet #801 is the first one for third stream, and so on so forth until SIP Call is ending.

This mechanism works well in two-armed testing. In fact, UE-A generates RTP traffic and UE-B receives it. However, in the case of conference server (ConfServ) as SUT the ConfServ may start sending audio data (silence) as soon as a member has joined conference group. Suppose it is a group of 3 participants: A, B, and C. As soon as they have joined the meeting ConfServ begins sending silent audio to A, B, C.  If A starts to speak (it sends Ref-Signal) and B & C listen and collecting it as Deg-Signal + subsequently accomplishing MOS score it is unlikely that first RTP packet A sends is the first one B & C get. So, the POLQA score C & B got will be not correct. 

This checkbox addresses this issue, it provides a way to recognize the beginning of Ref-Signal from RTP packet flow. The RTP packets will apply the POLQA engine (collection, Jitter Buffer processing, decoding, and finally get a POLQA score). In fact, if the very first RTP packet belonging to Ref-Signal is 1001. Then, for the example above of 400 packets, the first stream is of 1001-1400 RTP packet range, the second one is of 14001-1800, and etc.

The idea is to synthesize a wave form from predetermined audio tones and of known size (240 milliseconds). Let's refer to it as Pilot. This Pilot precedes the very first appearance of Ref-Signal in the transmission. Thus, if the Pilot is 240 milliseconds and Reg-Signal is of 8 seconds (400 RTP packets) then the transmission sequence from UE A may look as follows:

UE A: <Pilot-240msec><Ref-Signal_Stream1-8sec><Ref-Signal_Stream2-8sec><Ref-Signal_Stream3-8sec><........><Ref-Signal_StreamN-8sec>

The receive side (suppose, it's B), should recognize the Pilot from the incoming RTP flow. 

UE-B: <silence-12sec><Pilot-240msec><Deg-Signal_Stream1-8sec><Deg-Signal_Stream2-8sec><Deg-Signal_Stream3-8sec><........><Deg-Signal_StreamN-8sec>

POLQA Engine activates upon recognition of the Pilot. 

Requirements : 

  1. Pilot must precede the very first RTP stream in series only. 
  2. Pilot consists of 2 tones: 1974 Hz and 956 Hz. of 160 milliseconds. The tone part is braced by silence of 40 milliseconds. 
  3. Its time length is 240 milliseconds (usually it carries in 12 RTP packets)
  4. Pilot pattern is not configurable (hardcoded) as of LS 24.2.

As Pilot length is known the beginning of the first packet of the Deg-Signal is also known.

If some of the packets comprising Pilot are lost, then two scenarios are possible:

1. The Pilot is still recognized: wrong calculation of the Deg-Signal's very first packet may happen. A few packets shift may lead to slight POLQA MOS deviation. In fact, if true MOS is 4.2 then it may lower to 4.1. 

2. The Pilot was not recognized: POLQA MOS scores calculation will not be accomplished. 

Tcl Parameter : PolqaPilotEn
Record WAV Files

Available in AMF and MME Nodal test cases. Select to Store RTP audio in WAV file format.

The record file is produced when Polqa Mos score falls below the configured MOS Score Threshold (only one record per test run).

NOTE:  Users need to be cautious when enabling this option. Use only when running adhoc tests, this option should not be enabled for Continous Monitoring tests as it could fill the hard drive.

Enter MOS Score Thereshold.

Range : 0.01 to 4.75 

Default : 4.75

Tcl Parameter : PolqaRecWavEn

Tcl Parameter : PolqaMosScoreThr

 

Video - VMAF

PEVQ is depracated, replaced by VMAF - About VoLTE VMAF Testing

Enable (PEVQ)

Depracated, replaced by VMAF - see below

Enable (PEVQ) - Perceptual Evaluation of Video Quality. Can only be enabled if at least one of selected DMFs is of RTP Video type. When Selected the PEVQ subframe is enabled. See additional information in About VoLTE PEVQ Testing.

Tcl Parameter : PevqEn
UEs Performing PEVQ Measurements

Number of UEs - Default = 1, Range from 1 to 600 (or <= #UEs, whichever is smaller), specifies number of UEs performing PEVQ.

There is one-to-one or one-to-two relation between number of UE in VoLTE test and number of RTP sessions carrying video traffic. Some of the RTP sessions should be designated to perform PEVQ measurements:

Number of PEVQ sessions are depend on video file format (QCIF, CIF,?VGA?).

Tcl Parameter : PevqNumUes

Selection Mode -There are two options to select UEs for PEVQ measurements

  • Continuous subset starting at First UE. For example, a test with 20,000 UEs and 100 UEs performing PEVQ measurements, if First UE = 10,001 the range is 10,001 to 10,100. (Enter FIRST UE)

  • Random sampling of UEs in the Test. For example, a test with 20,000 UEs and 5 UEs performing PEVQ measurements, the set of PEVQ UEs could be {3, 127, 12321, 15386, 16391}

Tcl Parameter : PevqSelectMode

Tcl Parameter : PevqFirstUe
Enable (VMAF)

Enable (VMAF) - Video Multi-method Assessment Function. Can only be enabled if at least one of selected DMFs is of RTP Video type. When Selected the VMAF subframe is enabled.  VMAF is FR (Full Reference) tool to calculate MOS and auxiliary metrics of Video data transmitting through WiFi or 4G/5G Mobile Networks. It calculates the metrics by applying Reference (Original) Video file (signal) and the Degraded Video signal to the VMAF algorithm. It means that the Reference file must be known to the side receiving video traffic. So, Two-armed testing model is expecting: that is either two Landslide (LS) emulating subscribers (if SIP proxy) or one subscriber + one endpoint (if LS emulates a VoLTE Nodal and IMS Node/Endpoint).

See additional information in About VoLTE VMAF Testing.

Tcl Parameter : VmafEn
UEs Performing VMAF Measurements

Number of UEs - Default = 1, Range from 1 to 600 (or <= #UEs, whichever is smaller), specifies number of UEs performing VMAF.

There is one-to-one or one-to-two relation between number of UE in VoLTE test and number of RTP sessions carrying video traffic. Some of the RTP sessions should be designated to perform VMAF measurements:

Tcl Parameter : VmafNumUes

Selection Mode -There are two options to select UEs for VMAF measurements

  • Continuous subset starting at First UE. For example, a test with 20,000 UEs and 100 UEs performing VMAF measurements, if First UE = 10,001 the range is 10,001 to 10,100. (Enter FIRST UE)
  • Random sampling of UEs in the Test. For example, a test with 20,000 UEs and 5 UEs performing VMAF measurements, the set of VMAF UEs could be {3, 127, 12321, 15386, 16391}

Additional Quality Metrics - SSIM  -Enable to perform SSIM Quality Metrics. 

  • SSIM option double VMAF execution time. 
  • As PSNR measurements increases time just by %2 it will be included in VMAF measurement by default.

 

Tcl Parameter : VmafSelectMode

Tcl Parameter : VmafFirstUe

Tcl Parameter : VmafMetricsSsimEn

 

Voice Recognition

 

Enable IVR

Enable IVR - Interactive Voice Recognition. Can only be enabled if at least one of selected DMFs is of RTP Voice type, Enable Supplementary and Enable GETS Analysis are both enabled on the VoLTE Tab. 

Note: The IVR speech-to-text can be very accurate during engineer testing with GETS sample prompt. However, with any speech-to-text software, it cannot be guaranteed with 100% accuracy.

Tcl Parameter : IvrEn
UEs Performing IVR Measurements

Number of UEs - Default = 1, Range from 1 to 600 (or <= #UEs (Subscribers), whichever is smaller), specifies number of UEs performing IVR.

There is one-to-one or one-to-two relation between number of UE in VoLTE test and number of RTP sessions carrying video traffic. Some of the RTP sessions should be designated to perform IVR measurements:

Tcl Parameter : IvrNumUes

Selection Mode -There are two options to select UEs for IVR measurements

  • Continuous subset starting at First UE. For example, a test with 20,000 UEs and 100 UEs performing IVR measurements, if First UE = 10,001 the range is 10,001 to 10,100. (Enter FIRST UE)

  • Random sampling of UEs in the Test. For example, a test with 20,000 UEs and 5 UEs performing IVR measurements, the set of PEVQ UEs could be {3, 127, 12321, 15386, 16391}

Tcl Parameter : IvrSelectMode

Tcl Parameter : IvrFirstUe

 

^ Back to Top

© 2024 Spirent Communications Inc. All Rights Reserved.