Command Sequencer

The Test Case Command Sequencer allows you to create and specify the functions, parameters, and timers that indicate when the function should be triggered. With test case Command Sequencer window, you can define multiple commands, including a list of arguments. Select the Sequencer test activity on the configuration tab.

You may select pre-defined commands, loops counts, and delays to indicate the commands to be repeated. The pre-defined command sequence are executed on the Test Server.

Each test case has only one sequence of commands and the test server executes the commands one at a time in the order listed. The Sequencer executes the first command, when it completes, executes the next command and so on.

Sequencer and Execution of On Demand Commands

Each test case included has one thread of execution, that is, one set of onDemand Commands specified using the Sequencer Window. It includes Actions described in Manage Command Sequencer Steps, ODCs, Delays, Loops, and Start/Stop ALL Traffic, no ACTION_X or LOGs. They run in order, immediately following each other. Estimated Duration and Elapsed Time (in seconds) provide you with an estimate of how long an Action may take to execute and how long it actually took to execute. Additional details below.

Estimated Duration of an action (in seconds) is an estimate of how long the action will take to execute. It is an estimate and should only be relied upon for Delays and ODCs that have a “rate” argument. Do not rely on the Estimated Duration for things like Sync Points, Stop DMFs, Start DMFs, ODCs without a Rate. For On Demand Commands (ODCs) with Rates, the Estimated Duration is derived by:

The # of UEs / RATE UNITs math: Attach 100 UEs at 1 UE/S = 100s. UE / (UE/s) == s. Thus, for as long as the execution can keep upand the Test Server is not overloaded, we should expect it will take ~100s to get through this ODC. A Delay of 10s should take 10s (unless teh TS is overloaded and timers can’t keep up). Example of Command Sequencer tests listed below.

Some ODC commands such as SGW Nodal External Apps, detailed here Command Mode Test Activity , estimate duration time in different manner.

Elapsed Time in seconds is how long the action actually took to execute.

You may select the end of a subset of command (within the sequence of commands) to execute when a test case is stopped before the Sequencer completes on its own. This allows for graceful cleanup of SUTs when sequencer test cases are manually terminated prior to the end of a test. To set/indicate a the Graceful Stop sequence, select the first command of the end of the subset to be executed.

TIP: See Set Graceful Stop to set the graceful stop sequence.

NOTES: The following applies to the Sequencer OnDemand Command (ODC) execution/operation:

The Sequencer On Demand Command (ODC) execution does not allow user-specified ranges of MNs or DMFs. Each command will operate over the entire range of MNs and/or DMFs defined in the test case.
ODC execution just initiates an operation and moves on to the next sequencer step once all the MNs have been commanded. It does not wait for MNs to reach their expected state.
Stop All Traffic occurs very quickly, but Start All Traffic initiates DMFs at the Data Resume Rate defined on the Data Traffic Tab. If there is more than one DMF in a test case, they are started in the order of DMF defined. DMF 0 for all MNs, then DMF 1 for all MNs, etc, (# of DMFs defined * # of sessions at the defined Data Resume Rate).
You may Retrieve Test Logs to track your Sequencer execution.

TIP:

The Session Index arguments and the DMF Index arguments will be automatically updated by the Tcl API during validation, all you need to do is define arguments (placeholders) and set the STARTING index to 1 and ENDING index to last index (i.e. total number sessions).

Example: { OnDemandCommand { ControlBearer { Handover 1000.0 1 1 4 } } }

Where: The first two 1’s indicate the Session range, and these will be updated by the Tcl API to include the entire range of MNs, that is, 1 – total sessions.

Tcl API Command Syntax:

CommandSequence “\{ \{ ITEM \} \{ ITEM \} \{ ITEM \} …. \{ ITEM \}”

DELAY_ITEM = { Delay NUMBER_OF_SECONDS }

##See specific ODC formats described in the Test Case On Demand Commands.
ODC = { OnDemandCommand { ODC_NAME { ARGS… } }

START_TRAFFIC = { StartAllTraffic RATE_UNUSED }

STOP_TRAFFIC = { StopAllTraffic RATE_UNUSED }

WAIT == { Wait }

ControlAttero { op=Connect hostip=HOST_IP ip=ATTERO_IP }

ControlAttero { op=Start ip=ATTERO_IP profile=PROFILE }

ControlAttero { op=Stop ip=ATTERO_IP profile=PROFILE }

ControlAttero { op=Disconnect ip=ATTERO_IP }

LOOP_START = { LoopStart }

LOOP_END = { LoopEnd NUMBER_OF_LOOPS }

Manage Command Sequencer Steps

You may open an existing command sequence template, save the listed command sequence as a template, add, remove, or re-order the steps by using the sequence control buttons on the Command Sequencer window and the Add menu.

You may select any row and use the tool bar or right-click on any step to access the Sequence menu. Select multiple contiguous sequence of Commands and move, remove or copy/insert where desired (Add, Before, After the current selection, or choose a location within the sequence of steps). This allows you to move entire blocks of steps up or down, or delete entire blocks. Inserts would occur either before or after the entire block or at a chosen step.

Sequencer Control Buttons

Open

Open an existing/saved command sequence template. The Select Test Case Command Sequence window displays command sequence templates relevant to the Test Case type. Click the Test Case column to sort the displayed list.

NOTE: Opening an existing command sequence template replaces any command sequence you are in the process of adding.

Save

Allows you to save valid sequences as templates to a library. The Test Case column on the Save Dialog Command Sequence window identifies the test case to which the Command Sequences belong. When you save a TC Command Sequence Template the Save Dialog Command Sequence window displays all the templates in the library.

Add

Click Add to access the drop-down list or right-click to use the Sequence menu. Use to build a set of OnDemand Command Sequence and insert them at current position.

See Add Sequence of Commands for details.

Edit

To edit a step, select the step and click Edit or select Edit from the right-click menu. You may edit delay time, the OnDemand Command, and number of loops (in the End Loop step).

Delete

To Delete a step, select the step and click Delete or select Remove from the right-click menu.

Move Up

To move the sequence command one step higher in the list, select the step and click Move Up or select Move Up from the right-click menu.

Move Down

To move the sequence command one step lower in the list, select the step and click Move Down or select Move Down from the right-click menu.

NOTE: To move sequence step(s) one step higher in the list, select the step(s) and click Move Up or select Move Up from the right-click menu.

Set Graceful Stop

Allows you to set the selected command as the first of the sequence to gracefully stop the test case.

NOTE: You may define a subset of commands to execute when you stop the test before completion. This allows for graceful cleanup of SUTs when sequencer test cases are manually terminated prior to the end of a test.

Select the option from the Tool Bar or from the right-click menu option, Set Graceful Stop.

Select a command to be the start of a graceful stop sequence, so that if you click Stop or force a test case to stop, the sequencer does not end abruptly, but instead executes the commands of the sequence to clean up resources.

NOTE: The Command index/#'s are 0-based similar to the TS output Run Log.

Tcl Parameter: CommandSequenceStopIndex

(You set it to index of the first command in the stop sequence)

Range: 0 to Last Index, i.e., NumberOfCommands-1

The Set Graceful Stop command executes as follows when you stop the test case, depending on the test case state.

Test Sequencer step execution State	The Test Case State/Sub-State
If the test case is executing a command that is before the Graceful Stop Sequence	The test case completes the current command (RUNNING:INDEX#COUNT state), moves to the first command of the stop sequence, executes the remaining commands, and then transitions to STOPPING and STOPPED state.
If the test case is executing a command within the Graceful Stop Sequence	The test case continues (remains in RUNNING:INDEX#COUNT state) and completes executing the remainder of the sequence of commands, and then transitions to STOPPING and STOPPED state.
If the test case has completed executing the sequence of steps (OR no Set Graceful Stop command is set)	The test case transitions to STOPPING and STOPPED state.

See example illustrationexample illustration.
See Example 1: Stopping test case while sequencer is before stop sequenceStopping test case while sequencer is before stop sequence.

Stopping test case while sequencer is before stop sequence:

log: 07/03 00:09:07.786:TAS: Directly commanding test case: ts0::tc0::stop

log: 07/03 00:08:08.652:ts0(Coast22):p0(0):tc0: Graceful sequencer stop will begin after current command completes.

test_progress: 29:1 Min(s) 25 Sec(s) remaining (in step 5)

test_progress: 33:1 Min(s) 20 Sec(s) remaining (in step 5)

log: 07/03 00:08:17.284:ts0(Coast22):p0(0):tc0: Gracefully stopping sequencer, going to command #6

tc_state: TS[0] TC[0] -> RUNNING:6#2

log: 07/03 00:08:17.284:ts0(Coast22):p0(0):tc0: State=RUNNING:6#2

log: 07/03 00:08:17.284:ts0(Coast22):p0(0):tc0: Executing Command 6
See Example 2: Stopping test case while sequencer is within/after stop sequenceStopping test case while sequencer is within/after stop sequence.

Stopping test case while sequencer is within/after stop sequence:

log: 07/03 00:12:19.400:TAS: Directly commanding test case: ts0::tc0::stop

log: 07/03 00:11:20.265:ts0(Coast22):p0(0):tc0: Graceful sequencer stop will begin after current command completes.

log: 07/03 00:11:20.600:ts0(Coast22):p0(0):tc0: Detach done ...

log: 07/03 00:11:20.600:ts0(Coast22):p0(0):tc0: Getting the next command...

log: 07/03 00:11:20.600:ts0(Coast22):p0(0):tc0: Gracefully stopping sequencer, already within stopping sequence

tc_state: TS[0] TC[0] -> RUNNING:7#7

log: 07/03 00:11:20.600:ts0(Coast22):p0(0):tc0: State=RUNNING:7#7
See Example 3: Stopping the test case after sequencer is already completedStopping the test case after sequencer is already completed.

Stopping the test case after sequencer is already completed:

07/03 00:16:14.091:ts0(Coast22):p0(0):tc0: Getting the next command...

07/03 00:16:14.091:ts0(Coast22):p0(0):tc0: State=RUNNING:7#7

07/03 00:16:14.091:ts0(Coast22):p0(0):tc0: The end of Command Sequencer

07/03 00:17:26.292:TAS: Directly commanding test case: ts0::tc0::stop

07/03 00:16:27.156:ts0(Coast22):p0(0):tc0: State=STOPPING

07/03 00:16:27.156:ts0(Coast22):p0(0):tc0: Stop Activity Timer ...

07/03 00:16:27.156:ts0(Coast22):p0(0):tc0: Stopping Traffic...

07/03 00:16:27.156:ts0(Coast22):p0(0):tc0: Stop Session Control Timers

07/03 00:16:27.156:ts0(Coast22):p0(0):tc0: All Sessions Disconnected...

07/03 00:16:28.156:ts0(Coast22):p0(0):tc0: Disconnect complete.

07/03 00:16:28.156:ts0(Coast22):p0(0):tc0: Stop Complete...

07/03 00:16:28.156:ts0(Coast22):p0(0):tc0: State=STOPPED

Disable Graceful Stop

Clicking Disable Graceful Stop clears all the Set Graceful Stop sequence.

Supports Graceful Stop within a loop. If first stop command is within a loop, when you stop the test case, the loop does not occur.

NOTE: Any loops that start within the stop sequence will execute and loop as configured.

Auto-Stop

Select to automatically stop the test case when the sequence completes execution.

Chart

You can chart Events per Second from the steps defined in the Command Sequencer window (also from Test Session window by clicking Mobility Events, which appears only if you have defined sequence of commands in the Command Sequencer window).

Copy/Paste

Select one more rows in the sequence table, right-click and select Copy from the menu or press Control-C to copy the selected sequence items to the clipboard. Right-click and select Paste or Press Control-V to paste/insert within a sequence table of the same type of test case (for example, from one MME Nodal test case Sequence table to another MME Nodal test case Sequence table).

Table Height

Select the height of the table displayed Max, Tall, Medium, Short. Depending on the height you select and the number of rows of commands you enter, a scroll bar appears to help scroll through the table rows.

NOTE:

A command is complete/done when the Test Session has finished executing the action associated with the command, and NOT when the MNs are in the resultant state due to the action. A TS executing an On Demand Command (ODC) does not wait for each MN to reach the expected state before being done and ready to execute the next ODC.

For example, an Attach ODC is done when all of the attach requests have been sent out, and not when all of the MNs are attached.

You must include in (your) required Delay between commands to make sure they reach a certain State. In addition, if you require 15 second measurements to fully report, you need to make sure your commands are at least spaced 15 seconds apart.

When you are in the On Demand Command (ODC) Tab, using the F2 key, will provide you with API Parameters and arguments for both Tcl and REST APIs.

^ Back to Top

Add Sequence of Commands

OnDemand (ODC)

The Commands available in the OnDemand sequence has the same format as described in the Test Case On Demand Commands. See Sequencer and Execution of On Demand Commands section (above) for how the commands specified on the Sequencer Window works.

IMPORTANT:

StartAllTraffic, StopAllTraffic, StartDMF, and StopDMF options are available only when you enable Data Traffic on the Test Configuration tab and select at least a DMF on the Data Traffic tab. When you select multiple DMFs, you may specify when an individual DMF may be started or stopped.

The DMF configurations must be in a PAUSED state if you do not want data flow to start automatically, which it will, by default.

NOTES:

StartAllTraffic: Allows you to specify when you would like to Start All Traffic. Double-click or click Edit to specify the Traffic Resume Rate.
StopAllTraffic: Allows you to indicate that you wish to stop all traffic as soon as possible (you can resume the stopped traffic by using the StartAllTraffic command).
StartDMF: Allows you to start an individual DMF. Double-click or click Edit to select a DMF from the list and specify the Traffic Rate for the DMF.
StopDMF: Allows you to stop an individual DMF. Double-click or click Edit and select the DMF to be stopped. (you can resume the stopped DMF by using the StartDMF command).

Delay

Allows you to pause the command processing for the duration you specify. Double-click Delay and define the number seconds to wait/delay executing the next sequence of command. Click OK to commit the entry.

NOTE: The delay timer starts when the Delay command/action is executed, that is, when the previous command is completed.

TIP: In some cases, a delay may need to be added between steps to ensure contexts / bearers are in the appropriate state before continuing.

Wait

Allows you to pause the step processing until you decide to click the Continue button to execute the next step. See the Wait section below.

SyncPoint

Use a SyncPoint to keep multiple test cases running Sequencer steps in sync with each other.

All Test Cases that have SyncPoints defined with the same ID will wait at the shared SyncPoint until all test cases reach the SyncPoint.

NOTE: For example, multiple test cases may sync to the same SyncPoint ID. When one test case reaches a SyncPoint, it will wait until it is notified. When all the test cases share the same SyncPoint, they all reach that Sync-Point, the TAS sends a command to confirm the SyncPoint, and the test case sequencers will all resume at (nearly) the same time.

In addition, you may also define unique SyncPoint labels to track a test case similar to a User Log. That is, the SyncPoint names/labels are sent to the Run Log to help you track and ensure that the test case has executed Sequencer steps and reached the Sync-Point ( RUNNING:3#3:Sync_ID, e.g. RUNNING:3#3:Sync_Test1).

NOTE: When a test session is running, you cannot Pause a test case that is waiting at a SyncPoint. You can Pause it right before it reaches the SyncPoint. When you click Resume, the TC again wait at the SyncPoint.

Example SyncPoint Setup:

If you wish to group many test cases together within a test session to wait at a sync point such that you send just one Continue command to one Test Case to have them all start again, set it upas follows:

All of the test cases in the example above, stops at the SyncPoint Wait1, except TC0 will stop at the Wait. You can then send the ContinueSequencer to TC0 and all of the test cases will continue.

ControlAttero

Support for adding Attero Commands. Four operations are supported :Connect, Start, Stop, Disconnect.

Not available in Command mode, only Sequencer.

NOTE: Pre-configure your profile on Attero and ensure the Test Server can reach the Attero API interface.

LoopStart
LoopEnd

The LoopStart and LoopEnd allows you to enclose a sequence of test case commands in an iterative loop.

LoopStart: Select LoopStart from the Add Sequence drop-down list or the right-click menu.
LoopEnd: Select LoopEnd from the Add Sequence drop-down list or the right-click menu. Double-click # of Loops field and define the number of times the loop will be executed. Click OK to commit the entry.

NOTE: The following rules apply when using test loops:

Loop Start and Loop End steps must always be used in pairs.
Multiple loops may be used but nested loops are not allowed.

^ Back to Top

Wait

The Wait step allows you to pause the step processing until you decide to click the Continue button to execute the next step.

NOTE: If you have Wait as a step and when you run the test, the Sequencer reports a sub-state, and also a new panel with Pause, Resume or Continue buttons appears on the Session Builder tab.

The Continue button appears (on the Session Builder tab) only when you have selected a test case that is running the Wait Sequence.
The TS will not allow you to click Continue if the test case is in the Stopping... state and is in the sequence of Graceful Stop.
The TS will not allow you to Pause/Resume if the Sequencer is running the Wait step, that is in Waiting... state.

The Pause command does not immediately pause the step, it transitions to a Sequencer State of PAUSING, until the current command complete execution. Then reaches the PAUSED state, before executing the next command. You can resume a Sequencer when it is in either the PAUSING or PAUSED state.

When you Resume a sequence while the status is PAUSING, the next command step is executed. The following illustrates the steps executed in Pause/Resume command sequence: For example:

Pause a sequence, the sequencer state switches to PAUSING and continues to execute the current Command # (step) and increments the count #.

RUNNING:1#1

user paused

RUNNING:1#2:PAUSING (still executes 1 )

user resumed

RUNNING:1#3

State changes to PAUSED, the Command # (step)and count # increments

user paused

RUNNING:1#4:PAUSING (still executes 1)

RUNNING:2#5:PAUSED (paused waiting to run 2)

Resume a paused sequencer, it maintains the Command #, increments Command Count.

user resumed

RUNNING:2#6 (executing 2)

NOTE: The Command Counter increments twice when you Pause and Resume a test, 1 count for the Pause, and 1 count for the Resume. (see Tcl API exampleTcl API example).

Example Tcl API Wait/Continue, Pause/Resume:

% ls::perform Run -TestSession $test