Adding Automation Control

Use the Automation Control tab... to add/define the sequence of test events (steps) that your test will execute. You can define the order in which the test cases start and stop, or make those steps dependent on the state of another test case (the event's predecessor). You can also specify a number of seconds to wait before executing the step.

This topic includes the following sections:

Step Builder	Assists with building automation steps quickly.
Defining Test Steps	Describes how to define the different types of steps and when they can be used.
Compound Tests	Describes the completed sequence of automation steps that executes multiple test cases.
Test Loops	Describes how to run a set of test case commands in an iterative loop.
Running Tests Indefinitely	Describes using the Wait step and the Continue button to pause and continue running the test session.

Step Builder

The Step Builder tab provides a quick way to build certain patterns of automation control. You may add steps to a set of TCs by selecting a pattern of steps and the Step Builder will insert the automation steps. The Step Builder allows you to insert patterns of steps anywhere within in the current steps, which creates multiple levels of sets of patterns (Series, Pyramid, or Parallel).

To Build Automation Steps:

Select the test session, select the Automation Control tab... and click the Step Builder button. A blank Step Builder Button Opens window opens.

Select the test case instance to build automation steps and click the Add Instance button. This adds an instance of the test case.

NOTE: The Pattern dropdown list becomes available only after you select a test case and click the Add Instance button.

Currently, you may add a maximum of 300 ( 0-299) Automation Control steps.

With Automation Control definition, you do not have to add the Init, Start, Stop, and Cleanup steps as they are added automatically.

The TAS automatically performs Init and Starts all test cases in parallel and when all test cases are stopped, the TAS performs the Cleanup step.

Select the a Pattern from the dropdown list for the automation steps.

There are three patterns:

Series runs each TC separately (Init Start Stop Cleanup Init Start Stop Cleanup). See an exampleSee an exampleSeries Pattern.
.
Series-no-Init runs each TC separately without Init steps ( Start Stop Cleanup Start Stop Cleanup).
Pyramid starts all TCs and then stops all TCs in reverse order (Init Start Init Start Stop Cleanup Stop Cleanup). See an exampleSee an examplePyramid Pattern
Pyramid-no-Init starts all TCs and then stops all TCs in reverse order without Init steps ( Start Start Stop Cleanup Stop Cleanup).
Parallel initializes, starts, and runs test cases in parallel (Init, Init, Start Start Stop Stop Cleanup Cleanup). See an exampleSee an exampleParallel Pattern.
Parallel-no-Init initializes, starts, and runs test cases in parallel without Init steps (Start Start Stop Stop Cleanup Cleanup).

NOTE: The Init step is not usually necessary in automation control as it is automatically performed with the Start step. You can choose to omit the Init step by using the xxx-no-Init patterns listed above.

Click OK and a pattern of automation steps you selected is added on the Automation Control tab... You may change the sequence and the type of function of the steps. See also Compound Tests for more about the sequence of automation steps.

^ Back to Top

Define different types of steps

The instructions in the sections below explain in detail how to define the different types of steps and when they can be used:

Learn the basics
Add a command step
Define a predecessor (optional)
Set a delay time (optional)
Add a test loop (optional)
Add a Wait step (optional)
Manage steps — Describes how to add, remove, or re-order the command steps you have defined.

NOTES:

Automation Control is not required to simply execute multiple test cases in parallel. Test cases are started in the order in which they are displayed in the on the Session Builder tab.
Automation Control should always be used when a node test case will be responding to nodal test cases. As a general rule, the node test case should begin before the nodal test cases and stop after the nodal test cases have stopped.
A maximum of 300 steps can be defined in one test session.

Learn the basics:

Before you experiment with Automation Control, you should have a basic understanding of:

^ Back to Top

To add a test case command:

Click the Insert button and a new step is added to the end of the list.

TIP: If at least one step has already been defined, you can use the Step menu to add steps.

Use the Function or Test Case drop-down list to select one of the test session's test cases.

Select the step's command from the Test Case Command drop-down list. The command directs the test case to work towards achieving the next state and will be executed as soon as the step is reached unless a Predecessor Test Case or Delay is defined (see Monitoring the Test Progress for definitions of the different states). The following commands are available:

Init
Start
Stop
Cleanup

IMPORTANT: If you are defining a test that executes the same test case more than once, you must include a Cleanup step after the test case is stopped for the first time, and an Init step before issuing the Start for the second execution. The Cleanup and Init steps release and reserve the IP addresses and other identifiers used by the test case, and must be used for every subsequent execution of the test case. The Cleanup command requires that you also set the Stopped state of the same test case as a predecessor in that step or a previous step to ensure that a cleanup is not performed while the test case is still running.

^ Back to Top

To define a predecessor:

You can use a predecessor to create dependencies in the test execution and to control the flow of your test in many ways:

You can use dependencies to serially execute test cases by defining the first test case's Stopped state as the second test case's Start predecessor, and so on. If the test cases use common IP addresses, a Cleanup is required for the first test case and it must be completed before the next test case starts.

You can re-run a test case if you first Stop the test case, Cleanup the resources, Init the next iteration to reserve the resources, and Start the next iteration. In this case either the Cleanup step or a previous step must use ensure that the test case has Stopped to prevent triggering the cleanup while the test case is still running.

Define a dependency for the step, and cause it to wait until another test case reaches the running or stopped state.

Select a test case from the Predecessor Test Case drop-down list, and the state that the selected test case must reach before this step executes is displayed in the Predecessor State column. The only valid options are Running and Stopped and you can change the selection if necessary. If the list is empty, a predecessor cannot be used for this step.

^Back to Top

To define a delay time:

You can define the number of seconds to wait before executing the step. If you have defined a predecessor, the test waits the specified number of seconds after the predecessor has reached the specified state before executing the step. If no predecessor is defined, the delay timer begins when the step is reached.

Double-click the Delay or # of Loops field, and enter the number of seconds. Press Enter to commit the entry.

^ Back to Top

To add a test loop:

With a test loop, you can repeat a set of test case commands for a number of times. During the test, the Test Log displays a message, including the loop number, when loop execution begins.

Insert a step above the first command step that will be included in the loop.

Select Loop Start from the Function or Test Case drop-down list.

Double-click the Delay or # of Loops field and define the number of times the loop will be executed. Press Enter to commit the entry.

IMPORTANT: The Loop Count is the number of times the control loops back to Start Loop. That is, 0 indicates no loop (test executes once), Loop count 1 indicates loop-back once (that is, test executes once plus the number of loops). Max number of loops supported is 2,000,000,000 (Infinite loop)

Insert a step below the last command step in the loop.

Select Loop End from the Function or Test Case drop-down list.

The following rules apply when using test loops:

Loop Start and Loop End steps must always be used in pairs.
Multiple loops can be used in a test session, but nested loops are not allowed.
A test case must always enter the loop in the same state. If a test case is started before the loop starts, it must be started or running when it exits the loop. If a test case will be started and stopped within the loop, it must enter the loop un-initialized (either not started before the loop or stopped followed by a Cleanup step before the loop) and use an Init/Start/Stop/Cleanup progression within the loop.

^ Back to Top

To add a Wait step:

The Wait step allows the test session to run indefinitely in its current state until you direct it to move on to the next step.

Insert a step and select Wait from the Function or Test Case drop-down list.

When the test session reaches the step, a Continue button is displayed in the bottom portion of the Test Session window and the progress bar displays a "Waiting on User in Step x" message. When you click the button, the test moves on to the next step.

^ Back to Top

To manage the steps:

When more than one step has been defined, you can add, remove, or re-order the steps by using the step control buttons on the Automation Control tab or the Step menu.

You may select multiple contiguous automation steps and move, delete or copy/insert where desired. This allows you to move entire blocks of steps up or down, or delete entire blocks. Inserts would occur either before or after the entire block.

To access the Step menu, right-click on any step.

Add a step	Click Insert or use the Step menu and select Insert Step to add a new step Above or Below the step, or at the end (Last) of the step list.
Remove a step	Select the step and click Delete or select Delete Step from the Step menu.
Move Up the List	Select the step and click Move Up or select Move Up from the Step menu.
Move Down the List	Select the step and click Move Dn or select Move Down from the Step menu.
Move Up/Down the List (multiple Steps)	Left-click and drag the step to the new position. When you release the mouse button, the step will move.
Build automation control steps	To build a set of automation control steps quickly and insert them at current position, select Step Builder.

^ Back to Top

Test Loops

The Loop Start and Loop End steps allow you to enclose a set of test case commands in an iterative loop. The # of Loops is set in the last column of the table and indicates how many times the sequence is repeated. A loop is the transition from Loop End to Loop Start. The # of Loops (Loop Count) can be 1 to 2147483647. A Loop Count = 0 is not supported, as that would not require no looping at all, would execute section once. Loop Count = 1 means the section will execute twice, once during initial sequencing, then one loop back from Loop End to Loop Start (i.e. 1 loop), and execute a second time.

The sequence below is an example of a 72-hour test that executes four (number of times the control loops back; 0-loops = 1 run) 24-hour data traffic models. By using test case loops rather than repeating a 24-hour test session for four iterations, the test maintains a minimum constant load on the SUT. In the 24-hour model, different loads are placed on the SUT based on the time of day.

During low traffic hours the SUT is loaded to 30% capacity. During the normal busy hours it is loaded to 50% capacity. Normal busy hours increases the load to 70% and peak busy hours adds an additional 10% load in traffic bursts. Session Loading tests are used in all of the test cases to present the SUT with a mix of activities.

In all cases except the traffic burst model, the session loading runs are designed to maintain a constant number of active MN sessions. In the traffic burst model, all sessions are established, kept active for an hour, torn down, and then remain idle for a time. Time was compressed for the actual test run and minutes equate to hours.

NOTE: You may also use Pass/Fail Criterion to see that all multiple attempts set up in Automation Control Steps occurred.

For example, Setup a Criterion against the required measurement, with Value == 1, set the reset to the TestCase State as UNINITIALIZED and the start to TEST CASE State == INITIALIZED. At the end of the test, the PASS count or FAIL count indicates whether the test executes as set up.

NOTE:

The normal order for a single test case inside a Loop is: LOOP START (INIT START STOP CLEANUP) LOOP END
Loops must contain all 4 states (INIT, START, STOP, CLEANUP) or None at all.
You can put multiple Test Cases in the loop, but each Test Case in the loop must have all four States covered.
If a Loop does not contain all 4 states you will get this error:
Other orders are possible: INIT LOOP START ( START STOP CLEANUP INIT ) LOOP END STOP CLEANUP

^ Back to Top

Running Tests Indefinitely

The Wait step allows you to pause the step processing until you decide to click the Continue button to execute the next step.

When running a node and nodal test cases in the same test session, if you want to pause the nodal test case to finish, you would include a Predecessor step on the nodal test case waiting for the STOPPED state.

This is particularly useful when you are using node and nodal test cases in the same test session. You can specify that the node test case waits for the nodal test case to finish before stopping, ensuring that it is available to participate in session tear down, without setting a finite execution time. In the example below, the Simple IP test case waits until the AAA Server Node is running before starting. The test continues with both test cases running until the Continue button is clicked and then the Simple IP test case stops. The AAA Server Node waits until all sessions have been disconnected and the Simple IP test case reaches the stopped state before stopping.

^ Back to Top

Compound Tests

The completed sequence of automation steps shown below is an example of how you can use Automation Control to turn one test session into a compound test package that executes multiple test cases. The test session contains three AAA Nodal test cases, each of which runs a Capacity test that consumes a percentage of the SUT's capacity (20%, 50%, and 70%). The 20% Capacity test is executed twice — once in conjunction with the 50% Capacity test and again with the 70% Capacity test. After one Capacity Test has established all of its sessions (reached the RUNNING state), the test waits for 30 seconds and then runs the 20% Capacity test for 10 minutes. Both tests are then stopped and the next pair of tests is started. Notice that when you re-use a test case, additional Cleanup and Init steps are required to flush the resources used in the previous execution.

NOTE: After the TAS initializes all test cases, it first starts test cases which are not in the automation steps, and then it starts test cases in the automation.

Once the automation control completes execution, test cases that are not explicitly stopped will continue running, and the test session’s overall state will show as RUNNING. It is up to you to stop the test manually (you may Retrieve the Run Log for more details).

^ Back to Top