The table below describes the fields in the DTA Administration window that lists all the Device Test Agent (DTA) that are registered / connected with the TAS. The DTA Application running on Android and iOS UEs talking directly to the Landslide TAS. The DTA Application will utilize the UE Node Handset Test Case uses.
DTA Test Cases currently supported are: (UE Node App)
To enable the DTA Application you need a new license, that includes the necessary license key which will provide either unlimited tests or some limited # of tests 10K, 1M. In addition, the number of DTA devices registered with the TAS is also licensed.
It is also expected to be used with VisionWorks, as DTA results are converted to TEST.PROCS_UE files.
In order to deploy this application, a Google Firebase Account must be setup and related information from the Firebase Account must be placed into a firebase.conf file on the TAS. Once properly licensed, Firebase is setup and the firebase.conf file is added, when you start the TAS you will see this confirmation of Firebase communications in the TAS output: Initializing DTA services and connecting to Firebase
Status: Indicates if FireBase is connected or not.
And when you open the GUI you will see the DTA Admin toolbar and menu options:
DTA (Device Test Agent) via Admin -> Device Test Agents Administration
Field | Description | |
---|---|---|
Recycle DTA |
Select Recycle to send a command for the DTA to re-register with the TAS, without requiring user intervention on the phone. |
|
Logout DTA |
Select Logout to send a command for the DTA to logout and require the user to login again. I.E. use this command if have other phone you want to register and you have reached the maximum DTAs. |
|
Cancel All Pending Tests |
Select to Cancel all pending tests on the selected DTAs. When tests are cancelled via the Cancel button, the state of the test will change to Canceling. | |
Retrieve DTA Logs |
Select Retrieve Logs to retrieve the DTA Application Debug logs and place them in Results Website. |
|
Reboot MDM Supported DTAs
|
Select to reboot the MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Update DTA App for MDM Supported DTAs |
Select to update the DTA app for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Wipe Data for MDM Supported DTAs
|
Select to wipe data for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Lock DTA App for MDM Supported DTAs
|
Select to lock the DTA app for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Unlock DTA App for MDM Supported DTAs
|
Select to unlock the DTA app for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Enable WIFI for MDM Supported DTAs
|
Select to enable WIFI for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Disable WIFI for MDM Supported DTAs
|
Select to disable WIFI for MDM supported DTAs. The MDM flag must be set to YES. Only available for DTAs with MDM (Mobile Device Management) capability. |
|
Click the Add button to add a New DTA Test Session. A new window opens to create/execute a DTA Test Session. You can configure one or more DTA Test Cases:
Select Run Test Cases in Parallel When Possible to send a flag to DTA to allow tests to run at same time if they support it, otherwise each Test Case will execute one at a time, next one started when current one completes. Enter the Log Level to send a value to DTA to indicate what the logging level should be for the test execution. 0 (Indicated no Logs OFF) , up to 10. Turning on Logs will cause DTA to produce a “Run Log” for each DTA Test Case. Select Report Logs to send a flag to DTA to indicate to send the “Run Log” along with the results to the TAS. If this is not checked, the “Run Log” will be kept on the phone, retrievable using the “Retrieve Logs” on DTA Admin window. Enter Expiration (min) (Range: 1 to 30 minutes) - how long DTA will keep tests pending in the background. Used by a sleeping / background DTA to know if a test should be executed when it is activated. Any tests that are in the pending queue that have expired will not be executed. Tags - Click to enable VisionWorks Tags for a test session. See additional details in topics : About the Test Case Settings Window and Customizing Report Options. Intent of Test - Click to select the intent of test for a test session. See additional details in topics : About the Test Case Settings Window and Customizing Report Options
The Test Cases list (left panel / list) is where you can add / removed DTA Test Cases from the DTA Test Session. The test cases will be started in the order in which they are listed, in a round robin fashion. You can save a DTA test case individually and open it in another DTA Test Session at a later time. Clinking on one of the test case will open the editor on the right hand side of the screen. Each Test Case editor should match what you see in the UE Node Handset test case. See UE Node App topic for additional details on the supported Test Types:
Scheduling Test Sessions is now supported via the Test Scheduler. DTA Tests can be recurring and can be paused, but they do not support the Stop or Abort options. The Target DTAs (bottom panel / table) allows you to add or remove DTAs to include in the DTA Test Session.
See sample of Real-Time Logs below.
A DTA test session log file will be generated and can be found in Test Results - See Viewing Test Results. The log files will follow this naming convention : Year-Month-Day_Time_DTA-TEST-SESSION_NameofTest.log.txt Example name of log file : 20-06-02_08.24.54_DTA-TEST-SESSION_DTA_Multi_Test.log.txt and the contents which will include relevant information regarding your test. Sample file : 2020-06-02T08:21:05.354-05: Starting test session: DTA_Multi_Test with ID=325 2020-06-02T08:21:05.354-05: -Starting test case: ping 2020-06-02T08:21:05.354-05: --Starting DTA ID=1 2020-06-02T08:21:05.355-05: DTA(0001-123456789022222): Test start sent for ping 2020-06-02T08:21:05.355-05: Confirming tests started for ping 2020-06-02T08:21:06.355-05: -Test case ping startup completed 2020-06-02T08:21:06.355-05: -Starting test case: mobile_info 2020-06-02T08:21:06.355-05: --Starting DTA ID=1 2020-06-02T08:21:06.356-05: DTA(0001-123456789022222): Test start sent for mobile_info 2020-06-02T08:21:06.356-05: Confirming tests started for mobile_info 2020-06-02T08:21:07.356-05: -Test case mobile_info startup completed 2020-06-02T08:21:07.356-05: -Starting test case: dns 2020-06-02T08:21:07.356-05: --Starting DTA ID=1 2020-06-02T08:21:07.357-05: DTA(0001-123456789022222): Test start sent for dns 2020-06-02T08:21:07.357-05: Confirming tests started for dns 2020-06-02T08:21:08.357-05: -Test case dns startup completed 2020-06-02T08:21:08.357-05: -Starting test case: http 2020-06-02T08:21:08.357-05: --Starting DTA ID=1 2020-06-02T08:21:08.358-05: DTA(0001-123456789022222): Test start sent for http 2020-06-02T08:21:08.358-05: Confirming tests started for http 2020-06-02T08:21:09.358-05: -Test case http startup completed 2020-06-02T08:21:09.359-05: -Starting test case: latency 2020-06-02T08:21:09.359-05: --Starting DTA ID=1 2020-06-02T08:21:09.359-05: DTA(0001-123456789022222): Test start sent for latency 2020-06-02T08:21:09.359-05: Confirming tests started for latency 2020-06-02T08:21:10.359-05: -Test case latency startup completed 2020-06-02T08:21:10.360-05: -Starting test case: speedtest 2020-06-02T08:21:10.360-05: --Starting DTA ID=1 2020-06-02T08:21:10.360-05: DTA(0001-123456789022222): Test start sent for speedtest 2020-06-02T08:21:10.360-05: Confirming tests started for speedtest 2020-06-02T08:21:11.361-05: -Test case speedtest startup completed 2020-06-02T08:21:11.361-05: -Starting test case: iperf 2020-06-02T08:21:11.361-05: --Starting DTA ID=1 2020-06-02T08:21:11.361-05: DTA(0001-123456789022222): Test start sent for iperf 2020-06-02T08:21:11.361-05: Confirming tests started for iperf 2020-06-02T08:21:12.362-05: -Test case iperf startup completed 2020-06-02T08:21:12.362-05: Test Session startup completed
|
||
Open DTA Test Session
|
Select to open DTA Test Session opens a window to select a saved DTA Test Session to open. |
|
Selected UE Attribute / Value |
Clicking on a single DTA will show more details in the bottom table. Most of this information is sourced from the DTA Status message sent approximately every 30s, or else from other actions and messages. |
|
ID |
The ID will be a number from 1 to 10000. The TAS can handle up to 10,000 IMEIs, but only supports 300 registered/live DTAs at any given time. A system can support up to 10K different phones being used at different times, no more than 300 at once. |
|
IMEI |
DTAs are uniquely identified by their IMEI, but the TAS will assign a more human readable ID to each IMEI, and this assignment will be persistent. |
|
Phone Info |
Details of Phone - based on what the phone reports to the TAS. If the phone supports 5G NR, you will see - 5G NR in this field. |
|
Username |
the Landslide username that was used to register the phone. |
|
DTA User |
Currently a free-form text field that each user on the DTA/phone fills out with their actual name, to provide another unique ID for the DTA, i.e. the actual persons name using the phone. |
|
State |
Similar to Test Server State, it tracks if DTA is READY, RUNNING or if status messages are missing, IDLE then OFFLINE. When tests are cancelled via the Cancel button, the state will change to CANCELING. |
|
MDM |
YES - if a DTA has MDM (Mobile Device Management) capability, otherwise set to NO. |
|
Last Update |
Indicates a label/action and timestamp of the last event that occurred on the DTA (helps with understanding the State). |
|
Application |
The name of the DTA Application. |
|
Available Tests |
Lists all the tests that the phone supports, which could be more/less than TAS can support (this table is showing information reported by the phone). |
|
Running Tests |
Lists all the running or queued up tests (DTA test cases). |
|
Firebase ID |
The unique Firebase ID for the phone used for communications to/from. |
|
Selected DTA's Tests |
The table lists the tests per selected DTA. The table lists: RID - Run Id Type - Type of test (e.g. dns, http, etc) Username - User Name on Landslide DTA User - User Name on DTA App State - State of test (e.g. Running, Stopped, Queued, etc) Last Update - Last update of test (e.g. Log, Status, Running, etc) |
The TAS will prevent shutdown if DTA tests are running, it will require the force flag - Force Shutdown? The TAS is active, do you want to force the shutdown (Yes or No).
NOTEs:
|
Real-Time Logs: Using DTA you will want to watch/search the REAL-Time Logs, as that is where all actions are logged.
USER DATE-TIME SEVERITY TYPE ACTION MESSAGE
sms 2019-12-10 07:30:01 Informational Operations DTA DTS(1031): Starting androidlongtimetest
sms 2019-12-10 07:30:01 Informational Operations DTA DTS(1031): Confirming tests started for mobile_info
sms 2019-12-10 07:30:01 Warning Operations DTA DTS(1031): DTA(62-?): No longer registered, skipping start of mobile_info and any others
sms 2019-12-10 07:30:02 Informational Operations DTA DTS(1031): Test case mobile_info startup completed
sms 2019-12-10 07:30:02 Informational Operations DTA DTS(1031): Confirming tests started for video
sms 2019-12-10 07:30:03 Informational Operations DTA DTS(1031): Test case video startup completed
sms 2019-12-10 07:30:03 Informational Operations DTA DTS(1031): Confirming tests started for http
sms 2019-12-10 07:30:04 Informational Operations DTA DTS(1031): Test case http startup completed
sms 2019-12-10 07:30:04 Informational Operations DTA DTS(1031): Confirming tests started for dns
sms 2019-12-10 07:30:05 Informational Operations DTA DTS(1031): Test case dns startup completed
sms 2019-12-10 07:30:05 Informational Operations DTA DTS(1031): Confirming tests started for latency
sms 2019-12-10 07:30:06 Informational Operations DTA DTS(1031): Test case latency startup completed
sms 2019-12-10 07:30:06 Informational Operations DTA DTS(1031): Confirming tests started for ping
sms 2019-12-10 07:30:07 Informational Operations DTA DTS(1031): Test case ping startup completed
sms 2019-12-10 07:30:07 Informational Operations DTA DTS(1031): Confirming tests started for speedtest
sms 2019-12-10 07:30:08 Informational Operations DTA DTS(1031): Test case speedtest startup completed
sms 2019-12-10 07:30:08 Informational Operations DTA DTS(1031): Confirming tests started for dns
sms 2019-12-10 07:30:09 Informational Operations DTA DTS(1031): Test case dns startup completed
sms 2019-12-10 07:30:09 Informational Operations DTA DTS(1031): Test Session startup completed
<system> 2019-12-09 14:30:04 Informational Operations DTA DTA(0032-355257099884386): Test Start Confirmed ID:355257099884386+92
sms 2019-12-09 14:30:04 Informational Operations DTA DTA(0032-355257099884386): Test start sent for dns
sms 2019-12-09 14:30:05 Informational Operations DTA DTA(0032-355257099884386)-Test(93/latency): Test Added, queued
<system> 2019-12-09 14:30:05 Informational Operations DTA DTA(0032-355257099884386): Test Start Confirmed ID:355257099884386+93
sms 2019-12-09 14:30:05 Informational Operations DTA DTA(0032-355257099884386): Test start sent for latency
<system> 2019-12-09 14:30:06 Informational Operations DTA DTA(0032-355257099884386): Test Start Confirmed ID:355257099884386+94
sms 2019-12-09 14:30:06 Informational Operations DTA DTA(0032-355257099884386)-Test(94/ping): Test Added, queued
sms 2019-12-09 14:30:06 Informational Operations DTA DTA(0032-355257099884386): Test start sent for ping
sms 2019-12-09 14:30:07 Informational Operations DTA DTA(0032-355257099884386)-Test(95/speedtest): Test Added, queued
<system> 2019-12-09 14:30:07 Informational Operations DTA DTA(0032-355257099884386): Test Start Confirmed ID:355257099884386+95
sms 2019-12-09 14:30:07 Informational Operations DTA DTA(0032-355257099884386): Test start sent for speedtest
sms 2019-12-09 14:30:08 Informational Operations DTA DTA(0032-355257099884386)-Test(96/dns): Test Added, queued
<system> 2019-12-09 14:30:08 Informational Operations DTA DTA(0032-355257099884386): Test Start Confirmed ID:355257099884386+96
sms 2019-12-09 14:30:08 Informational Operations DTA DTA(0032-355257099884386): Test start sent for dns
sms 2019-12-09 14:30:19 Informational Operations DTA DTA(0032-355257099884386)-Test(89/mobile_info): Results Saved: 19-12-09_13.30.19__0032-355257099884386__89__TAS__mobile_info.txt
sms 2019-12-09 14:30:21 Informational Operations DTA DTA(0032-355257099884386)-Test(90/video): Started, RID=90
sms 2019-12-09 14:31:31 Informational Operations DTA DTA(0032-355257099884386)-Test(90/video): Results Saved: 19-12-09_13.31.31__0032-355257099884386__90__TAS__video.txt
sms 2019-12-09 14:31:32 Informational Operations DTA DTA(0032-355257099884386)-Test(91/http): Started, RID=91
sms 2019-12-09 14:31:44 Informational Operations DTA DTA(0032-355257099884386)-Test(91/http): Results Saved: 19-12-09_13.31.44__0032-355257099884386__91__TAS__http.txt
USER DATE-TIME SEVERITY TYPE ACTION MESSAGE
sms 2019-12-09 14:30:06 Informational Operations DTA DTA(0032-355257099884386)-Test(94/ping): Test Added, queued
sms 2019-12-09 14:31:57 Informational Operations DTA DTA(0032-355257099884386)-Test(94/ping): Started, RID=94
sms 2019-12-09 14:32:07 Informational Operations DTA DTA(0032-355257099884386)-Test(94/ping): Results Saved: 19-12-09_13.32.07__0032-355257099884386__94__TAS__ping.txt
The RUN ID is a semi-unique number 1-9999 created by the DTA for each DTA Test Case execution. Over a significant period of time, the RUN-ID is a unique ID for a test execution on any DTA. When looking for test results you can use DTA ID and DTA Test RUN-ID in both REAL-TIME logs and Results.
Results use similar IDs/Prefixes. The general prefix for DTA produced files: <START_DATETIME>__<DTA_ID>-<DTA_IMEI>__<RUN_ID>__<DTA USER>__<TEST CASE>, this can include the .log (Run Log) and .txt (Results). For the TEST_PROCS_UE produced by the TAS, it will be <START_DATETIME>__<DTA_ID>-<DTA_IMEI>__<RUN_ID>__TEST.PROCS_UE.<VW_TIMESTAMP>.csv.
Examples:
When executing tests from the TAS, the DTA Username is fixed at “TAS”. Also the test results will go to the results page for the user that is logged into the TAS. When executing tests from the DTA/Phone itself, test results will go to the results page for the Landslide Username that the phone registered with and show the “DTA User” used during DTA Login.
Example:
These naming standards can be useful for querying the results website, i.e. for __<DTA_USERNAME>__, or __<RUN_ID>__ or __<DTA_ID>-, or __<TEST_CASE>.txt