Skip to content

Device Lab Introduction

The Device Lab avoids the need to connect mobile devices directly to the training machine. In many scenarios, the training machine is located in a server room or data center, making it impractical to connect physical mobile devices. Now, with the ability of Device Lab, you can keep your devices local without plugging them in the training machines.

Overview

Device Lab architecture Device Lab Structure

As shown in the above diagram, the current version of XGen consists of two parts: the XGen client and the Device Lab module. The Device Lab module further includes two components: a Controller and one or more agents.

Device Lab Controller is the core of the device lab. The primary responsibility of the controller is to establish connections between XGen and the Agent, forwarding and logging messages between XGen and Agent. If these three components are not within the same network environment, the controller needs to expose its public IP address to the outside, in order for XGen and Agent to locate and connect to it.

Alongside the controller, there is also a Dashboard installed. The purpose of the Dashboard is to track tasks forwarded by the controller and to handle a portion of the configuration work related to the connections between XGen and the agent. In short, Dashboard is a web application that provides a user interface for monitoring devices and tasks.

Device Lab Agent is a daemon running on a host machine which can connect with many testing devices, like your phones. It is responsible for communicating with the controller and executing tasks.

XGen Client is a tool named xgend (serving as a service within the XGen container) that is utilized to interact with the controller. It is used to submit tasks, check task status, and download task results.

Installation

Please follow the steps below to install Device Lab. First install the controller and dashboard, then install the agent, and finally install the XGen client. For more details, please refer to Device Lab Installation.

Usage

About XGen usage, please refer to XGen Usage. The following content is about Device Lab usage.

Controller Usage

Controller service

After installation, the controller service will be started automatically. You can also use the following command to start the controller and enter controller container. Controller container is a docker container which runs controller service. 

$ run_controller
In most cases, you do not need to run run_controller command manually. if the 'xgen_controller' container already exists, running the run_controller command will enter the corresponding container.

You can use the following command to check controller service status: 

$ xgenctl status
You can use the following command to exit controller container, but it will not stop the controller container: 
$ exit

Dashboard Usage

After Device Lab Controller installation, you can vist the web page of dashboard by http://<controller-ip> in your browser.

Login page

Login Page

At the first time you visit the dashboard, you need to change the default password. The default username is admin, and the default password is admin. The web page will redirect to the change password page automatically.

After changing password, it will return to the home page.

In the top right corner of the home page, there are 3 buttons, it indicates that the web system includes 3 modules, Dashboard, Config, Admin. You can click the Dashboard button to go back to the home page. Additionally, you can click the Config button to access the configuration page, where you can configure controller public IP and the agent. Furthermore, the Admin button allows you to access three sub-pages: the document page, the password change page, and the system logout. Here is the structure:

  • Dashboard Page Root
    • Dashboard
      • Devices
      • Tasks
      • XGen
    • Config
      • Controller IP
      • Agent
    • Admin (logo icon)
      • Document
      • Change Password
      • Logout

Devices page

Home Page

On the home page (the default home page is the Devices page), you can view a list of devices grouped by agent. The status of each device is indicated by the color of its card. Devices can have the status of online (displayed as a light green card) or offline (displayed as a light gray card). The device status is updated automatically.

In each device card, you can find details such as the device name, device model, device OS version, and more.

Tasks page

Tasks Page

In the tasks page, you can see the task list, grouped by xgen-session. Different stages of the task are shown by the color of the task rows.

An XGen session refers to one-time execution of XGen or one-time execution of onnx_latency_benchmark.

For more information about a task, you can click the detail button to see the task details. and also you can click the Log button to see the task's runtime log.

Task Details:

Task Details

Each task consists of multiple phases, and each phase requires a certain amount of time to complete.

On the task details page, you can access detailed information about the task, including its status, task result, and the duration of each phase. This allows you to identify any bottlenecks in the task. Additionally, a description is provided for each phase at the bottom of the page.

XGen page

XGen Page

In the XGen page, all connected XGen instances are listed here, the status and some statistic information are show in the list.

Config Controller IP

If your agent, XGen, and controller are not within the same local network, then you need to configure the public IP of the controller so that the agent and XGen can connect to the controller via the internet. This is where you configure the external access IP for the controller.

You can configure Controller public IP in the config part of dashboard by clicking on the Config button in the top right corner of the page.

Config Controller

By clicking edit, you can update the configuration. Please note that update Controller IP configuration will cause all agents to disconnect and all tasks to stop.

Config Agent Info

Adding agent here is intended to control the connection of legitimate agents. You can configure the agent in the config page. You can add or edit agent configuration. This page controls which agents can connect to the controller. Please note that update agent configuration may cause agents to disconnect and stop current tasks.

Config Controller

If you meet any problem, please refer to Troubleshooting for help.

Reset dashboard web page password

Enter into the dashboard container by running command: 

$ docker exec -it xgen_dashoard bash
and in the dashboard container, run the following command to reset the password:

$ reset-web-passwd -p <new_password>

Agent Usage

Check devices

xgen_devices is a powerful command to check devices.

$ xgen_devices --help

Usage: xgen_devices [OPTIONS] [ COMMAND [ARGS] ]...

  Tool for XGen Device Lab Agent

Options:
  --help  Show this message and exit.

Commands:
  list    List all devices
  update  Update device info.

To list all devices, just simply run xgen_devices list:

$ xgen_devices list

Platform    UUID      Brand    Model     Task Status    Connection Status
----------  --------  -------  --------  -------------  -------------------
Android     eb24e4c4  Xiaomi   M2011K2C  idle           available

To see more information about devices

$ xgen_devices list -v

Platform    UUID      Brand    Model     CPU                                               GPU                        Memory      Battery %  Task Status      Task Num  Connection Status
----------  --------  -------  --------  ------------------------------------------------  -------------------------  --------  -----------  -------------  ----------  -------------------
Android     eb24e4c4  Xiaomi   M2011K2C  Venus based on Qualcomm Technologies, Inc SM8350  Qualcomm, Adreno (TM) 660  7.13 GB            99  idle                    0  available

To see devices without network connection

$ xgen_devices list -nm

By default, Agent will try to fetch device information by all available methods. However, missing some information is normal. Users can update device information by xgen_devices update:

xgen_devices update --help

Usage: xgen_devices update [OPTIONS]

  Update device info. Update device info may cause unexpected error, Please
  Use With Caution.

Options:
  -d, --device TEXT               Device ID of the device you want to update
                                  [required]
  -f, --field [gpu|cpu|brand|model]
                                  Field you want to update  [required]
  -v, --value TEXT                Value you want to update. ( No more than 64
                                  characters )  [required]
  --help                          Show this message and exit.

Update agent configuration

If you need to update the agent configuration. the user may edit the configuration in /root/.xgen_agent/agent.conf and restart the agent by running command xgenctl restart.

Daemon services

The agent runs as a daemon service. After updating the corresponding configuration, you need to restart the service to make the configuration take effect. You can use the following command to restart the service:

xgenctl restart

You can use the following command to check agent service status: 

$ xgenctl status

You can use the following command to exit agent container, but it will not stop the agent container: 

$ exit

If you meet any problem, please refer to Troubleshooting for help.

Troubleshooting

As Device Lab is a distributed system and many daemons are running in the background, problems may happen. Logs are useful. Please check the logs of controller, dashboard, and agent.

Controller logs are located in /root/.xgen_controller/logs.

Controller log structure:

controllerd  device  pack_monitor  queue  tasks  unpack  xgenctl
  • controllerd directory records the logs of controllerd service, which is the main controller daemon and responsible for doing initialization and starting other daemons.
  • device directory records the logs of the daemon which is responsible for transferring devices information from agents to all connected XGen instances.
  • pack_monitor directory records the logs of the daemon daemon which is responsible for packing all benchmark results from different agents and send them to the corresponding XGen instance.
  • queue directory records the logs of the daemon which is responsible for managing queue creating and deleting.
  • tasks directory records logs of all benchmark tasks.
  • unpack directory records the logs of the daemon which is responsible for unpacking benchmark requests for different devices from XGen instances.
  • xgenctl directory records the logs of the daemon which is responsible for communicating with other daemons.

Dashboard logs are located in /root/.xgen_dashboard/logs.

Dashboard log structure:

service  xgenctl
  • service directory records the logs of the daemon which is responsible for offering web service and API services.
  • xgenctl directory records the logs of the daemon which is responsible for communicating with other daemons.

Agent logs are located in /root/.xgen_agent/logs.

Agent log structure:

xgen_devices  agent_status  agentd  task_manager  xgenctl
  • xgen_devices directory records the logs of the daemon which is responsible for checking devices status in real-time.
  • agent_status directory records the logs of the daemon which is responsible for sending agent devices status to controller.
  • agentd directory records the logs of the daemon which is responsible for doing initialization and starting other daemons.
  • task_manager directory records the logs of the daemon which is responsible for managing benchmark tasks.
  • xgenctl directory records the logs of the daemon which is responsible for communicating with other daemons.

Health Check

For controller, dashboard and agent, just simply run xgenctl status for a health check.

If the controller is running normally, the output will be like this:

Settings check passed.
Redis check passed.
MQ check passed.
Supervisor services check passed.

If the dashboard is running normally, the output will be like this:

Redis check passed.
Supervisor services check passed.

If the agent is running normally, the output will be like this: 

Settings check passed.
Redis check passed.
MQ check passed.
Supervisor services check passed.

If there are any problems reported by the health check, please follow the presented instructions to fix the problem.