Skip to content

How to Install XGen and Device Lab on a Single Machine

If the user would like to install everything on a local machine, this part explains the process. It can be done simply through the all-in-one script of XGen.

Requirements

  • GNU/Linux x86_64 with kernel version > 3.10 (Ubuntu or CentOS or WSL2)

  • Network (Recommend): The controller needs to be accessible from the agent and XGen. Port 80 should be open and reserved for the controller. There are many ports are used internally by XGen components.

    If your firewall is enabled, please ensure following ports are allowed in the firewall rules: 22 80 28000 38000 19000 12379 6379 55672 45672 4369 5671 5672 15671 25672 27017 19001-19200 Please ensure following domains are also accessible: *.cocopie.ai *.cocopie.io *.bitbucket.org *.aliyuncs.com *.amazoncs.com

  • Docker

  • Memory size: 16GB or more
  • Devices: The agent needs to be able to connect to the devices via USB. The default suggested devices are 64-bit devices (phones/tablets) that run Android 9 and iOS 11 or above. Support of other devices can be added separately.
  • NVIDIA GPU with Architecture >= Kepler (or compute capability 3.0)

  • Disk size: 500GB or more. XGen by itself takes less than 20GB. But more space would be needed to hold AI models and datasets. The amount depends on the models and datasets the user uses. The AI models included in XGen's common AI are 500MB or less. The sizes of the default datasets included in XGen's common AI are as follows (downloadable on demand):

    • ImageNet: 144.02 GB

    • UCF101: 75.74 GB

    • CoCo2017: 25.01 GB

The requirements on the devices:

  • A feature of XGen is that it can ensure accurate assessment of the quality and speed of the generated AI model and code by including real devices in the optimization loop. To use this feature, the machine that runs XGen needs to connect to Device Lab. XGen uses the devices for speed measurement.

  • When no devices are connected, XGen can still be used to optimize AI models as explained in the Model Optimization Alone Without Running on Devices; the code generation functionality of XGen and on-device benchmarking are not usable in that mode though.

Step 1: Install Nvidia-Driver, Docker and Nvidia Container Toolkit

If the XGen machine is running the Ubuntu Linux operating system:

  1. Please make sure that nvidia-dirver (version >= 510) has been installed. Instructions: Nvidia-Driver Installation Guide (CUDA installation is not mandatory on the host, because the Nvidia Container Toolkit in the later step includes CUDA support.) There is another solution as well if there is no specific requirement for the driver:

    # install ubuntu-devices-common to check devices version
sudo apt install ubuntu-devices-common -y
# check devices version and determine CUDA driver version needed
sudo ubuntu-drivers devices # this command will list all available drivers and related apt package name
# output example:
vendor   : NVIDIA Corporation
driver   : nvidia-driver-470 - distro non-free
driver   : nvidia-driver-535-open - distro non-free
driver   : nvidia-driver-525 - distro non-free
driver   : nvidia-driver-535 - distro non-free recommended
driver   : nvidia-driver-525-server - distro non-free
driver   : nvidia-driver-535-server - distro non-free
driver   : nvidia-driver-535-server-open - distro non-free
driver   : nvidia-driver-470-server - distro non-free
driver   : nvidia-driver-525-open - distro non-free
driver   : xserver-xorg-video-nouveau - distro free builtin

# install the recommended driver or the driver you need
sudo apt install nvidia-driver-535 -y

  2. Please make sure that Docker Engine has been installed. Instructions: Install Docker Engine 

    curl -fsSL https://get.docker.com -o get-docker.sh
sudo bash ./get-docker.sh

  3. Please make sure that Nvidia Container Toolkit is installed. Instructions: Nvidia Container Toolkit Installation Guide

After completing the above steps, the user can verify whether the deployment requirements of XGen Docker Image are met by running the following command in Linux: 

 docker run --rm --gpus all nvidia/cuda:11.1.1-base-ubuntu20.04 nvidia-smi
When success, the NVIDIA GPU information will be output to the screen.

Hint: to run docker command without sudo authorization, the user needs to be granted the docker permission. The admin can do that through the following steps:

 sudo groupadd docker  # if docker group does not exist
 sudo usermod -aG docker $USER
Important: re-login to make the change take effect. 
 docker ps
If the above command runs successfully, the user would have the docker permission.

Install the following commands if they are not installed yet: 

 sudo apt install unzip curl git bc

Before installation, please ensure that usbmuxd service is not running on the host machine. usbmuxd is used by XGen in communications with iOS devices, similar to how adb-server is used for Android devices. In the previous version of XGen, usbmuxd was installed as a system service on the host machine. In the current version, we have included it within the Docker container. Therefore, if usbmuxd is present on your machine, it will cause the agent container to fail to start. Therefore, when running the agent container, please stop the usbmuxd service and remove it to avoid conflicts.

How to stop usbmuxd service on UBUNTU20.04: 

sudo systemctl stop usbmuxd
And how to delete usbmuxd service on UBUNTU20.04: 
sudo apt remove usbmuxd -y

Step 2: Installation

The installation script has many installation options. The user can run the following command to see the options:

bash xgen_aio_deploy.sh -h

Usage: bash xgen_aio_deploy.sh [OPTIONS]
A script for fetching and setting up the docker images of CoCoPIE XGen.
Options:
  -h, --help                    Show this help message.
  --region string               US or HK (default US; HK is faster for mainland China).
  --memory bytes                Memory limit for XGen (default 70% of System Memory).
  --xgen-path string            Home directory of XGen (default $HOME/XGen).
  --xgen-data string            Path in the host to store dataset (default $xgen-path/Data).
  --xgen-output string          Output path of XGen (default $xgen-path/Output).
  --controller-path string      Home directory of Controller (default $HOME/XGenController).
  --agent-path string           Home directory of Agent (default $HOME/XGenAgent).

Examples:
  bash xgen_aio_deploy.sh
  bash xgen_aio_deploy.sh --region HK
  bash xgen_aio_deploy.sh --xgen-path ~/XGen
  bash xgen_aio_deploy.sh --xgen-path ~/XGen --xgen-data ~/MLDataSets --controller-path ~/XGenController --agent-path ~/XGenAgent

For users in mainland China, the following command usually runs faster; for other users, use '--region US'.

 bash xgen_aio_deploy.sh --region HK

After installation, XGen and Device Lab are installed into the configured path. And commands run_xgen, run_controller, run_agent, upgrade_xgen, upgrade_controller, upgrade_agent are written into $HOME/.bashrc.

Update the shell environment:

 source ~/.bashrc

The user can now type the following command to enter XGen shell: 

 run_xgen --name <username>

run_xgen: This command will start all containers including Device Lab Controller with Dashboard, Device Lab Agent as well as XGen. Then will enter into the XGen instance. Each XGen container instance needs to be assigned a unique name with param --name to distinguish it from other instances. If the XGen container named with the --name parameter is already running, enter that container. Otherwise, start a new container with the specified name.

run_agent, run_controller: These two commands will start Device Lab Controller with Dashboard and Device Lab Agent and enter into the corresponding container. In All-in-One installation, these two commands are usually not needed to run; run run_xgen instead to start everything. Similar to the run_xgen command, if the xgen_controller container or xgen_agent container already exists, running the run_agent or run_controller command will enter the corresponding container.

upgrade_xgen, upgrade_controller, upgrade_agent: Executing the upgrade_xgen command will upgrade all containers, including the Device Lab Controller with Dashboard, Device Lab Agent, as well as XGen. However, you can also choose to upgrade the Device Lab components individually using the upgrade_controller and upgrade_agent commands.

uninstall_xgen, uninstall_controller, uninstall_agent: These commands are used to remove the corresponding container environments and images. The difference lies in the fact that uninstall_xgen encapsulates the logic of uninstall_controller and uninstall_agent. This is because, when installed on the same machine, the uninstallation process can be made more convenient and efficient.

Before connecting any devices, please read the How to connect Devices part below for more information. Connect your devices via USB to the host machine, and run the following command in XGen container to check the status of the devices: 

xgen_devices

Platform    UUID                       Brand    Model             Task Status    Connection Status
----------  -------------------------  -------  ----------------  -------------  -------------------
Android     3131612233002SC            vivo     V2072A            idle           available
Android     6c74106e                   Xiaomi   MI 9              idle           available
iOS         00008101-0009245134F9001E  Apple    iPhone 12 16.3.1  idle           available

Step 3: Configuration

All components in All-in-One script are already well configured. There is no more configuration required. To access the Device Lab Dashboard, simply visit http://deployed_machine_ip from your machine.

tips: If there are firewalls or other network restrictions, you can use an SSH tunnel to map port 80 on the deployed XGen machine to a port on your local machine, such as 8080. This way, you can access the remote machine's dashboard by visiting http://127.0.0.1:8080.

SSH tunnel port mapping example: ssh -L 8080:127.0.0.1:80 <username>@<host_ip>

At the first time, you'll be redirected to the login page. The default account and password are both admin. Then you'll be redirected to change password page to change the default password for security.

For more usage about Device Lab, please refer to Device Lab Usage.

How to Start XGen

To start XGen, the first step is to start its docker container by running the following command, which resides in the bin folder of XGen.

The underlying operational logic is as follows: start the controller and agent along with their related dependent containers. If these containers are not already started, they will be initiated. Once they are up and running, an XGen container will be launched and accessed.

Of course, you can also individually start the controller container and agent container using the run_controller and run_agent commands before launching the XGen container. The run_xgen command, however, simplifies this entire process for you:

 run_xgen --name <username>

The command creates and starts an XGen container named xgen_<version>_<username>. Meanwhile, it creates a folder Projects under the XGen home directory. It is for holding all AI models and optimization results; users' own to-be-optimized models should be put there as well. That folder is mapped to the /root/Projects folder inside the XGen container; the files in that folder are hence accessible both inside and outside the XGen container.

After run_xgen --name <username>, XGen's shell will be started.

XGen is proprietary software. During the initial login, the user needs to execute xgenctl status and adhere to the instructions for acquiring a license.

The license file must be named license and situated within the /etc/xgen/license directory. Users can verify the license status by executing xgenctl status.

After configuring the license, type the following to start XGen:

 XGen

Because the XGen container is a docker container, users can use the standard docker commands to manage the container, such as stopping it, checking its status, and so on. But users should avoid using docker commands to directly create XGen containers or rename the created XGen container. Always use run_xgen --name <username> to create XGen containers. Otherwise, there could be unexpected data overwritten and file corruptions. For more detailed info on XGen command usage, please see XGen Usage.

Sometimes, during an XGen session, users might experience a disconnection from the machine running XGen. Here are some helpful hints.

Exiting the XGen container is the same as all containers – running the exit command within the container will leave it. 

exit
The difference lies in the behavior after exiting. For Controller and Agent containers, exiting does not mean stopping the container; the container will continue to run in the background as services. However, for the XGen container, exiting means stopping the container.

xgen_controller, xgen_agent, and the underlying foundational data storage containers operate as services, thus running persistently in the background even if a user executes the exit command within the container. If there is a need to halt these services, the following Docker commands can be employed: 

docker stop xgen_controller xgen_dashboard xgen_agent
As for XGen, it supports concurrent usage by multiple users. In most cases, an XGen container functions as a client, with each user employing an individual container to accomplish their tasks. Upon completion, users exit the container, causing it to stop. This approach optimizes computational resources for other users' utilization.

How to Upgrade XGen

For users who already have version v1.1.0 installed, direct upgrading to the latest v1.3.0 version using the upgrade_xgen command is not supported. A fresh installation is required for v1.3.0 since there have been architectural and module organization optimizations.

For new users installing version v1.3.0, for future upgrades, simply use the upgrade_xgen command to directly upgrade.

For more details about v1.3.0, please see What's New.

After re-installation, the user can start XGen as before through run_xgen --name <username>.

In future version upgrades, the user can run the upgrade_xgen command. In an all-in-one installation of the XGen environment, upgrade_xgen will upgrade all container images, including the controller, dashboard, and agent. The user can upgrade the controller or the agent individually by simply running upgrade_controller or upgrade_agent.

How to Uninstall XGen

The user may uninstall XGen by running the uninstall_xgen command in the bin folder of XGen:

uninstall_xgen

The command will present users with options to choose the components (such as previously generated models) to retain. Similar to the upgrade_xgen command, the uninstall_xgen command will uninstall all container images, including the controller, dashboard, and agent.

How to Connect Devices

XGen measures the speed of the optimized models directly on connected devices (e.g., Android phones). The devices should be connected to the machine that runs an XGen Agent. This part explains how to connect devices.

Before explaining the detail, it is worth noting that regardless of the method of connection, the following apply:

  • For Android devices, the user should turn on the "Developer mode", "USB debugging". Please see instructions.

  • On the devices, at the first connection to the machine, an authorization window pops up; please authorize the machine to access this device and trust the machine always.

  • The USB cable used for connection must be a data cable rather than a charging cable.
  • Running command adb devices inside the agent shell would show the list of Android devices that are accessible by agent.
  • Running command sib devices inside the agent shell would show the list of iOS devices that are accessible by agent.

Connect Android devices via USB

Connect the devices with the agent machine via a USB data cable. Some USB cables are only for charging, which won't work. (When the phone is connected with the computer via a data cable, usually the computer prompts to select connection mode, select Transfer Files.) The phone needs to be configured as stated above.

ADB has been installed in Agent container. So after completing the devices connection, then run the following command to check if the devices are connected to the agent container machine:

# enter 
run_agent
# check if your android device is connected
xgen_devices list -nm

Connect iOS devices

  1. Install CoCoBench app in TestFlight. For installation, the user needs to have TestFlight installed on the iOS device to be used with XGen, and follow the invitation link in the XGen welcome email to install CoCoBench via TestFlight, and then launch CoCoBench on the iOS device. To guarantee the app runs correctly, here are some tips:

    • Make sure that CoCoBench of the latest version is installed.
    • Make sure that CoCoBench is running foreground.
    • Make sure that the screen of the iOS device is always on, which means that the screen is not locked and the device does not enter the sleep mode.
    • Make sure that there is no password set for the iOS device.
    • Make sure that the auto update of CoCoBench in TestFlight is turned on; otherwise, manually check it to keep the app up to date.
    • By default, CoCoBench expires 90 days after the publish date due to the limitation of TestFlight. The user must update CoCoBench to the latest version in TestFlight before the expiration date; enabling auto update would simplify this.

  2. Connect the iOS device to the host machine via a USB data cable (not a charging cable). Then the host machine might pop up a window to ask the user to trust the host machine. Click Trust.

  3. Enter Agent container, then run the following command to check if the iOS device is connected to the agent container machine:

    # enter 
run_agent
# check if your iOS device is connected
xgen_devices list -nm

  4. If the iOS device falls offline due to some reason (e.g., unplugged and replugin), the user needs to restart the usbmuxd services as shown below: 

    xgenctl restart -s usbmuxd
Service usbmuxd restarted.
    Alternatively, a simpler approach is to restart all services.

    xgenctl restart
    After that, reconnect the iOS device to the host machine.

Hints

Avoid issues from disconnection

Sometimes in the middle of an XGen session, the user may get disconnected from the machine that runs XGen. To ensure that the session is not interrupted by the disconnection and the user can get back to the session after reconnection, the user can use tmux:

  • Install tmux by, for instance, the following command (on Ubuntu): sudo apt-get install tmux

  • In the host terminal, run the following to start a new session: tmux new -s Some-Session-Name

  • In the tmux window, start XGen as before: run_xgen --name <username>

  • The session will continue running even if a disconnection happens. If the user is disconnected and then reconnected to the machine, run the following command will bring back the session: tmux attach -t Some-Session-Name.