Accelerating software testing is vital to speeding up embedded system development, especially for Android and Linux systems running on ARM-based hardware. Virtual prototyping strategies, including the use of VDKs, can help.
Accelerating software testing is vital to speeding up the development of embedded systems. There can be few markets in which this is truer than in the development of Android and Linux based systems running on ARM-based hardware.
To make the job easier, a consortium of companies known as the Linaro organization is developing a hardware testing infrastructure and software testing framework called LAVA.
The Linaro Automation and Validation Architecture (LAVA) is a system for deploying operating systems on physical and virtual hardware. LAVA can be used for simple boot testing, operating system testing and system-level testing, although extra equipment may be required for system tests of target hardware. Results are tracked over time and data can be exported for further analysis.
LAVA is a collection of participating components including:
- LAVA Scheduler is a web application for managing a collection of devices and processing a queue of test jobs to perform. This component is part of the LAVA Server.
- LAVA Dispatcher is a framework and command line tool for controlling test execution on a particular machine, including the deployment of the initial environment.
- LAVA Test is a framework and command line tool for wrapping existing test programs and translating results into a common format.
- LAVA Dashboard is a web application for storing and viewing test results. It has simple data mining and reporting features, and is part of the LAVA Server.
LAVA uses the concept of user jobs and tests. LAVA jobs are device-independent JSON files that define which software image to deploy on the device under test and what to do to execute it. LAVA tests are invoked by a job using the “lava_test_shell” action and pointing to a test file (written in YAML).
The LAVA component that interacts with the actual device is the LAVA dispatcher, which reads the user jobs and executes the generic actions in a specific way according to the target device. Integration between LAVA and the VDK technology happens at the dispatcher level.
Integrating VDKs into LAVA
Figure 1 shows how the integration works.
Figure 1 VDK integration with the LAVA Dispatcher (Source: Synopsys)
The first thing to look at is the user job. An example of a simple job file is shown in Figure 2, which shows how the job sets its name (vdk-test), the target device where to execute (01_VDK_ARMv8Base_CortexBL4x4) and the set of actions to be performed. The action “deploy_linaro_kernel” will load the software image, file system, bootloader, etc., indicated by its parameters, onto the target device. The software files can be in the local machine (using file://) or can be downloaded from the web (using http://). The action “boot_linaro_image” will tell the device to boot with the new deployed image.
“rootfs”: “file:/// path_to_file /Vanilla-Cortex/ARMv8/rootfs.ext2”,
“bl1”: “file:/// path_to_file /Vanilla-Cortex/ARMv8/bl1.bin”,
“firmware”: “file:/// path_to_file /Vanilla-Cortex/ARMv8/fip.bin”
Figure 2: A simple LAVA job example (Source: Synopsys)
The LAVA Dispatcher will start running the job by first picking the device information specified by the target parameter. By convention, the dispatcher will try to open the “01_VDK_ARMv8Base_CortexBL4x4.conf” file located in the folder “/etc/lava-dispatcher/devices”.
A LAVA job must always include a target device where the job will be executed, which represents an instance of a device type and can include configuration options that override the options of the device type. In this example, we define an instance of the “ARMv8Base_CortexBL4x4” VDK platform, which is an 8-core ARM-based SoC simulation model based on the big-LITTLE architecture.
The only mandatory entry in the target configuration file is the name of the device type that is being inherited. In this example, the target configuration file will inherit all its options from the “VDK_ARMv8Base_CortexBL4x4.conf” file. This is shown in Figure 3. This file is by convention located in the folder “/etc/lava-dispatcher/device-types”.
device_type = VDK_ARMv8Base_CortexBL4x4
Figure 3 – A VDK target configuration file example (Source: Synopsys)
The device type configuration file (VDK_ARMv8Base_CortexBL4x4.conf) contains the configuration options for the VDK. Those options are used to select and configure a LAVA client (part of the LAVA dispatcher) that manages the execution of a job on a specific target device. The most relevant configuration options are illustrated in Figure 4. The client_type is used to select what LAVA client to use for handling the target device. For this we select the generic “vdk” client (described later). The simulator_command tells the LAVA client which command to use to launch the VDK. Typically a shell script sets the appropriate variables and launches the simulator model and the VDK tools in batch mode. Finally, the tester_ps1_pattern option contains the Linux prompt that appears once the Linux boot of the software image under test is finished. This pattern is required for proper synchronization between LAVA and the software running on the VDK.
client_type = vdksim_bin = /path/VDK_ARMv8Base_CortexBL4x4/skins/Vanilla-Cortex/launch.tclsimulator_command = sudo %(sim_bin)s tester_ps1_pattern = root@genericarmv8:~#
Figure 4 – The device type configuration file for ARMv8Base_CortexBL4x4 VDK (Source: Synopsys)
A dedicated LAVA client implementation is required for the dispatcher to manage the execution of LAVA jobs in a VDK. Therefore, a generic VDK client has been implemented, as a Python file (“vdk.py”) and copied into the device folder of the LAVA dispatcher installation. Whenever a device type configures the client type as a VDK (i.e. client_type = vdk), the file is picked up by the dispatcher. The LAVA VDK client (or client from now on) works as follows:
- When the vdk.py file is invoked, the VDK is started. The client uses the simulator_command information from the VDK_ARMv8Base_CortexBL4x4.conf.conf file to launch the VDK.
- The client will hold its execution until the control connection of the VDK starts the TCP/IP server. Information about the TCP/IP port of the server is sent to the client using the standard output. When received, the client will open a Telnet session with the VDK, known as the simulation control connection. To enable multiple VDK simulations to run without conflict, a different TCP/IP port is assigned to each request.
- The client uses the control connection to send the VDK the actions (and parameters) specified on the user job. For instance, the client will send commands using Telnet strings such as “LAVA::commands::boot_linaro_image::run” or “LAVA::commands::deploy_linaro_ kernel::set_parameter_kernel <path_to_file>” to the VDK. These commands are implemented in a custom LAVA package on the VDK side, where they are received, evaluated and executed. The goal with this approach is to use one LAVA client (vdk.py) across many different VDKs, since model-specific details are taken into account in the VDK scripting framework.
- Job commands such as boot_linaro_kernel or lava_test_shell will invoke the power_on command in the client. When sent to the VDK, this command lets the VDK simulation continue and the simulation will periodically check for incoming control commands. Whenever a LAVA job is finished or there is an error condition, the client will execute the power_off command, which shuts down the VDK simulation and ends any remaining processes started by the client.
- Once the target device has been booted (power_on command), the LAVA client will wait until connecting with the serial terminal. LAVA uses this connection to send and receive information from the Linux shell. In this example, the Linux shell information uses the UART peripheral model of the VDK. This UART model is connected with the LAVA client using the high-speed data interfaces in the VDK. In this example, we use the ExtApp interface to connect the UART model to a simple application providing a TCP/IP server. This second TCP/IP server is required by the LAVA client to setup another Telnet connection. We name this the serial terminal connection.
- The client uses this connection to listen to the Linux boot and shell messages and to send commands to the Linux prompt. This is required to execute the actual software tests on the system, specified in the lava_test_shell action of the user job. The client uses the test pattern (see ‘tester_ps1_ pattern’ option) to know when the shell is ready to receive Linux commands.
With this infrastructure in place, the LAVA dispatcher can execute jobs using a VDK target with any other target device. The VDK targets and device types can be added to the LAVA Scheduler (in the LAVA Server) to automate the dispatch and execution of jobs.
The end result of this process is a system that enables users to manage and run many testing jobs in parallel.
For additional information, please view the webinar, "Better Testing Through Automation and Continuous Integration with Virtualized Development Kits"
Linaro website: www.linaro.org/about/
LAVA welcome page: https://validation.linaro.org/
Victor Reyes is a technical marketing manager in the System Level Solutions group at Synopsys. His responsibilities are in the area of virtual prototype technology and tools with special focus on automotive. Reyes received his MsC and PhD in electronics and telecommunication from University of Las Palmas, Spain, in 2002 and 2008 respectively. Before joining Synopsys, he held positions at CoWare, NXP Semiconductors and Philips Research.