Whether for a jumbo jet, self-driving vehicle or an MRI machine, the software that enables these systems to run reliably was developed by human engineers. In order to reduce risk and develop the best solutions possible, those engineers need a reliable set of tools for testing and automating software processes.
One solution for that is Robot Framework.
Robot Framework is a flexible, open source automation framework that is actively supported. It is written, in Python, to support different methods of automated testing, such as data-driven and test-driven methods.
This framework uses an abstraction mechanism called keywords to make tests much more human readable. Keywords work like functions in code. They can be called from either test cases or other keywords and they can be given arguments for even better abstraction. Robot Framework is also extensible using libraries which provide new keywords for test cases to call. These libraries can be implemented with Python or Java.
There are some terms that should be defined before we explain much more as they can differ between different test frameworks.
|Test Case||A single test case which should test some functionality. It should have as little dependency on other test cases as possible.|
|Test Suite||A file that contains related test cases.|
|Test Suite Directory||A directory that contains test suite files.|
|Test Libraries||Contains the lowest-level keywords.|
|Resource Files||Files that provide variables and higher-level user keywords.|
|Variable Files||Files that provide more flexible ways to create variables than resource files.|
Robot Framework provides several standard libraries that help with a lot of the common functionality that is needed to develop tests. Here are some of the standard libraries that are more useful for Hardware-in-the-Loop (HIL) testing.
|BuiltIn||Contains generic often needed keywords. Imported automatically and thus always available.|
|Collections||Contains keywords for handling lists and dictionaries.|
|DateTime||Supports creating and verifying date and time values as well as calculations between them.|
|Dialog||Supports pausing the test execution and getting input from users.|
|OperatingSystem||Enables performing various operating system related tasks, such as locating files, removing files or directories, or getting environment variables.|
|Process||Supports executing processes in the system. Useful to run other host software alongside the tests.|
|Remote||Part of the remote library interface. Does not have any keywords of its own. Helpful for running tests from different hosts or on multiple targets.|
|String||Library for manipulating strings and verifying their contents.|
|XML||Library for verifying and modifying XML documents.|
There are many custom libraries that are already developed and maintained by third parties that provide essential functionality for embedded testing. These include, but are not limited to, the following:
|Database||This can allow you to query your database after an action has been made to verify the results.|
|MQTT||Provides keywords for testing on MQTT brokers. MQTT is a lightweight protocol for machine-to-machine communication, typically used for IoT messaging.|
|RESTinstance||Library for RESTful JSON APIs. validates JSON using JSON Schema, guiding you to write API tests to base on properties rather than on specific values.|
|SeleniumLibrary||A web testing library that utilizes the Selenium tool internally. Allows you to automatically open web pages, press buttons, check objects on the page, validate user sign in, etc.|
|SerialLibrary||Very useful for early embedded platform testing. Getting a UART up and running to get to a known state on a board is usually one of the first things to test. This library makes that easy by providing simple keywords to open, close, write to, and read from serial ports.|
|SSHLibrary||Yet another incredibly useful interface for some embedded targets. If the target board is running SSH and is connected to the same network as the host machine, then this is a great way to run commands for testing on the target. This library provides support for both SSH and SFTP.|
This is just a small sample of the already available custom libraries. More can be found under the external tab here.
Sometimes, the available custom and standard libraries that already exist will not be enough to fully test your custom solution. Sometimes you can cobble the existing libraries together to get them to do what you need. However, it may make sense to develop your own custom library. Some examples include if you have a custom application to run on the host or need to use an interface that does not already have a supporting library. A custom implementation is likely easier to manage and maintain into the future if APIs change.
This guide will not cover how to write your own custom library. There is a comprehensive guide available from the Robot Framework developers. We have used this guide to develop our own custom library to control a networked power controller. It uses an HTTP interface, so we could have used an HTTP library to implement our own keywords. However, by making it a custom library, it becomes much easier to integrate into different projects, since it can just be installed to the developer’s machine once and used across projects. Another benefit of using a library is that since it is a class, when you include the library, you can provide configuration data. That way, fewer arguments need to be provided via keyword calls.
The next blog will provide a tutorial for using robot framework in HIL testing.