Code coverage is an important metric that allows for thorough validation of software products. It is also a requirement for many different certifications such as DO-178. Generating code coverage reports for software running on an OS such as Linux or Windows is trivial using current tools. However, generating code coverage reports on an embedded target, running a bare metal application presents a number of challenges, foremost of which is the lack of any kind of file system to store gcov results.
It can appear impossible with currently available tooling to generate coverage reports for embedded targets. Fortunately, there are options available for engineers to create and retrieve code coverage reports from embedded targets.
Code coverage is a metric generated during testing, it analyzes how many lines of code are hit and how many logical paths are taken through normal operation of the software. Code coverage results can help developers in several ways. First, to assess test coverage and identify gaps in testing, second, to identify the most frequently called functions as potential candidates for optimization, and third to identify dead code that could be removed or may indicate a logic bug.
During a recent DornerWorks project code coverage reports were required from baremetal software application running on a RISC-V target. The embedded gcov library developed by NASA Jet Propulsion Laboratory was identified as an option to generate the required code coverage. After some application customization and experimentation the embedded gcov library was used to successfully generate the necessary reports.
There are many tool options available to generate code coverage results. In this instance GNU Gcov and lcov were used to generate all reports. Gcov is an open source code coverage tool that can be used with GCC to generate code coverage results. Lcov is a utility that can manage gcov data generation and generate coverage reports that are organized and readable. Lcov output reports show line by line what code is being hit and what’s being missed, blue highlighted lines have been hit and orange lines have been missed.
Lcov summary pages show the number of lines and functions that have been hit and the total coverage percentage for each file.
There were multiple steps taken to generate code coverage reports, the first was to include the necessary embedded gcov source files in the image. These files can be found in the NASA-JPL github. Once the repo is cloned, the files gcov_public.c and gcov_gcc.c were added to the project’s source files and gcov_gcc.h and gcov_public.h were included main.c.
Next, build the profiling tools into the image, this can be done by building an image using the preprocessor flags -ftest-coverage and -fprofile-arcs. These flags will add instrumented code, in blocks to the image in order to track code usage. Once the instrumented code is included, profiling data will be collected and stored in memory while the software is running.
Once the gcov files were included it was necessary to ensure that the constructor functions built in with gcov were called to initialize the instrumented coverage code. The constructor gcov_init() needed to be called inside every code block that was being tracked by gcov. gcov_init() is a standard gcov lib function that has been customized for embedded targets. init_array is the list of constructors and destructor functions that exist on ELF based platform. Details of the init_array section of an ELF is beyond the scope of this article, there are many sources available to read more detailed explanations. To ensure that the necessary constructors were called an additional function was written to handle the contents of init_array. The init_ctors function must be called as early as possible to initialize the gcov constructors before the majority of the source code can run.
void init_ctors(void)
{
void (**p)(void);
extern uint32_t __init_array_start, __init_array_end;
/* Beginning of constructors listed in linker script */
uint32_t beg = (uint32_t)&__init_array_start;
/* End of constructors listed in linker script */
uint32_t end = (uint32_t)&__init_array_end;
while(beg<end) {
p = (void(**)(void))beg; /* get function pointer */
(*p)(); /* call constructor */
beg += sizeof(p); /* next pointer */
}
}
After adding the necessary files and the above function to initialize the constructors there were still hurdles to be overcome. The constructor functions were not being called at the appropriate time despite the addition of the init_ctors function. After some debugging it was discovered that the gcov constructors were being optimized away by GCC at compile time. In order to stop the necessary constructors from being optimized away, a section was added to the linker script to manually keep the gcov constructors from being removed. The linker script snippet below shows the use of the keyword KEEP, which stops the linker from optimizing away anything in the defined section.
.gcov_mem : ALIGN(0x10)
{
__gcov_mem_start = . ;
KEEP (*(.gcov_mem))
__gcov_mem_end = . ;
} > ram
It was then necessary to add an attribute to the gcov constructor definitions in gcov_public.h that needed to be kept, so that they would be placed in the .gcov_mem section that was defined in the linker script. The attribute section is used to place functions in specific sections, this attribute is a part of GCC and is not yet a part of the C standard. The gcov constructors with the attributes can be seen in the code snippet below.
void __gcov_init(struct gcov_info *info) __attribute__ ((section(".gcov_mem")));
void __gcov_exit(void) __attribute__ ((section(".gcov_mem")));
Once the application built successfully and was run, the gcov_exit function could be called and the code coverage results were dumped to the serial output of the target. gcov_exit is a standard gcov lib function that has been customized to dump the results to serial, an abbreviated snippet of the __gcov_exit function is shown below.
void __gcov_exit(void)
{
GcovInfo *listptr = gcov_headGcov;
while (listptr) {
...
gcov_convert_to_gcda(buffer, listptr->info);
for (u32 i=0; i<bytesNeeded; i++) {
if (i%16 == 0) GCOV_PRINT_HEXDUMP_ADDR(i);
GCOV_PRINT_HEXDUMP_DATA((unsigned char)(((unsigned char *)buffer)[i]));
if (i%16 == 15) GCOV_PRINT_STR("\n");
}
GCOV_PRINT_STR("\n");
GCOV_PRINT_STR(gcov_info_filename(listptr->info));
GCOV_PRINT_STR("\n");
listptr = listptr->next;
} /* end while listptr */
}
Once the results were captured lcov could be used to process the results and export a human readable summary.
Optimizing application size and efficiency is just one of the many challenges a developer faces when creating a product for an embedded system. Accurate code coverage results can help a developer narrow down improvements in application testing and opportunities for application optimization. Historically, code coverage results have been difficult to obtain from embedded targets that lacked file systems and operating systems. With some ingenuity and the right application of tools this is a hurdle that can be overcome.
