This is a comprehensive guide on measuring test coverage in embedded software using CTC++ Testwell by Verifysoft Technology. My name is Eyal Gerber, and I’m from Novodes, where we specialize in embedded software development solutions and testing. In this post, we’ll delve into the process of testing and measuring the coverage of your embedded software tests using a powerful tool called CTC++ Testwell and how to integrate it with Keil uVision.
Understanding Test Coverage
Before we dive deep into the process, let’s clarify what test coverage is and why it’s vital. Test coverage measures how much of your code is exercised by your tests. This is essential in ensuring your tests are comprehensive and effective in catching potential issues early in the development cycle.
Types of Coverage
One of the standout features of CTC++ is the variety of coverage types it supports. Let’s quickly outline them:
- Statement Coverage: Checks if each line of code is executed.
- Decision Coverage: Ensures every decision (true/false) in the code takes place.
- Multi-Condition Coverage: A more complex form that considers multiple conditions.
For the purpose of this guide, we’ll focus on Statement Coverage and Decision Coverage.
Getting Started with CTC++ Testwell
To begin, you need to install the necessary tools and set up your environment. Here are the steps:
Installation
- Download and Unzip: Download the relevant CTC++ files and unzip them on your computer.
- Install Additional Packages: If you’re using Keil uVision, you’ll also need to install additional files. In this case, it’s a zip named keilctc-2.2.
- Project Integration: You’ll receive a set of files that need to be added to your project. These include C files and header files.
Add these C files to your project and ensure the header files are included in the project’s included path.
4. Modify Template Function: Define ctc_send_data based on your capabilities. For instance, in our example, we use a custom API function called TraceC.
Handling Libraries and Functional Safety
If you’re using a functional safety compiler with limited std libraries, don’t worry. CTC++ Testwell handles it well by letting you define NO_STD_LIB in your project.
Defining NO_STD_LIB
Add the definition at the beginning of your project’s files:
#define NO_STD_LIB
This setting ensures CTC++ Testwell adapts to the absence of standard libraries, like malloc, automatically.
Instrumentation: Key to Coverage Measurement
Instrumentation is the process of inserting coverage code into your code, making it possible to track which parts of the code are executed during runtime.
Configuring Instrumentation
- Run Instrumentation: After building your project, you’ll need to generate a batch file and use it for instrumentation. The batch file may be generated with Keil uVision by going to the target options –> Output –>”Create Batch File” Checkbox and rerunning the build.
- Command Line Execution: Open CMD in your project folder and run the following command:
call keilctc -i d <batch_file>
3. Exclude Unwanted Files: Use exclude pattern to exclude third-party or unnecessary files from instrumentation so that you only measure coverage in the files you desire. This will also speed the instrumentation process. This is done in the ctc-keil.ini file.
Uploading and Collecting Serial Data
Once the code is instrumented, upload the resulting hex file to the target microcontroller using your programmer.
Serial Data Collection
Ensure your microcontroller only sends coverage data to prevent noise:
- Open Serial Terminal: Open your serial terminal and begin capturing the serial data printed by the microcontroller.
- Save Output: Copy the serial coverage data and save it into a file, typically
MON.txt.
Analyzing Coverage Data
With the captured data saved, run the following command to process it:
ctc2dat -i MON.txtThis command links the serial data with the instrumentation data, creating a coverage report.
Reviewing the Coverage Report
Finally, use ctcreport to generate a detailed report:
The generated report shows coverage percentages across different files, indicating which parts were tested successfully and which were not.
It is possible to browse each file that has been covered and see both the statement coverage and the decision coverage:
Conclusion
CTC++ Testwell proves to be a robust tool, making the process of measuring test coverage straightforward and efficient. Even complex integrations, such as adapting to limited std libraries or customizing data collection methods, are well-supported.
If you are also looking for powerful commercial unit test frameworks and mocking frameworks, you might consider options like Parasoft’s unit testing tool C/C++Test or QA Systems’ Cantata. However, CTC++ Testwell remains an excellent choice for hassle-free integration and support with regards to coverage-only tools.
Remember, detailed test coverage is not just a checkbox in your development process; it’s a crucial part of ensuring your code is reliable and bug-free. So, invest the time to integrate these tools properly and enjoy a more robust software development process.
Happy testing!
