Test automation in medical technology – saving costs with existing resources

This architecture requires that the connected tools have appropriate application program interfaces (API). ContinoProva does not require any specific form of test specification. It is able to execute the test steps defined by the test specification sequentially and feed the execution results back into the test sequence.

ContinoProva Client and Server each consist of individual software modules. The client contains a module called Test Control, which takes the data from the current test step of the test specification and generates an executable code unit from it. In another module called Test Execution, the tools involved in a specific test step are coordinated.

The separation of Test Control and Test Execution makes it easier to adapt the implementation to different types of test specifications. The Test Execution module is matched on the server side by the Execution Provider module. To enable easy interchangeability of tools, communication with the respective tool API is encapsulated in an extra module called Tool Adapter.

The module test 

Testing the modules of the test object required dividing the entire system to be tested into functionally meaningful units. In order to test the functionality of a module, its interfaces were connected to the test environment and a model was created that simulates the environment of the test object with a sufficient degree of abstraction.

Adapters, known as services, were developed to connect the modules to be tested to ContinoProva. They integrate external tools into ContinoProva and make their functions, such as stimulating or reading interfaces, available in the form of services.  The integration of external tools via services made it possible to implement and adapt module-specific adapters quickly and with little effort. This provided a flexible test environment that could be used for multiple modules.

Another advantage was the ability to reuse a test specification in different test environment constellations with different external tools. This is made possible by carefully defining the interfaces and skillfully structuring and nesting generic and tool-specific services. In this case, different bus simulators that read and write CAN messages were used for CAN bus communication.

The system test

A hardware-in-the-loop (HiL) system from dSpace and other external components such as PCAN and controllable relay boxes were used for the system test. These were connected to ContinoProva via services so that the ContinoProva user interface could be used as the front end for the test specification. As with the module test, functions that would have had to be simulated in the environment model were implemented in ContinoProva services. The battery simulation implemented in ContinoProva for various states of charge and other battery properties proved to be valuable at this point. This significantly reduced the test runtime and saved test resources.

It became apparent that the service implementation required less effort than developing or expanding the environment model in the HiL environment and was therefore the more efficient solution. Parts of the environment model were developed in modular services using easy-to-understand programming languages (C# and .NET). The test environment was thus up and running in a short time. The encapsulation of tool-specific adapters in services also enabled easy testability, a high reuse rate of test artifacts, and simple error analysis. A total of approximately 10 services were developed for the project to implement interfaces and simulate the behavior of the environment.

Lessons Learned

Feedback from product developers: Berlin Heart’s developers and testers benefited from the ease of use and comprehensibility of the test specification format, meaning that only a short training period was necessary before productive testing could begin.

Sensible demarcation between module and environment: A precise analysis of the module and its interfaces is necessary in order to set sensible boundaries between the module and its environment. This influences the number and types of external interfaces of the test object.

Simulation vs. implementation: Test automation inevitably raises the question of which parts of the test system should be simulated and which should be implemented using (real) systems. The answer to this question must take into account the fact that implemented systems can increase cross-effects. This can make it more difficult to analyze the causes of errors when they occur.

First module tests, then meaningful integration tests: Testing at different test levels is necessary in order to detect and locate errors at an early stage. Integration tests should be based on a well-considered integration strategy that is adapted to the test objective and test object.