Testing with hardware, what does that mean?
Projects that require testing with hardware include:- a piece of hardware
- firmware that is running on hardware
- software that communicates with the hardware
- a communication protocol between the hardware and software.
1. Close the (domain) knowledge gap
The first obstacle you may come across when starting the project is a potentially huge domain knowledge gap between your current knowledge and the demands of the project. So, you eventually realize that you need to start quickly closing this gap. Here are a few sources you can leverage to get what you are looking for:- Colleagues — Reach out to your colleagues to hear expert advice from someone who has already been involved in a similar project or is working on one currently.
- Customer — Usually, the client has more extensive knowledge about the business domain than you.
- International standards —These kinds of projects sometimes must be compliant with certain international standards. They might be a bit complex at first sight but will generate a lot of critical questions and provide answers.
- Certification bodies – Certification documentation represents the source of extensive knowledge you can further leverage for your future projects’ success.
2. When hardware is missing, emulate it
Another thing you learn quickly is that the hardware is elusive. From my own experience, we were usually able to start the development of a software piece of the system before hardware was ready. To make things more complex, it usually turns out that it takes 3 to 6 months, sometimes even a year, for hardware to be completed. So, what can you and your team do? You can build an emulator. There are a few important things to keep in mind here:- Emulators are complex and usually mimic the behavior of real hardware (device).
- The emulator is a piece of software you will have to maintain.
3. Advocate for testability
Once the emulator is in the house, you are slowly closing the domain knowledge gap, and the initial parts of the software are becoming available for you to test. For these kinds of projects, it is imperative to advocate for testability from the very beginning. Projects of this kind usually do not have a UI part (or it is developed in the later stages), so you should opt for testing that does not include this part, at least initially. In this case, we usually develop analytics engines, calculation engines, and data transformations first. Although you should ensure that the app is testable from the very beginning, you need to shift your focus to controllability and observability because it is crucial for controlling the system, bringing it into the right state easily, and observing what’s going on in that state. So how are you going to do that?- Provide the right inputs at the right time — Define the right scope and the right sample size from the very beginning because it will verify the very foundation of the product you are building. By making it so, with the help of an emulator, you will be able to control the application and achieve the desirable states.
- Creating an efficient logging strategy — Since you are testing a lot of things that are going without the UI (calculations, transformations, data flows), you should be able to find whatever you need rather quickly and more efficiently. So, make sure your logging strategy works for you by staying focused on what you need there and avoiding unnecessary information.
4. Automate what steals your time
Now that you feel confident because you are closing your domain knowledge gaps — you have the emulator, which is helping us test various features, and you are able to control the system and have a clear overview of what’s working or not. Everything seems to be running smoothly, right? Not really! You suddenly realize that the level of complexity of these systems is much higher than you think. No matter how much you prepare yourself for the unexpected turn of events, you can never know what challenges are lurking in the dark. The devices and instruments can run in different configurations, and, in the end, you need to test them all. All of this adds up to the complexity. And what do you do in this case? Automate whatever steals your time. When it comes to automation, projects like these are a bit specific. For instance, let’s say you are measuring blood cholesterol levels. Creating a result with some input data, preparing that data, analyzing what happened with it, and realizing that your cholesterol level is 4, takes a lot of time. To avoid this scenario, you should automate what takes most of your time, including data preparation, data transformation analysis, or analysis of end results. At this point, the real hardware is usually ready, and you can start working on a real thing and play with it finally. Now, the question arises: Emulators are good, but should you trust them? Emulators are good, but should you use them now that you have got the real thing? In most cases, you will not stop using them. As a matter of fact, you will probably use them more than expected. My team used to continue using emulators even when we had the hardware or the prototype ready — emulators are simply more convenient and faster. However, there are cases when you must play with the real thing because emulators do not have the required capacity to perform the needed activities. Here are a few things emulators cannot do:- Time-related testing
- Performance
- Reliability
