Artificial intelligence (AI) is increasingly being used to add new capabilities to embedded products across industrial, medical and autonomous applications. Advances in Arm Cortex processor technology, including Helium for accelerating digital signal processing (DSP) and machine learning (ML) workloads, are making increasingly sophisticated Edge AI implementations practical.

However, an AI model must still perform reliably when exposed to changing real-world conditions, sensor variations and environmental disturbances that may not have been represented during development. At the same time, the complete implementation must provide the required processing capability within the product’s power and cost constraints.

A considerable amount of attention is often given to the AI model rather than the product itself. As such, it is important to realise that the AI model is not the product: it is only one part of a complete system that must solve the stakeholder’s requirements, operate reliably in the physical world and satisfy its functional, engineering and regulatory requirements.

The following article introduces the Real-Time Edge Intelligence (RTEI) framework for designing first-time-right (FTR) Edge AI products, starting with stakeholder requirements and bringing together physical-world understanding, data intelligence and engineering implementation.

Start with the stakeholder requirements

Successful New Product Development (NPD) starts with understanding what the stakeholder actually wants the product to do. These needs then have to be progressively translated into requirements and, ultimately, an engineering specification.

The NPD requirements hierarchy can be summarised as follows:

• User requirements — define what the client and stakeholders expect the product to do and how it will be used by a typical user.
• Functional requirements — define what the system must do to satisfy the user requirements, without specifying its internal design or how those functions will be implemented.
• System specification — translates the user and functional requirements, together with information from stakeholder discussions and engineering know-how, into concrete design parameters and constraints. These may include the sensor type, system and signal-chain architecture, sampling rates and algorithmic concepts.
• Technical specification — defines how the system specification will be implemented using real-world technology, including detailed hardware design, component selection and software implementation. This typically contains proprietary engineering details and is therefore normally an internal document.

Once the user and functional requirements are understood, appropriate technologies can be investigated and the system specification developed, rather than assuming that AI is the solution from the outset.

Understand the physical world and the data

Edge AI systems interact directly with the physical world. Sensors do not produce perfect data, operating conditions change, components have tolerances, and the data collected during development will rarely represent every situation encountered after deployment. As such, understanding the physical system and the available data is essential. Some fundamental questions include:

• What does the sensor actually measure?
• What are the wanted and unwanted signal components?
• Which disturbances are expected?
• What can we determine from physics and mathematics before asking an ML model to infer it from data?

This latter point is where domain knowledge and DSP algorithms can play an important role. Filtering, spectral analysis, tracking and feature extraction are all DSP algorithms that can expose characteristics of the physical system and provide an ML model with better-quality information. In many cases, better feature extraction can also simplify the ML problem itself.

The objective is therefore not to choose between DSP and AI. It is to use signal processing, mathematics, domain knowledge and AI where each provides the greatest value. This can lead to smaller and more efficient ML models — particularly important when deploying onto embedded processors where processing performance, memory, latency and power consumption all have to be considered.

From AI model to engineered product

The Real-Time Edge Intelligence (RTEI) framework brings these elements together into a structured process for developing FTR Edge AI products.

The process begins by fully understanding what the stakeholders want the product to do. Available technologies can then be investigated to help complete the requirements and specifications before system design begins. More importantly, this does not assume from the outset that AI or ML is necessarily the solution.

A key part of the subsequent stages is the collaboration between human and data intelligence. Domain knowledge and an understanding of physical-world behaviour are then used to interpret the available data and determine which information is actually relevant to the problem. This allows high-quality features to be developed that improve ML accuracy and robustness, while reducing the need for a complex model with greater processing, memory and power requirements.

The resulting DSP algorithms, ML models and embedded software must then be brought together with the processor, sensors, electronics and wider hardware platform as a complete real-time system. Finally, that system must then be validated against the stakeholder, engineering and certification requirements established at the beginning of the development process before customer deployment.

AI as part of the engineering toolbox

As AI technology matures, I believe it will increasingly become part of the engineering infrastructure, alongside processor technology, DSP algorithms, communications, embedded operating systems and software libraries. AI should therefore be seen as an incredibly powerful technology, but one of several that engineers can select according to the problem they are trying to solve.

The important engineering question is therefore not “How do we put AI into this product?”, but rather “What combination of technologies gives us the best solution to meet the stakeholder’s requirements?” Sometimes the answer will involve significant AI and sometimes relatively little. Suffice to say, that in most successful Edge AI products, AI will remain an important component of the product, but not the product itself.

Further reading

These ideas, together with the NPD process and RTEI framework, are covered in depth in Real-Time Edge Intelligence Handbook, available from ASN’s bookshop.