The internet of things (IoT) has gained tremendous popularity over the last few years, as many organisations strive to add IoT smart sensor technologies to their product portfolios. The basic paradigm centres around connecting everything to everything, and exchanging all data. This could be house hold appliances to more blue sky applications, such as smart cities. But what does this particularly mean for you?

Almost all IoT applications involve the use of sensors. But how do SME and even multi-national organisations transform their legacy product offering into a 21st century IoT application? One the first challenges that many organisations face is how to migrate to an IoT application while balancing design time, time to market, budget and risk.

Sounds interesting? Then read further….

We recently completed a project for a client who manufactured their own sensors, but wanted to improve their sensor measurement accuracy from ±10% to better than ±0.5% without going down the road of a massive re-design project.


The question that they asked us was simply: “Is it possible to get high measurement accuracy performance from a signal that is corrupted with all kinds of interference components without a hardware re-design?”

Our answer: “Yes, but the winning recipe centres around knowing what architectural building blocks to use”.

Traditionally, many design bureaus will evaluate the sensor performance and try and improve the measurement accuracy performance by designing new hardware and adding a few standard basic filtering algorithms to the software. This sort of intuitive approach can lead to very high development costs for only a modest increase in sensor performance. For many SMEs these costs can’t be justified, but perhaps there’s a better way?

Algorithms: the winning recipe

Algorithms and mathematics are usually regarded by many organisations as ‘academic black magic’ and are generally overlooked as a solution for a robust IoT commercial application. As a consequence very few organisations actually take the time to analytically analyse a sensor measurement problem, and those who do invent something tend to come up with something that’s only useable in the lab. There has been a trend over the years to turn to Universities or research institutes, but once again the results are generally too  academic and are based more on getting journal publications, rather than a robust solution suitable for the market.

Our experience has been that the winning recipe centres around the balance of knowing what architectural blocks to use, and having the experience to assess what components to filter out and what components to enhance.  In some cases, this may even involve some minor modifications to the hardware in order to simplify the algorithmic solution. Unfortunately, due to the lack of investment in commercially experienced, academically strong (Masters, PhD) algorithm developers and the pressure of getting a project to the finish line, many solutions (even from reputable multi-national organisations) that we’ve seen over the years only result in a moderate increase in performance.

Despite the plethora of commercially available data analysis software, many organisations opt to do basic data analysis in Microsoft Excel, and tend to stay away from any detailed data analysis as it’s considered an unnecessary academic step that doesn’t really add any value.   This missed opportunity generally leads to problems in the future, where products need to be recalled for a ‘round of patchwork’ in order solve the so called ‘unforeseen problems’. A second disadvantage is that performance of the sensors may be only satisfactory, whereas a more detailed look may have yielded clues on how make the sensor performance good or in some cases even excellent.

 Algorithms can save the day!

 “Although many organisations regard data analysis as a waste of money, our experience and customers prove otherwise.”

Investing in detailed data analysis at the beginning of a project usually results in some good clues as to what needs to be filtered out and what needs to be enhanced in order to achieve the desired performance.   In many cases,  these valuable clues allow  experienced algorithm developers to concoct a combination of signal processing building blocks without re-designing any hardware – which is very desirable for many organisations! Our experience has shown that this fundamental first step can cut project development costs by as much as 75%, while at the same time achieving the desired smart sensor measurement performance demanded by the market.

So what does this all mean in the real world?

Returning the story of our customer, after undertaking a detailed data analysis of their sensor data, our developers were able design a suitable algorithm achieving a ±0.1% measurement accuracy from the original ±10% with only minor modifications to the hardware. This enabled the customer to present his IoT application at a trade show and go into production on time, and yes, we stayed within budget!

When looking at undertaking a NPD (new product development) it’s always tempting to take short cuts in order make the development more attractive to management or a client. I’ve lost count as to the number of times that I’ve heard,

 If we had the budget and time, of course we’d do it properly.

The problem with this, is that after doing this several times, it becomes the norm, and taking short cuts and in order to potentially save money usually leads to more problems in the long run.

 Follow the yellow brick road

For those of you who remember the Wizard of Oz, following the yellow brick road led Dorothy to Emerald city. The NPD process is the same, and is nicely described on Wiki and many other great resources, so there’s no reason to ignore it. If you can’t get the information out of clients, propose a workshop or private session, where you can discuss all requirements and get a clear picture of the clients vision/expectations. Remember that before taking on any new development, you should undertake the task of defining specifications:

  •  User specifications: a document that specifies what the user(s) expects the product (e.g. software) to be able to do.
  •  Functional specifications: are requirements that define what a system is supposed to do. A functional specification does not define the inner workings of the proposed system, and does not include the technical details of how the system function will be implemented.
  • Technical specifications: based on the Workshops, User and Functional requirements, you can finally construct the technical specifications documentation. This document(s) will contain all technical details of how the system will be implemented, and usually includes tables, equations and sketches of GUI layouts and hardware block diagrams.
  • Review: the specifications and findings with the client, and make sure that they understand what they will be getting (i.e. the deliverables).

Although it’s tempting to ignore these steps and start playing with software, following the aforementioned steps keeps you focused and almost always leads to shorter development times.

About the author: Sanjeev Sarpal is director of algorithms and analytics at Advanced Solutions Nederland BV. He holds a PhD in signal processing and has over 20 years commercial experience with the design and deployment of algorithms for smart sensor applications.

Many hi-tech companies that we have spoken with over the years have struggled to clarify this question, and almost all say that they fall into the D category. Where many say,

we’re not a University.

However, upon closer examination, it would appear that many companies are actually engaged in applied research activities but don’t even realise it or don’t like advertising it. From our experience, research activities can be broken up into two distinct categories:

  • Applied research
  • Fundamental research

Fundamental research is where Universities and research institutes are primarily focused, with the sole purpose of advancing theoretical knowledge in that field via publications and conferences.

Applied research on the other hand, takes the fundamental research findings and applies them to real world application for commercial exploitation. Many companies are constantly on the lookout for new concepts and methods to put them in front of their competition. This could take the form of implementing a radically new scientific method, or even improving their internal processes in order to reduce waste and boost profits.

The latter (also applied research) is currently where many companies are focused (mainly due to the current financial situation), but for the companies who can afford it, there is still some room for the first point.

How do they do it?

Many companies generally hire academics to help them with the translation of these concepts into products, but experience has shown that academics are not the best choice as product developers, so companies tend to use a mix of skills sets, i.e. academic and non-academics in order to get to their products to the finish line.

Academics generally prove to be invaluable at analysing problems and processes and highlighting any areas of weakness, while at the same time taking a pivotal role in the definition of technical specifications.

What about culture and egos?

There are the ego and cultural aspects to consider too, which is especially prevalent with academics. The mismatch with the academic and commercial models unfortunately leads to many focusing on providing the best solution (the academic world), rather than focusing on what the client actually needs (the commercial world).

Many academics consider management as too dumb to understand their fantastic idea.

Good leadership and emotional intelligence are needed to keep the R&D team focused on the big picture, and avoiding the department from turning into a playground for hobby projects. Many engineers and academics struggle with the concept of knowing when a product is good enough, and keep on fiddling with it until it’s perfect.

From our experience, getting clear and realistic customer requirements and then coming up with a careful plan, led by experienced developers is the only way of reaching a successful conclusion.

R, D, or both ?

Whether your company falls into R or D or a mixture of both depends on your business model, but perhaps you actually do more R than you realise. In all cases, selection of a good team that works well together is paramount for success.

So, you’ve had a great idea for a new software product, you’ve scribbled something down on paper and have a majority of the details in your head, and started coding – Right??

Not so fast!

make sure that you do your homework first!”

All successful products (software is no exception) are based on the company/coders doing their homework first. Although, it’s boring and sometimes very frustrating, find out why people would buy or even use your product, and what it is they miss or find unhandy with the existing products. Don’t send out an email and ask people, ‘why’ – most people struggle at explaining things, but use a short to the point questionnaire, focusing on the core functionalities.

Also, make sure that you have fully understood what your competition offers, by producing a sheet of pros and cons of each product. You should generally find that a competitor’s product is better at certain things, and worse others. Try to read any customer reviews too, as these can be invaluable. Collecting these pieces of information should help you fine-tune your ideas about your new product. Remember: the key to a product’s success lies in “giving them what they want”, not what you think that they want.

After completing this phase, you can objectively see if there is a business case for your product.

 Beginning the design

Great! You’ve decided that there is a business case.

Don’t start coding yet! Make sure that summarise the homework phase and use it to produce your product specifications and system architecture documentation. Get colleagues or peers to review that key concepts in order to see if you’ve missed something or if there is flaw in the concept.

 Now for the exciting bit

You can now start coding! Try and split the design phase up into manageable chunks, and don’t be afraid to leave out features for your first release.

Try and build up relationships with prospective clients by showing them your progress (posting short videos to blogs are good), and listening to any feedback that they may have. The key here is building up your mailing list of prospective clients before you release.

Sometime later, you’ve gone through the Alpha and Beta testing and are now ready to show the world what you’ve done, and start earning some money for your endeavours.

 Selling your product

Believe or not, this is actually the difficult bit!

Using the mailing list that you should have hopefully built up, you can email your prospective clients. Don’t just send one email and sit back, you need to be a little more proactive than that, and chase up leads. Remember, most people don’t care that you’ve created a new product or even share the same level of enthusiasm that you have. The solution: you need to convince them with examples, but how?

This is can get very tricky, as everybody hates receiving a, “hey, buy my stuff” email. The best advice here is don’t even mention price, focus on the ‘why’ question, i.e. just like your homework phase, why is this useful for solving their problems? Give practical examples (videos, whitepapers or even presentations) of how you can use your product in solving real world issues/problems – hopefully there will be some overlap with a problem that they’re currently facing, which arouses interest.

For many people, price becomes less important if the product solves their problem. However, money is money, and price is always important, so don’t be afraid to directly answer their question if they ask, “how much does it cost?”

 Uninterested clients

If you hear, “It’s too expensive” , “let me think about it” or “I’ve haven’t had the time to look at it yet” then the client has their serious doubts, so the practical examples (case studies) are essential. However, don’t overdo it, and remain polite at all times, even if the client is being rude.

If these hints helped you, please click on like or leave a comment below.

I recently attended a seminar on advanced instrumentation, where algorithms were heavily featured. The project pitches heavily emphasised implementation rather than analysis and design, which started an interesting discussion, and led me to think about providing some hints that we’ve successfully used over the years:

1. What do we want to achieve? This is perhaps obvious, but I’ve seen that many people do over look this step and jump into Matlab or C in order to try something out. I would urge some caution here, and suggest that you think very carefully about what you’re about to undertake before writing a single line of code. Don’t be afraid to ask your colleagues/network for advice, as their suggestions may save you months of development time. Also consider using established techniques such as, MoSCoW.

2. The specifications: After establishing the ‘big picture’, split up the specifications into ‘must haves’ and ‘nice to haves’. This may take some time to work out, but undertaking this step saves a considerable amount of time in the development process, and keeps the client in the loop. The specifications don’t need to be 100% complete at this stage (they’re always minor details to be worked out), but make sure that you’re clear about what you’re about undertake, and don’t be afraid to do some analysis or short experiments if required.

3. Algorithm design: Sketch out the algorithm’s building blocks (Visio is a good tool), and for each idea produce a short list of bullets (pros and cons) and computational complexity. This will allow you easily review each concept with your peers.

4. Test data: arrange for some test vectors data (from clients or design some of your own synthetic signals), and sketch out a simple test plan of test vectors that you aim to use in order to validate your concept.

5. Development: Depending on your programming ability, you may decide to implement in C/C++, but Matlab/Octave are very good starting points, as the dynamic data types, vector math and toolboxes give you maximum flexibility. Use the testplan and vectors that you’ve designed in step 4. However, in the case of how to best design your algorithm for streaming applications, I would say that many aspects of the algorithm can be tested with an offline (data file) approach. For a majority of our radar and audio work, we always begin with data file comprised of 10-30seconds worth of data in order to prove that the algorithm functions as expected. Subsequent implementation steps can be used to make the algorithm streaming, but bear in mind that this may take a considerable amount of time!

6. Avoid a quick fix! Depending on the complexity of your algorithm, there will be certain testvectors that degrade the performance of your algorithm or even cause it to completely fail. Allocate sometime to investigate this behaviour, but remember to prioritise the importance, and don’t spend months looking for a minor bug. Try and avoid looking for a quick fix or a patch, as they generally re-appear in the future and kick you up the backside.

7. Implementation: after verifying that your concept is correct, you can finally consider target implementation. This step couples back to the previous steps, as the algorithm complexity will have direct influence on the implementation platform and development time. Some good questions to ask yourself: Is the target platform embedded? In which case, do I need an FPGA, DSP or microcontroller? Will it be fixed point or floating point? Perhaps it will be PC based, in which case is it for Windows, Linux or Mac or for a tablet? What tools do you need in order to develop and test the algorithm?

8. Validation: Verify that your implemented algorithm works with your test vectors and that look for any difficult cases that you can find – remembering point 6.

9. Documentation: In all of the aforementioned steps, documentation is essential. Make sure that you document your results, and provide a paper trail such that a colleague can continue with your work if you get hit by a bus.