Spicule - Data Processing Experts

Network Switch

Real-Time Analytics and Data Processing for IoT and Device Metrics

A sensor can tell you when things are about to go wrong, but if your data processing platform can’t query, aggregate and search and filter that information as soon as the sensor transmits it, you’ll still be losing a lot of valuable time.

At Spicule, we have the knowledge and expertise to design, implement and support state-of-the-art data processing platforms which will seamlessly capture and analyse all the complex data your business relies upon. Here’s how we used Apache Druid to put the wind back in the energy sector’s sails.

The Challenge

The UK is a world leader in offshore wind energy. It already has an offshore wind turbine infrastructure capable of generating 7.5GW – more than any other country in the world – and, according to some estimates, the energy produced by offshore wind turbines could provide us with as much as a third of our electricity by 2030. But, regardless of all the sustainable environmentally-friendly benefits wind energy offers, the wind towers themselves are still just a gigantic set of complex moving parts. If one of those parts fails, the wind farm’s owners could have an extremely expensive problem on their hands – not to mention the downtime incurred before the wind tower could begin operating again.

Luckily, there are high-tech fail-safes to prevent that from happening – or at least to give the owners a heads-up before disaster strikes. A phenomenal amount of data is received from each wind tower every second: data streamed from sensors on the blades and sensors on the turbines, sensors that monitor the amount of electricity being produced, and sensors that keep track of the weather and the effects its having on the wind tower’s performance. What all that data will do (among many other things) is let the technicians predict when a failure is likely to occur. Armed with that information, they can get the turbine fixed in good time. But understanding which turbines are close to a failure state means monitoring that enormous amount of data on a real-time basis and being able to create alerts so you’ll know exactly when to take action.

Working on New Tech

Our challenge was to design and build a platform that could a) store the massive quantities of data each wind tower was producing on a second-by-second basis, b) stream and analyse the data in real-time while simultaneously comparing it to historical trends to assess changes and degradations in the wind tower’s functionality c) generate the results in a clearly understandable format, including incorporating alerts to notify technicians when anomalies were appearing.  

OUR SOLUTION

Apache Druid is a three-fold analytics engine that combines the best of timeseries databases, column-oriented analytic databases, and search indexing systems. That means it can instantly onboard all the data the wind tower is streaming into the platform and – in real-time – filter it, query it, and simultaneously perform all the numerical aggregations necessary to a) ensure the wind tower is operating correctly, and b) historically compare the incoming data to all the previously received data so that any negative trends will be immediately recognised and flagged to the wind tower’s technicians. We also incorporated a rising series of alerts according to the requirements of the wind tower’s owners, so they would instantly be aware of the level of urgency required. Druid’s fault-tolerant architecture also guarantees that no data or service will be lost if a server fails.

We also designed a custom front-end interface which was simple-to-use and provided a variety of different reporting options including charts, graphs and tables. Because of the complexity of the data, which could include anything from how the various parts of the tower were performing to how changes in weather patterns had affected the amount of electricity the turbine was generating, the reports had to be easily tailored to meet the needs of the audience receiving them.

green meadow with Wind turbines generating electricity

As a result, Druid was a tailor-made solution for our client. It gives them the ability to store, stream, analyse and quantify limitless amounts of data, and instantly be able to partition that data to lock down the various results they’re looking for. On a very basic level, the platform we provided has already paid for itself by saving them money in maintenance costs: rather than sending out an engineer to a single wind tower when something is going wrong, the platform’s early warning alerts mean the engineer can attend to issues at the other wind towers during the same visit. All in all, Spicule’s solution has provided instant actionable insights that let the wind farms owners make immediate good decisions about their clean energy-generating multi-million-pound investment.

APACHE DRUID: THE PERFECT DATA ANALYSIS SOLUTION FOR IoT AND DEVICE METRICS

Apache Druid


Druid’s three-fold analytics makes it the perfect platform for any kind of intensive data streaming – whether the data’s being received from the sensors on a wind turbine, the sensors on a car, or the sensors monitoring a piece of farm machinery. In fact, Druid’s applications in the arena of IoT and Device Metrics are virtually limitless.

Whatever you require, we’ll work with you to develop a solution that meets your needs and helps you find the answers to your questions.

WHAT ARE THE BENEFITS OF DRUID FOR IoT AND DEVICE METRICS?

WORK WITH REAL-TIME ANALYTICS EXPERTS

Spicule have offices in Norwich and Cambridge, and we work with clients in the UK and around the world. We pride ourselves on designing and delivering the best possible solution to meet each client’s specific need, and we are able to provide 24/7 on-site and remote support whenever required.

Whatever your real-time data management problem may be, we’ll work hard to make sure you find the results you’re looking for.

Let's Start Talking

Request a Callback