Spicule - Data Processing Experts

Mobile App Development

Apache Druid for Real-Time Monitoring of Application Performance

Mobile apps are big business. From checking the headlines, booking a taxi, ordering a pizza or playing a game that catapults a deeply unhappy cartoon bird into deep space, mobile apps are the perfect way to attract customers, sell services, and keep the money rolling in from lucrative in-app purchases. But only if your app functions reliably. If it falls over, crashes, or stops your user from doing what they want to do you’ve got a big problem in your hands.

At Spicule, we have the experience and expertise to build, implement and support state-of-the-art data processing platforms that help you monitor your app’s performance, track your traffic in real-time and anticipate and solve problems to ensure your satisfied customers keep coming back for more.

The Challenge

When customers use your app on their mobile phone, it streams endless amounts of data back to your servers for processing and analysis. In fact, even when they’re not physically interacting with your app, the app might still be sending back vital information it’s important for you to know. On a very basic level your app needs to chart their activity, predict their activity, and deliver such a fulfilling experience they’ll use your app again and again, make more purchases, and recommend it to others.

On a more advanced and subtle level, your app will also be collecting behind-the-scenes information that tells you more about who your user is including their geographical location and what they’re doing when your app isn’t being used. Your app will also be sending back technical data it’s vital for you to know if it’s going to perform optimally, i.e. errors that are preventing the app from working correctly, and spikes of activity that may be slowing it down or making it crash completely. That’s an enormous amount of incoming data for your servers to cope with. If you can’t stream it and analyse it in real-time so you can correct errors and anticipate problems before they arise, your app will perform badly and your user will soon be taking their business to your competitor. You’ll lose customers, sales, and potential advertising revenue.

In the case of a mobile news app, there’s an added level of difficulty. News apps don’t have the attraction value of a shopping app or a gaming app and, unless they’re one of the big-name news outlets, they’ll probably have a relatively small base of daily or active users. That means there will be lots of time when their app traffic remains steady and manageable. But what happens when a major news story unexpectedly breaks and the app needs to rapidly increase their resources and server capabilities?

Our challenge was to design and build a platform that could a) store, stream and analyse all incoming real-time data to track mobile app usage, monitor site reliability, troubleshoot bottlenecks and diagnose software crashes, and b) be instantly scalable up or down, with no downtime, to save resources when demand is low but to meet rising capacity when demand unexpectedly increases.

Our Solution

If your mobile news app only has two tiny servers under the hood it’s not going to last very long when a headline goes viral. That’s just one of the reasons why Apache Druid is a fool proof solution to this kind of problem, because Druid will scale as your app grows. In other words, if your app is happily coping with a few thousand users and then it’s suddenly hit with twenty million downloads, Druid will allow for that increased activity and provide you with all the analytics you need at the far end, with no negative impact on Druid’s performance. Druid can be scaled up or down simply by adding or removing servers and it will automatically rebalance. And, if there’s a server failure, Druid’s fault-tolerant architecture will instantly re-route around it without affecting your app’s performance.

As far as the other aspects of application performance management are concerned, Druid has very few limits to the volume, complexity and throughput of data it can stream and analyse in real-time. It can rapidly analyse application events with thousands of attributes and compute complex metrics on load, performance and usage, as well as performing fast ad-hoc slicing and dicing on any set of attributes to help you dig even deeper to the root of an issue. For our mobile news client, we also created dashboards and alerts to notify them as soon as Druid’s real-time analysis flagged an application performance problem, enabling their engineers to fix the problem before it became well… a problem!

web site codes on computer monitor

As a result, Druid has given our client the peace-of-mind that their app is delivering everything their customers are looking for: a user experience which is fast and reliable even when a sudden increase in demand takes their network by surprise. Druid’s scalability has also made it an extremely cost-effective solution because our client will never be charged for resources they’re not using but they know the resources will be instantly available if they ever need them.


Apache Druid

Druid’s three-fold analytics makes it the perfect platform for application performance management – that’s why global brands like eBay, Airbnb and Paypal already rely on Druid to keep their customers happy and their operations running smoothly.

Whatever your business requires, we’ll work with you to develop a performance management solution that meets your needs and helps you find the answers to your questions.



Spicule have offices in Norwich and Cambridge, and we work with clients in the UK and around the world. We pride ourselves on designing and delivering the best possible solution to meet each client’s specific need, and we are able to provide 24/7 on-site and remote support whenever required.

Whatever your real-time data management problem may be, we’ll work hard to make sure you find the results you’re looking for.

Let's Start Talking

Request a Callback