How Flox is Keeping Animals from Harm using AI and IoT
Flox needed a robust system to handle vast amounts of unstructured data, including photos and sensor data along with an efficient way to deploy and update AI models across a fleet of devices in the field.
The Golioth platform along with Pipelines and advanced OTA features allowed Flox to manage both devices and AI models which in turn is helping them build a strong data set for advanced wildlife management with IoT.
Golioth allows Flox to deliver firmware updates and AI models to remote devices via a Zephyr Application on an nRF-9160. Flox uses Golioth Pipelines to send data from devices back to the model to develop it further.
People and Animals Can Coexist Safely with the Right Technology
Flox needed a robust system to handle vast amounts of unstructured data, including image and sensor data. Efficient deployment and updating of AI models across a fleet of devices required a seamless over-the-air (OTA) update mechanism. Additionally, performing AI inference on the edge and continuously improving models with collected data were significant hurdles. Lastly, they needed a scalable solution that could adapt to their evolving embedded systems.
“Our mission is to enable animals and humans to coexist. Specifically keeping animals away from where they could potentially be harmed."
Flox uses a combination of IoT and AI to protect animals from dangerous situations. Their algorithm detects the animal type and then produces a sound that’s specific to that animal's location and other metadata. The emitted sound repels the animal from the location. Becklin explains, “We have a motion detector that detects motion on the edge device and then a camera is triggered and takes a photo. We then process that on device to see what type of animal it is using a model specifically trained for that location. Based on the result we have an algorithm deciding what unique acoustic signal to play back to repel that animal, and all of this is happening on the edge device in real time.”
“We are currently targeting the northern hemisphere primarily so we're protecting moose, elk, deer, wild boars, wolves and birds. A lot of birds,” Becklin explains. The Flox deterrence model can detect an animal, like geese at an airport, and use an acoustic signal to redirect the animal to a safe space.
Each of these edge devices has real-time detection artificial intelligence responsiveness and as a result, it also has network connectivity and interactivity with the cloud. “It's critical for us that these units are up to date. Obviously we're updating software but we also need to make sure we have the latest model on there that's trained on the latest dataset, especially if you're entering a new region. There'll be a lot of training data in the beginning to get the unit up to speed so being able to deploy quickly and securely to these units out in the field is critical to us,” Becklin shares.
Flox wants to avoid managing devices manually, which poses a huge challenge in keeping devices up to date, especially in low-latency areas where network connectivity is not strong. Because CoAP ensures message delivery, even in cases of limited network connectivity or device power, Flox determined that the flexibility of a Zephyr and CoAP app-enabled board would fit the bill for their use case. However, they needed to build the infrastructure to deliver firmware updates and secure devices. “We obviously tried doing this by ourselves, and we had ideas about doing this, but after speaking to a friend who has a lot of systems experience, he said you should check out Golioth or someone like them to solve this problem because it's a tough problem to manage,” Becklin confessed.
“Being good at detecting and repelling is what our customers need. They want to be able to keep areas safe from animals and keep animals safe. So for us being reliable is a big big thing. People are potentially putting these units up in really remote areas and they might not be able to service them that often, so for us to deliver the hardware software solution that's very reliable with as much uptime as possible is critical,” Becklin explains.
Remote AI Model Management for MCUs Unlocked
To overcome these challenges, Flox turned to Golioth's AI-ready IoT infrastructure. Becklin explained, "We needed a solution that could handle the complexity of our data and provide a flexible platform for deploying and managing AI models. Golioth's infrastructure fit the bill perfectly." Golioth provides flexible data pipelines, enabling Flox to stream and process large volumes of unstructured data efficiently. The Golioth OTA update feature allows Flox to deploy and manage AI models across their fleet with ease, ensuring all devices receive necessary updates without downtime. The infrastructure supports both edge and cloud inference, providing Flox with the flexibility to run AI models on-device for real-time decision-making or in the cloud using more complex models. Golioth also offers a scalable and secure platform that can grow with Flox's needs, ensuring secure data transmission and handling increasing data volumes and processing demands.
With tens of thousands of hardware units out in the wild at risk, Flox needed a reliable solution for updating their devices and models. “These units will be positioned way out in the forest in very remote areas so it's not always a good connection. Finding a reliable solution was priority number one in this and we either have to do it ourselves or find someone else to provide it and Golioth has been really good for us so far. There’s not a lot of friction,” Becklin noted. Flox relies on an embedded system. They use the OpenMV RT1062 MCU that’s connected to an nRF9160 board to communicate with Golioth. Becklin commented, “Each system comes with constraints and obviously here we have constraints. We have bandwidth constraints and battery life as well. But as far as the OpenMV platform goes, I mean, it's amazing what they can actually pack into these small units now. The camera and onboard compute has been a pleasant surprise to me but also an enabler. We could not do this on board maybe five years ago, but now you can actually do this in a pretty cheap solution. And while our target battery life is at least six months, we believe we can get even more than that. So yeah, I would say that constraints–while obviously challenging–create better technology because constraints help you to have to actually dig in and solve the problem in a more efficient way.”
The Golioth SDK enables Flox to use Zephyr to send messages over CoAP, known to be more reliable for sending messages in remote areas with low connectivity–a crucial part of their solution. Data is passed through Golioth into an Amazon S3 bucket to build a large dataset for creating a more accurate data model that’s hyper-localized for each deployed location.
“Golioth takes away a lot of work for us, to manage the networking part, so we can focus on what we're good at,” Becklin continued, “We can focus on the platform part and using Golioth and the SDK is a way for us to easily communicate with the platform in both directions, whether it's doing over-the-air updates or pushing data up to the cloud.”
Building a Data Loop for a Powerful Data Model
Golioth allows Flox to deliver firmware updates and models to a device, and then ultimately, Golioth Pipelines can send data from devices back to the cloud to develop the AI models further. “Besides doing the over-the-air updates, the communication between each unit and our platform is key. We might not do that real time, we might be batching that up, but we need to get the data back onto the platform so we can retrain the models,” Becklin stated.
“I think for most people cellular right now seems really good. You move around town, you move around wherever you usually go and you have good connectivity, but in reality if you go out in farmland or forest or remote areas it's not that good. We have to take that into consideration.” Golioth has been able to deliver reliability and provides Flox with the feedback mechanism to know whether data has actually been delivered or whether an acoustic signal has played.
The images we capture get pushed to a bucket for storage and then used to retrain the model if needed. We also perform analytics on all the data to let our clients know how they're how their edge devices are doing,” explains Becklin.
Data Delivered, Models Trained (and Re-trained)
The implementation process involved a collaborative effort between Flox and Golioth. Golioth helped configure data pipelines to stream and process data from Flox's systems efficiently. Koray Amico Kulbay, Robotics and Autonomous Systems Engineer at Flox noted, "The support from Golioth was exceptional. They assisted us in establishing communication between our two boards (OpenMV RT1062 and nRF9160) and integrating with the Golioth platform. This support enabled us to concentrate on deploying our AI models on our edge devices." Flox utilized Golioth's OTA update feature to deploy AI models across their test fleet, ensuring all devices had the latest models. With Golioth's infrastructure in place, Flox could continuously collect data and improve their AI models, leading to better performance and more accurate decision-making by their fleet.
“The support from Golioth was exceptional."
The results of this partnership were significant. Flox achieved enhanced efficiency through real-time data processing and analysis, leading to more efficient operations and faster response times. Continuous data streaming and real-time training allows Flox to refine their AI models, for better accuracy and reliability. Golioth’s OTA updates simplified the deployment and management of AI models, reducing downtime and maintenance efforts. Golioth's scalable infrastructure ensured FLOX could easily expand their operations without worrying about data management or processing bottlenecks and dual support for edge and cloud inference provides Flox with flexibility.
In conclusion, integrating Golioth's AI-ready IoT infrastructure has been transformative for Flox. By addressing the challenges of data management, model deployment, and real-time inference, Flox achieved greater efficiency, scalability, and flexibility. Amico Kulbay summed up the experience, saying, "Golioth's comprehensive solution has placed us at the forefront of wildlife management technology. Their embedded AI infrastructure equips us to meet the demands of the future." Golioth's robust and scalable solutions for integrating AI with IoT devices empower businesses like Flox Robotics to leverage AI for better decision-making and operational efficiency.
.svg)
