Providing a foundation for the development and distribution of new geospatial analytics products
The Problem
As the world’s premier provider of satellite-based AIS data, exactEarth delivers the latest positions for hundreds of thousands of vessels to their vast customer base. By using GA-CCRi’s Optix platform, exactEarth can monetize new data products by selling access to these products to their customers, including customized feeds and analytics, while reducing the latency in their data processing pipeline from minutes (and sometimes hours) to seconds / milliseconds. The Optix platform provides a foundation for the distribution of their products and provides a path for growth in the future of derivative geospatial analytics.
exactEarth has been collecting and aggregating AIS data from maritime vessels globally since 2011. In 2017, exactEarth and GA-CCRi teamed up to optimize the ETL (Extract, Transform, Load) process using distributed power available in cloud computing. This research yielded massive gains which led to exactEarth contracting GA-CCRi to provide a new platform for their data and products. They had three key goals: (1) performant and resilient cloud storage of all data, (2) a platform that could distribute these products to their customers with low latency and (3) flexibility to define and create new products from new data sources.
Our approach
With the successful prototype in hand, GA-CCRi worked with exactEarth to refine the requirements for the production environment. Once we had a clear understanding and agreement of those requirements, GA-CCRi began work. GA-CCRi leveraged the Optix platform (an enterprise offering of GeoMesa) to ensure production reliability, repeatability and maintainability. GA-CCRi deployed Optix.Geo, which leverages Apache HBase for the distributed datastore, and configured it for the existing exactEarth products and some new ones. The Optix platform’s use of open standards let GA-CCRi leverage Open Geospatial Consortium standard APIs to enable exactEarth to integrate the Optix back end with their existing front end.
Results
The resulting system significantly improved exactEarth’s offering. By leveraging Optix as the back end datastore, exactEarth was able to reduce the processing time necessary to ingest data and make it available to their customers. In some cases, the new platform had data about a vessel hours ahead of the legacy platform. Query performance (both in speed returned and concurrency) also improved, enabling more users to get results back quicker. On the operational costs side, the use of cloud computing resources enabled exactEarth to scale back their on-premise infrastructure. Since that initial deployment, GA-CCRi and exactEarth have demonstrated the platform’s flexibility by adding several new products.
New insights
Real-time maritime data has many use cases we would love to explore with you. Since beginning with exactEarth, we have used maritime data to determine economic impacts of weather events, build global economic indicators, steer ships around hurricanes, and build predictions of COVID models. Since so much of the world’s trade relies on maritime transport, understanding maritime traffic unlocks answers to previously unanswered questions. We would love to explore your problem and collaborate on how we can leverage these methods, technologies and data to find a solution.
Interested in how maritime data can help you? Have a massive IoT dataset to analyze?
