EMSA Runs a Tight Ship with Azure-Based Maritime Data Platform
Azure services are helping in investigation of maritime incidents, maritime accidents, pollution, and illegal activity by delivering reliable data to help EU Member States and the Commission.
- Expertise and solution in this case
-
Azure PaaS Solutions and Consultancy
all Expertises
Azure PaaS Solutions and Consultancy
Azure solutions can only be made efficient, scalable and secure with the right know-how and the right project methodology. Codit gives you the tools to efficiently explore new business models.
See more -
Data Connectivity Suite for Vessels
all Accelerators
Data Connectivity Suite for Vessels
Streamline communications, cut costs, keep an eye on operations and monitor your vessels, wherever they are. Make the best use of your data and gain insights to accelerate your business.
See more
The European Maritime Safety Agency (EMSA) augmented its maritime monitoring data services with a more scalable solution. The cloud-native platform ensures swift delivery of positions ship data and storage, allowing more relevant views to end-users monitoring and investigations.
The Challenge
With the volume of vessel data to drastically increase in the future, EMSA realized that its existing platform was not scalable to handle the increase in data volume. The agency needed a more agile solution to effectively manage the increasing data flow and data storage needs. However, upgrading its existing on-premises infrastructure would be too costly and time-consuming. To expand its infrastructure’s capacity and ensure scalability for the future – all while staying within the necessary timeline and budget – EMSA decided to explore cloud-native technologies for a more enhanced and efficient solution.
The European Maritime Safety Agency (EMSA) facilitates the exchange of EU vessel traffic information via their data platform IMS to EU Member States and the Commission. IMS provides ship identity details, latest positions and other status information in near-real-time for around 17,000 vessels operating in and around EU waters, helping stakeholders in a broad range of activities such as anti-piracy, maritime safety, pollution control, fisheries monitoring, law enforcement, and customs and border control.
IMS works via tracking Automatic Identification System (AIS) radio signals transmitted by ships. Tracking vessels outside the range of AIS coastal networks requires the use of satellites. Long-Range Identification and Tracking (LRIT) is a mandatory international system to track ships around the world. Emerging technologies now enable AIS signals to be received by satellite. This will progressively extend the geographical range of the AIS system. EMSA is at the forefront of exploring how this can support the European vessel traffic monitoring community.
With the volume of vessel data to increase in the future, EMSA realized that processing additional satellite messages through its existing platform would have performance issues for data delivery and storage. The agency needed a more agile solution to effectively manage the increasing data flow. However, upgrading its existing on-premises infrastructure would be too costly and time-consuming. To expand its infrastructure’s capacity and ensure scalability for the future – all while staying within the necessary timeline and budget – EMSA decided to explore cloud-native technologies for a more enhanced and efficient solution.
Codit was always flexible, available, and reactive to our requests – even if things on our side were changing day by day. ”
The Approach
The organization chose Codit to help transform its roadmap into reality, relying on the team’s in-depth expertise in Microsoft Azure technologies and proven experience in developing seamless integration solutions.
Miguel Terra Homem, Project Officer for Application Development, Maritime Digital Services, EMSA, says, “We had an overall strategy for what direction we wanted to go, for example implementing Azure Databricks, but we needed Codit’s expertise for the finer details of our roadmap in order to determine which technology we should use for the final solution and to help us put it in place.”
Codit worked with the team at EMSA to configure its existing infrastructure with a hybrid integration platform using Microsoft Azure PaaS and in a few short weeks it was possible to configure the Azure solution, which could process data three times faster than the existing on-premises infrastructure.
The first port of call was setting up the infrastructure and connecting the Automatic Identification System (AIS) data flow. The team improved the data quality checks, for visibility into potential problems.
“Each EU member state or data provider supplies its own ID for every ship. To prevent duplicate IDs in the EMSA database, the on-premises infrastructure assigns each ship a unique, EMSA-specific ID, which unfortunately caused many anomalies in the tracking information,” explained Gillian Verstraete, Codit Azure Architect “We set up the new solution to examine the ingested data, proactively identify anomalies and missing data, and alert EMSA of any issues. The solution also now has an integrated dashboard so that the team can keep an eye on all traffic coming in, going out, and any messages that could create a problem.”
The Codit team also strengthened the solution’s back-end storage with a large-capacity Microsoft Azure SQL Database, which acts as a fast, first-line data provider for the most up to date (real time maritime picture) data to end-users.
The next challenge was to connect the database to the front-end application. The team leveraged Microsoft Azure API Management and built APIs that call and retrieve the position data requested by end-users and securely visualize this data in maps and other visual formats. Security was a prime consideration when developing this solution.
“Whenever a user requests information, such as a ship’s position, the solution must know if that user is authorized to see that particular ship,” says Joachim De Roissart, Codit Business Consultant and Project Manager, “We ensured this security in a way that doesn’t slow down the process for users, seamlessly integrating the solution with the EMSA IDMV2, a service that exposes the EMSA users and user roles, along with our data stores.”
Once the initial infrastructure was in place and the remaining data streams connected, Codit revised the solution’s resampling strategy to offer richer data views to end users, allowing visualization of extended vessels’ tracks.
“Every 6 seconds, the EMSA system gathers a massive amount of data points from vessels’ movements,” says Verstraete, “Logical resampling is essential to ensure that end users can see the full picture of the information that they need. To provide a richer view, we redistributed the gathered information over different partitions and restructured the data so that it’s rearranged to show the 10,000 most important positions from the requested range of time. Additionally, this sampling strategy is configurable so that it can change in line with any future increase or decrease in that limit.”
With its cloud-based solution in place, EMSA saw an opportunity to further enhance its storage for the longer term and explore big data through Microsoft Azure Databricks, a highly performant technology for running queries on a large amount of data. Codit integrated the big-data technology into the new solution, storing every piece of position data inside Azure Databricks, from where EMSA can query the data through workbooks and front-end applications can retrieve the information through APIs.
The Results
The resulting solution reinforces EMSA’s position as a technology leader in the maritime sector by supporting its stakeholders with global coverage maritime data.
Miguel Terra Homem, Project Officer for Application Development, Maritime Digital Services, EMSA, says, “Thanks to this project and collaboration, EMSA is one of the first EU agencies to develop a completely cloud-focused application.”
The high-performant Azure-based platform has laid the foundation for this next project, providing EMSA with a deeper understanding of cloud-native benefits, which the organization is using to define its ideal approach and strategy for cloud technologies moving forward.
Terra Homem adds, “Our collaboration with Codit has equipped us with knowledge, through which we can continue to discover more about cloud-native technologies and to define our strategy for leveraging them in the future.”
A word from the client
“Our collaboration with Codit has equipped us with knowledge, through which we can continue to discover more about cloud-native technologies and to define our strategy for leveraging them in the future.”
Miguel Terra Homem Project Officer for Application Development, Maritime Digital Services @ EMSA