Imagine that you need to create an environmental and statistical view of the largest port of Europe, Rotterdam. At your disposal is a variety of (real-time) data sources.
This presentation is a walk-through on how we as a team of engineers and scientists combined the different data formats its attributes and infrastructure to create the required views. This resulted in an architecture based on Apache Kafka Stream processing and utilizing Time-series and Document stores incorporated in a Kubernetes cluster.
Harbourmaster data (Cargo, docks, etc.) and Radar, AIS (GPS, Ship information) are the main sources. They are processed in realtime so we can present (near real-time) dashboards about Co2 emissions, ship movements, passages, and visits.
The efforts from the Port Of Rotterdam are a part of their ambition to become the smartest, efficient and most environmental friendly port of the world.