Our oceans and seas are a substantive source of economic activity, and it is within this context that ESA InCubed has released a thematic call dedicated to the globally significant maritime sector. Open until March of this year, the campaign focuses on developing innovative products and services using Earth observation for areas such as maritime security, transportation, pollution and fisheries.
As part of its remit to co-fund European commercial development across the entire Earth observation (EO) value chain, the ESA InCubed programme periodically issues thematic calls with particular focus areas. Last year saw the launch of a Cultural and Natural Heritage call, while the latest campaign is on the maritime sector.
The world’s oceans are home to a major component of the global economy, not only in terms of trade and transportation, but also energy and food production. Around 90% of international trade occurs through maritime routes, and some 50 000 merchant vessels are currently in operation across the globe. Europe’s blue economy is worth nearly €650 billion per annum, and the continent has some of the world’s largest harbours and boasts a number of major shipping companies. But our oceans are also subject to sizeable societal issues and challenges, including marine plastic pollution, climate change effects, overfishing, security issues and illegal migration.
The InCubed Maritime Call seeks to address these challenges by inviting participants to submit ideas for innovative EO solutions that will help to ensure a green and sustainable maritime sector. The call divides the sector into specific areas of interest:
Security and emergency
Finance and Insurance
Renewable Energy
Construction
Transportation
Pollution monitoring
Fisheries and acquaculture
Further details on these areas, along with the global and European maritime landscapes and how the sector can derive enormous benefits from EO services, can be found in two highly valuable new reports commissioned by ESA:
These reports give a wealth of useful information, helping applicants from both within and outside the maritime sector to submit ideas for the call under the seven areas of interest. Responses are welcome from individuals, companies and academia in InCubed participating states.
Idea pitches will be assessed based on criteria such as the level of innovation, the credibility of the business opportunity and the quality and completeness of the proposal.
Details of the call can be found on ideas.esa.int, and the closing date for entries is 29 March 2024.
Φ-lab contributed to the recent United Nations COP 28 Climate Change Summit in Dubai, sharing ESA’s experience in tackling climate issues through AI-driven modelling and analysis of Earth observation (EO) data. The team played a major role in sessions on climate adaptation in the areas of health monitoring and risk transfer mechanisms.
COP 28, the 28th meeting of the Conference of the Parties, is the UN climate change forum that took place between 30 November and 13 December in Dubai. With over 70 000 delegates from around 200 countries, COP 28 was a truly worldwide gathering that included the first ever global stocktake on climate change.
ESA has or is developing a number of EO climate-related activities that provide valuable insight to scientists, the private sector and policy makers alike, notably the RECCAP-2 and World Emission projects and the Copernicus Sentinel-5P and Anthropogenic Carbon Dioxide Monitoring missions. Within ESA’s Directorate of Earth Observation, Φ-lab has established a formidable reputation as a centre of excellence for transformative innovation, facilitating groundbreaking EO research through enabling technologies such as machine learning and edge and quantum computing.
One of the objectives of the COP global stocktake is to evaluate our collective progress towards climate adaptation, a theme that was taken up in many sessions at the conference, including two that benefited from Φ-lab’s expertise. Satellite Technologies for Climate Adaptation and Health was organised by the ESA Climate Office and chaired and led by Φ-lab AI Applications Lead Rochelle Schneider, a researcher with extensive experience in health monitoring and prediction in projects such as UNICEF-Φ-lab dengue fever modelling.
Φ-lab’s Rochelle Schneider (left) with fellow participants in the ESA booth at COP 28
Rochelle’s talk began by citing World Health Organisation (WHO) statistics on predicted future climate-related fatalities and current levels of poor air quality. The discussions then went into detail on the role of satellites in providing a fundamental source of environmental hazard and air quality data. When these data products are fed into machine learning models, the result is high-resolution air pollution maps of cities and countries that both highlight areas surpassing WHO guideline limits and enable the assessment of human exposure to such pollution.
In their concluding remarks, Rochelle and fellow speaker Maarten Kappelle of UNEP’s Science Division underlined the importance of data scientists and a broad spectrum of experts joining forces to improve how society combats climate change. In this way these multidisciplinary teams will be able to inform adaptation strategies and directly contribute to public health policies.
Rochelle also took part in the Collaborative Futures: Bridging Gaps with AI and EO in Climate Risk Management and Adaptation session. Here the panellists discussed the potential for AI and data from space to aid adaptation through insurance and financial mechanisms, with particular emphasis on the part that the private sector has to play in providing new solutions for climate risk management. In fact this is a strategic development area for the EO industry, which ESA is addressing and supporting via the InCubed commercialisation programme.
The session included several examples of how AI modelling can help overcome challenges for the risk transfer industry in data gaps and accessibility, leading to the possibility of innovative insurance products to help protect exposed communities.
Both panellists and audience were surprised to hear that AI onboard satellites, rather than being a future development, is already a reality at ESA. Rochelle gave examples of the Agency’s extensive work in this area, including Φ-lab-led projects like RaVAEn, FDL NIO and Cognitive Cloud Computing in Space.
“It’s clear that useful climate monitoring is not possible without the global, near-real-time measurement that Earth observation affords,” Rochelle remarked in summing up her participation. “But as these discussions at COP 28 have shown, when we add AI to the mix, we can create a pipeline that takes raw data, processes it intelligently and creates models of trends, empowering governments, institutions and commercial enterprises in their climate-action decision making.”
With support and guidance from ESA Φ-lab, the Earth Systems Predictability (ESP) Forum earlier this year brought together over 150 top experts to explore how Earth observation (EO) in concert with artificial intelligence (AI) and other emerging technologies can help us make decisions on the future of our planet. The final report, which sets forth a far-reaching vision for ESP and its development, is now available.
ESA and the European Commission’s recent announcement on joint efforts to address climate change highlights once again the crucial importance of EO in advancing our understanding of the Earth’s systems and their impact on global warming. ESA has many climate initiatives already in progress, including the Space for a Green Future Accelerators and the Climate Change Initiative, along with developing comprehensive modelling of the planet through Digital Twin Earth and its pivotal role in DestinE. The ESP Forum adds a significant piece to the jigsaw by seeking to bridge the gap between the knowledge gained through climate change sensing and modelling on the one hand and the required climate action and decision-making on the other.
ESP is described as an emerging discipline that combines elements from EO, AI, simulation, visualisation and decision support. It aims to build on Earth digital twins and provide an accessible, distributed platform that is integrated with everyday life. The forum was held in May of this year, organised by Trillium Technologies, ESA Φ-lab and Oxford University. With its solid experience in groundbreaking innovation in AI, machine learning and digital twinning based on EO assets, Φ-lab was well placed to offer a guiding hand on the proceedings and reporting.
The forum was the result of months of extensive preparation, with the specialist participants engaged in shaping the agenda, the open research questions and the suggested innovation approaches to be discussed and developed during the three-day event.
After releasing provisional conclusions from the event in July, the full ESP Forum report has now been published and can be viewed directly at the foot of the page, or online here.
The report affirms that the foundations of ESP rest on fundamental building blocks such as data from Earth and space, trustworthy and generative AI, AI-based weather and climate models and scaled systems. A detailed deployment vision is set out in the text, drawing on existing work in a variety of disciplines comprising Earth digital twinning, large-scale simulations, finance, environmental, social, and corporate governance (ESG), social return on investment, and decision intelligence.
The final section of the report is an exhaustive ESP roadmap that seeks to provide context for the reader to take action. The roadmap includes technical elements linked to areas such as digital twin Earth visualisation, mobile interface technology, onboard ML, MLOps for ESP, and anthropospheric models and data.
Other key findings include:
ESP should be envisioned as a ubiquitous decision-support technology, spanning orders-of-magnitude in scale, from lightweight software modules running on mobile devices, to sophisticated cloud-native Earth digital twins
Compelling visualisations and interactive interfaces are essential components of ESP technology, helping to craft narratives around planetary stewardship and support confident mission-critical decisions
ESP must be developed from the ground-up in close cooperation with the people who will use the technology to make everyday decisions.
The global company Spire, which specialises in using continuous global monitoring to track aircraft, ships and weather patterns using a large constellation of CubeSats, is now an ESA Third Party Mission.
As Earth’s population continues to grow, acquiring high quality data to help to predict the movement of the world’s resources is a priority. A specialist in this field, providing radio frequency datasets in near real-time, Spire Global recently announced that it has officially joined ESA’s prestigious Earthnet Third Party Mission (TPM) programme. The data portfolio that Spire provides will include GNSS-RO polarimetric data (PRO) from the ESA InCubed co-funded PROGRES activity.
MANTIS, the first satellite mission to be supported from concept to liftoff by ESA’s Earth Observation InCubed programme, has been launched on a SpaceX Falcon 9 rocket. MANTIS carries a high-resolution multispectral camera coupled with a powerful AI processing unit. Intuition-1 was also launched on the same rocket and will similarly demonstrate the advantages of onboard AI capabilities, in this case in tandem with a hyperspectral imager. The satellite’s machine learning algorithms were developed under the ESA-funded Genesis project.
The two satellites lifted off from the Vandenberg Space Force Base in California, US, at 18:49 GMT (19:49 CET, 10:49 PST) on 11 November.
Hot on the heels of the first Earth Observation Commercialisation Forum, now is a good time to take a look at the all-embracing support that ESA gives to the commercial sector in Europe. With funding programmes, business guidance for companies, and multiyear contracts, ESA provides a vital springboard for continued growth in commercial Earth observation.
The global Earth observation data and service market is estimated to be worth around €2 billion every year and predicted to increase to €7–9 billion by 2032. But despite this wealth of opportunity at a global level, companies in the commercial Earth observation sector in Europe may find getting off the ground difficult owing to the complexity of navigating the investment landscape.
As part of its joint initiative with ESA Φ-lab, UNICEF appointed two researchers to work on a project that forms part of the UN Secretary-General’s Digital Cooperation Roadmap. The project maps current access to electricity and the Internet for schools around the world, in support of an ambitious UNICEF target to provide connectivity for every child by 2030.
In an effort to close the digital gap of countries with no Internet access and children without connectivity at home, UNICEF and the International Telecommunication Union created Giga, a global initiative whose goal is to understand the current connectivity status. This is where remote sensing and ESA Φ-lab come in—with its expertise in enhancing the value of satellite-derived data through artificial intelligence. The team at Φ-lab is employing Artificial Intelligence (AI) to analyse multi-modal Earth Observation and terrestrial datasets.
ESA’s first Earth Observation Commercialisation Forum took place at ESA Headquarters in Paris on 30–31 October 2023. The event saw investors, institutions, entrepreneurs and different-sized companies from the Earth observation sector come together to discuss the commercial potential and challenges of Earth observation. Revisit the event by watching the streaming replay.
In his opening address, ESA’s Director General, Josef Aschbacher, said, “Advancing commercial space in Europe is one of the key components of Agenda 2025, which I set as an ambitious vision when I took over the position of ESA Director General.
“The fast-growing Earth observation sector has an abundance of possibilities that private businesses can capitalise on, from satellites and ground infrastructure through to value-added services that address real-world needs with information from space.”
At a recent showcase event, FDL Europe presented the fruits of this year’s summer research sprint. With technical and financial support from ESA Φ-lab, the team demonstrated how digital twins and foundation models can enable solutions in the areas of early detection of solar events, data gathering for disaster management, and facilitating downstream applications based on Synthetic Aperture Radar (SAR) data.
The Frontier Development Lab (FDL) Europe is a collaboration that works with ESA and a number of commercial partners such as Google Cloud and NVIDIA. The initiative focuses on accelerated research that applies artificial intelligence (AI) to space science in order to push frontiers and develop new tools that help solve major human challenges.
Φ-lab spearheads the collaboration within ESA, giving strategic and expert input into FDL Europe’s annual development cycle and individual projects. Each year, FDL Europe puts machine learning specialists together with Earth and planetary scientists for an intensive summer research sprint, drawing on bold thinking and rapid iteration and prototyping to produce stimulating outcomes.
For the 2023 run, the FDL Europe experts’ focus was heavily driven by the power of emerging AI technologies, as Head of the Φ-lab Explore Office Pierre Philippe Mathieu explains: “Foundation and large language models are examples of exciting new methodologies that are radically changing the way we – and machines – retrieve and interface with data. Part of FDL Europe’s remit is to apply these techniques safely and ethically for the benefit of humanity, and the three challenges this year were selected for their potentially wide-ranging impact in protecting our planet.”
Challenge 1: creating an SSA twin for space weather
The first FDL Europe team was tasked with creating a Space weather Situational Awareness (SSA) twin for solar events. Early detection of space weather is crucial for safeguarding both terrestrial and orbiting infrastructure, an endeavour that ESA’s Vigil mission will take on by providing an unprecedented view of the sun from its orbit at the fifth Lagrange point. But understanding the propagation of coronal mass ejections (CMEs) is complex, and Vigil’s distance from the Earth will limit its connectivity.
The team on this challenge developed a machine learning pipeline that leverages the advantages of edge computing in space with a CME-aware compression model, significantly reducing the downlink bandwidth requirements and the delay in alerting dangerous events. On the ground, their SSA twin can provide a three-dimensional reconstruction of solar activity through a combination of physics-based equations and Neural Radiance Fields, a deep learning method for synthesising 3D views of complex scenes by optimising an underlying continuous volumetric scene function using a sparse set of 2D input views. The result amply attests to the power of onboard intelligence paired with digital models.
Challenge 2: building a foundation model adaptor for disasters
As the recent growth in AI chatbots has shown, Large Language Models (LLMs) are having an enormous effect on society as a whole, but their ability to act as a knowledge store, summarise a large array of inputs and adapt to unseen tasks is also profoundly influencing software development. The second FDL team took on the challenge of harnessing the possibilities of LLMs for collating and interpreting information on current and past disasters, providing reliable and detailed reporting to authorities.
The team chose to focus on floods as one of the most prevalent and damaging classes of extreme weather events. While flood relief decision-making can often be hampered by the time and resources required to bring all the relevant insights together, the FDL team’s foundation model combines Earth observation (EO) data, flood prediction and mapping models, human-in-the-loop inputs and weather monitoring systems to create accurate, instantaneous flood reports through natural language processing. A prototype of the interface can be seen at floodbrain.com.
Challenge 3: a foundation model for analysing SAR EO data
With its all-weather capabilities, SAR remote sensing is an essential tool for Earth observation. However, in addition to the extensive preprocessing required, SAR data is difficult for deep learning to adapt to, and existing deep learning models struggle both to remain robust and to function reliably over new regions or different time frames.
The third FDL Europe team sought to develop SAR foundation models that could be generalised to a multitude of analysis tasks. The resulting pipeline was flexible enough to handle the huge breadth of data from SAR imagery and was also fine-tuned on a variety of labelled downstream tasks, including processing complex data and extending models across time and space domains. As a forerunner of a full foundation model for SAR, this pipeline has pioneered the way for a potentially transformative asset for the data-science community.
Giuseppe Mandorlo is the Project Manager for Vigil at ESA and so naturally was following the development of Challenge 1 particularly closely: “The results achieved by the FDL Europe Heliophysics team are extremely impressive – even more so when you take into account the short duration of the sprint. Some of the findings are truly astounding and could influence numerous aspects of the Vigil Project, from the selection of instrument imaging detector bands right up to the design of the overall Space Weather Network architecture.”
“All three of these challenges can be seen as precursor projects that lay the groundwork for the further development of key AI technologies for EO and science more generally,” added Φ-lab data scientist Nicolas Longépé. “The combination of multi-disciplinary research teams and internationally recognised mentors has led to remarkable outcomes and much food for thought, not least the fact that the user community for EO data will now include not only humans but also chatbots.”
A video of the showcase is now available, and the full report will be published on fdleurope.org shortly. Researchers interested in joining FDL Europe 2024 can now register their interest.
ESA’s first-ever Earth Observation Commercialisation Forum will open its doors at ESA Headquarters in Paris on 30 October. Registration is still available for this premier two-day event, which will bring together institutions, investors and businesses to explore the commercial potential and funding landscape in Earth observation.
The forum is open to all Earth observation (EO) stakeholders, including entrepreneurs, start-ups, established companies and public and private investment bodies. Attendees will be treated to keynote addresses from ESA speakers such as Director General Josef Aschbacher, Director of Earth Observation Simonetta Cheli and Director of Commercialisation, Industry and Competitiveness Géraldine Naja, along with representatives from the European Commission, venture capitalists and industry leaders.
The speeches, panel discussions and networking sessions will provide a unique opportunity to understand the market trends, major drivers and challenges in commercial EO. With such a wealth of opportunity for connecting up the many and varied players in the sector, the event is sure to be the go-to platform for delving into the state of the art and direction of travel in commercial Earth observation. Full details can be found on the dedicated website.
Register for the Earth Observation Commercialisation Forum here.
