Described as the new electricity fuelling the 2000 industrial revolution in the AI for the Earth report presented at the 2018 World Economic Forum, AI research is a real driver of innovation, future growth and competitiveness for societies and industries worldwide, and as such can bring enormous opportunities.
AI is currently in the midst of a true renaissance, driven by Moore’s Law and now super-fed by Big Data. AI is deeply entrenched in our society and routinely used in everyday life in applications ranging from digital assistants (e.g. ChatGPT and BARD), natural language processing (e.g. translation and speech recognition), image recognition and generation, virtual assistants (e.g. Siri and Alexa) and autonomous vehicles (e.g. self-driving cars, spacecraft and robotics), all of which are transforming the way we work and live.
Added to this, over the last decade Machine Learning (ML) has undergone a major revolution driven by the unique convergence of large-scale computing capability (e.g. cloud computing, Graphics Processing Unit (GPU) architectures and High Performance Computing (HPC)), easy access to large volumes of data through the Internet, and the availability of new algorithms enabling robust training of large-scale deep neural networks (e.g. foundation models).
In Earth observation, AI applications have so far mainly involved Computer Vision techniques for interpreting and understanding satellite imagery. However, many other areas could benefit from AI, including Earth Science models and prediction, EO Big Data analytics, on-board and in-situ edge computing, Quantum Computing for EO, Decision Intelligence and Digital Twins of the Earth.
There is a pressing need to increase Europe’s strength and positioning in the area of AI research applied to space. In particular, EO Big Data presents a niche area of the data economy. Much more could be gained from these valuable data by using AI, thereby increasing opportunities for a growing variety of applications and services. The investment, research and innovation effort required to achieve such an ambitious goal is huge, and can only be achieved through teaming and partnership.
In this context, and as part of a first element of a wider ESA response, Φ-lab aims to operate as a catalyst, enabling a network of AI and EO labs across Europe to advance our AI4EO knowledge and develop new innovative applications and solutions with EO. In particular, our vision is to become a hub connecting a growing ecosystem of AI talents and capabilities across Europe to tackle a suite of challenges
To do so, Φ-lab is hosting talents via a suite of collaborative schemes supporting the exchange of ideas and developing new solutions that are described here.
The coming together of AI and EO is a perfect synergy, albeit with its own strengths and weaknesses. We are now at an inflection point or crossroads of opportunities, where on the one side AI has become one of the most transformative technologies of the century, while on the other a growing European EO capability is delivering a totally unique, comprehensive and dynamic picture of the planet, thereby generating big open data sets to be explored by AI in an automatic, flexible and scalable way.
Today, EO data remotely sensed from space are particularly suited to AI processing, but also present challenges. EO data are:
Illustration of the innovative adaptative product engine for EO data based on Machine Learning
The innovative adaptative product engine for EO data illustrated above identifies a new type of value chain, whereby the algorithms are engineered by learning from the data as opposed to the traditional way of explicitly coding knowledge. In AI-powered value chains, most of the effort is made up-front (i.e. model architecture definition, data preparation and learning) and the result or product derives from the inference executed by a trained ML model.
Machine learning algorithms are therefore critical to accelerating insight extraction from the data, but domain experts are also essential in order to properly interpret the statistical correlations and data and derive the correct causal relationship. Also, although very powerful, ML techniques suffer from their own inherent limitations, namely that they are data hungry while lacking transparency, unable to distinguish causation, have difficulty in generalising and are opaque in understanding internal structure.
This calls for more research that connects together AI expertise in order to address EO problems. An objective of Φ-lab is to foster the development of a seamless EO Data Engine comprising AI4EO use cases, datasets and pre-trained algorithms (referred to as apps) that can then be used in a modular fashion. These apps can also be chained and regularly updated with new information, in much the same way as an app on a mobile phone. Developing such a data engine will open new opportunities for innovative business models to provide users with updated information. Φ-lab, in collaboration with teams across ESA, provides a number of elements of this huge and highly complex process
Towards scalable Big Data analytics
How to augment EO capabilities with AI? How to fuse and analyse data from heterogenous sources? How to turn Big Data into actionable information? How to exploit Generative AI to build Virtual Sensors
Towards trustworthy and explainable AI
How to trust AI? What is in the AI black box? How to quantify the error and uncertainty of the ML model? How to make AI decision-making more transparent? How to resist adversarial attacks? How to mitigate or eliminate hallucinations?
Towards physics-aware AI
How to integrate first principles and domain knowledge into the AI statistical approach? How to bring together models, emulators and AI? How to satisfy physics by design?
Towards self-learning AI
Can machines learn by themselves? How to build new learning principles in AI? How to develop unsupervised learning for EO data without labels? Can we transfer learning to different domains in order to generalise?
Towards AI-based EO data fusion and prediction for Digital Twin Earth (DTE)
How to leverage the combination of EO, models and AI techniques to support high-resolution prediction and informed decision-making with DTE? How to improve prediction of the Earth system and develop smart satellite learning? How to optimally target satellite observations to improve forecasting? How to use DTE for scenario prediction and Decision Intelligence?
