ESA title

Category: Philab

A bold new chapter for AI4EO with ‘ESA Φ-lab Challenges’

Posted on by mavelar

With a new look and the same ambition, the rebranded ‘ESA Φ-lab Challenges’ return with a fresh momentum, inspiring the Earth observation and AI communities to drive innovation through a new series of impactful competitions.

Earth observation (EO) is a powerful window that shows how the physical, chemical, and biological systems of our planet are interconnected. Until now, initiatives like ESA’s AI4EO have demonstrated how the combination of remote sensing technologies and AI can reveal hidden patterns and drive environmental, technological, and social innovation at a larger scale – from measuring biodiversity and soil health, to urban city planning or disaster response.

But to keep up with the evolving nature of innovation, we must also broaden our approach. This is why AI4EO is evolving into ‘ESA Φ-lab Challenges’. While AI is a fundamental technology, it is just a piece of a bigger puzzle. This new initiative will open the door to a wider range of cutting-edge technologies and approaches, encouraging innovation in all areas of Earth observation.

At the same time, these challenges are a platform for researchers and innovators to showcase their work, contribute with practical solutions to shared global issues, and help build a dynamic and engaged Φ-lab community.

Ready to make a difference? Here are three challenges you cannot miss:

From orbit to action: AI for Earthquake Response

    What if you could help first responders and support life-saving decisions right after an earthquake?

    Earthquakes are among one of the most unpredictable and destructive natural disasters, capable of destroying buildings, severing power lines, and bringing entire cities to a standstill in seconds. In the aftermath, time is critical and swift action is needed to contain further destruction, rescue survivors and restore order.

    Despite having access to terabytes of high-resolution satellite imagery, mapping affected areas still relies heavily on human interpretation, making it a time-consuming task when there is no time to lose. Together with Earth observation data, artificial intelligence emerges as a promising tool to automate and potentially accelerate disaster response.

    To support humanitarian and disaster relief efforts, ESA Φ-lab and the International Charter ‘Space and Major Disasters’ invite data scientists, AI researchers, students, geologists and developers around the world – solo or in a team – to join the ‘AI For Earthquake Response’ challenge.

    This initiative challenges you to develop state-of-the-art AI models that will automatically detect damaged vs. undamaged buildings, by analysing pre- and post-event satellite imagery. Participants will have exclusive access to a curated archive of multi-mission, high-resolution satellite imagery collected from previous Charter activations. All the EO data products of the virtual constellation used in past Charter activations concerning earthquakes are seamlessly ingested and processed by the on-line platform ‘Charter Mapper’, and made available through the Earth Observation Training Data Lab (EOTDL).

    This challenge foresees two main phases: one ‘training and live scoring phase’, where participants will have the possibility to train and test their models on partially annotated scenes (closing on 5 September), and a ‘stress test phase’, where participants will have to deal with fully annotated imagery from previously unforeseen sites, like in a real earthquake scenario.

    A webinar about this challenge is available here. The deadline is 15 September 2025, 17:00 CEST. Winning models will gain visibility in open-science forums and may be considered for integration into the ESA Charter Mapper, potentially becoming tools used by the Charter community in future disaster response activations. The first, second and third place will be awarded € 3000, € 2000 and € 1000, respectively, during the 54th Charter Meeting, from 6 to 10 October 2025 in Strasbourg, France.

    HYPERVIEW2: explainable artificial intelligence

      Earth observation is transforming agricultural practices by providing timely, large-scale insights into crop health, soil conditions, water availability and land use/land cover. As these Earth observation systems rely increasingly on AI to process vast amounts of data, it is essential that the models used are not only accurate but also explainable.

      Explainable AI (XAI) ensures that farmers, agronomists, and decision makers can understand and trust the reasoning behind these outputs. This transparency is key to building confidence in digital tools, allowing for their responsible and effective use in agriculture.

      Following the success of the HYPERVIEW challenge in 2022, the HYPERVIEW2 challenge is now back to develop new XAI systems applied to agriculture, using airborne hyperspectral images, Sentinel-2 multispectral images and PRISMA hyperspectral images.

      The goal is to develop an XAI model to estimate the concentration of six important contaminants/trace elements in soils – Boron (B), Copper (Cu), Zinc (Zn), Iron (Fe), Sulphur (S) and Manganese (Mn) – using Earth observation imagery. In the right balance, these elements boost plant health, productivity, and resilience to stress – important information that farmers need to optimise crop nutrition and yield.

      This challenge was launched by Φ-lab, together with KP Labs, the Warsaw University of Technology, and the Poznan University of Technology. The deadline for applications is 14 September 2025 and the award ceremony will take place at the EASi Workshop, during the European Conference on Artificial Intelligence, from 25 to 30 October in Bologna, Italy.

      PANGAEA: testing geospatial foundation models’ capabilities with a cutting-edge benchmark dataset

        If you want to dive deeper into benchmarking or tackle targeted geospatial tasks, the PANGAEA challenge will be the right one for you.

        PANGAEA is a highly curated, comprehensive benchmark dataset for Earth observation, designed to evaluate the performance of machine learning models across a broad range of geospatial tasks, such as land cover classification, change detection, environmental monitoring, and multi-sensor/multi-temporal analysis, among others.

        What makes it so unique is its diversity and structure: while it covers a wide spectrum of resolutions, sensor types, and temporal layers, it also provides a standardised protocol for evaluating the performance of a model, which is crucial for comparing results from different researchers, institutions, and AI approaches. Additionally, PANGAEA is designed to test and refine geospatial Foundation Models, a new generation of AI models with a wide range of applications across Earth observation.

        This will be an open-ended challenge: participants will have the opportunity to continuously explore, experiment, and iterate their models over time. Within this challenge, there will be regular Data Sprints: short, high-intensity mini-challenges that will focus on specific real-world tasks using the PANGAEA dataset, with clear goals and metrics, and their own prize pool and recognition opportunities. These are ideal for teams looking to make a mark, try something new, or just have fun competing under pressure.

        The community should stand ready: the next Data Sprint will be announced later in 2025, promising fresh challenges, new opportunities, and a chance to shine.

        You can know more about these three challenges here.

        ‘ESA Φ-lab Challenges’ is an initiative created by ESA Φ-lab and implemented by Novaspace, Planetek Italia, Sinergise, GMATICS, and EarthPulse.

        To know more: ESA Φ-lab, Φ-lab Challenges

        Photo courtesy of ESA Φ-lab Challenges

        Help ESA redefine the future of space computing

        Posted on by mavelar

        Due to the growing volume of data produced by Earth observation (EO), traditional computing architectures struggle to process information efficiently and promptly. To mitigate this issue, prepare Europe for the future of space computing, and grow from Earth observation into Earth action, ESA is seeking innovative mission concepts that use disruptive computing paradigms, potentially coupled with matching sensing technologies that could either bring new capabilities for Earth-orbiting satellites, or significantly improve current mission constrains. 

        Artificial intelligence (AI) and novel computing paradigms such as quantum, photonic or neuromorphic computing have the potential to transform space-based applications by dramatically increasing mission autonomy and decision making without humans. To consolidate Europe’s position as a leader in sustainability and remote sensing, ESA is launching the new SysNova challenge “Innovative mission concepts enabled by disruptive computing paradigms“.

        The call builds on multiple past and ongoing initiatives at ESA. “Through missions like Φ-sat, OPS-SAT, and initiatives such as 3CS, ESA has explored the benefits of embedding intelligence in orbit. In parallel, disruptive paradigms like quantum and neuromorphic computing have shown potential to enhance processing of vast amount of data efficiently. Yet, few have examined how these technologies could redefine entire missions. It’s time to take that next step”, says Gabriele Meoni, Innovation Officer at ESA Φ-lab and one of the campaign managers.

        Read the full article on www.esa.int.

        Living Planet Symposium Extra News: Day 5

        Posted on by mavelar

        ESA’s Living Planet Symposium came to a close today, concluding a week of networking, discussions and meeting of curious, scientific minds.   

        Today, one of the focal points was thermal imaging instruments, which are critical for monitoring land-surface temperature – and will be carried on upcoming missions such as the upcoming Copernicus Land Surface Temperature Mission. ESA’s Soil Moisture and Ocean Salinity (SMOS) mission celebrated passing its 15-year milestone in orbit – the mission has helped improve weather and climate models.

        Three new contracts were signed for ESA’s InCubed programme, which is central to the agency’s efforts to turn promising concepts into successful Earth observation services, strengthening Europe’s position in this rapidly evolving sector. 

        Read the full article on www.esa.int.

        Unmissable Φ-lab moments at LPS 2025

        Posted on by mavelar

        At this year’s Living Planet Symposium, ESA Φ-lab will present its innovations and initiatives at the forefront of Earth Observation. Make sure not to miss our key moments.

        The European Space Agency’s Living Planet Symposium (LPS) is one of the world’s largest events dedicated to Earth Observation (EO). LPS25 – this year’s edition – will take place from 23 to 27 June, in Vienna (Austria), gathering scientists, policymakers, and industry experts to share the latest research, satellite-based applications, and innovative technologies addressing environmental and societal challenges.

        As part of this dynamic programme, ESA Φ-lab will be actively involved, presenting its next-generation solutions at the intersection of EO, transformational innovation, commercialisation for human prosperity, climate action, and sustainability, among others.

        Curious about Φ-lab? Make sure to pass by the Φ-lab corner at ESA’s stand (Main Hall, ground floor) to meet the team, discover how Φ-lab drives cutting-edge research and disruptive technologies in Earth Observation, and learn about opportunities for collaboration.

        Here are Φ-lab’s must-sees:

        1. The future of geospatial data discovery and use

        The rapid growth of EO data availability calls for new approaches to efficiently manage, analyse and extract meaningful insights from heterogenous and enormous satellite datasets. In this context, self-supervised learning and foundation models are emerging as transformative tools, offering unprecedented capabilities for detecting patterns, changes, and anomalies across the planet.

        Φ-lab has built a strong expertise in using powerful AI models. The latest example is the joint ESA/IBM Research Europe release of TerraMind, a next-generation geospatial foundation model designed to help us better understand and protect our planet.

        AI is reshaping EO, enhancing data analysis, discovery, and interaction through multimodal data and language models. In the session “AI and Earth observation – where to now?”, Φ-lab Visiting Professors will share the latest advances and lead a thought-provoking debate on the future of AI in remote sensing.

        During the session “Foundation Models for Earth Observation: Current solutions with less labelled data to improve environment monitoring and future perspectives to revolutionize geospatial data discovery and utilization”, you will learn about three of the latest Φ-lab-supported projects on the topic of foundation models for EO: TerraMind, FM4CS (Foundation Models for Climate and Society) and PhilEO.

        EVE (Earth Virtual Expert) is a large language model (LLM) developed to support the EO and Earth Science communities. It builds on open-source LLMs, trained on billions of curated EO data tokens and fine-tuned with tailored datasets. Designed to assist both expert and non-specialists, EVE makes complex EO knowledge accessible to everyone through natural language processing. In the session “EVE: A Comprehensive Suite of LLMs and Data for Earth Observation and Earth Sciences”, attendees will discover how EVE was built, explore its capabilities, and learn how to interact with it for their own applications.

        Ensuring AI technologies are explainable, trustworthy, and physics-aware is essential. “Explainable AI for Earth Observation and Earth Science” will explore cutting-edge advancements in Explainable AI methods across diverse data types, including SAR, optical, and hyperspectral data. Attendees will discover innovative strategies to bridge data gaps, address physical inconsistencies, and promote responsible, ethical AI use in support of Earth Action initiatives.

        2. Advancing weather and climate forecasting with Machine Learning

        The monitoring and prediction of Earth’s weather and climate systems have seen remarkable progress in recent years. With the growing availability of high-resolution satellite data and sophisticated in situ sensors, we have now access to an unprecedented amount of data about our planet’s interconnected systems. Machine learning and deep learning techniques are transforming the way we interpret, model, and forecast the complex dynamics of Earth.

        “Machine Learning for Earth System Observation and Prediction” will bring together researchers exploring the latest AI-driven approaches in environmental science. It will highlight innovations in data assimilation, climate prediction, and the development of large-scale, data-driven Earth system models.

        3. Investing in commercial ideas that change the way we see our planet

        Great ideas need more than ambition. They need backing, and this is where the ESA InCubed programme steps in. By blending co-funding, technical expertise, and commercial and industrial guidance, InCubed is a key tool of ESA’s EO commercialisation strategy to effectively bridge the gap between vision and commercial success in the EO sector.

        As the demand for agile EO solutions grows, public-private partnerships are emerging as a powerful model to accelerate innovation and optimise resources. “New approaches to support commercialisation” will gather industry and institutional voices to explore the opportunities of new approaches, but also address challenges such as goal alignment, intellectual property, and data access.

        “Enhancement of EO products using advanced multi-instrument and multi-platform synergies” will focus on methods that exploit synergies between complementary observations, modelling and multi-sensor data, using data from missions like Copernicus’ Sentinels, EarthCARE, MTG, EPS-SG, PACE, among others.

        “Commercial Earth Observation Missions: Embracing New Paradigms and Innovative Models” will explore new commercial EO mission concepts designed to meet both institutional and market needs. With examples of business models – from public-private partnerships to fully private ventures – this session will offer insights into the evolving commercial EO landscape.

        Driven by new climate regulations such as EU ETS and CBAM, the demand for accurate GHG monitoring is rising. “Opportunities in the Earth Observation Market: A Focus on GHG Monitoring” will focus on the commercial opportunities at the intersection of Earth Observation and climate policy, featuring an overview of the regulatory landscape, and a panel discussion with EO companies developing services to meet emerging compliance needs.

        4. Harnessing Quantum Computing for a smarter, greener future

        Quantum Computing (QC) promises to process vast amounts of information more efficiently than classical systems, creating new opportunities for climate modelling, environmental monitoring, and the analysis of highly complex, interconnected natural systems. By accelerating data processing and enabling new types of simulations, quantum technologies could improve the accuracy of climate predictions and support more responsive decision making in the face of global challenges.

        In the “HPC and Quantum Computing” Insight Session, experts will discuss how quantum technologies are beginning to transform the way we work with EO data. It will also explore hybrid quantum-classical approaches, which combine the strengths of both computing paradigms.

        “Joint ESA-GRSS initiatives for the exploitation of Earth Observation data” will focus on the work developed by the Quantum Computing for Earth Observation Working Group, an initiative that is part of the IEEE-GRSS Technical Committee QUEST (Quantum Earth Science and Technology) and operated in collaboration with Φ-lab. This project fosters collaboration between the QC and EO communities and is working to turn the promise of QC into practical, impactful applications for EO, through joint research, open knowledge exchange and hands-on projects.

        5. Smarter satellites, faster insights: inside the Φsat-2 mission

        Designed to test new mission concepts and onboard data processing using advanced AI processors, ESA’s Φsat-2 is an innovative nanosatellite that runs multiple applications directly in orbit. Equipped with a high-resolution multispectral instrument, it supports tasks like cloud detection, vessel classification, wildfire monitoring and image compression.

        “The ESA Φsat-2 mission: an AI empowered 6U Cubesat for Earth Observation” session will present the mission’s status, AI demonstrations, and opportunities for the community to engage. Make sure not to miss the Φ-lab-led presentation: “All4One or One4All? Tailoring Onboard AI with NAS and Foundation Models”.

        6. Where future careers in Earth Observation begin

        To secure the future of Earth Observation, it is essential to engage and inspire young professionals today. The “Exploring Space Opportunities with ESA Φ-lab and EUSPA: Pathways for Students and Young Professionals” session is designed for students and young professionals eager to enter the European space sector, with a focus on innovation, technology, and entrepreneurship.

        The session will introduce two key institutions: ESA Φ-lab and EUSPA, the European Union Agency for the Space Programme, which manages operational EU space services like Galileo, EGNOS, and Copernicus.

        The Grand Marathon, organised by Φ-lab, is an innovation challenge rewarding scalable, market-ready solutions addressing climate events and infectious diseases, with a focus on younger populations. Launched in November 2024 and held in partnership with Save The Children and Hello Tomorrow, the competition celebrates the power of AI-based technologies and blockchain for global resilience.

        The “Grand Marathon Finalists pitching and award” session will host the final pitch between the two top teams – GEOMATYS and Plastic-i – who will receive € 50.000 each and compete for the € 150.000 first prize.

        Join Φ-lab at LPS25 and take the opportunity to connect, explore new possibilities, and be part of the conversation driving the next wave of Earth Observation innovation.

        To know more: Living Planet Symposium, ESA Φ-lab, TerraMind, FM4CS, PhilEO, EVE, InCubed, QUEST, Φsat-2

        Photo courtesy of ESA

        Join ‘Call for Φdeas’ and make your transformative mark in Earth Observation

        Posted on by mavelar

        ‘Call for Φdeas’ is a call for ideas sponsored by ESA Φ-lab to stimulate transformative innovation in the Earth observation sector. This call encourages the submission of groundbreaking ideas that can make an impact in scientific fields like Earth Science, green-tech, climate-tech, and sustainability, institutions, NGOs or in the commercial sector. Selected ideas can receive up to € 1.000.000 in funding and the deadline for submissions is 31 August 2025.

        In a rapidly evolving world, Earth observation (EO) plays a vital role in understanding and addressing global and local challenges. To keep pace with emerging needs and technological advancements, it is essential to look for fresh perspectives and novel approaches. Funding calls such as ESA Φ-lab’s ‘Call for Φdeas’ create valuable opportunities to explore untapped potential for transformative innovation in the Earth Observation domain.

        ‘Call for Φdeas’ is open to research and academic institutions, NGOs, commercial entities (start-ups, SMEs and LSIs), international collaborators, among others, to propose ambitious, forward-thinking initiatives that will have an impact in scientific fieldslike Earth Science, green-tech, climate-tech, and sustainability, institutions, NGOs or in the commercial sector.

        The main targets for this call are ideas with transformative potential, not incremental innovation. Selected ideas are expected to deliver significant technology progress and/or impact the reference sector/market/system by a different use of current technologies.

        Selected ideas can be used to populate future ESA Φ-lab workplans or as input for other ESA programmes (e.g., FutureEO, InCubed). ‘Call for Φdeas’ offers a maximum funding of € 1.000.000 per idea and co-funding is encouraged. For more information, please refer to ‘Evaluation Criteria’ in the dedicated Call for Φdeas channel on the Open Space Innovation Platform (OSIP).

        Ideas should fall into three categories: ‘Exploratory Ideas’ (to investigate novel, unconventional or unproven EO-related concepts, including technologies, mission studies or EO applications), ‘Capacity Building’ (to build the competences, techniques, or ecosystems needed to mature promising disruptive EO ideas), and ‘Innovation Impact’ (to translate a mature idea into a transformative solution ready for adoption for an identified use case).

        This is a recurrent call and accepts new submissions twice a year. For 2025, a single round of submissions is foreseen, and the submission phase deadline is 31 August 2025 COB.

        By encouraging diversity of ideas, ‘Call for Φdeas’ helps ensure that the Earth observation sector remains dynamic, relevant, and responsive to the complex realities of our planet.

        To know more: ‘Call for Φdeas’ OSIP, ESA Φ-lab

        Photo courtesy of ESA

        Strengthening space ties: new ESA International Fellowship for a Brazilian researcher

        Posted on by mavelar

        The European Space Agency (ESA) and Brazil have maintained a collaborative relationship in space exploration and technology over the years. In 2002, ESA and Brazil signed a Framework Cooperation Agreement to expand joint efforts in space science, Earth Observation, telecommunications, microgravity experiments, and life sciences, facilitating the exchange of experts and collaborative studies, and strengthening the scientific and technical ties between the two entities.

        Since then, other joint initiatives followed. In 2011, Brazil’s National Institute for Space Research (INPE) became a member of the International Charter ‘Space and Major Disasters’, an ESA co-founded global initiative that provides rapid satellite data access to support disaster management efforts. In 2018, ESA and the Brazilian Space Agency (AEB) signed an Implementing Arrangement to establish and use telemetry and tracking facilities in Natal, Brazil.

        In March 2024, Simonetta Cheli, Director of Earth Observation Programmes at ESA, and Clezio Marcos De Nardin, Director of the National Institute for Space Research, signed a Protocol of Intent to strengthen the relationship between ESA and INPE/AEB.

        As part of the research outcomes stemming from the Protocol of Intent signed between ESA and INPE, Gabriel da Rocha Bragion, ESA International Research Fellow, will spend 12 months at ESA Φ-lab developing methods for estimating carbon stocks from biomass, using SAR data and machine learning techniques. 

        Read the full article on www.eo4society.esa.int.

        Emerging European thermal imaging firm enjoys global success

        Posted on by mavelar

        OroraTech, a leading supplier of thermal sensing data and predictive wildfire solutions, has been subcontracted by Spire Global to develop a Canadian wildfire monitoring service as part of a major government contract. 

        Spire Global, a space-based data, analytics and space services provider, is leading the project, with OroraTech – which has worked closely with ESA in recent years to hone its capabilities – preparing the constellation’s instrumentation. This marks an important milestone in the growth of the emerging Germany-based company. 

        This achievement demonstrates the shared objective of ESA and the European Commission to empower growing European data providers to scale their operations and achieve commercial success in the global Earth observation marketplace. 

        Read the full article on www.rapidresponse.copernicus.eu.

        Norwegian EO industry lifts off with new ESA InCubed national call

        Posted on by mavelar

        Norway-based companies of any size are invited to participate in the latest InCubed call by submitting proposals for the development of groundbreaking and commercially promising products in the Earth Observation (EO) field. The call opens on 15 April and the deadline for submissions is 27 May 2025 at 14:00 CET.

        ESA InCubed is an Earth Observation commercialisation programme managed by ESA Φ-lab that aims to boost entrepreneur initiatives in the Earth Observation (EO) sector, supported by its signatory Participating States. With a focus on leveraging the benefits of EO data and services, InCubed provides co-funding for the development of any elements of the EO value chain, namely full satellites and constellations, platforms, payloads and instruments, ground segment-based systems, and downstream applications and value-added services. 

        Depending on the type of activity, companies can apply for co-funding support and will be guided by ESA top-tier experts to create sound products/services from a technical, commercial, and financial standpoint, with ESA acting as the de-risking partner.

        In collaboration with the Norwegian Space Agency (NOSA), the upcoming Norway national call will open on 15 April 2025 and has a budget of €2.1 million. Proposals should be focused in developing innovative and commercially successful EO products and services.

        Norwegian companies with innovative ideas leveraging EO data or developing relevant EO technologies may apply. Universities and research institutes with no commercial interest in the project may be funded up to 100% of their costs if those do not surpass 30% of the total activity funding. Business ideas currently being funded by other ESA/NOSA programmes are not eligible for this call, unless for complementary tasks.

        Interested Norwegian companies should submit an idea pitch on the Open Space Innovation Platform (OSIP), which will be ranked based on defined criteria, including the understanding of InCubed’s objectives; quality and completeness of the business and technical proposals; adequacy of the management approach, planning and co-funding; and level of experience. Qualifying pitches will be invited to submit a full proposal to ESA. Successful applicants will be contacted directly by ESA to discuss further contract negotiations. The first evaluation cycle will be in June 2025.

        Interested companies can find out more about this call during the InCubed meeting at NOSA, in Oslo, on 7 April, and enrol via the NOSA dedicated link. The call opens on 15 April 2025 and the closing date for submissions is 27 May 2025 at 14:00 CET.

        To know more: ESA InCubed, Norwegian Space Agency

        Photo courtesy of ESA

        Talking to Earth: a first-generation AI digital assistant

        Posted on by mavelar

        A recent initiative funded by ESA Φ-lab focuses on creating an AI-powered digital assistant that allows users to access and explore complex EO data through a natural language interface. In the long term, the aim is to integrate this tool into digital twins of Earth, supporting decision-making in areas such as climate monitoring, disaster management and urban planning.

        A digital twin of the Earth environment is an interactive “digital replica” that allows us to understand the various relationships between the physical and natural Earth environments and society. It enables scientists to quantify past, present and future events on our planet, integrating models, observations, and technologies such as Artificial Intelligence (AI) to improve our understanding of the human impact on global environment and society.

        Through data and simulations, digital twins allow for real-time prediction, monitoring, control and optimisation of Earth’s natural and physical processes. Two related flagship programmes are the European Commission’s Destination Earth (DestinE) and the European Space Agency’s Digital Twin Earth (DTE).

        In the past years, rapid advances in Earth Observation (EO) have led to an increase in the amount of available EO data to be processed for the benefit of society, creating an opportunity to harness the power of new technologies like AI. The growing European EO capability is delivering a unique and dynamic picture of Earth, but this information remains largely unexploited due to current limitations in querying and retrieval capabilities and does not include an appropriate interface for non-experts to interact with digital twins.

        Technologies such as Computer Vision and Natural Language Processing – used for object detection/EO image segmentation and for interpretation of human language, respectively – have been treated as separate areas, without benefitting from each other. But what if we could use these technologies together to create a new way to interact with and understand EO data?

        In that sense, ESA Φ-lab is funding the creation of a digital assistant interface for digital twins of Earth – Demonstrator Precursor Digital Assistant Interface for Digital Twin Earth (DA4DTE) – in collaboration with e-GEOS (Italy), the National and Kapodistrian University of Athens (Greece) and the Technical University of Berlin (Germany). Φ-lab has already a proven track record in the development of foundation models, which serve as the basis for digital assistants.

        The developed digital assistant prototype interacts with users via two modalities, text and satellite images, and consists of four back-end engines: search-by-image, search-by-text, knowledge graph question answering (KGQA), and visual question answering (VQA) engines. These engines are orchestrated by the task interpreter to answer complex requests of users looking for EO data.

        The Knowledge Graph Question-Answering Engine is built on a knowledge graph that integrates geospatial data from widely used geographical knowledge bases, such as OpenStreetMap, along with metadata from Copernicus missions. This knowledge graph is combined with Large Language Models (LLMs) such as Llama, GPT, and Mistral.

        The search-by-image engine is based on a newly developed technique called Cross-Modal Masked AutoEncoder (CM-MAE). This method features modal-agnostic, self-supervised feature characterisation, making it well-suited for cross-modal image retrieval tasks. Additionally, the engine incorporates deep hashing modules to map cross-modal embeddings into compact binary hash codes, ensuring efficient and scalable data storage and retrieval.

        In the end, the idea is that users, whether they are EO experts or not, will be able to perform a semantic query on EO data archives such as “Show me 3 pictures of rivers in Italy, with a vegetation coverage over 20%, taken after May 2020” or “Count the number of buildings in this area”. This digital assistant will help to answer questions in several EO-related data domains – agriculture, forest, urban, marine, cryosphere, among others – contributing to improved decision-making.

        “As digital twins of Earth become increasingly sophisticated, the ability to interact with them is crucial. The digital assistant for Digital Twin Earth represents a major step forward in making complex Earth Observation data more accessible”, comments Nicolas Longépé, Earth Observation Data Scientist at ESA Φ-lab. “By allowing users to ask questions in natural language and receive insightful, data-driven responses, this tool lowers the barrier to entry and accelerates the use of EO data for decision-making. Whether it is for climate monitoring, disaster response, or environmental management, having an intelligent interface to navigate and interpret vast datasets is essential for informed action.”

        All the developed back-end engines are available as open source with a permissive licence here. The precursor digital assistant prototype is available here. A presentation titled “A digital assistant for digital twins of the Earth” will take place during Living Planet Symposium, from 23 to 27 June 2025 in Vienna, Austria.

        To know more: ESA Φ-lab, Digital Twin Earth, Destination Earth

        Photo courtesy of Unsplash/Carl Wang

        Where there’s smoke, there’s data: SeasFire’s mission to predict wildfires in Europe

        Posted on by mavelar

        SeasFire, an ESA Φ-lab-supported initiative, uses cutting-edge deep learning algorithms to explore the spatio-temporal connections between Earth system variables and fire regimes, gaining valuable insights into predicting potential wildfires. This project is aligned with ESA’s mission to develop innovative applications of Earth Observation (EO) data that address important societal and environmental challenges. 

        Planet Earth undergoes several complex physical processes that occur at variable spatial and temporal scales. Wildfires are notable examples of such processes, since they do not behave the same way in different areas and years. As major hazards, wildfires are deeply influenced by a combination of multiple human and natural factors, such as temperature, soil moisture, relative humidity, wind speed, vegetation – commonly referred to as ‘fire drivers’.

        Wildfires disrupt natural ecosystems and cause the loss of lives, properties and infrastructure. Due to climate change, an increase in the number of fires in Europe and around the world is expected, with major wildfire events extending to evergreen forests and boreal regions. Therefore, it is important to improve our capabilities to anticipate fire danger and understand its driving mechanisms at a global scale.

        SeasFire – Earth System Deep Learning for Seasonal Fire Forecasting in Europe – emerges as an innovative solution for fire forecasting. Supported by ESA Φ-lab and implemented by the National Observatory of Athens, the National Technical University of Athens, the Harokopio University of Athens and the Max Planck Institute for Biogeochemistry, SeasFire proposes to explore and capture the potential spatio-temporal asynchronous links between pre-occurring and non-overlapping fire-driving forces in the Earth system and European fire regimes to predict seasonal burned area extent in Europe.

        How has this been accomplished? SeasFire has made use of two major advancements of our time, namely the availability of a huge amount of satellite data with good spatio-temporal resolution, and Deep Learning (DL) techniques that have proven capable of capturing the spatio-temporal interactions of Earth system variables, treating Earth as an interconnected system.

        Some of the DL models developed in this project include an encoder-decoder architecture that takes as input snapshots of fire drivers and is trained to predict burned area patterns in the future; FireCastNet is a Graph Neural Network that can leverage local, mid-range and long-range spatial connections; and TeleViT is a transformer-based architecture, which combines information from local fire drivers and teleconnections to improve long-term forecasting.

        Current CO2 estimates rely on factors such as burned areas, vegetation carbon stocks, and combustion completeness. While Machine Learning algorithms have been developed to forecast burned area patterns, prior estimates from vegetation carbon stocks and combustion completeness are model-based and assume fixed emission factors that may not accurately capture larger changes in carbon stocks over longer periods.

        SeasFire overcomes these limitations by integrating Earth Observation (EO) data with climate datasets in a hybrid modelling framework. This approach combines process-oriented modelling with observation-based learning, enabling more accurate model parameterisation and reducing biases from model initialisation. By comparing existing carbon models with data from the Copernicus Atmosphere Monitoring Service (CAMS), SeasFire enhances the reliability and precision of carbon cycle predictions.    

        The SeasFire DataCube, a public global analysis-ready and cloud-friendly dataset for seasonal fire forecasting, between the years 2001-2021 and at a spatio-temporal resolution of 0.25° x 0.25° x 0.25° x 8 days, includes a combination of variables describing seasonal fire drivers, namely climate, vegetation, oceanic indices, human factors, land cover and the burned areas. In the future, this datacube can also be exploited as a template for modelling different natural hazards like floods, heatwaves and droughts.

        From this initiative resulted also an interactive toolkit that allows the visualisation of EO data and model outputs stored in a Zarr file, accessed via the SeasFire GitHub organisation. In this repository, it is possible to find all trained models regarding seasonal wildfire forecasting and modelling of CO2 emissions.

        “This project explores a key gap by attempting to capture the interannual variability of seasonal, high-impact wildfire events — an area where traditional numerical models often fall short. By leveraging Deep Learning and teleconnections which dictate climate dynamics, we explore novel machine learning methods that treat the Earth as a system to enhance long term forecast capabilities. Moving forward, we aim to refine models for more precise regional predictions, establish clear benchmarks for comparing our models with existing and future approaches, and strengthen collaborations to validate and operationalise our methodology”, says Ilektra Karasante, SeasFire Project Manager.

        “Φ-lab’s support has been essential from the project’s conception to its implementation. Beyond the initial trust in funding this high-risk/high-gain research, Φ-lab has further accelerated our developments by providing access to computational resources through the Network of Resources (NoR), fostered collaborations with leading scientists in the field, and enhanced the project’s visibility through ESA and ECMWF workshops, strengthening our research network and helping integrate our work into the broader scientific community. Through all the above Φ-lab-supported activities, SeasFire outcomes reached a much broader audience.”

        Φ-lab hosted a workshop about “Innovations in Data-Driven Seasonal Fire Forecasting: From Models to Visuals” on 7 February 2025. Patrick Ebel, Internal Research Fellow at Φ-lab and SeasFire Technical Officer comments: “It was my pleasure being ESA’s Technical Officer for this activity and its enthusiastic consortium. SeasFire shows how recent advances in Deep Learning and weather forecasting can be harnessed to model wildfires and their impact to tackle one of the greatest challenges Europe will be facing in the coming decades under a changing climate. I appreciated the public’s scientific interest in SeasFire and look forward to future advances building on the achievements of the activity.”

        To know more: ESA Φ-lab, SeasFire

        Photo courtesy of Unsplash/Mike Newbry