ESA title

Category: Philab

Φ-lab gives major impetus to ESA-ECMWF ML4ESOP workshop

Posted on by mavelar

With record attendance and a host of new topics covered, the 2024 joint workshop on Machine Learning for Earth System Observation and Prediction (ML4ESOP) has just drawn to a close. Organised by ESA Φ-lab and ECMWF, the workshop explores synergies between machine learning, deep learning and conventional tools as applied to satellite observations and weather and climate models.

Machine learning and deep learning (ML/DL) are already proven technologies for enhancing our understanding of the planet’s complex dynamics, leveraging new methodologies and high-performance computing to process and analyse enormous datasets automatically. The intersection between Earth observation (EO), Earth System Prediction and AI is a point where a number of disciplines can come together and experts from each field can synergistically advance the state of the art.

This forms the guiding principle of the ESA-ECMWF ML4ESOP workshop, which each year gathers a diverse community of practitioners to look at the progress and challenges in the fusion of traditional ESOP techniques with ML/DL. This year’s event was held from 7 to 10 May and boasted almost 800 registrations online and in person, comprising academia, public institutions and industry.

Apart from hosting the 2024 workshop at ESA’s ESRIN establishment, Φ-lab played a fundamental role through its participation in the organising and scientific committees, the chairing of sessions, the presentation of a number of research initiatives and the lively discussions throughout the four days.

ECMWF’s Massimo Bonavita, who organised the event with Φ-lab AI Applications Lead Rochelle Schneider, was impressed by the turnout: “Each year we see a higher number of participants and an ever-broader range of research subject areas. The new and exciting thematic areas that were selected as the framework of the workshop ensured that the key elements and hurdles of ML4ESOP were addressed, with engaging contributions on weather and climate from a truly international cohort of researchers.”

Indeed, this year’s thematic areas (TAs) gave an enticing structure to the proceedings. TA1 was a new addition and covered Destination Earth (DestinE), reflecting the importance of the initiative’s platform, digital twin and data lake components. ESA and ECMWF are both development partners of DestinE, as is EUMETSAT, another contributor at the workshop.

TA2, entitled Multidomain ML4ESOP, delved into the integration of natural language processing and DL techniques into satellite image analysis. TA3 and TA4 both looked into weather and climate prediction, the former with respect to end-to-end ML and the latter in terms of hybrid approaches that combine ML with physics-based methodologies. Last but certainly not least, TA5 dealt with the emerging enablers of powerful ML4ESOP: high-performance, edge and quantum computing.

A wide range of applications were presented in the sessions, including volcanic cloud tracking, sea-ice detection, permafrost carbon dynamics, precipitation nowcasting, tree height estimation and European summer drought prediction to name but a few. Φ-labbers gave talks on the ESA-UNICEF child connectivity project and vision-based solar forecasting models.

Two poster sessions were distributed across the four days. Over 70 posters were on display, giving representatives from both the private and public sectors the opportunity to showcase ML4ESOP research that in many cases focused on extreme weather modelling and prediction.

Researchers showcasing their projects at the poster session

One of the guests at the gathering was Stephan Hoyer, Applied Sciences Research Lead at Google, who was also treated to a visit to Φ-lab: “This workshop has given us great exposure to other perspectives on EO, weather forecasting and AI, including some fascinating conversations with delegates from a huge variety of backgrounds.”

Each TA had an accompanying working group to discuss the topics presented, and the final afternoon featured a plenary session that brought the event to a conclusion. “The plenary underlined just how much had been covered during the workshop, together with the strategies we need to adopt as a community to propel ML4ESOP forward,” Rochelle commented. “It’s clear that machine learning is an indispensable tool in our quest to understand and predict the behaviour of Earth systems, and I’m confident that the work discussed will help determine the optimum approaches for future research in this domain.”

The workshop content is available on the event website.

To know more: Φ-lab, ECMWF, EUMETSAT

Photos courtesy of ESA and ECMWF/Mike Morrissey

Dedicated InCubed national calls give rise to innovative EO development activities

Posted on by mavelar

Following last year’s calls for proposals in the United Kingdom and Spain, seven new InCubed activities have now commenced. Addressing needs over the full range of the Earth observation value chain, the activities demonstrate how customer-oriented Earth observation solutions provide technological advances and tangible societal benefits.

With its co-funding of market-oriented development activities, the ESA InCubed programme is a key enabler for the financing of ventures that improve European competitiveness in Earth observation (EO). In cooperation with its participating Member States, InCubed periodically issues national calls in order to boost EO innovation and investment in specific countries.

InCubed national calls were launched in Spain and the UK last year. Both were met with an enthusiastic response, and InCubed then initiated a comprehensive evaluation process in conjunction with the space agencies of each country. The result so far is that five Spanish and two British companies have been awarded InCubed co-funding for new activities.

These activities cover both upstream and downstream applications, from instruments, data transmission and onboard software in the space segment to data processing and even an entire satellite mission. They respond to significant technical challenges, such as achieving timely intelligence extraction from Synthetic Aperture Radar (SAR) imagery, the need for higher resolution sensing in Thermal Infrared (TIR) cameras and the difficulty of reducing aerosol interference in methane detection.

Commercial EO España-style

The new activities from Spain encompass important developments in satellite technology. SATLANTIS’s TALISMAN is a 16U cubesat mission carrying a short-wave infrared (SWIR) optical instrument coupled with a liquid crystal polarimeter. The combination is designed to enable point-source methane detection – a vital activity for climate change mitigation – while minimising the impact of aerosol scattering. EODDL-LYNX from SENER builds on the company’s previous InCubed activity to integrate an active electronically steerable antenna with a payload data transmitter. The antenna will not only enhance downlink throughput and flexibility, but also provide a more compact, lightweight solution compared with mechanically steered variants.

Madrid-based Crisa will develop the power conditioning module of its NICE instrument control unit for satellites. The power conditioning module will complete the jigsaw of NICE’s core functions, alongside the payload controller and standard discrete interface module. Deimos Space’s Insight4EO picks up on the emerging trend of edge computing onboard satellites by providing a processing and intelligence services package for LEO EO missions. The activity is aimed at facilitating realtime decision-making, increased mission autonomy and higher data throughput.

In the downstream data processing segment, ATR4PAZ from Hisdesat will tackle the thorny issue of rapidly deriving meaningful insight from SAR data. Focusing on the security and intelligence sector, ATR4PAZ will simulate objects of interest on existing SAR imagery in order to train a machine learning to detect those features.

UK-side sustainability solutions

Like its Spanish counterparts, the British contingent also includes an innovative instrument in the form of SPIRIT, an ultra-compact, high-resolution, wide-field-of-view thermal infrared camera to be developed by Supersharp. SPIRIT imagery will be used to produce heat maps with an unparalleled revisit rate to monitor the energy usage of buildings worldwide.

London company Messium will develop an application for monitoring the Nitrogen concentration in wheat crops. Based on satellite-derived hyperspectral data and custom machine learning models, the interface will complement in-situ measurements and help in the adoption of variable-rate fertiliser distribution for more efficient nutrient management and reduced nitrous oxide production.

“Launching targeted national calls enables us to work hand in hand with our Participating States to stimulate EO innovation in individual countries,” commented Michele Castorina, InCubed Programme Manager and head of the ESA Φ-lab Invest Office. “These latest Spanish and British calls have yielded groundbreaking activities that not only include contributions to the green agenda, but have also captured the direction of travel in EO technologies with such topics as edge compute, payload downsizing and high-capacity data transmission. I therefore expect these developments to make a valid contribution to Europe’s commercial space arena.”

To know more: ESA InCubed, Spanish Space Agency, UK Space Agency

Photo courtesy of Pixabay/Efraimstochter, Pixabay/Nerivill

Applying artificial intelligence to raw satellite imagery for time-critical applications

Posted on by nchimonides

An ESA Φ-lab-funded project is showing the benefits of using raw Copernicus Sentinel-2 data and artificial intelligence to improve response times in early warning systems for catastrophic events.

An abundance of useful satellite imagery is now available, for a realm of applications. The Sentinel-2 mission of the European Copernicus programme provides high-resolution optical imagery with good re-visit times over large areas, that are invaluable for land monitoring. However, for applications that require rapid response, such as the monitoring of catastrophic events or the detection of illegal vessels, traditional data handling and processing schemes usually exhibit excessive latency. The processing is typically encompassed within the ground segment, which causes delay.

Read the full article on sentinel.esa.int.

Copernicus Sentinel-2 image courtesy of ESA

AI in Earth observation: a force for good

Posted on by mavelar

The upcoming launch of the Φsat-2 mission is a prime example of the pioneering work that ESA does in the field of AI in Earth observation. But when it comes to AI, hopes and fears abound in equal measure. In this interview, ESA’s Rochelle Schneider sets the record straight on how this transformational technology is improving access to crucial information on the state and future of our planet.

The success of generative AI tools like ChatGPT has brought with it new questions of what awaits us if the power and capabilities of AI continue to grow. Fortunately, conferences such as AI For Good, in which ESA often participates, demonstrate that rather than causing harm, AI has a highly positive impact on society and sustainable development.

As an expert in machine learning, ESA Φ-lab AI Applications Lead Rochelle Schneider is well placed to explain why AI is a force for good in Earth observation. Rochelle has extensive experience in retrieving vital information from Earth observation data for the benefit of disease prevention and child development.

Read the full article on www.esa.int.

Φ-lab’s continuing EO innovation drives AI-derived methane detection

Posted on by nchimonides

ESA Φ-lab, an established prime mover for disruptive innovation in European Earth observation (EO), is supporting a project on identifying methane pipeline leakage using satellite data. The research moves the state of the art forward through a data-fusion approach, employing machine learning algorithms to combine various remote sensing image and other time-series data sources.

Φ-lab has a proven track record for driving innovation in the Earth observation sector, including through the financial and technical sponsorship of cutting-edge research via PhDs.

Methane detection from space is one such area of Φ-lab’s research. Albeit with a shorter lifespan in the atmosphere, methane gives rise to up to 30 times more heat retention than carbon dioxide and so is proving to be a significant contributor to global warming. Detecting and fixing methane leaks is therefore a priority in terms of climate-change mitigation, but in the case of gas pipelines, management of leaks can be hampered by the vast expanses of remote locations that the networks of pipes often traverse.

In recent years Φ-lab has overseen two projects, STARCOP and LEO-GEO4GHG, both of which explored how methane plumes could be detected from satellites using onboard AI. In new research that Φ-lab is jointly supervising with German aerospace company OHB Digital Connect and the Technical University of Munich, PhD student and OHB employee Enno Tiemann is exploiting machine learning to analyse a combination of data sources for plume detection and quantification.

“Currently, methane detection often relies on experts who manually inspect the EO data and create a map of the plume that is then used to determine the emission rate – sometimes with a high degree of uncertainty,” Enno explains. “Whereas previous work has adopted a cascading approach, whereby initial AI detection of large plumes is followed by human analysis of higher-resolution optical or infrared data, our research involves algorithms that simultaneously interrogate multiple satellite datasets to improve detection and reduce uncertainty.”

Although Sentinel-5P is a well-established methane detector, giving daily coverage of the entire planet, it has a fairly coarse resolution that means the minimum measurable emission rate is around 25 tonnes per hour. Sentinel-2 and Landsat 8 and 9 by contrast have a threshold of 1.5-1.8 tonnes per hour, but lack daily coverage. By fusing time series data from multiple satellites, in conjunction with wind-vector data, the new research is expected to yield more robust estimates of plume location and gas flows, together with insight on the evolution of plumes over time.

Φ-lab provides essential support to the project through technical supervision, expertise sharing and joint funding with OHB. Φ-lab AI Applications Lead Rochelle Schneider is the project supervisor for ESA: “In addition to giving vital input to the project, Φ-lab is able to grant external researchers access to our pool of world-leading AI experts and data scientists. We’re very happy to help guide this research initiative, which like STARCOP and LEO-GEO4GHG is an example of AI-driven EO that is sure to make a significant contribution to combatting methane emissions in the energy sector.”

To know more: Φ-lab, OHB, TUM

Photo courtesy of Robzor

Φsat-2 gets two new AI apps

Posted on by nchimonides

Φsat-2, ESA’s groundbreaking cubesat scheduled for launch in June, will now include two new AI-driven apps destined to demonstrate a crucial role in future environmental monitoring from space. The apps, which focus on marine pollution and wildfires, were the winning entries in ESA’s OrbitalAI Challenge.

AI is already established as a vital tool in Earth observation, helping to sift through terabytes of satellite data to provide useful insight for scientists, policy makers and commercial operators. But while most AI processing of information is carried out on the ground once the data have been downloaded, there are significant advantages to be had from integrating AI capabilities on the satellite.

Read the full article on www.esa.int.

Building ChatGPT-style tools with Earth observation

Posted on by nchimonides

Imagine being able to ask a chatbot, “Can you make me an extremely accurate classification map of crop cultivation in Kenya?” or “Are buildings subsiding in my street?” And imagine that the information that comes back is scientifically sound and based on verified Earth observation data. ESA, in conjunction with technology partners, is working to make such a tool a reality by developing AI applications that will revolutionise information retrieval in Earth observation.

A digital helping hand for data

Earth observation generates vast volumes of vital data every day, but it is difficult for humans alone to ensure that we obtain the best value from that data. Fortunately, AI helps in interacting with such large and complex datasets, identifying key features and presenting the information in a user-friendly format.

I*STAR for example, an activity co-funded by the ESA InCubed programme, developed a platform that uses AI to monitor current events like earthquakes or volcano eruptions so that satellite operators can automatically plan the next data acquisitions for customers. The SaferPlaces AI tool, again supported by InCubed, creates flood maps for disaster response teams by merging in situ measurements with satellite data. SaferPlaces was crucial to damage assessment efforts during last year’s floods in Emilia-Romagna in Italy.

Read the full article on www.esa.int.

Hello Major TOM: ESA Φ-lab releases largest ML-ready Sentinel-2 dataset ever published

Posted on by nchimonides

ESA Φ-lab has launched Major TOM (Terrestrial Observation Metaset), a community-oriented project that allows researchers to share, use and combine large Earth observation (EO) datasets. The Major TOM framework will help unlock the huge potential of satellite imagery by offering users the largest ever quality-controlled and globally distributed sample of data, with future expansions to multiple satellites and modalities planned from both Φ-lab and the wider community.

Recent years have seen a marked trend towards larger, more general EO and geospatial models, known as foundation models, which require massive volumes of high-quality training data. These large models present unique opportunities in that they have the potential to help solve many pressing scientific and societal problems.

But there are also challenges, including the risk of deepening the reproducibility crisis seen in AI research, whereby published models are often difficult to recreate due to closed data sources and opaque technical details. Bias is another issue, since all models are skewed by the data they learn from, and this may lead to biases being embedded into the systems that foundation models form part of.

ESA Φ-lab believes that these issues can be alleviated through the creation of high-quality globally distributed and collaborative ML-ready datasets, and has begun to integrate them under the moniker Major TOM. These ML-ready datasets are a means to steer the development of large models in a positive direction, democratising them and helping to make systems that are more reproducible and with a lower bias by virtue of the dataset’s global sampling. To achieve this, Φ-lab has partnered with Hugging Face to host and freely distribute Major TOM on the Hugging Face Hub. With its open and community-driven platform for datasets and models, Hugging Face is a leading light for the democratisation of machine learning technology.

The creation of such a large dataset presented the team with several technical hurdles. “Satellite data is often held and delivered in very large products – over 100 km across – which many people find difficult to work with for machine learning applications, especially when trying to combine different satellites whose products overlap to differing extents,” explains Φ-lab research fellow Alistair Francis. “By contrast, Major TOM uses a fixed, 10 km grid across the entire globe, meaning that data from one Major TOM dataset will fit neatly on top of another.”

Whilst the sheer volume of data processing involved was a challenge, the need to ensure its quality was equally difficult. For example, optical satellite imagery often contains clouds that hide the surface below. Although not eliminated from the Major TOM dataset entirely, cloudy imagery was minimised by using Φ-lab’s state-of-the-art AI cloud mask, soon to be released publicly.

Illustration of the global coverage of Major TOM Core. Regions in colour denote sampled areas (green for land and light blue for sea).

Major TOM’s inaugural core dataset has now been released on Hugging Face. It constitutes the largest ML-ready collection of Copernicus Sentinel-2 images ever published. Covering over 50% of the Earth’s surface (including almost all dry land) with nearly 50 TB of data and 2.5 trillion pixels, Major TOM Core is a game-changer for those seeking to train large models with satellite data. It is expected that future expansions from the broader EO community, spearheaded by ESA Φ-lab, will spawn a diverse ecosystem of combinable datasets that will be invaluable in creating the next generation of large deep learning models from satellite data.

Giuseppe Borghi is the Head of ESA Φ-lab: “We want to build an open community of contributors and end users who can create a data landscape that ensures EO derives the largest benefits from the AI revolution. If we want to make sure that EO models are reliable, reproducible, traceable and in turn, trustworthy, then it stands to reason that we need to start with high-quality trustable data.”

An interview on Major TOM with Alistair Francis and fellow Φ-lab researcher Mikolaj Czerkawski can be found here.

To know more: Φ-lab, Hugging Face, Major TOM paper preprint

Header image contains modified Copernicus Sentinel data (2022), processed by ESA

FOREST-2 to deliver thermal-sensing insights to Copernicus

Posted on by nchimonides

ESA is working with European New Space company OroraTech to demonstrate how data from its temperature-sensing FOREST-2 mission will facilitate the aims of the Copernicus programme.

The Munich-based thermal intelligence specialist was one of nine firms selected as European Emerging Copernicus Contributing Missions (CCMs) in June 2023, following a recruitment drive designed to encourage – and capitalise on – New Space growth in Earth observation. In complement to the Sentinel family, this group of providers will soon supply commercial data to Copernicus to help address key environmental and societal challenges impacting European citizens.

OroraTech was first supported by incubation programmes ESA BIC Bavaria and ESA Kick-Start, and in 2022 was awarded funding from ESA InCubed for the development of its upcoming FOREST-3 CubeSat – all of which served as important stepping stones to the firm joining Copernicus.

Read the full article on spacedata.copernicus.eu.

Image courtesy of OroraTech

ESA Φ-lab broadens its cloud-service support to start-ups

Posted on by nchimonides

As part of its extensive efforts to nurture innovation in Earth observation technologies and applications, ESA Φ-lab is providing its supported companies with preferential rates agreed with two leading cloud-service suppliers in collaboration with the ESA Commercialisation, Industry and Competitiveness Directorate. Scaleway will provide start-ups with a cloud storage and business services package, while Ellipsis Drive is offering deals on its cloud spatial-data management platform.

ESA Φ-lab is a major driving force behind innovation and commercialisation in European Earth observation, lending support to businesses of all shapes and sizes through the ESA InCubed programme and various research initiatives. Help comes not only in the form of funding and technical and commercial support, but also in the creation of purpose-driven partnerships that deliver essential business and commercialisation services. Two such arrangements, drawn up in collaboration with the ESA Partnership Initiative for Commercialisation (EPIC), have now been agreed with premier cloud-service providers Scaleway and Ellipsis Drive.

French company Scaleway supplies cloud infrastructure and services to over 25 000 customers, including more than 700 European start-ups. CEO Damien Lucas explains the nature of Scaleway’s offer within the ESA partnership agreement: “Qualifying businesses from ESA’s portfolio will be fast-tracked through the selection process of our Startup Programme. Upon acceptance, they will have access to a wide range of perks, including cloud credits to be spent on our public cloud products, dedicated expert advice and access to our global community.

“We’re very proud to play our part in helping European space-sector entrepreneurs establish their business systems and we look forward to working with the commercialisation teams in ESA on joint promotional activities.”

Ellipsis Drive provides a cloud-based B2B tool for ingesting, organising and accessing spatial data, with web-based visualisations and integrated use via a myriad of plug-ins, packages and applications. Based in the Netherlands, Ellipsis Drive has hosted data for over 400 customers to date and currently has around 2500 users managing and consuming spatial content from its platform.

“We see partnering with ESA as a clear win-win scenario, giving start-ups access to our spatial data management, visualisation and integration service on very favourable terms, while also enabling us to tap into a broader customer community in the space sector,” says Ellipsis Drive’s Rosalie van der Maas. “We’re offering up to 100GB of free storage for the first year, with substantial discounts for larger plans and subsequent periods.”

Further details on these offers and how to take advantage of them will be communicated to InCubed and other ESA Φ-lab-supported companies in the near future, and will also be distributed via the ESA BIC and ESA Technology Broker Network.

Michele Castorina is InCubed Programme Manager and head of the ESA Φ-lab Invest Office: “The fact that ESA creates partnerships for the benefit of European start-ups is testimony to the across-the-board helping hand that the Agency provides to the space industry. Cloud storage and spatial data hosting are especially relevant for commercial Earth observation, and I fully expect these services to be a significant asset to InCubed-co-funded businesses as they scale up their operations.”

“Having the right tools for their business infrastructure is a vital stepping stone to market success for early-stage enterprises,” added Joana Kamenova, Commercialisation Officer and lead for EPIC at ESA. “This type of collaboration helps to advance the growth of the European space ecosystem, and we will actively promote Scaleway’s and Ellipsis Drive’s offer packages to our networks.”

To know more: ESA Φ-lab, InCubed, EPIC, ESA BICs, ESA Technology Brokers, Scaleway, Ellipsis Drive

Photo courtesy of Fauxels