ESA title

Category: Philab

Irish minister commends InCubed contribution to entrepreneurial space sector

Posted on by nchimonides

During a visit yesterday to ESA’s ESRIN establishment, the Irish National Delegation to ESA took a tour of Φ-lab and discussed the importance of the ESA InCubed programme. The day’s schedule also included the signing of a contract for the InCubed-supported PROTELUM activity.

The Irish Delegation was received yesterday at ESRIN by Simonetta Cheli, Director of Earth Observation Programmes and head of the establishment. The visitors were given a tour of the site’s facilities and participated in a number of sessions covering ESA’s Earth observation (EO) programmes, Ireland’s space policy and technical discussions with the Earth Observation Directorate management team.

The tour featured a visit to Φ-lab, where Division Head Giuseppe Borghi explained the lab’s mission and highlighted some of its flagship programmes. Φ-lab’s focus on transformational innovation in commercial EO was a key theme of the day, with a number of managers from Irish space-sector businesses among the guests. Several of these companies have benefited directly from Φ-lab support with the co-funding of development activities through InCubed.

“Ireland has a strong tradition of entrepreneurship in many sectors and, as our industry representatives showed during the visit to ESRIN, New Space is no exception,” commented Damien English TD, Irish Minister of State for Business, Employment and Retail. “We continue to be impressed by the work that Φ-lab and its InCubed programme are doing in nurturing private-sector Research and Development in Earth observation. It is enabling Irish companies to realise their potential by accelerating the commercialisation of their products and services, which is a key deliverable highlighted in Ireland’s National Space Strategy for Enterprise.”

One such InCubed initiative is the PROTELUM activity, which was launched yesterday at a signing ceremony during the visit. Developed by Dublin-based Davra, PROTELUM is a management tool for the ongoing compliance assessment and monitoring of mining sites. The platform will cover the entire mining life cycle and will enable operators and regulators to continually identify safety risks, both underground and at the surface. The solution will apply analytical methods to data from sources such as industrial Internet of Things (IoT) sensors, EO satellites and drones in order to provide actionable insights and predictive modelling.

County Cork’s Treemetrics also attended the event and gave a brief overview of its Satforcert product in one of the technical sessions. Satforcert uses EO-derived data in combination with Global Navigation Satellite Systems (GNSS) to create more efficient and transparent processes for sustainable forest management certification. Currently enjoying its second stint of InCubed co-funding, the product has successfully completed end-user validation and is now being expanded to include features related to forest carbon storage and credits.

Other InCubed-supported companies present during the visit included mBryonics, Skytek, TechWorks Marine and Icon Geo.

Simonetta Cheli added: “It has been a pleasure to welcome the Irish Delegation today in what has been an extremely fruitful exchange of ideas on the current status and future direction of European Earth observation. The contribution from industry partners has been particularly stimulating, with for example Davra and Treemetrics both demonstrating how commercial EO can contribute to sustainable development by providing vital monitoring tools. We are therefore delighted to support the PROTELUM and Satforcert initiatives through the ESA InCubed programme.”

To know more: ESA InCubed, Davra, Treemetrics, mBryonics, Skytek, TechWorks Marine, Icon Geo

InCubed to be represented at ESA’s Industry Space Days 2022

Posted on by gaiamaggi

Michele Castorina, Head of the Φ-lab Invest Office, will talk about EO Market trends and present an overview of the ESA InCubed programme at the Industry Space Days (ISD) event, taking place on 28–29 September at ESA’s ESTEC establishment.

The ISD is an annual event organised by the ESA SME Office of the Directorate of Commercialisation, Industry and Procurement. Aimed at fostering cooperation between actors in the space sector, the gathering is free of charge and open to entities and investors from ESA Member States, Associate States, Cooperating States and the European Union. Participants can register here.

This year, Φ-lab’s Michele Castorina will be at the ISD to give attendees a flavour of the mission and activities of InCubed, including how companies, innovators and entrepreneurs can apply for co-funding of product/service development and commercialisation initiatives in the Earth observation sector. Michele will also be on hand for informal discussions with interested entities.

To know more: ISD 2022, ESA SME Office, InCubed

The AI effect: high-performing Sentinel-2 cloud mask goes global

Posted on by esaphilabincubed@2020_admin

After releasing its free and open-source cloud mask for Copernicus Sentinel-2 data, Estonian company KappaZeta is now working on enlarging the model from the Northern European summer season to year-round global coverage. Developed in conjunction with ESA Φ-lab, KappaMask is already outperforming similar approaches and uses Artificial Intelligence (AI) and active learning techniques to optimise automatic data labelling.

Although well established as a gold-standard provider of Earth observation (EO) insight, the data from Sentinel-2, like all optical satellite imagery, needs to have cloud and cloud-shadow areas identified and filtered out. Creating a cloud mask, effectively a stencil that removes unwanted data, is an essential step for virtually any EO application, and as ESA has shown with the Φ-sat-1 experiment, masking can be done effectively at source through onboard processing on the satellite. For Sentinel-2 users however, free, accurate and user-friendly cloud masks are currently few and far between. While masking is relatively simple when studying small areas, large stacks of imagery require automated pre-processing in order to provide timely, valid data to the user.

KappaZeta set out to provide an AI-powered, free-of-charge solution for Sentinel-2 data users worldwide. In an initiative funded by ESA Φ-lab, development commenced in 2020, with Phase 1 focused on Northern European summer-season conditions. Refining the mask was aided by the adoption of active learning, an approach which selects the highest impact samples for labelling. “We needed a reference dataset to train and test our model,” explains KappaZeta CEO Kaupo Voormansik. “Manual labelling of satellite imagery is a slow and expensive process, but as interest has grown in Deep Learning, the active learning methodology has proven to be a powerful tool for efficiently creating high-variety reference cloud masks using limited resources.”

Phase 1 was completed in August 2021, with the outcome published in a research paper and the initial version of KappaMask released to the public. The European model proved to be highly accurate and in fact performed better than comparable products, with particularly noteworthy results in the detection of cloud shadows and small fragmented clouds – a problematic area for some previous cloud masks.

The successful release of the Northern European summertime mask was followed by Phase 2, which aims to extend the model to the rest of the world over all seasons. This entails both improving the Phase-1 model architecture and obtaining a global reference dataset. For the latter, KappaZeta has used a combination of existing labelled datasets and its own labelling, the plan being to add 5000 newly segmented sub-tiles (each consisting of a 512 by 512 pixel area) to improve model accuracy. With an eye once again on efficient working, the team has picked the sub-tile locations based on Sentinel-2 data download statistics, thereby selecting according to user interest rather than aiming for a blanket global coverage.

Nicolas Longépé, Φ-lab data scientist and one of the ESA supervisors for KappaMask, recognises the effectiveness of the company’s research paradigm: “KappaZeta has used a smart approach for developing its cloud mask, with active learning and demand-based coverage helping to achieve the right trade-offs in terms of precision versus effort. Indeed the Phase-1 results have already shown KappaMask to be one of the most accurate free-to-use cloud masks, and once complete we fully expect the product to significantly enrich the analysis toolbox of Sentinel-2 data users.”

“We are also happy to support the project to see how KappaMask compares with other available solutions,” added Valentina Boccia of the ESA EO Ground Segment Department. “KappaZeta’s work convincingly illustrates how innovative AI techniques could be integrated into the mainstay of Sentinel-2 data processing.”

KappaMask is scheduled for release as a cloud-masking web service later this year. The reference dataset and source code will be freely available, and details of the model and the accuracy validation will be published in a forthcoming paper.

To know more: Sentinel-2, KappaZeta, Φ-lab Explore Office

ESA explores cognitive computing in space with FDL breakthrough experiments

Posted on by nchimonides

In a series of world firsts, the Frontier Development Lab (FDL) programme in collaboration with ESA has achieved significant results in the field of Cognitive Cloud Computing in Space (3CS). In experiments on a D-Orbit InOrbit NOW satellite carrier mission, FDL has shown how a Machine Learning (ML) payload can reduce downlink latency, easily adapt to different optical instruments and be updated directly in space, while enabling fast information extraction and delivery to end users. FDL has also created an ML payload for third-party application hosting, launched on the latest D-Orbit mission in January.

It can take considerable time to fuse and extract actionable insights from space-derived data streams, which are often enormous. Just one image tile of Earth from ESA’s Sentinel-2 spacecraft, covering a 100 km by 100 km square, is 2.5 GB – the same size as a movie download. Flood information for emergency response is a prime example of where updated satellite-derived insights are indispensable for directing relief efforts, but bottlenecks in the downloading and analysis of images can lead to delays of several hours or even days in making actionable insights available.

Enter Cognitive Cloud Computing in Space (3CS), which has the potential to relieve such challenges by drawing on advances in federated Machine Learning combined with the provisioning of high-powered computational hardware in orbit. 3CS envisages large-scale intelligent swarm systems in space, where multiple spacecraft and instruments come together to empower Earth observation (EO), returning relevant and verified actionable insights to the ground within seconds. ESA is giving significant attention to 3CS and has already made inroads into understanding its onboard computing requirements through the Φ-sat concept, with the Φ-sat-1 experiment launching in 2020. ESA Discovery is funding 12 projects related to 3CS, and in the commercial EO environment, ESA InCubed is co-funding the AI-express (AIX) activity being developed by Planetek, D-Orbit and AIKO.

To prove the viability of some elements of the 3CS vision, FDL set up its NIO (Networked Intelligence in Orbit) experiments with funding and support from ESA Φ-lab. The NIO trials are based around the WorldFloods ML payload, which was developed by young data scientists in partnership with ESA Φ-lab in a 2019 FDL Europe sprint and subsequently published in Nature. The payload was launched in June of last year on the D-Orbit InOrbit NOW (ION) Wild Ride mission and runs on D-Orbit’s own Nebula cloud environment in tandem with Unibap’s SpaceCloud computer.

Firstly, in an emulation of onboard intelligent data processing, the WorldFloods payload took a pre-loaded Sentinel-2 tile of flood images and converted the pixel data to bounded polygons of flood areas. This resulted in a 10 000-fold reduction in data packet size, with the processed tile then rapidly downlinked and shown to perform comparably with conventionally produced flood maps.

Next, the WorldFloods ML payload was adapted to a different instrument. Instead of the high-resolution images from Sentinel-2, the ML payload received images from the Wild Ride onboard RGB D-Sense camera. Despite the fact that the camera has a fairly coarse resolution and was not designed for EO, the ML payload was successfully fine-tuned with only a few images and generated reasonably accurate vector maps of waterbodies, land and clouds. The maps were then downloaded in just 36 seconds.

Finally, the NIO team achieved another breakthrough by deploying an updated ML payload in orbit. Any ML pipeline needs to be maintained and refined, and so the ability to upload new model parameters or ‘weights’ to an onboard processor is a key component of any future 3CS infrastructure. The new model weights were uplinked to the D-Orbit/Unibap platform without a hitch, and subsequent functional testing demonstrated increased flood detection performance.

James Parr, whose company Trillium Technologies runs the FDL programme, sums up the value of the NIO results: “Taken together, these experiments give a tantalising glimpse of the promise of 3CS – how spacecraft working together with in-orbit cloud infrastructure can enable hybrid observation and adaptive space services. We have the potential to revolutionise how we respond to disasters, manage emissions and pollution, improve weather forecasts and foster next-gen space situational awareness.”

In a further NIO development, a set of advanced services tests has been funded by ESA for another D-Orbit ION satellite carrier. Launched in January, the Dashing Through the Stars mission includes a miniaturised hyperspectral camera, designed by research institution VTT. As part of the ESA-commissioned tests, FDL has created a customisable ML payload that allows third parties to upload and run Deep Learning models for onboard processing of the VTT camera images, delivering insights tailored to many different applications directly to the ground.

“The idea of intelligent, federated satellites operating in concert to provide insight faster and more accurately is a fundamental enabler for the future of EO, opening up new avenues for shaping the future of software-defined missions,” says Φ-lab data scientist Nicolas Longépé. “The NIO experiments on Wild Ride have served as a compelling proof of concept for 3CS, and with the launch of Dashing Through the Stars we’ll be able to see how access to such a system can be extended to new downstream applications.”

David Steenari, a data processing engineer from the ESA Directorate of Technology, Engineering and Quality, underlines the broad-based ESA support for Dashing Through the Stars: “The camera development, along with its integration on to the ION carrier and in-flight calibration, have been developed under a combination of GSTP and TDE funding. Coupling the imager to FDL’s ML payload will amply demonstrate exactly what tomorrow’s commercially accessible payloads will be able to deliver.”

To know more: Trillium, FDL Europe, SpaceCloud, WorldFloods

Access to Earth observation data to improve with AI-based I*STAR platform

Posted on by nchimonides

Telespazio has signed a contract with the ESA InCubed programme to develop an innovative service for improving access to Earth observation data. Derived from artificial intelligence (AI) models, I*STAR will allow new user groups to request customised, smart data acquisitions from satellite constellations simply, efficiently and promptly.

The community of Earth observation players is growing and diversifying, as is the number of missions and business models in the sector. Users need a simple solution that recommends intelligent acquisitions of satellite data, while national and international space agencies could benefit from a platform which enables them to promote their missions to those same users. All stakeholders are naturally keen to reduce operational costs.

I*STAR is being developed to address these needs by Telespazio, a joint venture between Leonardo (67%) and Thales (33%). The as-a-service solution will provide one-click access for organisations to acquire EO data. Using Deep Learning and Machine Learning algorithms, I*STAR can model user preferences with regard to satellite platforms, sensors, areas of interest and types of products, ensuring that even non-specialist customers can make specific requests from missions without the need for direct support from space operations. Through the automation of data acquisition processes, human intervention is reduced and resources are freed up, giving rise to tangible cost savings.

A further advantage of the service will be the ability to improve response times for disaster relief, allowing authorities and civil protection entities to react more efficiently and effectively.

Marco Brancati is Telespazio’s Head of Innovation and Technical governance: “I*STAR introduces a brand new solution in the Earth observation ground segment –  the ability to request products or acquisitions according to user profiles while minimizing the need to know specific mission or to have operational skills. We’re very pleased and encouraged that the ESA InCubed programme has recognised our novel approach and given us the opportunity to bring I*STAR to market.”

“I*STAR is built on the idea that AI is a key enabler for new ways to exploit EO data,” commented Michele Castorina, Head of the Φ-lab Invest Office. “Improving usability and access for an ever-wider user community will help to invigorate commercial EO by providing a marketspace for both downstream and institutional operators.”

The I*STAR activity kicked off in April and is expected to hold its first major development review in October.

To know more: Telespazio, InCubed

Photo courtesy of Telespazio

Teach an Earth-observing satellite to know what it sees

Posted on by nchimonides

For decades now Earth observation satellites have been monitoring our ever-changing home planet; the next step is to enable them to recognise what they see. The latest public challenge for the machine learning community from ESA’s Advanced Concepts Team is to train satellite software to identify features within the images it acquires – with the winning team getting the unique opportunity to load their solution to ESA’s OPS-SAT nanosatellite and test it in orbit.

Edge computing expert Gabriele Meoni of ESA’s Ф-lab at ESRIN, focused on Earth Observation – which has developed this challenge jointly with the ACT – explains: “ESA’s AI-equipped Ф-sat-1, aboard the Federated Satellite Systems (FSSCat) CubeSat, has already demonstrated the benefits of AI on-board – it is able to detect images filled with cloud cover and set these aside. With our new ‘OPS-SAT case’ competition, we seek to take this approach further. Participating teams receive 26 full-sized images acquired by the OPS-SAT CubeSat, which include small 200×200-pixel crops or ‘tiles’ identified with one of eight different classifications – Snow, Cloud, Natural, River, Mountain, Water, Agricultural, or Ice – with a total of ten examples of each type, representing a baseline for feature identification.”

The dataset for the challenge was collected in 2021 and specifically not published in order to keep the competition fair.

Read the full article on www.esa.int.

Photo courtesy of ESA, CC BY-SA 3.0 IGO

LPS22: Φ-lab shows how new digital tech and computing paradigms will reshape EO

Posted on by nchimonides

With its themes of emerging technologies and the competitive business sector in Earth observation (EO), this year’s Living Planet Symposium (LPS22) championed topics that are fundamental to the mission of ESA Φ-lab. Indeed both the Explore and Invest offices of Φ-lab made major contributions to the event in areas such as Artificial Intelligence for Earth observation (AI4EO), Virtual Reality (VR), New-Space missions and quantum and neuromorphic computing.

“ESA Φ-lab’s defining mantra of transformative innovation in Earth observation, from idea creation through to supporting leading-edge product and service development up to market adoption, has allowed us to play a significant role at the 2022 Living Planet Symposium. Through presentations, discussions and practical demonstrations, our researchers and business innovators have provided a comprehensive view of disruptive activities in EO in both the institutional and commercial sectors.” – Giuseppe Borghi, Head of Φ-lab.

As one of the largest Earth observation conferences in the world, ESA’s Living Planet Symposium is a critically important forum for understanding our environment and climate. The 2022 edition was held last week at the World Conference Center in Bonn, and one of its central topics dealt with Enabling the Earth Observation digital transformation using emerging technologies, AI and data analytics to create new opportunities across the entire sector.

ESA Φ-lab was well represented at the event with a very broad range of offerings, including two sessions dedicated to showcasing the wealth of activities covered by the ESA InCubed Programme­­­­. In one of the Agora interactive forums, an overview of InCubed was presented within the context of the EO commercial sector as a whole and the forthcoming ESA Ministerial Council (CMIN22). There were also examples of current initiatives and strategic partnerships that Φ-lab is either developing or has put in place to support innovation and investment, such as the recently announced free cloud services from partner OVHcloud. The second session gave a detailed view of several commercial EO satellite missions that are benefitting from InCubed co-funding.

A key area of innovation and cutting-edge research within Φ-lab is the future of computing for EO, and the subject featured in sessions chaired or co-chaired by Φ-lab researchers. In ‘AI@edge and Emerging Computing Paradigms for the Future of Earth Observation’, speakers presented research in a field which offers both significant computational opportunities and some daunting challenges. Andrzej Kucik was one of the session’s conveners: “With talks from both industry and academia, this session revealed the state of the art in deploying AI models at the ultimate edge – space. We also had a demonstration by IniVation of their neuromorphic event camera, which is inspired by biological retinas and has enormous potential for high-resolution, power-efficient sensing.” The discussion on neuromorphic algorithms continued in a subsequent Agora session co-hosted by Gabriele Meoni, where participants made some fascinating contributions on other technologies that are revolutionising information processing, including quantum, distributed and hybrid computing.

The Φ-lab stand at LPS22

Attendees were particularly keen to see Φ-lab’s sensor-derived virtual reality simulations. Located in the ESA exhibition booth, the VR demonstration was run by the Explore Office and based around two research initiatives. In one experience, users explored a virtual 3D model of the Earth through the VR headset and were able to overlay and analyse various EO datasets as they swept across the surface of the planet.

A second headset was on hand displaying the first prototype of a digital twin of ESRIN (the ESA establishment that is home to Φ-lab). “Digital Twin ESRIN is a complete 3D reconstruction of our site, created from drone imagery in combination with in-situ data from vegetation health and air quality sensors,” explained Head of the Explore Office Pierre Philippe Mathieu. “This simulation aims to provide a live twin of a geographical area in order to enable real-time quantification of our environment and its evolution, all delivered within a high-impact VR experience.”

Φ-lab’s presence was also highly evident in a number of other gatherings at LPS22. Division head Giuseppe Borghi discussed transformative innovation at the Digital Copernicus session, promoting its role in generating a unique competitive advantage in the European EO ecosystem. Following on from her team’s UNESCO-award-winning research on dengue fever monitoring, Rochelle Schneider dos Santos was well placed to co-host the ‘Earth Observation for Health’ session, but also facilitated a forum for young scientists called ‘Meet the Next GenEO’. Nicolas Longépé helped organise three sessions on super-resolution methods and SAR data analytics, and many other Φ-researchers shared results and ideas in discussions and on poster boards throughout the week.

The plenaries of LPS22, including ‘Future EO’ and ‘New Space & EO (Commercial Markets)’, are all available for streaming here.

To know more: Living Planet Symposium, ESA Φ-lab, Φ-lab Explore Office, ESA InCubed Programme

Main photo courtesy of ESA/JürgenMai

Boosting commercial Earth observation

Posted on by nchimonides

One of the objectives of the Living Planet Symposium, taking place this week in Bonn, is to foster interaction between the institutional and commercial sectors to boost the Earth observation space economy. This is being achieved by highlighting existing partnerships, expanding the number of data users and facilitating access to private funds for companies.

With numerous contributions from industry, investors and ESA’s Commercialisation, Industry and Procurement and Earth Observation Programme directorates, the symposium has brought the business side of observing our planet to the fore. Several sessions featured the ESA InCubed programme, which helps start-ups, mid-cap and large Earth observation players bring their innovative ideas to market through commercial, technical support and investment advice.

Read the full article on www.esa.int.

Living Planet Symposium kicks off

Posted on by nchimonides

ESA’s Living Planet Symposium has opened with a flourish with over 4000 participants including scientists, academics, space industry representatives, institutional stakeholders, data users, students and citizens gathered to discuss the latest findings on our changing planet, as well as advances in satellite technologies, new opportunities in the commercial world, and ESA’s plans for the future.

The symposium – one of the biggest Earth observation conferences in the world – takes place every three years. With the urgency to understand and monitor our planet from space to address the climate crisis and the growing interest in satellite data for all manner of uses that benefit society and the economy, each Living Planet Symposium garners more interest than the last.

Read the full article on www.esa.int.

Prestigious UNESCO award given for Φ-lab AI-powered dengue fever research

Posted on by nchimonides

The International Research Centre in Artificial Intelligence under the auspices of UNESCO (IRCAI) has recognised a Φ-lab initiative as part of its Global Top 100, a list of projects solving problems related to the United Nations Sustainable Development Goals (SDGs). The research, carried out in collaboration with UNICEF, developed an Artificial Intelligence (AI) solution for quantifying dengue fever outbreaks from space.

As one of the world’s most common and rapidly spreading arboviral diseases, dengue fever causes major public health and economic consequences in tropical and sub-tropical regions. Forecasting outbreaks is an essential tool for aid agencies and health authorities, but the complex dynamics of the spread of dengue present major challenges for modelling.

A Φ-lab team worked with UNICEF to create an AI-ensemble model for predicting dengue outbreaks based on Earth Observation (EO) satellite data and products. Φ-lab research leader Rochelle Schneider dos Santos explains: “It’s crucial to factor in long-term dependency in time series data when modelling outbreaks, but this is problematic at country level due to geographical variations in dengue incidence. Our climate-based ensemble model used multiple Machine Learning approaches to predict the incident rate one month in advance for each administrative area in Peru.”

IRCAI named the Φ-lab/UNICEF research on its 2021 Global Top 100 list of AI solutions for sustainable development. The project came under the Promising category in recognition of the successful piloting of the dengue predictive model in Peru (the pilot has since been extended to Brazil).

“This project is a perfect example of collaboration between a humanitarian organisation and a research entity to support the UN SDGs,” said Dohyung Kim, Lead Data Scientist at the UNICEF Office of Global Innovation. “Not only is the outcome of the project expected to deliver information and insights for policy makers, it also provides tools which can be directly used by field operators to track the changes in dengue prevalence at a finer scale, thanks to the powerful Earth observation data from ESA.”

Full details on the Φ-lab entry for the IRCAI Global Top 100 can be found here, and the complete list of projects is here. A research paper on the initiative was published in the proceedings of the International Conference on Machine Learning 2021.

To know more: UNICEF, IRCAI, UNESCO