A thunderous shift in foundation model architecture with THOR

Foundation models are enabling new ways to use Earth observation data, but most existing models struggle to handle data from diverse sensors and are limited to fixed patch sizes. This makes them hard to use in real-world applications that require flexibility. Funded by ESA Φ-lab and developed by the Norwegian Computing Centre, THOR is a new foundation model designed to overcome both the challenges of heterogeneous inputs and rigid deployment constraints.

Foundation models are driving a paradigm shift in Earth observation, moving the field away from specialised models towards general-purpose geospatial intelligence. Although they promise to revolutionise the way we interact with satellite data, most current foundation models are architecturally rigid.

This means they are trained using a fixed input image size and a fixed patch size (the size of small, non-overlapping segments into which input images are divided before being fed to the model), making it more difficult to process data that differs, even slightly, from the format they saw during training.

Their rigidity creates a bottleneck for data-efficient adaptation: when the workflow breaks down the data into smaller patches, it produces a low-resolution sequence of tokens – units of data that foundation models process to understand the input they were given and then generate an output. Subsequent, dense pixel-level tasks like segmentation will then require large, complex decoders to upsample features. These decoders often require large amounts of data for fine-tuning, undermining the efficiency of foundation models.

Inspired by the Norse god of thunder and his legendary hammer, THOR (Transformer-based foundation model for Heterogeneous Observation and Resolution) is a versatile multi-modal foundation model that will shatter these shortcomings. This model has been developed by the Norwegian Computing Center, funded and supported by ESA Φ-lab through ESA’s Foundation Models for Climate and Society (FM4CS) project.  

THOR is the first foundation model with an architecture that unifies the 10 – 1000m ground sampling distance range of Sentinel-1, -2 and -3, including the OLCI (Ocean and Land Colour Instrument) and SLSTR (Sea and Land Surface Temperature Radiometer) sensors.

This model has been trained on the LUMI high-performance computer using the THOR Pretrain dataset, a 22TB-dataset that has been aligned spatio-temporally and across modalities, and that contains diverse land cover products, digital elevation models, and ERA5-Land variables. By incorporating a randomised patch size and input image size during pre-training, THOR becomes ‘computer-adaptive’.

Other state-of-the-art models like TerraMind, DOFA or Copernicus-FM are flexible in handling diverse inputs, but not so versatile when it comes to deployment. These models have a fixed internal resolution, meaning that, for very fine‑grained tasks like detailed floods or crop boundaries, they often rely on large, complex task‑specific decoders to recover detail.

Instead of locking the model into a fixed image size and resolution, THOR can change its internal resolution at inference time, allowing users to trade accuracy for computational cost without retraining the model: coarser patches could be used for faster, global analyses, while smaller patches can be used for more detailed, local maps.

This way, THOR solves both input heterogeneity and deployment versatility, focusing on making a single model adaptable and efficient across resolutions, data availability, and deployment constraints. THOR achieved state-of-the-art performance and demonstrated its superior data efficiency in the PANGAEA 10% benchmark, a standardised, open-source benchmarking framework designed specifically to evaluate the performance of geospatial foundation models (GFMs). The 10% benchmark refers to a specific, low-data evaluation scenario within PANGAEA designed to assess the effectiveness of GFMs when they are trained using only 10% of the labelled data for downstream tasks. 

Valerio Marsocci, Internal Research Fellow at ESA Φ-lab, comments the importance of THOR for real-world scenarios: “With dense, high‑quality features produced directly from the encoder, THOR often requires much simpler downstream models, which improves robustness and reduces costs. By providing a flexible pre-training starting point, we empower scientists to solve both local and global problems – whether it is mapping disasters or monitoring crop health – without needing to reinvent the architectural wheel.”

For Arnt-Børre Salberg, Chief Research Scientist at the Norwegian Computing Center, THOR sets a new standard for foundation models in the European space ecosystem: “We developed THOR to be a global ‘go-to’ foundation model for Earth observation. This open-access tool transforms satellite data into vital intelligence for maritime security, hydropower energy management and emergency preparedness against floods and avalanches, being an essential tool for a safer, more sustainable future driven by Norwegian innovation.”

THOR is helping Norway consolidate its strategic position in the Arctic region, according to Dag Anders Moldestad, Lead, Earth Observation at the Norwegian Space Agency: “Norway occupies a unique vantage point in the Northern Hemisphere. For us, satellites are not just tools, but our eyes on the ground.”

“What makes THOR a game-changer is its flexibility. It allows us to develop and deploy services in real time with significantly less computing power, so we can respond to crises as they happen. In disaster management, where every second counts, or in tracking the rapid shifts of our climate, THOR provides the speed and efficiency necessary to turn raw data into valuable information”, Moldestad added. 

Find more information about THOR’s technical details in this arXiv paper. The model and pretrain dataset are now available on Hugging Face. Its source code and TerraTorch extension are available on GitHub. A showcase of THOR can be found here.

To know more: FM4CS, ESA Φ-lab, Norwegian Computing Center

Photo courtesy of Unsplash/Mark Kӧnig