The NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography (AI2ES) is developing trustworthy AI for a wide variety of weather, climate, and coastal phenomena. In this talk, I will briefly overview AI2ES and then focus specifically on our recent developments in developing explainable and interpretable AI for the earth sciences. Specifically, I will discuss recent work (by Mamalakis et al, 2021; 2022) developing benchmark datasets to objectively assess XAI methods for geoscientific applications, work by Flora et al (in prep) on developing a standard toolkit for XAI for the earth sciences and to assess the validity of XAI methods, applications of XAI to a 3D CNN coastal fog predictive model by Krell, Kamangir et al (submitted), and quantification of the sources of XAI uncertainty from deep ensembles by Gagne et al. (in prep). Because AI2ES is focused on developing AI that is deemed trustworthy by environmental forecasters and managers, I will also describe our preliminary findings on weather forecasters' perceptions of AI trustworthiness, explainability, and interpretability (Cains et al., 2022), in the comparative context of reviews of theoretical and empirical research on explainability, trust, and trustworthiness (Smith et al. in prep, Wirz et al. in prep).

The extremely limited communications bandwidth between Earth and distant spacecraft forms one of the greatest challenges to planetary science advancement. Rather than the 100’s of TB’s of remote sensing data now common in terrestrial remote sensing, a mission to distant ocean worlds like Enceladus or Europa may have only 75 MB of total downlink for both science observations and engineering data. Onboard science capabilities, a unique new form of autonomy, can mitigate much of this bottleneck by recognizing, summarizing, and prioritizing science observations based on their utility to ground science teams. As this requires building a new level of trust with mission scientists, these onboard systems must be constructed with reconfigurability and interpretability as primary requirements, as well as the capability to provide overlapping lines of evidence for any drawn conclusions, all with very limited onboard compute power. As these systems often incorporate machine learning and other data-driven solutions, they form a unique challenge area advancing the definition and boundaries of model interpretability for scientific insight generation.

Astrophysical light curves are particularly challenging data objects due to theintensity and variety of noise contaminating them. Yet, despite the astronomi-cal volumes of light curves available, the majority of algorithms used to processthem are still operating on a per-sample basis. To remedy this, we propose a sim-ple Transformer model –called Denoising Time Series Transformer (DTST)– andshow that it excels at removing the noise in datasets of time series when trainedwith a masked objective, even when no clean targets are available. Moreover, theuse of self-attention enables rich and illustrative queries into the learned represen-tations. We present experiments on real stellar light curves from the TransitingExoplanet Space Satellite (TESS), showing advantages of our approach comparedto traditional denoising techniques.

The upcoming Daniel K. Inouye Solar Telescope (DKIST) will produce unprecedented high-cadence, high-resolution, and multi-line spectropolarimetric observations of the Sun. New computational techniques are needed to infer the state of the Sun's atmosphere from these observations. Deep learning is a promising approach to this spectropolarimetric inversion problem that can both provide real-time visualizations to astronomers and potentially improve upon existing algorithms by combining spatial, temporal, and multi-spectral information. Here we investigate group equivariant deep learning as a method for inferring the three-dimensional photospheric structures, training on magnetohydrodynamic (MHD) simulations of two types of solar features: sunspots and active regions. Our results demonstrate that including multiple lines improves the mean relative error from 18.6% to 14.4%, averaged over all MHD state variables, and that using group equivariant convolution architectures further improves the mean relative error to 12.5%.

High resolution remote sensing imagery is used in a broad range of tasks, including detection and classification of objects. It is, however, expensive to obtain, while lower resolution imagery is often freely available and can be used for a range of social good applications. To that end, we curate a multi-image multi-spectral dataset for super-resolution of satellite images. We use PlanetScope imagery from the SpaceNet-7 challenge as the high resolution reference and multiple Sentinel-2 revisits of the same location as the low-resolution imagery. We provide baselines for both single imager super-resolution and multi-image super-resolution. We also provide an ablation on how number of scenes, cloud cover and dynamism in different scenes in the dataset affect performance. Finally, we provide our code to construct the dataset along with implementations of baselines for the community to build upon.

Learning latent representations has aided operational decision-making in several disciplines. Its advantages include uncovering hidden interactions in data and automating procedures which were performed manually in the past. Representation learning is also being adopted by earth and environmental sciences. However, there are still subfields that depend on manual feature engineering based on expert knowledge and the use of algorithms which do not utilize the latent space. Relying on those techniques can inhibit operational decision-making since they impose data constraints and inhibit automation. In this work, we adopt a case study for nitrogen response rate prediction and examine if representation learning can be used for operational use. We compare a Multilayer Perceptron, an Autoencoder, and a dual-head Autoencoder with a reference Random Forest model for nitrogen response rate prediction. To bring the predictions closer to an operational setting we assume absence of future weather data, and we are evaluating the models using error metrics and a domain-derived error threshold. The results show that learning latent representations can provide operational nitrogen response rate predictions by offering performance equal and sometimes better than the reference model.

Numerical simulators are essential tools in the study of natural fluid-systems, buttheir performance often limits application in practice. Recent machine-learningapproaches have demonstrated their ability to accelerate spatio-temporal predictions, although, with only moderate accuracy in comparison. Here we introduce MultiScaleGNN, a novel multi-scale graph neural network model for learning to infer unsteady continuum mechanics in problems encompassing a range of length scales and complex boundary geometries. We demonstrate this method on advection problems and incompressible fluid dynamics, both fundamental phenomena in oceanic and atmospheric processes. Our results show good extrapolation to new domain geometries and parameters for long-term temporal simulations. Simulations obtained with MultiScaleGNN are between two and four orders of magnitude faster than those on which it was trained.

Subsurface simulations use computational models to predict the flow of fluids (e.g., oil, water, gas) through porous media. These simulations are pivotal in industrial applications such as petroleum production, where fast and accurate models are needed for high-stake decision making, for example, for well placement optimization and field development planning. Classical finite difference numerical simulators require massive computational resources to model large-scale real-world reservoirs. Alternatively, streamline simulators and data-driven surrogate models are computationally more efficient by relying on approximate physics models, however they are insufficient to model complex reservoir dynamics at scale.Here we introduce Hybrid Graph Network Simulator (HGNS), which is a data-driven surrogate model for learning reservoir simulations of 3D subsurface fluid flows. To model complex reservoir dynamics at both local and global scale, HGNS consists of a subsurface graph neural network (SGNN) to model the evolution of fluid flows, and a 3D-U-Net to model the evolution of pressure. HGNS is able to scale to grids with millions of cells per time step, two orders of magnitude higher than previous surrogate models, and can accurately predict the fluid flow for tens of time steps (years into the future). Using an industry-standard subsurface flow dataset (SPE-10) with 1.1 million cells, we demonstrate that HGNS is able to reduce the inference time up to 18 times compared to standard subsurface simulators, and that it outperforms other learning-based models by reducing long-term prediction errors by up to 21%.

We have a poster session and a breakout room. In the breakout room, you can meet and chat with other workshop participants.

FourCastNet, short for Fourier Forecasting Neural Network, is a global data-driven weather forecasting model that provides accurate short to medium-range global predictions at $0.25^{\circ}$ resolution. FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning wind energy resources, predicting extreme weather events such as tropical cyclones, extra-tropical cyclones, and atmospheric rivers. FourCastNet matches the forecasting accuracy of the ECMWF Integrated Forecasting System (IFS), a state-of-the-art Numerical Weather Prediction (NWP) model, at short lead times for large-scale variables, while outperforming IFS for variables with complex fine-scale structure, including precipitation. FourCastNet generates a week-long forecast in less than 2 seconds, orders of magnitude faster than IFS. The speed of FourCastNet enables the creation of rapid and inexpensive large-ensemble forecasts with thousands of ensemble-members for improving probabilistic forecasting. We discuss how data-driven deep learning models such as FourCastNet are a valuable addition to the meteorology toolkit to aid and augment NWP models.

Accurately predicting air quality levels at a fine resolution is a critical task to ensuring the public's health is not at risk. In this work, we construct a graph representation of the road network in Mitcham, London, and model nitrogen dioxide levels using a Gaussian process defined on the vertices of our graph. We introduce a heteroscedastic noise process into our model to capture the complex variations that exist in nitrogen dioxide observations. Defining our model in this way offers superior predictive performance to its spatial analogue. Further, a graph representation allows us to infer the air pollution exposure that an individual would experience on a specific journey and their subsequent risk. We demonstrate this approach for the district of Mitcham, London.

This work introduces a wavelet neural network to learn a filter-bank specialized to fit non-stationary signals and improve interpretability and performance for digital signal processing. The network uses a wavelet transform as the first layer of a neural network where the convolution is a parameterized function of the complex Morlet wavelet. Experimental results, on both simplified data and atmospheric gravity waves, show the network is quick to converge, generalizes well on noisy data, and outperforms standard network architectures.

Stochastic partial differential equations (SPDEs) are significant tools for modelling dynamics in many areas including atmospheric sciences and physics.Neural Operators, generations of neural networks with capability of learning maps between infinite-dimensional spaces, are strong tools for solving parametric PDEs. However, they lack of ability to modeling stochastic PDEs which usually have poor regularity \footnote{Roughly speaking, regularity describes the smoothness of a function.} due to the driving noise. As the theory of regularity structure has achieved great successes in the analysis of SPDEs and provides the concept \emph{model} that well-approximate SPDEs' solutions, we propose the Neural Operator with Regularity Structure (NORS) which incorporates the models for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic $\Phi^4_1$ model and the 2d stochastic Navier-Stokes equation, and the results demonstrate that the NORS is resolution-invariant, efficient, and can achieve one order of magnitude lower error with a modest amount of data.

Ensemble weather predictions typically show systematic errors that have to be corrected via post-processing. Even state-of-the-art post-processing methods based on neural networks often solely rely on location-specific predictors that require an interpolation of the physical weather model's spatial forecast fields to the target locations. However, potentially useful predictability information contained in large-scale spatial structures within the input fields is potentially lost in this interpolation step. Therefore, we propose the use of convolutional autoencoders to learn compact representations of spatial input fields which can then be used to augment location-specific information as additional inputs to post-processing models. The benefits of including this spatial information is demonstrated in a case study of 2-m temperature forecasts at surface stations in Germany.

Deep Learning (DL) has recently emerged as a promising Empirical Statistical Downscaling perfect-prognosis technique (ESD-PP), to generate high-resolution fields from large-scale climate variables. Here, we analyze two state-of-the-art DL topologies for ESD-PP of different levels of complexity over North America. Besides classical validation leaning on accuracy metrics (e.g., Root Mean Squared Error (RMSE)), we evaluate several interpretability techniques to gain understanding on the inner functioning of the DL models deployed. Taking as reference the RMSE both topologies show similar values. Nonetheless, by analyzing the resulting interpretability maps, we find that the simplest model fails to capture a realistic physics-based input-output link, whilst the complex one describes a local pattern, characteristic of downscaling. In climate change scenarios, where weather extremes are exacerbated, erroneous patterns can lead to highly biased projections. Therefore, including interpretability techniques as a diagnostic of model functioning in the evaluation process can help us to better select and design them.

Starting a decade ago, the Responsive Environments Group at the MIT Media Lab began to develop and deploy different types of sensors in a soon-to-be-retired cranberry bog in Plymouth MA. This grew into the backbone of the Tidmarsh Living Observatory, which streams data from over a hundred diverse sensors at what is now a MA Audubon Wildlife Sanctuary. This talk will overview the sensor infrastructure and describe several applications that we have built atop it that include machine learning for real-time wildlife identification, manifestation and interpretation of the data in a virtual ‘Digital Twin’ avatar landscape, dynamically separating audio from a dispersed microphone array into localized foreground and background representations, and new ’sensory enhancing' interfaces designed to augment the visitor experience.

Recent advances in deep learning, in particular enabled by hardware advances and big data, have provided impressive results across a wide range of computational problems such as computer vision, natural language, or reinforcement learning. Many of these improvements are however constrained to problems with large-scale curated data-sets which require a lot of human labor to gather. Additionally, these models tend to generalize poorly under both slight distributional shifts and low-data regimes. In recent years, emerging fields such as meta-learning or self-supervised learning have been closing the gap between proof-of-concept results and real-life applications of machine learning by extending deep-learning to the semi-supervised and few-shot domains. We follow this line of work and explore spatio-temporal structure in a recently introduced image-to-image translation problem for storm event imagery in order to: i) formulate a novel multi-task few-shot image generation benchmark in the field of AI for Earth and Space Science and ii) explore data augmentations in contrastive pre-training for image translation downstream tasks. We present several baselines for the few-shot problem and discuss trade-offs between different approaches.

High-dimensional nonlinear systems such as atmospheric or oceanic flows present a computational challenge for data assimilation (DA) algorithms such as Kalman filters. A potential solution is to rely on a reduced-order model (ROM) of the dynamics. However, ROMs are prone to large errors, which negatively affects the accuracy of the resulting forecast. Here, we introduce the reinforcement learning reduced-order estimator (RL-ROE), a ROM-based data assimilation algorithm in which the correction term that takes in the measurement data is given by a nonlinear stochastic policy trained through reinforcement learning. The flexibility of the nonlinear policy enables the RL-ROE to compensate for errors of the ROM, while still taking advantage of the imperfect knowledge of the dynamics. We show that the trained RL-ROE is able to outperform a Kalman filter designed using the same ROM, and displays robust estimation performance with respect to different reference trajectories and initial state estimates.

Multi-output Gaussian processes (MOGPs) have been widely used to model small geographic and oceanographic data sets, because of their ability to provide confidence estimates for predictions. Causal relationships in oceanographic data mean that certain variables are primarily influenced by a small number of others, but existing MOGPs learn correlations between outputs that are actually unrelated, leading to significantly reduced predictive accuracy. We introduce the AcyGP model, which composes latent GPs using a directed acyclic graph (DAG) structure that is learned from the data. The algorithm prevents spurious correlations by only introducing inter-output correlations when improvement in likelihood justifies the increase in structure complexity. Evaluation of the AcyGP model demonstrates state of the art predictive performance on real geographic and oceanographic data.

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN's superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Oceanographic observations exist with different spatio-temporal resolutions and can be assimilated at various precision. The availability of numerous numerical simulations like ocean re-analysis make supervised machine learning appealing to deal with scale-related inverse problems. However data assimilation at finest resolutions using detailed oceanographic models is computationally intensive and building an exhaustive database may not be practical. Here we investigate the deep image prior method to downscale sea surface height observation and characterize estimation uncertainty in a fully-unsupervised manner. To do so, we set up a twin experiment using high resolution simulation from the NEMO Ocean engine and up-scale degraded data with multiple factors. Finally we give further perspectives of the method and make the link with data assimilation.

Sea ice charts, an important tool for navigation in the Arctic, are to this day manually drawn by professional ice analysts. The primary descriptor of the charts -- the Sea Ice Concentration (SIC) - indicates the ratio of ice to open-water in an area. Automating the SIC chart production is desired but the optimal representation of the corresponding machine learning task is ambivalent. Here, we explore it with either regressional or classification objective, each with two different (weighted) loss functions: Mean Square Error and Binary Cross-Entropy, and Categorical Cross-Entropy and the Earth Mover's Distance, respectively. While all perform well, the regression-based models achieve higher numerical similarity to the ground truth, whereas classification results in more visually pleasing and consistent charts. Weighting the loss functions improves the performance for intermediate classes at expense of open-water and fully-covered sea ice areas.

Rapid, accurate wave forecasts are critical to coastal communities and nearshore research. Observational data assimilation improves predictive skill, but is difficult to implement in current adjoint variational systems. Machine learning offers an alternative. Here, a previously proposed framework SWRL Net (Mooneyham et al. 2020) is applied to an array of buoys along the U. S. West Coast to quantify the effect of training data size, determine the impacts of transfer learning using archived wave prediction hindcasts, and evaluate the potential skill on recent wave forecasts. Results across buoy locations show diminishing returns for training data sets greater than 5-years, with error reductions of 10-60%. Experiments trained with shorter (1-year) forecast records have higher error, but the application of transfer learning using wave hindcasts substantially improves model performance.

I will summarize our work into automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. The focus of the talk will be both on what has gone well in our efforts, as well as open challenges that, if we can solve, will dramatically improve our ability to study and conserve natural ecosystems and the animals within them.

We apply an interpretable Long Short-Term Memory (iLSTM) network for land-atmosphere carbon flux predictions based on time series observations of seven environmental variables. iLSTM enables interpretability of variable importance and variable-wise temporal importance to the prediction of targets by exploring internal network structures. The application results indicate that iLSTM not only improves prediction performance by capturing different dynamics of individual variables, but also reasonably interprets the different contribution of each variable to the target and its different temporal relevance to the target. This variable and temporal importance interpretation of iLSTM advances terrestrial ecosystem model development as well as our predictive understanding of the system.

Predictor inputs and label data for crop yield forecasting are not always available at the same spatial resolution. We propose a deep learning framework that uses high resolution inputs and low resolution labels to produce crop yield forecasts for both spatial levels. The forecasting model is calibrated by weak supervision from low resolution crop area and yield statistics. We evaluated the framework by disaggregating regional yields in Europe from parent statistical regions to sub-regions for five countries (Germany, Spain, France, Hungary, Italy) and two crops (soft wheat and potatoes). Performance of weakly supervised models was compared with linear trend models and Gradient-Boosted Decision Trees (GBDT). Higher resolution crop yield forecasts are useful to policymakers and other stakeholders. Weakly supervised deep learning methods provide a way to produce such forecasts even in the absence of high resolution yield data.

We apply an invertible neural network (INN) for E3SM land model calibration and simulation with eight parameters at the Missouri Ozark AmeriFlux forest site. INN provides bijective (two-way) mappings between inputs and outputs, thus it can solve probabilistic inverse problems and forward approximations simultaneously. We demonstrate INN's inverse and forward capability in both synthetic and real-data applications. Results indicate that INN produces accurate parameter posterior distributions similar to Markov Chain Monte Carlo sampling and it generates model outputs close to the forward model simulations. Additionally, both the inverse and forward evaluations in INN are computationally efficient which allows for rapid integration of observations for parameter estimation and fast model predictions.

The panel discussion will focus on the future of model interpretability in the Earth and Space Sciences. We will cover topics such as where the most promising and/or urgent Earth Science applications lie, what the current state-of-the-art is with regards to model interpretability and explainability, and where the field is heading.

Our panelists:

Leilani Gilpin: Dr. Gilpin is an Assistant Professor in the Department of Computer Science and Engineering at UC Santa Cruz. Her research focuses on the design and analysis of methods for autonomous systems to explain themselves. Her work has applications to robust decision-making, system debugging, and accountability. Her current work examines how generative models can be used in iterative XAI stress testing.

Andrew Ross: Dr. Ross is a postdoctoral fellow at NYU researching how to improve ocean and climate models with hopefully-interpretable ML methods, working under Laure Zanna as part of M²LInES. Previously, he did his PhD in interpretable ML at Harvard University with Finale Doshi-Velez.

Antonios Mamalakis: Dr. Mamalakis is a postdoctoral researcher at the Department of Atmospheric Science of Colorado State University, working with professors Imme Ebert-Uphoff and Elizabeth Barnes. His research focuses on the application of machine learning (ML) and ML interpretability methods to climate problems, on climate predictability and teleconnections, climate change impacts, and hydrology. Dr Mamalakis holds a PhD in Civil and Environmental Engineering from the University of California, Irvine, and a MSc and a diploma from the University of Patras, Greece.

Moderators: Professor Karianne Bergen, Brown University Dr. Natasha Dudek, McGill University and Mila

Astrophysical light curves are particularly challenging data objects due to theintensity and variety of noise contaminating them. Yet, despite the astronomi-cal volumes of light curves available, the majority of algorithms used to processthem are still operating on a per-sample basis. To remedy this, we propose a sim-ple Transformer model –called Denoising Time Series Transformer (DTST)– andshow that it excels at removing the noise in datasets of time series when trainedwith a masked objective, even when no clean targets are available. Moreover, theuse of self-attention enables rich and illustrative queries into the learned represen-tations. We present experiments on real stellar light curves from the TransitingExoplanet Space Satellite (TESS), showing advantages of our approach comparedto traditional denoising techniques.

Ensemble weather predictions typically show systematic errors that have to be corrected via post-processing. Even state-of-the-art post-processing methods based on neural networks often solely rely on location-specific predictors that require an interpolation of the physical weather model's spatial forecast fields to the target locations. However, potentially useful predictability information contained in large-scale spatial structures within the input fields is potentially lost in this interpolation step. Therefore, we propose the use of convolutional autoencoders to learn compact representations of spatial input fields which can then be used to augment location-specific information as additional inputs to post-processing models. The benefits of including this spatial information is demonstrated in a case study of 2-m temperature forecasts at surface stations in Germany.

Recent advances in deep learning, in particular enabled by hardware advances and big data, have provided impressive results across a wide range of computational problems such as computer vision, natural language, or reinforcement learning. Many of these improvements are however constrained to problems with large-scale curated data-sets which require a lot of human labor to gather. Additionally, these models tend to generalize poorly under both slight distributional shifts and low-data regimes. In recent years, emerging fields such as meta-learning or self-supervised learning have been closing the gap between proof-of-concept results and real-life applications of machine learning by extending deep-learning to the semi-supervised and few-shot domains. We follow this line of work and explore spatio-temporal structure in a recently introduced image-to-image translation problem for storm event imagery in order to: i) formulate a novel multi-task few-shot image generation benchmark in the field of AI for Earth and Space Science and ii) explore data augmentations in contrastive pre-training for image translation downstream tasks. We present several baselines for the few-shot problem and discuss trade-offs between different approaches.

Deep Learning (DL) has recently emerged as a promising Empirical Statistical Downscaling perfect-prognosis technique (ESD-PP), to generate high-resolution fields from large-scale climate variables. Here, we analyze two state-of-the-art DL topologies for ESD-PP of different levels of complexity over North America. Besides classical validation leaning on accuracy metrics (e.g., Root Mean Squared Error (RMSE)), we evaluate several interpretability techniques to gain understanding on the inner functioning of the DL models deployed. Taking as reference the RMSE both topologies show similar values. Nonetheless, by analyzing the resulting interpretability maps, we find that the simplest model fails to capture a realistic physics-based input-output link, whilst the complex one describes a local pattern, characteristic of downscaling. In climate change scenarios, where weather extremes are exacerbated, erroneous patterns can lead to highly biased projections. Therefore, including interpretability techniques as a diagnostic of model functioning in the evaluation process can help us to better select and design them.

Numerical simulators are essential tools in the study of natural fluid-systems, buttheir performance often limits application in practice. Recent machine-learningapproaches have demonstrated their ability to accelerate spatio-temporal predictions, although, with only moderate accuracy in comparison. Here we introduce MultiScaleGNN, a novel multi-scale graph neural network model for learning to infer unsteady continuum mechanics in problems encompassing a range of length scales and complex boundary geometries. We demonstrate this method on advection problems and incompressible fluid dynamics, both fundamental phenomena in oceanic and atmospheric processes. Our results show good extrapolation to new domain geometries and parameters for long-term temporal simulations. Simulations obtained with MultiScaleGNN are between two and four orders of magnitude faster than those on which it was trained.

Accurately predicting air quality levels at a fine resolution is a critical task to ensuring the public's health is not at risk. In this work, we construct a graph representation of the road network in Mitcham, London, and model nitrogen dioxide levels using a Gaussian process defined on the vertices of our graph. We introduce a heteroscedastic noise process into our model to capture the complex variations that exist in nitrogen dioxide observations. Defining our model in this way offers superior predictive performance to its spatial analogue. Further, a graph representation allows us to infer the air pollution exposure that an individual would experience on a specific journey and their subsequent risk. We demonstrate this approach for the district of Mitcham, London.

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN's superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Predictor inputs and label data for crop yield forecasting are not always available at the same spatial resolution. We propose a deep learning framework that uses high resolution inputs and low resolution labels to produce crop yield forecasts for both spatial levels. The forecasting model is calibrated by weak supervision from low resolution crop area and yield statistics. We evaluated the framework by disaggregating regional yields in Europe from parent statistical regions to sub-regions for five countries (Germany, Spain, France, Hungary, Italy) and two crops (soft wheat and potatoes). Performance of weakly supervised models was compared with linear trend models and Gradient-Boosted Decision Trees (GBDT). Higher resolution crop yield forecasts are useful to policymakers and other stakeholders. Weakly supervised deep learning methods provide a way to produce such forecasts even in the absence of high resolution yield data.

Subsurface simulations use computational models to predict the flow of fluids (e.g., oil, water, gas) through porous media. These simulations are pivotal in industrial applications such as petroleum production, where fast and accurate models are needed for high-stake decision making, for example, for well placement optimization and field development planning. Classical finite difference numerical simulators require massive computational resources to model large-scale real-world reservoirs. Alternatively, streamline simulators and data-driven surrogate models are computationally more efficient by relying on approximate physics models, however they are insufficient to model complex reservoir dynamics at scale.Here we introduce Hybrid Graph Network Simulator (HGNS), which is a data-driven surrogate model for learning reservoir simulations of 3D subsurface fluid flows. To model complex reservoir dynamics at both local and global scale, HGNS consists of a subsurface graph neural network (SGNN) to model the evolution of fluid flows, and a 3D-U-Net to model the evolution of pressure. HGNS is able to scale to grids with millions of cells per time step, two orders of magnitude higher than previous surrogate models, and can accurately predict the fluid flow for tens of time steps (years into the future). Using an industry-standard subsurface flow dataset (SPE-10) with 1.1 million cells, we demonstrate that HGNS is able to reduce the inference time up to 18 times compared to standard subsurface simulators, and that it outperforms other learning-based models by reducing long-term prediction errors by up to 21%.

Rapid, accurate wave forecasts are critical to coastal communities and nearshore research. Observational data assimilation improves predictive skill, but is difficult to implement in current adjoint variational systems. Machine learning offers an alternative. Here, a previously proposed framework SWRL Net (Mooneyham et al. 2020) is applied to an array of buoys along the U. S. West Coast to quantify the effect of training data size, determine the impacts of transfer learning using archived wave prediction hindcasts, and evaluate the potential skill on recent wave forecasts. Results across buoy locations show diminishing returns for training data sets greater than 5-years, with error reductions of 10-60%. Experiments trained with shorter (1-year) forecast records have higher error, but the application of transfer learning using wave hindcasts substantially improves model performance.

FourCastNet, short for Fourier Forecasting Neural Network, is a global data-driven weather forecasting model that provides accurate short to medium-range global predictions at $0.25^{\circ}$ resolution. FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning wind energy resources, predicting extreme weather events such as tropical cyclones, extra-tropical cyclones, and atmospheric rivers. FourCastNet matches the forecasting accuracy of the ECMWF Integrated Forecasting System (IFS), a state-of-the-art Numerical Weather Prediction (NWP) model, at short lead times for large-scale variables, while outperforming IFS for variables with complex fine-scale structure, including precipitation. FourCastNet generates a week-long forecast in less than 2 seconds, orders of magnitude faster than IFS. The speed of FourCastNet enables the creation of rapid and inexpensive large-ensemble forecasts with thousands of ensemble-members for improving probabilistic forecasting. We discuss how data-driven deep learning models such as FourCastNet are a valuable addition to the meteorology toolkit to aid and augment NWP models.

Learning latent representations has aided operational decision-making in several disciplines. Its advantages include uncovering hidden interactions in data and automating procedures which were performed manually in the past. Representation learning is also being adopted by earth and environmental sciences. However, there are still subfields that depend on manual feature engineering based on expert knowledge and the use of algorithms which do not utilize the latent space. Relying on those techniques can inhibit operational decision-making since they impose data constraints and inhibit automation. In this work, we adopt a case study for nitrogen response rate prediction and examine if representation learning can be used for operational use. We compare a Multilayer Perceptron, an Autoencoder, and a dual-head Autoencoder with a reference Random Forest model for nitrogen response rate prediction. To bring the predictions closer to an operational setting we assume absence of future weather data, and we are evaluating the models using error metrics and a domain-derived error threshold. The results show that learning latent representations can provide operational nitrogen response rate predictions by offering performance equal and sometimes better than the reference model.

Understanding natural symmetries is key to making sense of our complex and ever-changing world. Recent work has shown that neural networks can learn such symmetries directly from data using Hamiltonian Neural Networks (HNNs). But HNNs struggle when trained on datasets where energy is not conserved. In this paper, we ask whether it is possible to identify and decompose conservative and dissipative dynamics simultaneously. We propose Dissipative Hamiltonian Neural Networks (D-HNNs), which parameterize both a Hamiltonian and a Rayleigh dissipation function. Taken together, they represent an implicit Helmholtz decomposition which can separate dissipative effects such as friction from symmetries such as conservation of energy. We train our model to decompose a damped mass-spring system. Then we apply it to two real-world datasets including a large ocean current dataset where decomposing the velocity field yields scientific insights.

Stochastic partial differential equations (SPDEs) are significant tools for modelling dynamics in many areas including atmospheric sciences and physics.Neural Operators, generations of neural networks with capability of learning maps between infinite-dimensional spaces, are strong tools for solving parametric PDEs. However, they lack of ability to modeling stochastic PDEs which usually have poor regularity \footnote{Roughly speaking, regularity describes the smoothness of a function.} due to the driving noise. As the theory of regularity structure has achieved great successes in the analysis of SPDEs and provides the concept \emph{model} that well-approximate SPDEs' solutions, we propose the Neural Operator with Regularity Structure (NORS) which incorporates the models for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic $\Phi^4_1$ model and the 2d stochastic Navier-Stokes equation, and the results demonstrate that the NORS is resolution-invariant, efficient, and can achieve one order of magnitude lower error with a modest amount of data.

This paper describes a novel machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple ML techniques and utilizing diverse data sources. Our multimodal framework, called Hurricast, efficiently combines spatial-temporal data with statistical data by extracting features with deep-learning encoder-decoder architectures and predicting with gradient-boosted trees. We evaluate our models in the North Atlantic and Eastern Pacific basins on 2016-2019 for 24-hour lead time track and intensity forecasts and show they achieve comparable mean absolute error to current operational forecast models while computing in seconds. Furthermore, the inclusion of Hurricast into an operational forecast consensus model could improve over the National Hurricane Center's official forecast, thus highlighting the complementary properties with existing approaches. In summary, our work demonstrates that utilizing machine learning techniques to combine different data sources can lead to new opportunities in tropical cyclone forecasting.

Detecting changes on the Earth, such as urban development, deforestation, or natural disaster, is one of the research fields that is attracting a great deal of attention.One promising tool to solve these problems is satellite imagery.However, satellite images require huge amount of storage, therefore users required to set Area of Interests first, which was not suitable for detecting potential areas for disaster or development.To tackle with this problem, we develop the novel tool, namely GRASP EARTH, which is the simple application based on Google Earth Engine.GRASP EARTH allows us to handle satellite imagery easily and it has used for disaster monitoring and monitoring urban development.

In various fields of natural science, there exists a high demand for accurate simulations of physical systems. For example, the weather forecasting requires a large-scale simulation of physical systems described by partial differential equations (PDEs). To reduce the computational cost, recent studies have attempted to build a coarse-grained model of the systems by using deep learning. Many training strategies for deep learning have developed for images or natural languages, but they are not necessarily suited for physical systems. A physical system demonstrates similar phenomena in most points (e.g., sunny days) but exhibits a drastic behavior occasionally (e.g., typhoons). Roughly speaking, a physical system dataset suffers from the class imbalance, whereas previous studies have rarely focused on this aspect. In this paper, we propose an imbalance-aware loss for learning physical systems, which resolves the class imbalance in a physical system dataset by focusing on the hard-to-learn parts. We evaluated the proposed loss using physical systems described by PDEs, namely the Cahn-Hilliard equation and the Korteweg-de Vries (KdV) equation. The experimental results demonstrate that models trained using the proposed loss outperform baseline models with a large margin.

Simulating wildfire propagation in near real-time is difficult due to the high computational cost and inappropriate choices of physics parameters used in the forecasting models. In this work, we first proposed a data-model integration scheme for fire progression forecasting, that combines deep learning models: reduced-order modelling, recurrent neural network (Long-Short-Term Memory) and data assimilation techniques. Capable of integrating real-time satellite observations, the deep learning-based surrogate model run about 1000 times faster than the Cellular Automata model used to forecast wildfires in real world scenarios. We then addressed the bottleneck of efficient physics parameter estimation by developing a novel inverse approach, relying on data assimilation in a reduced order space. Both the fire prediction and the parameter estimation approaches are tested over recent massive wildfire events in California with satellite observation from MODIS and VIIRS to adjust the fire forecasting in near real-time.

Where seabed substrate is rocky, comprising of bedrock, boulders and cobbles, sampling and analysis of benthic ecosystems often relies on still images and video from underwater cameras. Processing benthic imagery to generate ecosystem information typically involves manual interpretation and annotation, which is a time consuming and expensive process and prone to human errors and biases. Machine learning can step in here to assist, if not fully replace manual annotation. Here, we develop an object detection model using Faster R-CNN to identify various epibenthic species from a high energy marine site with a rocky substrate. The model achieves an overall F1-score of 66.28\% across 7 different benthic species. The work is a significant step in identifying various learnings and challenges associated with data in non-ideal conditions and we suggest methods to further improve the existing framework.

Typical deep learning approaches to modeling high-dimensional data often result in complex models that do not easily reveal a new understanding of the data. Research in the deep learning field is very actively pursuing new methods to interpret deep neural networks and to reduce their complexity. An approach is described here that starts with linear models and incrementally adds complexity only as supported by the data. An application is shown in which models that map global temperature and precipitation to years are trained to investigate patterns associated with changes in climate.

Climate models encode our knowledge of the Earth system, enabling research on the earth’s future climate under alternative assumptions of how human-driven climate forcings, especially greenhouse gas emissions, will evolve. One important use of climate models is to estimate the impacts of climate change on natural and societal systems under these different possible futures. Unfortunately, running many simulations on existing models is extremely computationally expensive. These computational demands are particularly problematic for characterizing extreme events, which are rare and thus demand numerous simulations in order to precisely estimate the relevant climate statistics. In this paper we propose an approach to generating realistic global precipitation requiring orders of magnitude less computation, using a conditional generative adversarial network (GAN) as an emulator of an Earth System Model (ESM). Specifically, we present a GAN that emulates daily precipitation output from a fully coupled ESM, conditioned on monthly mean values. The GAN is trained to produce spatio-temporal samples: 28 days of precipitation in a 92x144 regular grid discretizing the globe. We evaluate the generator by comparing generated and real distributions of precipitation metrics including average precipitation, average fraction of dry days, average dry spell length, and average precipitation above the 90th percentile, finding the generated samples to closely match those of real data, even when conditioned on climate scenarios never seen during training.

Climate control of buildings makes up a significant portion of global energy consumption, with groundwater heat pumps providing a suitable alternative. To prevent possibly negative interactions between heat pumps throughout a city, city planners have to optimize their layouts in the future. We develop a novel data-driven approach for building small-scale surrogates for modelling the thermal plumes generated by groundwater heat pumps in the surrounding subsurface water. Building on a data set generated from 2D numerical simulations, we train a convolutional neural network for predicting steady-state subsurface temperature fields from a given subsurface velocity field. We show that compared to existing models ours can capture more complex dynamics while still being quick to compute. The resulting surrogate is thus well-suited for interactive design tools by city planners.

One of the most effective approaches to mitigating climate change is to monitor the carbon stock in the tropical rainforests. However, biomonitoring of carbon in the forests is expensive and challenging due to inaccessibility. To improve carbon stock monitoring and the evaluation of fine-scale forest loss, we established a rapid, automatic, and cost-efficient generalized machine learning framework that uses diverse remote sensing data and satellite imagery to accurately estimate aboveground carbon density, at fine-grained resolution (tens of meters), in remote tropical rainforests. The study area first focused on rainforests in Indonesian Borneo. In our preliminary tests on 80 sites in Indonesian Borneo, our machine learning models were capable of producing ACD estimates with R2 of 0.7-0.8, which is a significant improvement from the comparable works (0.5-0.6 at best). This machine learning framework will be used to facilitate further carbon stock modeling in other forest regions (e.g. Brazil) as well as for the general purpose of climate change mitigation.

The Geostationary Lightning Mapper (GLM) instrument onboard the GOES 16 and 17 weather satellites has been shown to be capable of detecting bolides (very bright meteors) in the Earth's atmosphere. Due to its large, continuous field of view and immediate public data availability, GLM provides a unique opportunity to detect a large variety of bolides, including those in the 0.1 to 3 m diameter range that complements current ground-based bolide detection systems, which are typically sensitive to smaller objects. Our goal is to generate a large catalog of calibrated bolide light curves to provide an unprecedented data set for three purposes: 1) to inform meteor entry models on how incoming bodies interact with the atmosphere, 2) to infer the pre-entry properties of the impacting bodies and 3) to statistically analyse bolide impact populations across the globe. We have deployed a machine learning based bolide detection and light curve generation pipeline on the NASA Advanced Supercomputer Facility. Detections are promptly published on a publicly available website, https://neo-bolide.ndc.nasa.gov. The pipeline has now been operational for almost 3 years and we have amassed a catalogue of over 3300 bolides. We first summarise the end-to-end development life cycle of the machine learning based bolide detection pipeline. We then present a statistical analysis of the bolides detected and assess the reliability of the automated detection pipeline.

Antarctic geothermal heat flow has often been derived indirectly from geophysical data with assumptions about a simplified and undifferentiated lithosphere, which resulted in weakly constrained and inconsistent models. From other continents, we know that thermal parameters and heat flow can exhibit large spatial variations depending on geology and tectonic history. Combining gravity and magnetic data in a joint inversion approach yields information on the crustal structure of Wilkes Land, East Antarctica, and possible geological features become more evident. Both datasets are combined through a coupling method which increases the mutual information to get similar and statistically compatible inversion results. Therefore, we minimize data misfit and variation of information under the coupling constraint. The results show matching features of high magnitude density and susceptibility anomalies. Prominent structures are visible in NE – SW direction along the edge of the Mawson craton and at the presumed Australo-Antarctic and Indo-Antarctic terrane boundary.Applying the same method to Australia, formerly connected to Wilkes Land, we can exploit the much better-known geology there and identify coherent structures along the adjacent margins. The inverted parameter relationship between susceptibility and density can be used as input for machine learning techniques to define a spatially variable heat production map, which in turn would lead to improved heat flow estimates. For this, we rely on existing petrophysical and geochemical databases to correlate and confine thermal parameters with our results.

Influenced by complex interactions at the intersection of air and water, the fate of objects floating in the ocean is difficult to predict even a few days into the future (DARPA-FFT, 2021). Despite the complexity of long-term drifter trajectory forecasting, accurate predictions are critically important for missions such as search and rescue, ecological studies, and disaster remediation. Inspired by the DARPA Forecasting Floats in Turbulence (FFT) Challenge (DARPA-FFT, 2021), we present an open-source machine learning benchmark dataset for measuring progress in ocean trajectory modeling based on a collation of drifter trajectories and archival weather and ocean forecasts. Given the recent success in deep generative models (Ravuri et al., 2021; Erichson et al., 2019; Sosanya & Greydanus, 2022) for forecasting physical processes, we hope that a formal dataset will enable further development of models tuned for the complexities of drifter trajectory forecasting. In addition to introducing benchmarks for drifter forecasting, we provide baseline solutions built off of OpenDrift (Dagestad et al., 2018) an open-source software package for modeling the fate of objects in the ocean or atmosphere.

The Southern sea otter (Enhydra lutris nereis) is a keystone predator and a protected marine mammal that inhabits the California coast and beyond. Current methods to study otters are expensive, inaccurate, and inefficient. To improve the use of resources for government officials, ecologists, and other researchers, we are proposing an auto-detection algorithm using drone aerial imagery as inputs. Using 842 images of sea otters from Monterey Bay and 1018 background images extracted from other open course oceanographic publications, we labeled, clustered, and augmented images to train YOLOv5 with a modified architecture for tiny objects. The resulting model with a 75% F1-score and 76% mAp_0.2:0.5. Future work includes adding background images from newly acquired datasets, using a GAN model to generate images, and using a two-stage detector to improve results.

We train a convolutional neural network, APOGEE Net, to predict Teff, logg, and, for some stars, [Fe/H], based on the APOGEE spectra. This is the first pipeline adapted for these data that is capable of estimating these parameters in a self-consistent manner not only for low mass stars, (such as main sequence dwarfs, pre-main sequence stars, and red giants), but also high mass stars with Teff in excess of 50,000 K, including hot dwarfs and blue supergiants. The catalog of ~650,000 stars presented in this paper allows for a detailed investigation of the star forming history of not just the Milky Way, but also of the Magellanic clouds, as different type of objects tracing different parts of these galaxies can be more cleanly selected through their distinct placement in Teff-logg parameter space than in previous APOGEE catalogs produced through different pipelines.

The most dominant mode of oceanic climate variability on an interdecadal scale is the El Ni\~no Southern Oscillation (ENSO). ENSO events are characterized by anomalous sea surface temperatures (SSTs) in the equatorial Pacific affecting global climate, ecosystem, and society. The spatial structure and dynamics of SST fields associated to ENSO however show strong variability between different events, known as ENSO diversity. Whether this diversity consists of distinct types or merely a continuous process is to date an open question.Using Gaussian mixture variational autoencoders, we analyze the multi-modality of the non-linear low dimensional representation of SST anomalies in the tropical Pacific. We show that hard classification approaches are not suitable to describe ENSO diversity. Analyzing the low dimensional representations allows us to identify two main factors describing the spatial variability of ENSO, namely the average maximum temperature anomaly and the zonal gradient. We suggest a weighting of ENSO events based on these factors to yield more expressive composites that describe the variability of ENSO.

Long-term monitoring of northern ecosystems is necessary to address the challenges posed by climate change and evolving human needs and stressors. Such efforts to survey wildlife in hostile and remote areas can however be hindered by several logistical challenges. A notable example is the lockdown imposed by COVID-19 in 2020-2021, which has had tremendous impacts on research activities, especially for long-term monitoring projects. This was particularly the case in the Canadian Arctic, where Northern authorities drastically restricted travel within and outside each territory. Gaps in time series can limit our understanding of ecosystem functioning and jeopardize our ability to detect population trends. To cope with such exceptional situations and limit the loss of data, alternative monitoring strategies should be explored. We propose a machine learning pipeline to monitor from space a High-Arctic food web composed of the Arctic fox, the Greater Snow Goose, the Snowy Owl, the brown lemming, and the collared lemming. We first trained a Faster R-CNN neural network model to detect snow geese on WorldView-3 satellite images of Bylot Island (Nunavut), home to the world's largest Greater Snow Goose breeding colony. A mean F1-score of >90% is achieved on all of the six vegetation types found in Bylot, which suggests that our model is able to generalize well over the entire study area. Our pipeline then leverages the structuring role of the Greater Snow Goose within the food web to infer some key parameters on the remaining species. Notably, we are able to infer snowy owls nest density and lemming abundance. We successfully validate each individual step of our monitoring pipeline by comparing our estimates with historical field data. To our knowledge, our approach is one of the first of its kind that effectively combines ecological knowledge with machine learning algorithms in order to obtain a method that could represent a realistic alternative to Arctic field work.

Data acquired by satellites with low revisit periods are crucial for detecting and tracking changes in land cover due to natural hazards globally and accelerating decision-making for disaster mitigation. The multispectral aspect of such datasets allows capturing rich information unique to a given type of change and can be viewed as a signature of the change, enabling its use for detecting and distinguishing changes. However, the large volume of such datasets and the variations in change types and their signatures necessitates the use of unsupervised approaches without prior knowledge of these signatures. Moreover, for assisting domain scientists and transparently informing hazard mitigation strategies, explaining these detections are also vital. To this end, this work explores unsupervised change detection approaches to detect natural hazards and examines model-agnostic methods to explain change decisions. Multispectral land surface reflectance observations from the Moderate Resolution Imaging Spectroradiometer are used in this study. Change detection and its explanations are obtained with a two-step approach. The change detection step learns a pixel-based, pre-change multispectral model of a region characterizing the spectral signature before a change and monitors incoming observations for deviations caused by changes using reconstruction-based techniques such as the autoencoder and variational autoencoder. As this assumes no prior knowledge of the type of deviation caused by a change, it is extendible to different change types. The reconstruction error then is viewed as an explanation of change using Shapley values of each band or feature. The bands most relevant for explaining a change type are observed to match with bands commonly used in forming change-specific band ratios and reconstruction errors naturally extract this information without supervision. As a baseline, the reconstruction error magnitude is also used to explain change decisions.

Geo-structures built on expansive clay, such as highway slopes in MS, are vulnerable to soil moisture variations and impacted by the increasing trend of high-intensity rainfall events. Early Identification of vulnerable or failing assets is crucial to strategize repairs and maintenance activities. Therefore, the first step of geotechnical asset management is to compile an inventory of the geostructures. Furthermore, frequent geo-structure health monitoring is crucial to avert risks and ensure asset longevity. Remote sensing-based infrastructure monitoring techniques have grown in popularity lately. However, identifying the vulnerable geostructures still requires boots on ground inspection. Such manual inspection after each weather event is expensive and impractical when quick action is needed. There is a need to automate the Geotechnical asset management methodology.To this end, this study used deep learning models integrated with ArcGIS Pro to detect and classify failed highway slopes in central MS. The failures studied included excessive deformations, shallow slides, sinkholes, etc. Training sample chips were first created by identifying failed slopes within Uncrewed Aerial Vehicle (UAV) based orthomosaic rasters and satellite imagery data. Comparative tests with different object detection and classification models with varying epochs were executed and documented. The results proved promising and assured capabilities of expanding the methodology to other infrastructure assets. The model can be run after each rainfall event to quickly spot any new slope instabilities. This innovative technology will ensure a quick inventory of failed and vulnerable geo-structures and help strategize maintenance and repair funds allocation. The geospatial deep learning methodology developed in this study is valuable for structural health monitoring and will help streamline Geotechnical Asset Management.