We present JAX-XC, an open-source library that provides exchange-correlation functionals in Jax. JAX-XC is built from LIBXC, its correctness has been verified numerically against LIBXC. Thanks to Jax, JAX-XC is end-to-end differentiable, computationally more efficient thanks to the vectorization provided by XLA, and also portable on various accelerators. More importantly, as more research is focusing on machine learning for density functional theory, we hope that JAX-XC could serve as a deep learning-friendly tool and a stepping-stone for researchers working in the intersection of deep learning and density functional theory.

Crystal Structure Prediction (CSP) is crucial in various scientific disciplines. Existing learning-based generative approaches seldom capture the full symmetries of the crystal structure distribution---the invariance of translation, rotation, and periodicity. In this paper, we propose DiffCSP, a novel diffusion method to learn the stable structure distribution from data, incorporating the above symmetries. To be specific, DiffCSP jointly generates the lattice and the fractional coordinates of all atoms by employing a periodic-E(3)-equivariant denoising model to better model the crystal geometry. Notably, DiffCSP leverages fractional coordinates other than traditional Cartesian coordinates to represent crystals, remarkably promoting the diffusion and the generation process of atom positions. Extensive experiments on crystal structure prediction verify the effectiveness of DiffCSP against existing learning-based counterparts.

Discovering high-entropy alloys (HEAs) with high yield strength is an important yet challenging task in material science. However, the yield strength can only be accurately measured by very expensive and time-consuming real-world experiments, hence cannot be acquired at scale. Learning-based methods could facilitate the discovery process, but the lack of a comprehensive dataset on HEA yield strength has created barriers. We present X-Yield, a large-scale material science benchmark with 240 experimentally measured (“high-quality”) and over 100K simulated (imperfect or “low-quality”) HEA yield strength annotations. Due to the scarcity of experimental annotations and the quality gap in imperfectly simulated data, existing transfer learning methods cannot generalize well on our dataset. We address this cross-quality few-shot transfer problem by leveraging model sparsification “twice” — as a noise-robust feature learning regularizer at the pre-training stage, and as a data-efficient learning regularizer at the few-shot transfer stage. We then propose a bi-level optimization framework termed Bi-RPT that jointly learns optimal masks and automatically allocates sparsity levels for both stages. The effectiveness of Bi-RPT is validated through extensive experiments on our new challenging X-Yield dataset, alongside other synthesized testbeds. Specifically, we achieve an 8.9-19.8% reduction in terms of the test MSE and 0.98-1.53% in terms of test accuracy, merely using 5-10% of the experimental data.

Local atomic environment descriptors (LAEDs) are used in the materials science and chemistry communities, for example for the development of machine learning interatomic potentials. Despite the fact that LAEDs have been extensively studied and benchmarked for various applications, global structure descriptors (GSDs), i.e., descriptors for entire molecules or crystal structures, have been mostly developed independently based on other approaches. Here, we propose a systematically improvable methodology for constructing GSDs from local atomic environment descriptors by incorporating statistical information and information about chemical elements. We apply the method to construct GSDs of varying complexity for lithium thiophosphate structures that are of interest as solid electrolytes and use an information-theoretic approach to obtain an optimally compressed GSD. Finally, we report the performance of the compressed GSD for energy prediction tasks.

This paper introduces SimuStruct: Simulated Structural Parts Dataset, a dataset that contains 2D structural parts, their respective meshes and the outputs of numerical simulations for different properties for linear and elastic material, boundary and loading conditions, and for varying levels of refinement. SimuStruct comprises the classic case of plates with holes since it is a 2D simple case with analytical resolution and which is found in different mechanical design applications. The SimuStruct dataset comprises many different cases, where each case is solved using standard Finite Element Methods (FEMs) with the open-source package FEniCS. Compared to other datasets similar in purpose, SimuStruct is more diversified and realistic because it aims to comprises diverse real cases for different loading and boundary conditions, different properties for linear and elastic material, and different levels of refinement. In addition, SimuStruct is more flexible, versatile, and scalable because all algorithms and codes are implemented using open-source libraries. The main goal of the SimuStruct dataset is to serve both as training and evaluation data for Machine Learning (ML)-based methods in structural analysis and optimal mesh generation and therefore support the development of ML-based optimal mechanical design solutions. An application of SimuStruct is presented to train and test an ANN model to predict stress-strain fields. SimuStruct will contribute to the connection of the Mechanical Engineering and ML communities, which will allow accelerating and exploitation the research in the computational design field.

Even though machine learning force fields are quite accurate in the prediction of forces and energies in the sampled region, they fail to extrapolate, which results in the unphysical behavior of the system during molecular dynamics simulations. We propose to overcome this problem by performing data augmentation. To expand the original dataset random perturbations of atoms were performed. The corresponding increase in the energy of the system was calculated under the assumption of harmonicity. The required spring constants were obtained from the original dataset by fitting a gaussian mixture model to the bond lengths distribution. The resulting force field performance was improved in the regions far from training data.

We developed a multi-step workflow for the discovery of next-generation conventional superconductors. 1) We started with a Bardeen–Cooper–Schrieffer (BCS) inspired pre-screening of 55000 materials in the JARVIS-DFT database resulting in 1736 materials with high Debye temperature and electronic density of states at the Fermi-level. 2) Then, we performed density functional theory (DFT) based electron-phonon coupling calculations for 1058 materials to establish a systematic database of superconducting properties. 3) Further, we applied forward deep-learning (DL) using atomistic line graph neural network (ALIGNN) models to predict properties faster than direct first-principles computations. Notably, we find that by predicting the Eliashberg function as an intermediate quantity, we can improve the model performance versus a direct DL prediction of $T_C$. Finally, 4) we used an inverse deep-learning method with a crystal diffusion variational autoencoder (CDVAE) model to generate thousands of new superconductors with high chemical and structural diversity. 5) We screened these CDVAE-generated structures using ALIGNN to identify candidates that are stable with high $T_C$. 6) We verified the top superconducting candidates with DFT.

In this paper, we present a novel approach to knowledge extraction and retrieval using Natural Language Processing (NLP) techniques for material science. Our goal is to automatically mine structured knowledge from millions of research articles in the field of polycrystalline materials and make it easily accessible to the broader community. The proposed method leverages NLP techniques such as entity recognition and document classification to extract relevant information and build an extensive knowledge base, from a collection of 9.5 Million publications. The resulting knowledge base is integrated into a search engine, which enables users to search for information about specific materials, properties, and experiments with greater precision than traditional search engines like Google. We hope our results can enable material scientists quickly locate desired experimental procedures, compare their differences, and even inspire them to design new experiments.

In order to deploy machine learning in a real-world self-driving laboratory where data acquisition is costly and there are multiple competing design criteria, systems need to be able to intelligently sample while balancing performance tradeoffs and constraints. For these reasons, we present an active learning process based on multiobjective blackbox optimization with continuously-updated machine learning models. This workflow is built upon open-source technologies for real-time data streaming and modular multiobjective optimization software development. We demonstrate a proof-of-concept for this workflow through the autonomous operation of a continuous-flow chemistry laboratory, which identifies ideal manufacturing conditions for the electrolyte 2,2,2-trifluoroethyl Methyl carbonate.

Solid-state materials, which are made up of periodic 3D crystal structures, are particularly useful for a variety of real-world applications such as batteries, fuel cells and catalytic materials. Designing solid-state materials, especially in a robust and automated fashion, remains an ongoing challenge. To further the automated design of crystalline materials, we propose a method to learn to design valid crystal structures given a crystal skeleton. By incorporating Euclidean equivariance into a policy network, we portray the problem of designing new crystals as a sequential prediction task suited for imitation learning. At each step, given an incomplete graph of a crystal skeleton, an agent assigns an element to a specific node. We adopt a behavioral cloning strategy to train the policy network on data consisting of curated trajectories generated from known crystals.

We propose a novel, physically-constrained and differentiable approach for the generation of D-dimensional qudit states via spontaneous parametric down-conversion (SPDC) in quantum optics. We circumvent any limitations imposed by the inherently stochastic nature of the physical process and incorporate a set of stochastic dynamical equations governing its evolution under the SPDC Hamiltonian. We demonstrate the effectiveness of our model through the design of structured nonlinear photonic crystals (NLPCs) and shaped pump beams; and show, theoretically and experimentally, how to generate maximally entangled states in the spatial degree of freedom. The learning of NLPC structures offers a promising new avenue for shaping and controlling arbitrary quantum states and enables all-optical coherent control of the generated states. We believe that this approach can readily be extended from bulky crystals to thin Metasurfaces and potentially applied to other quantum systems sharing a similar Hamiltonian structures, such as superfluids and superconductors.

Spatial Light Modulators (SLMs) are devices that can modulate the amplitude or the phase of a beam of light. These devices are used in applications such as beam front aberration and microscopic manipulation with optical tweezers. Here, we study the problem of learning to modulate light in a new type of temperature-controlled SLM. These SLMs are panels that use a thin viscous film in which shallow wave patterns can be induced by varying the temperature of the panel. This method can be used for modulating light such as high-power lasers. The problem here is to learn which input temperature signal is necessary in order to induce a given pattern in the reflected light. We propose a deep equivariant model to learn this relationship. We generate a synthetic dataset consisting of temperature signals and corresponding light patterns by simulating the thin film lubrication equation that governs the phenomenon of thermocapillary dewetting. We use this dataset to train our networks. We demonstrate the advantage of using equivariant neural networks over convolutional neural networks in order to learn the mapping.

Transforming a design into a high-quality product is a challenge in metal additive manufacturing due to rare events which can cause defects to form. Detecting these events in-situ could, however, reduce inspection costs, enable corrective action, and is the first step towards a future of tailored material properties. In this study a model is trained on laser input information to predict nominal laser melting conditions. An anomaly score is then calculated by taking the difference between the predictions and new observations. The model is evaluated on a dataset with known defects achieving an F1 score of 0.821. This study shows that anomaly detection methods are an important tool in developing robust defect detection methods.

Experimentally synthesizing predicted materials in a reproducible manner remains a key bottleneck in materials science progress. Autonomous synthesis and closed loop integration of prediction and characterization can address these issues, however, this requires autonomous characterization methods for all analysis including crystallographic phase identification which currently remains a rate-limiting step. Here we benchmark several machine learning techniques for X-ray Diffraction spectra interpretation (spectral clustering, convolutional neural networks, and invertible neural networks) and compare the relative strengths and weaknesses of each approach. Future work will involve deploying these techniques across the entire high-throughput experimental materials database.

We present a new machine learning (ML) benchmark for materials stability predictions named Matbench Discovery.A goal of this benchmark is to highlight the need to focus on metrics that directly measure their utility in prospective discovery campaigns as opposed to analyzing models based on predictive accuracy alone.Our benchmark consists of a task designed to closely simulate the deployment of ML energy models in a high-throughput search for stable inorganic crystals.We explore a wide variety of models covering multiple methodologies ranging from random forests to GNNs, and from one-shot predictors to iterative Bayesian optimizers and interatomic potential-based relaxers. We find M3GNet to achieve the highest F1 score of 0.58 and $R^2$ of 0.59 while MEGNet wins on discovery acceleration factor (DAF) with 2.94. Our results provide valuable insights for maintainers of high throughput materials databases to start using these models as triaging steps to more effectively allocate compute for DFT relaxations.

Many collective systems exist in nature far from equilibrium, ranging from cellular sheets up to flocks of birds. These systems reflect a form of active matter, whereby individual material components have internal energy. Under specific parameter regimes, these active systems undergo phase transitions whereby small fluctuations of single components can lead to global changes to the rheology of the system. Simulations and methods from statistical physics are typically used to understand and predict these phase transitions for real-world observations. In this work, we demonstrate that simulation-based inference can be used to robustly infer active matter parameters from system observations. Moreover, we demonstrate that a small number (from one to three) snapshots of the system can be used for parameter inference and that this graph-informed approach outperforms typical metrics such as the average velocity or mean square displacement of the system. Our work highlights that high-level system information is contained within the relational structure of a collective system and that this can be exploited to better couple models to data.

The density of states (DOS) is a spectral property of materials, which provides fundamental insights on various characteristics of materials. In this paper, we propose to predict the DOS by reflecting the nature of DOS: DOS determines the general distribution of states as a function of energy. Specifically, we integrate the heterogeneous information obtained from the crystal structure and the energies via multi-modal transformer, thereby modeling the complex relationships between the atoms in the crystal structure, and various energy levels. Extensive experiments on two types of DOS, i.e., phonon DOS and electron DOS, with various real-world scenarios demonstrate the superiority of DOSTransformer.

In computational chemistry, crystal structure prediction (CSP) is an optimization problem that involves discovering new crystal structures. This problem is challenging for machine learning (ML) methods: it requires discovering globally optimal designs that attain the smallest energy in complex non-Euclidean manifolds. One approach to tackle this problem involves building simulators based on density functional theory (DFT), but these simulators are painfully slow. More recent approaches are exploring the alternate paradigm of relying on learned graph neural networks (GNNs) surrogate models as a proxy for simulation. We propose a method that leverages GNNs to reduce the complexity of the problem. Concretely, we reduce the non-Euclidean optimization search space to a standard vector one with Graph Variational Autoencoders (GVAEs), and we combine that with techniques from offline model-based optimization. This prevents the optimization procedure from producing unstable structures that erroneously appear to have low energies under the learned model. We show that this procedure outperforms current alternatives, both in terms of success rate of structure prediction, and computational cost. In addition, it provides a generic recipe to apply offline optimization techniques for optimizing in non-Euclidean spaces.

In this extended abstract we compare the performance of different families of descriptors – \textit{molar composition descriptor, weight composition descriptor and elemental descriptor} – for regression tasks and include examples of a classification task for perovskite oxide materials with general formulas $ABO_3$, $A_2BB’O_6$, and $A_xA’_{1-x}B_yB’_{1-y}O_6$. The best performance was observed for our elemental descriptor which consisted of $A$-site and $B$-site element information on: Shannon’s ionic radius, ideal bond length, electronegativity, van der Waals radius, ionization energy, molar volume, atomic number, and atomic mass. The weight composition descriptor showed superior results over a simpler molar composition descriptor. The results of principal component analysis, regression models with the hyperparameters optimized using an autoML software and Wasserstein autoencoders are briefly discussed for a possible use in inverse materials design.

Training deep neural networks (DNNs) in solid mechanics is challenging due to data scarcity. Even when using synthetic datasets obtained from computational simulations, these datasets have limited size because each simulation is time-consuming. One way to address this challenge is to use transfer learning, i.e. finetuning a model pretrained in a different context (e.g., computer vision) such that it can learn a new task while using less data (e.g., learning the behavior of a material with a given microstructure). Unfortunately, a model obtained by transfer learning loses the ability to solve the original task. Therefore, each new task that is being learned destroys the ability to perform the previous one with the same model. We present a Cooperative Data-driven Modeling (CDDM) network that can continually learn tasks without forgetting, accumulating knowledge such that less training data is required when facing a new task or that leads to smaller prediction test error for each new task. We provide our numerical experiments on predicting the plastic behavior of different materials using recurrent neural networks, as they have been shown to handle history-dependent problems.

A general method for learning the conditional distribution $p(G | \hat{G})$ of two 3D graphs is proposed. The method is designed to be invariant to rigid body transformations and to permutations of the vertices of either graph. The core of the method is a continuous normalizing flow and semi-equivariance conditions are established to ensure the aforementioned invariance conditions. The utility of the technique is demonstrated as a conditional generative model for the molecular setting.

In recent years, graph neural network (GNN) based approaches have emerged as apowerful technique to encode complex topological structure of crystal materials inan enriched representation space. These models are often supervised in nature andusing the property-specific training data, learn relationship between crystal structureand different properties like formation energy, bandgap, bulk modulus, etc. Mostof these methods require a huge amount of property-tagged data to train the systemwhich may not be available for different properties. However, there is an availabilityof a huge amount of crystal data with its chemical composition and structural bonds.To leverage these untapped data, this paper presents CrysGNN, a new pre-trainedGNN framework for crystalline materials, which captures both node and graphlevel structural information of crystal graphs using a huge amount of unlabelledmaterial data. Further, we extract distilled knowledge from CrysGNN and injectinto different state of the art property predictors to enhance their property predictionaccuracy. We conduct extensive experiments to show that with distilled knowledgefrom the pre-trained model, all the SOTA algorithms are able to outperform theirown vanilla version with good margins. We also observe that the distillation processprovides a significant improvement over the conventional approach of finetuningthe pre-trained model.

Retrosynthesis analysis aims to design reaction pathways and intermediates for a target compound. Emerging works have been developed to automate this process using artificial intelligence (AI). Although there are multiple synthesis pathways to synthesize one target product, most of the existing models were optimized to select only one of them. One of the potential reason is due to the absence of other potential reactions in the available reaction dataset, as one product usually only appears one time in the reaction dataset. In this work, we generate virtually validated pseudo-reactions using local reaction templates and reaction outcome prediction model to optimize the retrosynthesis model to predict multiple synthesis pathways. With the aid of newly generated pseudo-reactions, the top-10 exact match accuracy is increased from 93.1% to 94.2% and the top-10 round trip accuracy is increased from 81.3% to 87.6% with higher prediction confidence and diversity on a public reaction dataset.

Molecular representation learning is a fundamental task for AI-based drug design and discovery. Self-supervised contrastive learning on molecular graphs, which aims to learn good representations via semantic-preserving transformations, is an attractive framework for this task. However, it is relatively under-explored to design such transformations for molecules under consideration of their chemical semantics. In this paper, we consider fragmentation which decomposes a molecule into a set of chemically meaningful fragments (e.g., functional groups) as the semantic-preserving transformation. Here, we also utilize the 3D geometric views of molecules as another source of such transformation. Based on these molecule-specialized semantic-preserving transformations, we propose fragment-based multi-view molecular contrastive learning (FragCL), an effective framework that learns chemically meaningful molecular representations. Through extensive experiments, we demonstrate that our framework outperforms prior molecular representation learning methods across various molecular property prediction tasks.

Polymers are important and numerous. While the structure synthesis and property annotation for polymers require expensive equipment and a long time of effort, small molecules without annotations have been collected from various sources and at a large scale. However, there is a lack of studies for effective transfer learning from molecules without labels (as the source domain) to polymers with labels (as the target domain). This paper proposes to extract the knowledge underlying the large set of source molecules as a specific set of useful graphs to augment the training set for target polymers. We learn a diffusion probabilistic model on the source data and design two new objectives to guide the model's denoising process with target data to generate target-specific labeled graphs. Experiments from unlabeled molecules to labeled polymers demonstrate that our transfer learning approach outperforms existing semi/self-supervised learning approaches.

Machine learning has been applied to the problem of X-ray diffraction phase prediction with promising results. In this paper, we describe a method for using machine learning to predict crystal structure phases from X-ray diffraction data of transition metals and their oxides. We evaluate the performance of our method and compare the variety of its settings. Our results demonstrate that the proposed machine learning framework achieves competitive performance. This demonstrates the potential for machine learning to significantly impact the field of X-ray diffraction and crystal structure determination.

We present JAX-XC, an open-source library that provides exchange-correlation functionals in Jax. JAX-XC is built from LIBXC, its correctness has been verified numerically against LIBXC. Thanks to Jax, JAX-XC is end-to-end differentiable, computationally more efficient thanks to the vectorization provided by XLA, and also portable on various accelerators. More importantly, as more research is focusing on machine learning for density functional theory, we hope that JAX-XC could serve as a deep learning-friendly tool and a stepping-stone for researchers working in the intersection of deep learning and density functional theory.

Discovering high-entropy alloys (HEAs) with high yield strength (YS) is an important yet challenging task in materials science. However, YS can only be accurately measured by very expensive and time-consuming real-world experiments, hence cannot be acquired at scale. Learning-based methods could facilitate the discovery process, but the lack of a comprehensive dataset on HEA yield strength has created barriers. We present X-Yield, a materials science benchmark with 240 experimentally measured ("high-quality") and over 100K simulated (imperfect or "low-quality") HEA yield strength data. Due to the scarcity of experimental data and the quality gap with imperfectly simulated data, existing transfer learning methods cannot generalize well on our dataset. We address this cross-quality few-shot transfer problem by leveraging model sparsification "twice" --- as a noise-robust feature learning regularizer at the pre-training stage, and as a data-efficient learning regularizer at the few-shot transfer stage. We then propose a bi-level optimization framework termed Bi-RPT that jointly learns optimal masks and automatically allocates sparsity levels for both stages. The effectiveness of Bi-RPT is validated through extensive experiments on our new challenging X-Yield dataset, alongside other synthesized testbeds. Specifically, we achieve an 8.9-19.8% reduction in terms of the test MSE and 0.98-1.53% in terms of test accuracy, merely using 5-10% of the hard-to-generate experimental data.

Crystal Structure Prediction (CSP) is crucial in various scientific disciplines. Existing learning-based generative approaches seldom capture the full symmetries of the crystal structure distribution---the invariance of translation, rotation, and periodicity. In this paper, we propose DiffCSP, a novel diffusion method to learn the stable structure distribution from data, incorporating the above symmetries. To be specific, DiffCSP jointly generates the lattice and the fractional coordinates of all atoms by employing a periodic-E(3)-equivariant denoising model to better model the crystal geometry. Notably, DiffCSP leverages fractional coordinates other than traditional Cartesian coordinates to represent crystals, remarkably promoting the diffusion and the generation process of atom positions. Extensive experiments on crystal structure prediction verify the effectiveness of DiffCSP against existing learning-based counterparts.