The discovery of new materials drives the development of key technologies like solar cells, batteries, carbon capture, and catalysis. While there has been growing interest in materials discovery with machine learning, the specific modeling challenges posed by materials have been largely unknown to the broader community. Compared with drug-like molecules and proteins, the modeling of materials has the following two major challenges. First, materials-specific inductive biases are needed to develop successful ML models. For example, materials often don’t have a handy representation like 2D graphs for molecules or sequences for proteins. Second, there exists a broad range of interesting materials classes, such as inorganic crystals, polymers, catalytic surfaces, nanoporous materials, and more. Each class of materials demands a different approach to represent their structures, and new tasks/data sets to enable rapid ML developments.This workshop aims at bringing together the community to discuss and tackle these two types of challenges. In the first session, we will feature speakers to discuss the latest progress in developing ML models for materials focusing on algorithmic challenges, covering topics like representation learning, generative models, pre-training, etc. In particular, what can we learn from the more developed field of ML for molecules and 3D geometry and where might challenges differ and opportunities for novel developments lie? In the second session, we will feature speakers to discuss unique challenges for each sub-field of materials design and how to define meaningful tasks that are relevant to the domain, covering areas including inorganic materials, polymers, nanoporous materials, catalysis, etc. More specifically, what are the key materials design problems that ML can help tackle?
Thu 6:00 a.m. - 6:10 a.m.
|
Openning
|
🔗 |
Thu 6:10 a.m. - 6:40 a.m.
|
Invited talk
SlidesLive Video » |
Boris Kozinsky 🔗 |
Thu 6:40 a.m. - 7:10 a.m.
|
Machine learning approaches to improve the exchange and correlation functional in Density functional Theory
(
Invited talk
)
SlidesLive Video » |
Marivi Fernandez-Serra 🔗 |
Thu 7:10 a.m. - 7:30 a.m.
|
Break
|
🔗 |
Thu 7:30 a.m. - 8:00 a.m.
|
Harnessing the properties of equivariant neural networks to understand and design materials
(
Invited talk
)
SlidesLive Video » |
Tess Smidt 🔗 |
Thu 8:00 a.m. - 8:30 a.m.
|
Machine learning-guided directed evolution of functional proteins
(
Invited talk
)
SlidesLive Video » |
Andrew Ferguson 🔗 |
Thu 8:30 a.m. - 8:40 a.m.
|
JAX-XC: Exchange Correlation Functionals Library in Jax
(
Spotlight
)
link »
SlidesLive Video » We present JAX-XC, an open-source library that provides exchange-correlation functionals in Jax. JAX-XC is built from LIBXC, its correctness has been verified numerically against LIBXC. Thanks to Jax, JAX-XC is end-to-end differentiable, computationally more efficient thanks to the vectorization provided by XLA, and also portable on various accelerators. More importantly, as more research is focusing on machine learning for density functional theory, we hope that JAX-XC could serve as a deep learning-friendly tool and a stepping-stone for researchers working in the intersection of deep learning and density functional theory. |
Kunhao Zheng · Min Lin 🔗 |
Thu 8:40 a.m. - 8:50 a.m.
|
Crystal Structure Prediction by Joint Equivariant Diffusion on Lattices and Fractional Coordinates
(
Spotlight
)
link »
SlidesLive Video » Crystal Structure Prediction (CSP) is crucial in various scientific disciplines. Existing learning-based generative approaches seldom capture the full symmetries of the crystal structure distribution---the invariance of translation, rotation, and periodicity. In this paper, we propose DiffCSP, a novel diffusion method to learn the stable structure distribution from data, incorporating the above symmetries. To be specific, DiffCSP jointly generates the lattice and the fractional coordinates of all atoms by employing a periodic-E(3)-equivariant denoising model to better model the crystal geometry. Notably, DiffCSP leverages fractional coordinates other than traditional Cartesian coordinates to represent crystals, remarkably promoting the diffusion and the generation process of atom positions. Extensive experiments on crystal structure prediction verify the effectiveness of DiffCSP against existing learning-based counterparts. |
Rui Jiao · Wenbing Huang · Peijia Lin · Jiaqi Han · Pin Chen · Yutong Lu · Yang Liu 🔗 |
Thu 8:50 a.m. - 9:00 a.m.
|
Cross-Quality Few-Shot Transfer for Alloy Yield Strength Prediction: A New Material Science Benchmark and An Integrated Optimization Framework
(
Spotlight
)
link »
SlidesLive Video » Discovering high-entropy alloys (HEAs) with high yield strength is an important yet challenging task in material science. However, the yield strength can only be accurately measured by very expensive and time-consuming real-world experiments, hence cannot be acquired at scale. Learning-based methods could facilitate the discovery process, but the lack of a comprehensive dataset on HEA yield strength has created barriers. We present X-Yield, a large-scale material science benchmark with 240 experimentally measured (“high-quality”) and over 100K simulated (imperfect or “low-quality”) HEA yield strength annotations. Due to the scarcity of experimental annotations and the quality gap in imperfectly simulated data, existing transfer learning methods cannot generalize well on our dataset. We address this cross-quality few-shot transfer problem by leveraging model sparsification “twice” — as a noise-robust feature learning regularizer at the pre-training stage, and as a data-efficient learning regularizer at the few-shot transfer stage. We then propose a bi-level optimization framework termed Bi-RPT that jointly learns optimal masks and automatically allocates sparsity levels for both stages. The effectiveness of Bi-RPT is validated through extensive experiments on our new challenging X-Yield dataset, alongside other synthesized testbeds. Specifically, we achieve an 8.9-19.8% reduction in terms of the test MSE and 0.98-1.53% in terms of test accuracy, merely using 5-10% of the experimental data. |
Xuxi Chen · Tianlong Chen · Everardo Olivares · Kate Elder · Scott McCall · Aurelien Perron · Joseph McKeown · Bhavya Kailkhura · Zhangyang Wang · Brian Gallagher 🔗 |
Thu 9:00 a.m. - 10:00 a.m.
|
Poster Session 1
(
Poster Session
)
|
🔗 |
Thu 10:00 a.m. - 10:30 a.m.
|
Break
|
🔗 |
Thu 10:30 a.m. - 11:00 a.m.
|
Machine learning to generate molecules and materials and their synthesis predictions
(
Invited talk
)
SlidesLive Video » |
Yousung Jung 🔗 |
Thu 11:00 a.m. - 11:30 a.m.
|
Invited talk
SlidesLive Video » |
Rafael Gomez-Bombarelli 🔗 |
Thu 11:30 a.m. - 11:50 a.m.
|
Break
|
🔗 |
Thu 11:50 a.m. - 12:20 p.m.
|
A potential of everything
(
Invited talk
)
SlidesLive Video » |
Shyue Ping Ong 🔗 |
Thu 12:20 p.m. - 12:50 p.m.
|
Open datasets/models in catalysis: recent progress their use to massively accelerate adsorption energy workflows
(
Invited talk
)
SlidesLive Video » |
Zachary Ulissi 🔗 |
Thu 12:50 p.m. - 1:00 p.m.
|
Break
|
🔗 |
Thu 1:00 p.m. - 2:00 p.m.
|
Panel Discussion
SlidesLive Video » |
Boris Kozinsky · Tess Smidt · Rafael Gomez-Bombarelli · Marivi Fernandez-Serra · Zachary Ulissi · Shyue Ping Ong · Yousung Jung · Andrew Ferguson 🔗 |
Thu 2:00 p.m. - 2:55 p.m.
|
Poster Session 2
(
Poster Session
)
|
🔗 |
Thu 2:55 p.m. - 3:00 p.m.
|
Closing remarks
|
🔗 |
-
|
Constructing and Compressing Global Moment Descriptors from Local Atomic Environments
(
Poster
)
link »
Local atomic environment descriptors (LAEDs) are used in the materials science and chemistry communities, for example for the development of machine learning interatomic potentials. Despite the fact that LAEDs have been extensively studied and benchmarked for various applications, global structure descriptors (GSDs), i.e., descriptors for entire molecules or crystal structures, have been mostly developed independently based on other approaches. Here, we propose a systematically improvable methodology for constructing GSDs from local atomic environment descriptors by incorporating statistical information and information about chemical elements. We apply the method to construct GSDs of varying complexity for lithium thiophosphate structures that are of interest as solid electrolytes and use an information-theoretic approach to obtain an optimally compressed GSD. Finally, we report the performance of the compressed GSD for energy prediction tasks. |
Vahe Gharakhanyan · Max Aalto · Aminah Alsoulah · Nongnuch Artrith · Alexander Urban 🔗 |
-
|
SimuStruct: Simulated Structural Plate with Holes Dataset with Machine Learning Applications
(
Poster
)
link »
This paper introduces SimuStruct: Simulated Structural Parts Dataset, a dataset that contains 2D structural parts, their respective meshes and the outputs of numerical simulations for different properties for linear and elastic material, boundary and loading conditions, and for varying levels of refinement. SimuStruct comprises the classic case of plates with holes since it is a 2D simple case with analytical resolution and which is found in different mechanical design applications. The SimuStruct dataset comprises many different cases, where each case is solved using standard Finite Element Methods (FEMs) with the open-source package FEniCS. Compared to other datasets similar in purpose, SimuStruct is more diversified and realistic because it aims to comprises diverse real cases for different loading and boundary conditions, different properties for linear and elastic material, and different levels of refinement. In addition, SimuStruct is more flexible, versatile, and scalable because all algorithms and codes are implemented using open-source libraries. The main goal of the SimuStruct dataset is to serve both as training and evaluation data for Machine Learning (ML)-based methods in structural analysis and optimal mesh generation and therefore support the development of ML-based optimal mechanical design solutions. An application of SimuStruct is presented to train and test an ANN model to predict stress-strain fields. SimuStruct will contribute to the connection of the Mechanical Engineering and ML communities, which will allow accelerating and exploitation the research in the computational design field. |
João Alves Ribeiro · Bruno Alves Ribeiro 🔗 |
-
|
Expanding the Extrapolation Limits of Neural Network Force Fields using Physics-Based Data Augmentation
(
Poster
)
link »
Even though machine learning force fields are quite accurate in the prediction of forces and energies in the sampled region, they fail to extrapolate, which results in the unphysical behavior of the system during molecular dynamics simulations. We propose to overcome this problem by performing data augmentation. To expand the original dataset random perturbations of atoms were performed. The corresponding increase in the energy of the system was calculated under the assumption of harmonicity. The required spring constants were obtained from the original dataset by fitting a gaussian mixture model to the bond lengths distribution. The resulting force field performance was improved in the regions far from training data. |
Yuliia Orlova · Gavin Ridley · Frederick Zhao · Rafael Gomez-Bombarelli 🔗 |
-
|
Forward and Inverse design of high $T_C$ superconductors with DFT and deep learning
(
Poster
)
link »
We developed a multi-step workflow for the discovery of next-generation conventional superconductors. 1) We started with a Bardeen–Cooper–Schrieffer (BCS) inspired pre-screening of 55000 materials in the JARVIS-DFT database resulting in 1736 materials with high Debye temperature and electronic density of states at the Fermi-level. 2) Then, we performed density functional theory (DFT) based electron-phonon coupling calculations for 1058 materials to establish a systematic database of superconducting properties. 3) Further, we applied forward deep-learning (DL) using atomistic line graph neural network (ALIGNN) models to predict properties faster than direct first-principles computations. Notably, we find that by predicting the Eliashberg function as an intermediate quantity, we can improve the model performance versus a direct DL prediction of $T_C$. Finally, 4) we used an inverse deep-learning method with a crystal diffusion variational autoencoder (CDVAE) model to generate thousands of new superconductors with high chemical and structural diversity. 5) We screened these CDVAE-generated structures using ALIGNN to identify candidates that are stable with high $T_C$. 6) We verified the top superconducting candidates with DFT.
|
Daniel Wines · Kevin Garrity · Tian Xie · Kamal Choudhary 🔗 |
-
|
MatKB: Semantic Search for Polycrystalline Materials Synthesis Procedures
(
Poster
)
link »
In this paper, we present a novel approach to knowledge extraction and retrieval using Natural Language Processing (NLP) techniques for material science. Our goal is to automatically mine structured knowledge from millions of research articles in the field of polycrystalline materials and make it easily accessible to the broader community. The proposed method leverages NLP techniques such as entity recognition and document classification to extract relevant information and build an extensive knowledge base, from a collection of 9.5 Million publications. The resulting knowledge base is integrated into a search engine, which enables users to search for information about specific materials, properties, and experiments with greater precision than traditional search engines like Google. We hope our results can enable material scientists quickly locate desired experimental procedures, compare their differences, and even inspire them to design new experiments. |
Xianjun Yang · Stephen Wilson · Linda Petzold 🔗 |
-
|
A framework for fully autonomous design of materials via multiobjective optimization and active learning: challenges and next steps
(
Poster
)
link »
In order to deploy machine learning in a real-world self-driving laboratory where data acquisition is costly and there are multiple competing design criteria, systems need to be able to intelligently sample while balancing performance tradeoffs and constraints. For these reasons, we present an active learning process based on multiobjective blackbox optimization with continuously-updated machine learning models. This workflow is built upon open-source technologies for real-time data streaming and modular multiobjective optimization software development. We demonstrate a proof-of-concept for this workflow through the autonomous operation of a continuous-flow chemistry laboratory, which identifies ideal manufacturing conditions for the electrolyte 2,2,2-trifluoroethyl Methyl carbonate. |
Tyler Chang · Jakob Elias · Stefan Wild · Santanu Chaudhuri · Joseph Libera 🔗 |
-
|
Behavioral Cloning for Crystal Design
(
Poster
)
link »
Solid-state materials, which are made up of periodic 3D crystal structures, are particularly useful for a variety of real-world applications such as batteries, fuel cells and catalytic materials. Designing solid-state materials, especially in a robust and automated fashion, remains an ongoing challenge. To further the automated design of crystalline materials, we propose a method to learn to design valid crystal structures given a crystal skeleton. By incorporating Euclidean equivariance into a policy network, we portray the problem of designing new crystals as a sequential prediction task suited for imitation learning. At each step, given an incomplete graph of a crystal skeleton, an agent assigns an element to a specific node. We adopt a behavioral cloning strategy to train the policy network on data consisting of curated trajectories generated from known crystals. |
Prashant Govindarajan · Santiago Miret · Jarrid Rector-Brooks · mariano Phielipp · Janarthanan Rajendran · Sarath Chandar 🔗 |
-
|
Designing Nonlinear Photonic Crystals for High-Dimensional Quantum State Engineering
(
Poster
)
link »
We propose a novel, physically-constrained and differentiable approach for the generation of D-dimensional qudit states via spontaneous parametric down-conversion (SPDC) in quantum optics. We circumvent any limitations imposed by the inherently stochastic nature of the physical process and incorporate a set of stochastic dynamical equations governing its evolution under the SPDC Hamiltonian. We demonstrate the effectiveness of our model through the design of structured nonlinear photonic crystals (NLPCs) and shaped pump beams; and show, theoretically and experimentally, how to generate maximally entangled states in the spatial degree of freedom. The learning of NLPC structures offers a promising new avenue for shaping and controlling arbitrary quantum states and enables all-optical coherent control of the generated states. We believe that this approach can readily be extended from bulky crystals to thin Metasurfaces and potentially applied to other quantum systems sharing a similar Hamiltonian structures, such as superfluids and superconductors. |
Eyal Rozenberg · Aviv Karnieli · Ofir Yesharim · Joshua Foley-Comer · Sivan Trajtenberg-Mills · Sarika Mishra · Shashi Prabhakar · Ravindra Singh · Daniel Freedman · Alex Bronstein · Ady Arie
|
-
|
Controlling Dynamic Spatial Light Modulators using Equivariant Neural Networks
(
Poster
)
link »
Spatial Light Modulators (SLMs) are devices that can modulate the amplitude or the phase of a beam of light. These devices are used in applications such as beam front aberration and microscopic manipulation with optical tweezers. Here, we study the problem of learning to modulate light in a new type of temperature-controlled SLM. These SLMs are panels that use a thin viscous film in which shallow wave patterns can be induced by varying the temperature of the panel. This method can be used for modulating light such as high-power lasers. The problem here is to learn which input temperature signal is necessary in order to induce a given pattern in the reflected light. We propose a deep equivariant model to learn this relationship. We generate a synthetic dataset consisting of temperature signals and corresponding light patterns by simulating the thin film lubrication equation that governs the phenomenon of thermocapillary dewetting. We use this dataset to train our networks. We demonstrate the advantage of using equivariant neural networks over convolutional neural networks in order to learn the mapping. |
Sumukh Vasisht Shankar · Darrel D'Souza · Jonathan Singer · Robin Walters 🔗 |
-
|
In-situ Anomaly Detection in Additive Manufacturing with Graph Neural Networks
(
Poster
)
link »
Transforming a design into a high-quality product is a challenge in metal additive manufacturing due to rare events which can cause defects to form. Detecting these events in-situ could, however, reduce inspection costs, enable corrective action, and is the first step towards a future of tailored material properties. In this study a model is trained on laser input information to predict nominal laser melting conditions. An anomaly score is then calculated by taking the difference between the predictions and new observations. The model is evaluated on a dataset with known defects achieving an F1 score of 0.821. This study shows that anomaly detection methods are an important tool in developing robust defect detection methods. |
Sebastian Larsen · Paul Hooper 🔗 |
-
|
Machine Learning for XRD Spectra Interpretation in High-Throughput Material Science
(
Poster
)
link »
Experimentally synthesizing predicted materials in a reproducible manner remains a key bottleneck in materials science progress. Autonomous synthesis and closed loop integration of prediction and characterization can address these issues, however, this requires autonomous characterization methods for all analysis including crystallographic phase identification which currently remains a rate-limiting step. Here we benchmark several machine learning techniques for X-ray Diffraction spectra interpretation (spectral clustering, convolutional neural networks, and invertible neural networks) and compare the relative strengths and weaknesses of each approach. Future work will involve deploying these techniques across the entire high-throughput experimental materials database. |
Hilary Egan · Davi Febba · Andriy Zakutayev 🔗 |
-
|
Matbench Discovery - Can machine learning identify stable crystals?
(
Poster
)
link »
We present a new machine learning (ML) benchmark for materials stability predictions named Matbench Discovery.A goal of this benchmark is to highlight the need to focus on metrics that directly measure their utility in prospective discovery campaigns as opposed to analyzing models based on predictive accuracy alone.Our benchmark consists of a task designed to closely simulate the deployment of ML energy models in a high-throughput search for stable inorganic crystals.We explore a wide variety of models covering multiple methodologies ranging from random forests to GNNs, and from one-shot predictors to iterative Bayesian optimizers and interatomic potential-based relaxers. We find M3GNet to achieve the highest F1 score of 0.58 and $R^2$ of 0.59 while MEGNet wins on discovery acceleration factor (DAF) with 2.94. Our results provide valuable insights for maintainers of high throughput materials databases to start using these models as triaging steps to more effectively allocate compute for DFT relaxations.
|
Janosh Riebesell · Rhys Goodall · Anubhav Jain · Kristin Persson · Alpha Lee 🔗 |
-
|
Graph-informed simulation-based inference for models of active matter
(
Poster
)
link »
Many collective systems exist in nature far from equilibrium, ranging from cellular sheets up to flocks of birds. These systems reflect a form of active matter, whereby individual material components have internal energy. Under specific parameter regimes, these active systems undergo phase transitions whereby small fluctuations of single components can lead to global changes to the rheology of the system. Simulations and methods from statistical physics are typically used to understand and predict these phase transitions for real-world observations. In this work, we demonstrate that simulation-based inference can be used to robustly infer active matter parameters from system observations. Moreover, we demonstrate that a small number (from one to three) snapshots of the system can be used for parameter inference and that this graph-informed approach outperforms typical metrics such as the average velocity or mean square displacement of the system. Our work highlights that high-level system information is contained within the relational structure of a collective system and that this can be exploited to better couple models to data. |
Namid Stillman · Silke Henkes · Roberto Mayor · Gilles Louppe 🔗 |
-
|
Predicting Density of States via Multi-modal Transformer
(
Poster
)
link »
The density of states (DOS) is a spectral property of materials, which provides fundamental insights on various characteristics of materials. In this paper, we propose to predict the DOS by reflecting the nature of DOS: DOS determines the general distribution of states as a function of energy. Specifically, we integrate the heterogeneous information obtained from the crystal structure and the energies via multi-modal transformer, thereby modeling the complex relationships between the atoms in the crystal structure, and various energy levels. Extensive experiments on two types of DOS, i.e., phonon DOS and electron DOS, with various real-world scenarios demonstrate the superiority of DOSTransformer. |
Namkyeong Lee · Heewoong Noh · Sungwon Kim · Dongmin Hyun · Gyoung S. Na · Chanyoung Park 🔗 |
-
|
Latent Conservative Objective Models for Offline Data-Driven Crystal Structure Prediction
(
Poster
)
link »
In computational chemistry, crystal structure prediction (CSP) is an optimization problem that involves discovering new crystal structures. This problem is challenging for machine learning (ML) methods: it requires discovering globally optimal designs that attain the smallest energy in complex non-Euclidean manifolds. One approach to tackle this problem involves building simulators based on density functional theory (DFT), but these simulators are painfully slow. More recent approaches are exploring the alternate paradigm of relying on learned graph neural networks (GNNs) surrogate models as a proxy for simulation. We propose a method that leverages GNNs to reduce the complexity of the problem. Concretely, we reduce the non-Euclidean optimization search space to a standard vector one with Graph Variational Autoencoders (GVAEs), and we combine that with techniques from offline model-based optimization. This prevents the optimization procedure from producing unstable structures that erroneously appear to have low energies under the learned model. We show that this procedure outperforms current alternatives, both in terms of success rate of structure prediction, and computational cost. In addition, it provides a generic recipe to apply offline optimization techniques for optimizing in non-Euclidean spaces. |
Han Qi · Stefano Rando · Xinyang Geng · Iku Ohama · Aviral Kumar · Sergey Levine 🔗 |
-
|
Compositional and elemental descriptors for perovskite materials
(
Poster
)
link »
In this extended abstract we compare the performance of different families of descriptors – \textit{molar composition descriptor, weight composition descriptor and elemental descriptor} – for regression tasks and include examples of a classification task for perovskite oxide materials with general formulas $ABO_3$, $A_2BB’O_6$, and $A_xA’_{1-x}B_yB’_{1-y}O_6$. The best performance was observed for our elemental descriptor which consisted of $A$-site and $B$-site element information on: Shannon’s ionic radius, ideal bond length, electronegativity, van der Waals radius, ionization energy, molar volume, atomic number, and atomic mass. The weight composition descriptor showed superior results over a simpler molar composition descriptor. The results of principal component analysis, regression models with the hyperparameters optimized using an autoML software and Wasserstein autoencoders are briefly discussed for a possible use in inverse materials design.
|
Jiri Hostas · Maicon Lourenço · John Garcia · Hatef Shahmohamadi · Alain Tchagang · Karthik Shankar · Venkataraman Thangadurai · Dennis Salahub 🔗 |
-
|
Cooperative data-driven modeling: continual learning of different material behavior
(
Poster
)
link »
Training deep neural networks (DNNs) in solid mechanics is challenging due to data scarcity. Even when using synthetic datasets obtained from computational simulations, these datasets have limited size because each simulation is time-consuming. One way to address this challenge is to use transfer learning, i.e. finetuning a model pretrained in a different context (e.g., computer vision) such that it can learn a new task while using less data (e.g., learning the behavior of a material with a given microstructure). Unfortunately, a model obtained by transfer learning loses the ability to solve the original task. Therefore, each new task that is being learned destroys the ability to perform the previous one with the same model. We present a Cooperative Data-driven Modeling (CDDM) network that can continually learn tasks without forgetting, accumulating knowledge such that less training data is required when facing a new task or that leads to smaller prediction test error for each new task. We provide our numerical experiments on predicting the plastic behavior of different materials using recurrent neural networks, as they have been shown to handle history-dependent problems. |
Aleksandr Dekhovich · Ozgur Turan · Jiaxiang Yi · Miguel A. Bessa 🔗 |
-
|
3D Graph Conditional Distributions via Semi-Equivariant Continuous Normalizing Flows
(
Poster
)
link »
A general method for learning the conditional distribution $p(G | \hat{G})$ of two 3D graphs is proposed. The method is designed to be invariant to rigid body transformations and to permutations of the vertices of either graph. The core of the method is a continuous normalizing flow and semi-equivariance conditions are established to ensure the aforementioned invariance conditions. The utility of the technique is demonstrated as a conditional generative model for the molecular setting.
|
Eyal Rozenberg · Ehud Rivlin · Daniel Freedman 🔗 |
-
|
CrysGNN : Distilling pre-trained knowledge to enhance property prediction for crystalline materials.
(
Poster
)
link »
In recent years, graph neural network (GNN) based approaches have emerged as apowerful technique to encode complex topological structure of crystal materials inan enriched representation space. These models are often supervised in nature andusing the property-specific training data, learn relationship between crystal structureand different properties like formation energy, bandgap, bulk modulus, etc. Mostof these methods require a huge amount of property-tagged data to train the systemwhich may not be available for different properties. However, there is an availabilityof a huge amount of crystal data with its chemical composition and structural bonds.To leverage these untapped data, this paper presents CrysGNN, a new pre-trainedGNN framework for crystalline materials, which captures both node and graphlevel structural information of crystal graphs using a huge amount of unlabelledmaterial data. Further, we extract distilled knowledge from CrysGNN and injectinto different state of the art property predictors to enhance their property predictionaccuracy. We conduct extensive experiments to show that with distilled knowledgefrom the pre-trained model, all the SOTA algorithms are able to outperform theirown vanilla version with good margins. We also observe that the distillation processprovides a significant improvement over the conventional approach of finetuningthe pre-trained model. |
KISHALAY DAS · Bidisha Samanta · Pawan Goyal · Seung-Cheol Lee · Satadeep Bhattacharjee · Niloy Ganguly 🔗 |
-
|
Learning single-step retrosynthesis with pseudo-reactions
(
Poster
)
link »
Retrosynthesis analysis aims to design reaction pathways and intermediates for a target compound. Emerging works have been developed to automate this process using artificial intelligence (AI). Although there are multiple synthesis pathways to synthesize one target product, most of the existing models were optimized to select only one of them. One of the potential reason is due to the absence of other potential reactions in the available reaction dataset, as one product usually only appears one time in the reaction dataset. In this work, we generate virtually validated pseudo-reactions using local reaction templates and reaction outcome prediction model to optimize the retrosynthesis model to predict multiple synthesis pathways. With the aid of newly generated pseudo-reactions, the top-10 exact match accuracy is increased from 93.1% to 94.2% and the top-10 round trip accuracy is increased from 81.3% to 87.6% with higher prediction confidence and diversity on a public reaction dataset. |
Shuan Chen · Yousung Jung 🔗 |
-
|
Fragment-based Multi-view Molecular Contrastive Learning
(
Poster
)
link »
Molecular representation learning is a fundamental task for AI-based drug design and discovery. Self-supervised contrastive learning on molecular graphs, which aims to learn good representations via semantic-preserving transformations, is an attractive framework for this task. However, it is relatively under-explored to design such transformations for molecules under consideration of their chemical semantics. In this paper, we consider fragmentation which decomposes a molecule into a set of chemically meaningful fragments (e.g., functional groups) as the semantic-preserving transformation. Here, we also utilize the 3D geometric views of molecules as another source of such transformation. Based on these molecule-specialized semantic-preserving transformations, we propose fragment-based multi-view molecular contrastive learning (FragCL), an effective framework that learns chemically meaningful molecular representations. Through extensive experiments, we demonstrate that our framework outperforms prior molecular representation learning methods across various molecular property prediction tasks. |
Seojin Kim · Jaehyun Nam · Junsu Kim · Hankook Lee · Sungsoo Ahn · Jinwoo Shin 🔗 |
-
|
Transfer Learning with Diffusion Model for Polymer Property Prediction
(
Poster
)
link »
Polymers are important and numerous. While the structure synthesis and property annotation for polymers require expensive equipment and a long time of effort, small molecules without annotations have been collected from various sources and at a large scale. However, there is a lack of studies for effective transfer learning from molecules without labels (as the source domain) to polymers with labels (as the target domain). This paper proposes to extract the knowledge underlying the large set of source molecules as a specific set of useful graphs to augment the training set for target polymers. We learn a diffusion probabilistic model on the source data and design two new objectives to guide the model's denoising process with target data to generate target-specific labeled graphs. Experiments from unlabeled molecules to labeled polymers demonstrate that our transfer learning approach outperforms existing semi/self-supervised learning approaches. |
Gang Liu · Meng Jiang 🔗 |
-
|
Machine learning-assisted close-set X-ray diffraction phase identification of transition metals
(
Poster
)
link »
Machine learning has been applied to the problem of X-ray diffraction phase prediction with promising results. In this paper, we describe a method for using machine learning to predict crystal structure phases from X-ray diffraction data of transition metals and their oxides. We evaluate the performance of our method and compare the variety of its settings. Our results demonstrate that the proposed machine learning framework achieves competitive performance. This demonstrates the potential for machine learning to significantly impact the field of X-ray diffraction and crystal structure determination. |
Maksim Zhdanov · Andrey Zhdanov 🔗 |
-
|
JAX-XC: Exchange Correlation Functionals Library in Jax
(
Poster
)
link »
We present JAX-XC, an open-source library that provides exchange-correlation functionals in Jax. JAX-XC is built from LIBXC, its correctness has been verified numerically against LIBXC. Thanks to Jax, JAX-XC is end-to-end differentiable, computationally more efficient thanks to the vectorization provided by XLA, and also portable on various accelerators. More importantly, as more research is focusing on machine learning for density functional theory, we hope that JAX-XC could serve as a deep learning-friendly tool and a stepping-stone for researchers working in the intersection of deep learning and density functional theory. |
Kunhao Zheng · Min Lin 🔗 |
-
|
Cross-Quality Few-Shot Transfer for Alloy Yield Strength Prediction: A New Material Science Benchmark and An Integrated Optimization Framework
(
Poster
)
link »
Discovering high-entropy alloys (HEAs) with high yield strength (YS) is an important yet challenging task in materials science. However, YS can only be accurately measured by very expensive and time-consuming real-world experiments, hence cannot be acquired at scale. Learning-based methods could facilitate the discovery process, but the lack of a comprehensive dataset on HEA yield strength has created barriers. We present X-Yield, a materials science benchmark with 240 experimentally measured ("high-quality") and over 100K simulated (imperfect or "low-quality") HEA yield strength data. Due to the scarcity of experimental data and the quality gap with imperfectly simulated data, existing transfer learning methods cannot generalize well on our dataset. We address this cross-quality few-shot transfer problem by leveraging model sparsification "twice" --- as a noise-robust feature learning regularizer at the pre-training stage, and as a data-efficient learning regularizer at the few-shot transfer stage. We then propose a bi-level optimization framework termed Bi-RPT that jointly learns optimal masks and automatically allocates sparsity levels for both stages. The effectiveness of Bi-RPT is validated through extensive experiments on our new challenging X-Yield dataset, alongside other synthesized testbeds. Specifically, we achieve an 8.9-19.8% reduction in terms of the test MSE and 0.98-1.53% in terms of test accuracy, merely using 5-10% of the hard-to-generate experimental data. |
Xuxi Chen · Tianlong Chen · Everardo Olivares · Kate Elder · Scott McCall · Aurelien Perron · Joseph McKeown · Bhavya Kailkhura · Zhangyang Wang · Brian Gallagher 🔗 |
-
|
Crystal Structure Prediction by Joint Equivariant Diffusion on Lattices and Fractional Coordinates
(
Poster
)
link »
Crystal Structure Prediction (CSP) is crucial in various scientific disciplines. Existing learning-based generative approaches seldom capture the full symmetries of the crystal structure distribution---the invariance of translation, rotation, and periodicity. In this paper, we propose DiffCSP, a novel diffusion method to learn the stable structure distribution from data, incorporating the above symmetries. To be specific, DiffCSP jointly generates the lattice and the fractional coordinates of all atoms by employing a periodic-E(3)-equivariant denoising model to better model the crystal geometry. Notably, DiffCSP leverages fractional coordinates other than traditional Cartesian coordinates to represent crystals, remarkably promoting the diffusion and the generation process of atom positions. Extensive experiments on crystal structure prediction verify the effectiveness of DiffCSP against existing learning-based counterparts. |
Rui Jiao · Wenbing Huang · Peijia Lin · Jiaqi Han · Pin Chen · Yutong Lu · Yang Liu 🔗 |