FOR DEVELOPERS
Deep Learning for Quantum Chemistry
Deep learning or neural networks have played a powerful role in revolutionizing industries. This includes contributing to self-driving cars, manufacturing, e-commerce, healthcare, robotics, etc. The power of neural networks to identify unknown and important patterns in data has resulted in important breakthroughs and inventions in these industries.
Currently, the application of deep learning has been very effective in the processing of visual data. CNNs or convolutional neural networks have been extensively used to obtain state-of-the-art results on image-based datasets. In medical imaging analysis, deep learning algorithms are helping doctors by examining CT scans and X-rays and assisting in the diagnosis of diseases such as cancer.
Research companies like DeepMind have used deep learning to create algorithms and systems that have broken limitations in science and enabled advanced research. For example, DeepMind’s AlphaFold2 algorithm is helping scientists to decode the shape and structure of proteins.
This article will explore the potential applications of deep learning in the domain of scientific research, particularly quantum chemistry.
What is quantum chemistry?
Also known as molecular quantum mechanics, quantum chemistry is a branch of physical chemistry that focuses on the application of quantum mechanics to chemical systems, especially in the quantum-mechanical calculation of electronic contributions and effects. These calculations include systematically applied approximations intended to make calculations computationally feasible while capturing as much information as possible about important contributions to wave functions and observable properties.
Density-functional theory (DFT)
Density-functional theory or DFT is an important computational quantum mechanical modeling used in physics, chemistry, and mechanical sciences. It is used to investigate the electronic structure or nuclear structure of many-body systems, particularly atoms, molecules, and condensed phases. Using this theory, the properties of a many-electron system can be determined using functionals, i.e., the functions of another function. In this case, these are functionals of spatially dependent electron density.
Deep learning is being used to model the mapping of the energy density functional component of DFT, which describes the quantum mechanical behavior of molecules. Since DFT plays a major role in simplifying quantum chemistry calculations for scientists, it requires an accurate functional mapping between the electronic probability densities to energy. Here, deep learning models are trained to act as a functional to establish the mapping between the 3D electron density and chemical interaction energy.
DeepMind has developed a neural network model called DM21 for the same application. Let’s explore this model in detail.
DM21: DeepMind quantum chemistry AI model
Quantum chemistry is an application of the fundamental rules of quantum physics to predict the chemical properties of molecules. DFT enables scientists to simplify quantum chemistry calculations, but it needs a functional or mapping electronic probability density to energy. While there is no known exact functional, approximate functionals have been used for many years in fields ranging from solid-state physics to nuclear spectroscopy.
However, most of the previous approximations or methods in use encounter ‘pathological errors’ in systems that have characteristics of fractional electrons, in particular, fractional charge (FC) or fractional spin (FS). Since manually designed functionals are prone to more errors and it is difficult to design functionals to satisfy every edge case, the DeepMind team approached the problem using deep learning.
DM21 is a neural network model for mapping electron density to chemical interaction energy, a key component of quantum mechanical simulation. It is trained and developed by DeepMind, a Google subsidiary. This model is open-source and has been made available as an extension to the PySCF simulation framework.
DM21 uses a neural network to approximate the energy density functional component of density functional theory. The model is based on a multilayer perceptron (MLP) architecture which takes a grid of electron densities as input. DM21 addresses the systemic problems with previous functional approximations which cannot handle systems with fractional electron character.
Model training
The DM21 model was trained using the supervised learning approach with the MLP neural network architecture. The training dataset consists of 1161 examples. The inputs contained a spatial grid of Kohn-Sham (KS) orbital features, i.e., electron densities, and the output values/labels were "high accuracy reaction energies"
The model training was done with regression loss as the loss function. A gradient regularizer term was also included in training so that the model could be used in self-consistent field (SCF) calculations.
Results and benchmark
The DM21 model was evaluated on three benchmark datasets, i.e., Bond-breaking Benchmark (BBB), GMTKN55, and QM9, which contained data for chemical tasks that were distinct from the training data. The DM21 model set new state-of-the-art performance on these benchmarks by outperforming four of the best-performing existing implementations.
According to researchers, “DM21 is better than the best hybrid functional and approaches the performance of the much more expensive double-hybrid functionals.”
We have seen that the DM21 model is the best performing functional whose approximations of the chemical interaction energies of the molecules are state-of-the-art compared to the previous methods of manually designing the functionals.
This is a perfect example of how effective deep learning models are at being able to detect and learn patterns from data. This method will not only result in models that will provide accurate predictions of energy states, but also cover the edge cases that result in pathological errors due to fractional electrons.
DL-driven breakthroughs in quantum chemistry
The use of deep learning and machine learning in physics and chemistry is an active and evolving research area. There have been various breakthroughs in quantum chemistry using deep learning.
In 2019, researchers at Stanford trained a convolutional neural network for use as a functional that achieved good results for ‘a large set of organic molecules’. Similarly, in 2020, Caltech used machine learning to solve the Navier-Stokes equation, while in 2021, DeepMind’s AlphaFold2 AI was used to predict protein structure.
Looking at the research and the outcomes so far, we can confidently conclude with a strong vision for a future where deep learning and machine learning will be extensively used to discover more breakthroughs in the field of scientific research.
