Who would have thought that thanks to deep learning we could predict the properties of drugs that do not yet exist? This is of great importance to the pharmaceutical industry.
With regard to artificial intelligence, complaints can sound something like this: “Almost eight years have passed since the invention of the AlexNet neural network [прим. переводчика: в 2012 году Алексей Крижевский опубликовал дизайн сверточной нейросети AlexNet, которая с большим отрывом победила в соревновании ImageNet ], so where is my self-driving car? ” Indeed, it may seem that the expectations of the mid-2010s have not been met. Among pessimists, predictions about .
The purpose of this essay is to discuss the significant progress of machine learning in the real-world drug discovery challenge. I want to remind you of another old adage, this time from AI researchers. To rephrase slightly, it sounds like this: “AI is called AI until it works, then it’s just software.”
What until a few years ago was considered cutting-edge fundamental research in machine learning is now often referred to as “just data science” (or even analytics) – and is revolutionizing the pharmaceutical industry. There is a solid chance that the use of deep learning to discover drugs will dramatically change our lives for the better.
In the 1980s, there was a shift towards , but they still relied on hand-set tags. Simple supervised learning models (such as linear regression or polynomial fit) are trained on features extracted by algorithms such as SIFT (Scale Invariant Feature Transformation) and HOG (Histogram of Directed Gradients). It should come as no surprise that the developments that have led to the practical application of deep learning today started decades ago.
Convolutional neural networks were first used to analyze biomedical images in 1995, when presented a model for the recognition of cancerous tumors in the lungs on fluorograms. Their method was a little different from what we are used to today, the derivation of the result took about 15 seconds, but the concept was essentially the same – with learning through backpropagation all the way to the convolutional cores of the neural network. Their model involved two hidden layers, whereas today’s popular deep network architectures often have one hundred or more layers.
Fast forward to 2012. Convolutional neural networks made a splash with the arrival of the AlexNet system, which led to a leap in performance of the now famous ImageNet dataset. The success of AlexNet, a network with five convolutional and three tightly coupled layers trained on game GPUs, has become so famous in machine learning that people are now talking about “moments of ImageNet»In different niches of machine learning and AI.
For example, “Natural Language Processing may have outlived its ImageNet moment with the development of large transformers in 2018” or “Reinforcement Learning is still waiting for its ImageNet moment.”
Almost ten years have passed since AlexNet. Computer vision and deep learning models are gradually improving. Applications have gone beyond the classification. Today they have learned how to segment images, estimate depth, and automatically reconstruct 3D scenes from multiple 2D images. And this is not a complete list of their capabilities.
Deep learning for biomedical imaging analysis has become a hot area of research. A side effect is an inevitable increase in noise. Published in 2019 … Of course, not all of them are worth reading. It is likely that many researchers overfit models too much on their modest datasets.
Most of them have not made any contributions to basic science or machine learning. A passion for deep learning has gripped academic researchers who had previously shown no interest in it, and for good reason. It can do the same thing as classical computer vision algorithms (see. Tsybenko and Hornik), and often does it faster and better, saving engineers from the tedious manual design of each new application.
The cost of developing a new drug can reach … Sometimes due to high cost and relatively low profitability .
It is also leading to a spike in morbidity in the aptly named category of “neglected diseases”, including a disproportionate number of people in the poorest countries who are considered disadvantageous to treatment, and with low incidence rates. Relatively few people suffer from each of them, but the total number of people with all rare diseases is quite large. It is estimated that about 300 million people. And even this number may be underestimated due to the gloomy assessment of experts: about 30% of those suffering from a rare disease do not live up to five years.
«»Rare diseases have significant potential to improve the lives of huge numbers of people, and this is where machine learning and big data come to the rescue. The blind spot for rare (orphan) diseases that do not have an officially approved treatment opens up an opportunity for innovation from small teams of biologists and machine learning developers.
One such startup in Salt Lake City, Utah is trying to do just that. Founders view the lack of drugs for the treatment of rare diseases as a gap in the pharmaceutical industry. They receive huge amounts of data by analyzing the results of microscopy and laboratory tests. With the help of neural networks, it is possible to identify the features of diseases and seek treatment methods.
By the end of 2019, the company had completed and collected over 4 petabytes of information. A small subset of this data (46 GB) they posted for the NeurIps 2019 competition, you can download it from and play around on your own.
The workflow described in this article is heavily based on information from the official docs [] Recursion Pharmaceuticals, but this approach may well serve as inspiration for other areas.
Other startups in this area include (diseases associated with aging), (oncology) and (various diseases for which there are no treatment options). Typically, young startups are engaged in innovative data processing techniques and apply a variety of machine learning methods in addition to or instead of deep learning with computer vision.
Next, I’ll describe the image analysis process and how deep learning fits into the rare disease drug discovery workflow. We will look at a high-level process that is applicable to a variety of other areas of drug discovery.
For example, it can be easily used to screen cancer drugs for their effect on tumor cell morphology. Perhaps even to analyze the response of cells of specific patients to different drug options. This approach uses concepts from , semantic hashing [] and good old convolutional neural network image classification.
Maybe climate control in a laboratory works differently in summer and winter? Maybe someone had lunch next to the slides before inserting them into the microscope? Maybe the supplier of one of the ingredients of the culture medium has changed? Or has the supplier changed its own supplier? A huge number of variables affect the result of an experiment. Tracking and highlighting unintentional noise is one of the main challenges in data-driven drug discovery.
Microscopic images can be very different in the same experiments. The brightness of the image, the shape of cells, the shape of organelles, and many other characteristics change due to the corresponding physiological effects or random errors.
So, the images in the figure below are obtained from the same metastatic cancer cells, collected by Scott Wilkinson and Adam Markus. Variations in saturation and morphology should reflect the uncertainty of the experimental data. They are created by introducing distortions into the processing. It is a kind of analogue of augmentation, which researchers use to regularize deep neural networks in classification problems. Therefore, it should come as no surprise that the ability to generalize large models to large datasets is a logical choice for looking for physiologically significant features in a sea of noise.
These mutations can be mimicked by suppressing gene expression with (siRNA). It’s a bit like babies grab your ankles: even if you can run fast, your speed will drop dramatically with your niece or nephew hanging from each leg. siRNA works in a similar way: a small sequence of interfering RNAs sticks to the corresponding parts of the messenger RNA of specific genes, preventing their full expression.
By learning from thousands of mutations instead of a singular cellular model of a specific disease, the neural network learns to code phenotypes in a multidimensional hidden space. The resulting code makes it possible to evaluate drugs by their ability to bring the disease phenotype closer to a healthy phenotype, each of which is represented by a multidimensional set of coordinates. Likewise, the side effects of drugs can be embedded in the encoded representation of the phenotype, and drugs are evaluated not only for the disappearance of symptoms of the disease, but also for minimizing harmful side effects.
Also, this image-based drug discovery method works well with the same DenseNet or ResNet architecture with hundreds of layers, which provides optimal performance on datasets like ImageNet.
Layer activation values encoded in a multidimensional space reflect phenotype, disease pathogenesis, relationships between treatments, side effects, and other ailments. Therefore, all these factors can be analyzed by displacement in the coded space. This phenotypic code can be subjected to special regularization (for example, by minimizing covariance between different activations of layers) to reduce coding correlations or for other purposes.
The figure below shows a simplified model. Black arrows represent the operations of convolution + pooling. Blue lines represent tight connections. For simplicity, the number of layers has been reduced and residual connections are not shown.
The effectiveness of this approach has been proven. We are seeing significant research progress and several drugs are already in the first phase of clinical trials. For example, teams of just a few hundred scientists and engineers at companies such as Recursion Pharmaceuticals achieve this. Other startups are close by: TwoXAR has several drug candidates undergoing preclinical trials in other categories of diseases.
The deep learning and computer vision approach to drug development can be expected to have a significant impact on large pharmaceutical companies and healthcare in general. We will soon see how this will affect the development of new treatments for common diseases (including heart disease and diabetes), as well as rare ailments that have remained out of sight to this day.
Source:
ORIGINAL PAGE –