Connect with us

Science & Technology

Future Goals in the AI Race: Explainable AI and Transfer Learning

Avatar photo

Published

on

Recent years have seen breakthroughs in neural network technology: computers can now beat any living person at the most complex game invented by humankind, as well as imitate human voices and faces (both real and non-existent) in a deceptively realistic manner. Is this a victory for artificial intelligence over human intelligence? And if not, what else do researchers and developers need to achieve to make the winners in the AI race the “kings of the world?”

Background

Over the last 60 years, artificial intelligence (AI) has been the subject of much discussion among researchers representing different approaches and schools of thought. One of the crucial reasons for this is that there is no unified definition of what constitutes AI, with differences persisting even now. This means that any objective assessment of the current state and prospects of AI, and its crucial areas of research, in particular, will be intricately linked with the subjective philosophical views of researchers and the practical experience of developers.

In recent years, the term “general intelligence,” meaning the ability to solve cognitive problems in general terms, adapting to the environment through learning, minimizing risks and optimizing the losses in achieving goals, has gained currency among researchers and developers. This led to the concept of artificial general intelligence (AGI), potentially vested not in a human, but a cybernetic system of sufficient computational power. Many refer to this kind of intelligence as “strong AI,” as opposed to “weak AI,” which has become a mundane topic in recent years.

As applied AI technology has developed over the last 60 years, we can see how many practical applications – knowledge bases, expert systems, image recognition systems, prediction systems, tracking and control systems for various technological processes – are no longer viewed as examples of AI and have become part of “ordinary technology.” The bar for what constitutes AI rises accordingly, and today it is the hypothetical “general intelligence,” human-level intelligence or “strong AI,” that is assumed to be the “real thing” in most discussions. Technologies that are already being used are broken down into knowledge engineering, data science or specific areas of “narrow AI” that combine elements of different AI approaches with specialized humanities or mathematical disciplines, such as stock market or weather forecasting, speech and text recognition and language processing.

Different schools of research, each working within their own paradigms, also have differing interpretations of the spheres of application, goals, definitions and prospects of AI, and are often dismissive of alternative approaches. However, there has been a kind of synergistic convergence of various approaches in recent years, and researchers and developers are increasingly turning to hybrid models and methodologies, coming up with different combinations.

Since the dawn of AI, two approaches to AI have been the most popular. The first, “symbolic” approach, assumes that the roots of AI lie in philosophy, logic and mathematics and operate according to logical rules, sign and symbolic systems, interpreted in terms of the conscious human cognitive process. The second approach (biological in nature), referred to as connectionist, neural-network, neuromorphic, associative or subsymbolic, is based on reproducing the physical structures and processes of the human brain identified through neurophysiological research. The two approaches have evolved over 60 years, steadily becoming closer to each other. For instance, logical inference systems based on Boolean algebra have transformed into fuzzy logic or probabilistic programming, reproducing network architectures akin to neural networks that evolved within the neuromorphic approach. On the other hand, methods based on “artificial neural networks” are very far from reproducing the functions of actual biological neural networks and rely more on mathematical methods from linear algebra and tensor calculus.

Are There “Holes” in Neural Networks?

In the last decade, it was the connectionist, or subsymbolic, approach that brought about explosive progress in applying machine learning methods to a wide range of tasks. Examples include both traditional statistical methodologies, like logistical regression, and more recent achievements in artificial neural network modelling, like deep learning and reinforcement learning. The most significant breakthrough of the last decade was brought about not so much by new ideas as by the accumulation of a critical mass of tagged datasets, the low cost of storing massive volumes of training samples and, most importantly, the sharp decline of computational costs, including the possibility of using specialized, relatively cheap hardware for neural network modelling. The breakthrough was brought about by a combination of these factors that made it possible to train and configure neural network algorithms to make a quantitative leap, as well as to provide a cost-effective solution to a broad range of applied problems relating to recognition, classification and prediction. The biggest successes here have been brought about by systems based on “deep learning” networks that build on the idea of the “perceptron” suggested 60 years ago by Frank Rosenblatt. However, achievements in the use of neural networks also uncovered a range of problems that cannot be solved using existing neural network methods.

First, any classic neural network model, whatever amount of data it is trained on and however precise it is in its predictions, is still a black box that does not provide any explanation of why a given decision was made, let alone disclose the structure and content of the knowledge it has acquired in the course of its training. This rules out the use of neural networks in contexts where explainability is required for legal or security reasons. For example, a decision to refuse a loan or to carry out a dangerous surgical procedure needs to be justified for legal purposes, and in the event that a neural network launches a missile at a civilian plane, the causes of this decision need to be identifiable if we want to correct it and prevent future occurrences.

Second, attempts to understand the nature of modern neural networks have demonstrated their weak ability to generalize. Neural networks remember isolated, often random, details of the samples they were exposed to during training and make decisions based on those details and not on a real general grasp of the object represented in the sample set. For instance, a neural network that was trained to recognize elephants and whales using sets of standard photos will see a stranded whale as an elephant and an elephant splashing around in the surf as a whale. Neural networks are good at remembering situations in similar contexts, but they lack the capacity to understand situations and cannot extrapolate the accumulated knowledge to situations in unusual settings.

Third, neural network models are random, fragmentary and opaque, which allows hackers to find ways of compromising applications based on these models by means of adversarial attacks. For example, a security system trained to identify people in a video stream can be confused when it sees a person in unusually colourful clothing. If this person is shoplifting, the system may not be able to distinguish them from shelves containing equally colourful items. While the brain structures underlying human vision are prone to so-called optical illusions, this problem acquires a more dramatic scale with modern neural networks: there are known cases where replacing an image with noise leads to the recognition of an object that is not there, or replacing one pixel in an image makes the network mistake the object for something else.

Fourth, the inadequacy of the information capacity and parameters of the neural network to the image of the world it is shown during training and operation can lead to the practical problem of catastrophic forgetting. This is seen when a system that had first been trained to identify situations in a set of contexts and then fine-tuned to recognize them in a new set of contexts may lose the ability to recognize them in the old set. For instance, a neural machine vision system initially trained to recognize pedestrians in an urban environment may be unable to identify dogs and cows in a rural setting, but additional training to recognize cows and dogs can make the model forget how to identify pedestrians, or start confusing them with small roadside trees.

Growth Potential?

The expert community sees a number of fundamental problems that need to be solved before a “general,” or “strong,” AI is possible. In particular, as demonstrated by the biggest annual AI conference held in Macao, “explainable AI” and “transfer learning” are simply necessary in some cases, such as defence, security, healthcare and finance. Many leading researchers also think that mastering these two areas will be the key to creating a “general,” or “strong,” AI.

Explainable AI allows for human beings (the user of the AI system) to understand the reasons why a system makes decisions and approve them if they are correct, or rework or fine-tune the system if they are not. This can be achieved by presenting data in an appropriate (explainable) manner or by using methods that allow this knowledge to be extracted with regard to specific precedents or the subject area as a whole. In a broader sense, explainable AI also refers to the capacity of a system to store, or at least present its knowledge in a human-understandable and human-verifiable form. The latter can be crucial when the cost of an error is too high for it only to be explainable post factum. And here we come to the possibility of extracting knowledge from the system, either to verify it or to feed it into another system.

Transfer learning is the possibility of transferring knowledge between different AI systems, as well as between man and machine so that the knowledge possessed by a human expert or accumulated by an individual system can be fed into a different system for use and fine-tuning. Theoretically speaking, this is necessary because the transfer of knowledge is only fundamentally possible when universal laws and rules can be abstracted from the system’s individual experience. Practically speaking, it is the prerequisite for making AI applications that will not learn by trial and error or through the use of a “training set,” but can be initialized with a base of expert-derived knowledge and rules – when the cost of an error is too high or when the training sample is too small.

How to Get the Best of Both Worlds?

There is currently no consensus on how to make an artificial general intelligence that is capable of solving the abovementioned problems or is based on technologies that could solve them.

One of the most promising approaches is probabilistic programming, which is a modern development of symbolic AI. In probabilistic programming, knowledge takes the form of algorithms and source, and target data is not represented by values of variables but by a probabilistic distribution of all possible values. Alexei Potapov, a leading Russian expert on artificial general intelligence, thinks that this area is now in a state that deep learning technology was in about ten years ago, so we can expect breakthroughs in the coming years.

Another promising “symbolic” area is Evgenii Vityaev’s semantic probabilistic modelling, which makes it possible to build explainable predictive models based on information represented as semantic networks with probabilistic inference based on Pyotr Anokhin’s theory of functional systems.

One of the most widely discussed ways to achieve this is through so-called neuro-symbolic integration – an attempt to get the best of both worlds by combining the learning capabilities of subsymbolic deep neural networks (which have already proven their worth) with the explainability of symbolic probabilistic modelling and programming (which hold significant promise). In addition to the technological considerations mentioned above, this area merits close attention from a cognitive psychology standpoint. As viewed by Daniel Kahneman, human thought can be construed as the interaction of two distinct but complementary systems: System 1 thinking is fast, unconscious, intuitive, unexplainable thinking, whereas System 2 thinking is slow, conscious, logical and explainable. System 1 provides for the effective performance of run-of-the-mill tasks and the recognition of familiar situations. In contrast, System 2 processes new information and makes sure we can adapt to new conditions by controlling and adapting the learning process of the first system. Systems of the first kind, as represented by neural networks, are already reaching Gartner’s so-called plateau of productivity in a variety of applications. But working applications based on systems of the second kind – not to mention hybrid neuro-symbolic systems which the most prominent industry players have only started to explore – have yet to be created.

This year, Russian researchers, entrepreneurs and government officials who are interested in developing artificial general intelligence have a unique opportunity to attend the first AGI-2020 international conference in St. Petersburg in late June 2020, where they can learn about all the latest developments in the field from the world’s leading experts.

From our partner RIAC

Continue Reading
Comments

Science & Technology

From rockets to spider silk, young scientists wow the jury – and each other!

Avatar photo

Published

on

The winners of the top prize at this year’s EU Contest for Young Scientists. © European Commission, 2023

The 34th annual edition of an EU contest for teenage researchers wrapped up this past week with participants from Canada, Denmark, Poland and Portugal claiming the top prize. 

By Sofía Manzanaro 

Inês Alves Cerqueira of Portugal just spent five days in Brussels and left with a top EU prize for young scientists. 

But ask 17-year-old Cerqueira what she remembers most about the event, which featured 136 contestants from three dozen countries in Europe and beyond, and the much-coveted award gets hardly any mention. 

No worries 

‘I loved listening to all the projects and having conversations about science without having to worry about people judging me or anything like that,’ she said as the 34th annual EU Contest for Young Scientists (EUCYS) drew to a close in the Belgian capital. 

Worries or not, Cerqueira and the other contestants aged 14 to 20 years were judged by a jury of 22 distinguished scientists and engineers from across Europe as part of the official competition. It featured 85 science projects in the running for first, second and third awards that shared a total of €62 000 in prize money. 

The rewards also include scholarships and visits to institutions such as the European Space Agency, nuclear-research organisation CERN and a forum that brings together eight of the largest research bodies in Europe. 

All the participants had already won first prizes in national science competitions. At EUCYS, four projects won the top prize and received €7 000 each. 

Cerqueira claimed hers with two teammates: Afonso Jorge Soares Nunes and Mário Covas Onofre. The three Portuguese, who come from the northern coastal city of Porto, are exploring the potential of spider silk to treat bone diseases including osteoporosis. 

The EUCYS projects, which ranged from rocket science and chronic-pain drugs to climate demographics and river pollution, were as varied as the backgrounds of the participants, who came from as far away as Canada and South Korea. 

Canadian Elizabeth Chen was another first-prize winner for a project on a cancer therapy. The two other top-award recipients were Maksymilian Gozdur of Poland for an entry on judicial institutions and Martin Stengaard Sørensen of Denmark for an initiative on rocket propulsion systems. 

Bright minds 

‘EUCYS is about rewarding the enthusiasm, passion and curiosity of Europe’s next generation of bright minds finding new solutions to our most pressing challenges,’ said Marc Lemaître, the European Commission’s director-general for research and innovation. 

Eagerness and spirit were on general display at the event. So was camaraderie. 

Noemi Marianna Pia, Pietro Ciceri and Davide Lolla, all 17 year olds from Italy, said they felt themselves winners by having earned spots at EUCYS for a project on sustainable food and described the event as a once-in-a-lifetime chance to mix with fellow young scientists from around the world. 

The three Italians want to develop plant-based alternatives to animal proteins. At their exhibition stand, they talked with contagious excitement about their research while holding dry chickpeas and soybeans. 

Lolla said that, while his pleasures include tucking into a juicy steak, he feels a pressing need to reduce meat consumption to combat climate change and preserve biodiversity. 

Sparkling ideas  

On the other side of the venue, 16-year-old Eleni Makri from Cyprus recalled how a classroom chat about summer plans sparked an idea to use seagrass on many of the island’s beaches to produce fertiliser. 

Her project partner, Themis Themistocleous, eagerly joined the conversation to explain how seagrass can recover phosphate from wastewater. The process involves thermal treatment of the seagrass. 

Themistocleous also expressed pride at having been chosen by Makri as her teammate for the competition. 

‘There were a thousand people, but she chose me!’ he said with a wide grin as Makri playfully shook her head in response. 

Science can also be the outcome of a partnership rather than its trigger. Metka Supej and Brina Poropat of Slovenia were brought together by sports, particularly rowing. 

After years of training on the same team, they decided to research the impact of energy drinks on heart-rate recovery. 

Multiple paths 

As they cheered for one another while preparing to say goodbye, the participants at EUCYS 2023 offered a glimpse of the combination of qualities – personal, intellectual, social and even professional – that turn young people into pioneering researchers. 

Gozdur, the Polish top-prize winner, discovered his passion for judicial matters while working at a law firm. Before that, he wanted to study medicine and even dabbled in the film industry. 

His EUCYS project drew on French and Polish criminal-procedure codes to examine the prospects for “restorative justice” – a central element of which is rehabilitation of the convict. The conclusion reached was that ‘penal populism is not beneficial to any party, especially to the victim’s,’ according to a description

Now 19 years old and a law student in Warsaw, Gozdur said he would like international institutions to take up his work so that it influences ‘real-life’ legal norms in the future. 

‘EUCYS showed me that my idea is actually relevant and that it may help societies,’ he said. ‘I would like to fight more for my project.’ 

For Sørensen, the Danish recipient of the top prize, venturing into rocket science as a teenager was no surprise. From the city of Odense, he began computer programming at the age of 10 and was inspired by his father – an electrical engineer – to look into engineering. 

Now 19 years old, Sørensen is striving in his research to create cheaper rocket engines. His project, entitled “Development of small regeneratively cooled rocket propulsion systems”, demonstrated how small rocket engines can be cooled by using a fuel that is a mixture of ethanol and nitrous oxide. 

Sørensen said he’s unsure what his future path will be while expressing interest in pursuing his rocket research.  

‘I would like to continue working on this project,’ he said. ‘And I would like to do something that matters in the world.’ 

Chen, the top-award winner from Canada, has long had a passion for cancer research. 

From childhood, she became involved in fundraisers for a Canadian cancer association and was puzzled about why significant donations had produced no cure. Now 17 years old and in high school, Chen is seeking a therapy that would avoid the often-considerable side effects of conventional treatments. 

Her project focuses on a novel form of immunotherapy based on “CAR-T cells”, which are genetically altered so they can fight cancer more effectively. 

‘I am really interested in going into university right away and then hopefully getting involved in some cancer research because that is just so interesting to me,’ said Chen, who comes from Edmonton. 

The three Portuguese winners – Cerqueira, Nunes and Onofre – said they have developed a partnership as strong as their spider silk and plan to pursue their research while at university with the hope – one day – of conducting clinical studies. 

Called “SPIDER-BACH2”, their project reflects an awareness that osteoporosis will become a growing health challenge worldwide as people live longer. It aims for in vitro production of bone-building cells known as osteoblasts. 

‘The future is bright for us,’ said Nunes. This article was originally published in Horizon, the EU Research and Innovation Magazine.

Continue Reading

Science & Technology

Space Exploration: The Unification of Past, Present and Future

Avatar photo

Published

on

An accreting SMBH in a fairly local galaxy with very large and extended radio jets. © R. Timmerman; LOFAR & Hubble Space Telescope

The enchanting realm of space exploration continues to unfold new wonders with every passing day, sparking a growing interest among individuals to embark on their own cosmic journeys. While exploring space with the aid of private companies that charge fortunes is a privilege usually reserved for billionaire adventurers, there are occasional exceptions that captivate our attention.

Just a few days ago on 8th September, Virgin Galactic’s third spaceflight set out on a brief mission that seized the spotlight due to some interesting details. Three private explorers, Ken Baxter, Timothy Nash, and Adrian Reynard, two pilots and one instructor, were onboard ‘VSS Unity’. However, the presence of two different and unique passengers added a twist to the journey: fossils of our ancient human ancestors. The fossil remains of two ancient species, two-million-years-old Australopithecus sediba and 250,000 years old Homo naledi,  held in carbon fiber, emblazoned with the South African flag,  were part of the Virgin Galactic’s spacecraft ‘crew’ for a one-hour ride, making them the oldest human species to visit space. Australopithecus sediba’s clavicle (collarbone) and Homo naledi’s thumb bone were chosen for the voyage. Both fossil remains were discovered in the Cradle of Humankind – home to human ancestral remains in South Africa.

The episode undoubtedly prompts questions regarding the underlying reason behind sending these fossil remains into the vast expanse of space in the first place. It profoundly underscores the immense power of symbols, speaking to us in ways words cannot. This voyage was not just a journey through space, but a soulful homage to our ancestors. Their invaluable contributions have sown the seeds of innovation and growth, propelling us to unimaginable heights. Now, as we stretch our hands towards the heavens, we remember them – and in this gesture, we symbolise our eternal gratitude and awe for the path they paved, allowing humanity to quite literally aim for the skies. As Timothy Nash said, ‘It was a moment to contemplate the enterprising spirit of our earliest ancestors, who had embarked on a journey toward exploration and innovation years ago.’

Moreover, the clavicle of the Australopithecus sediba was deliberately chosen given that it was discovered by nine-year-old Mathew Berger, son of Lee Berger, a National Geographic Society explorer, who played a major role in discovering both species and handed over the remains to Timothy Nash for the journey. This story serves as a touching testament to the boundless potential of youth, showing us that even the young can be torchbearers in the realm of science, lighting the path of discovery with their boundless curiosity. The unearthing of Homo naledi in 2013 wasn’t just about finding bones; it was a window into our past. This ancient ancestor, with its apelike shoulders and human-like feet, hands, and brain, wasn’t just a distant relative. They were artists and inventors, leaving behind symbols and tools in their cave homes as a silent testament to their legacy. This led to the discovery of more than 1,500 specimens from one of the biggest excavations in Africa’s history. It wasn’t just about digging up the past; it was about piecing together the jigsaw of our very essence, deepening our understanding of the roots and journey of our kind, especially in the heartland of South Africa. Each discovery, each bone, whispered tales of our shared journey, of beginnings, growth, and the undying spirit of exploration.

For those involved in the venture, the occasion was awe-inspiring as it connected our ancient roots to space exploration. However, not everyone is pleased. The event has sparked criticism from  archaeologists and palaeoanthropologists, many of whom have called it a mere publicity stunt and raised serious concerns over such an act given that it poses risks to the care of the precious fossils. It was further argued that the act was ethically wrong, and lacked  any concrete scientific justifications.

Setting aside this debate, the episode connects chronicles of our past with the boundless potential of humankind’s future. It celebrates the age-old quest for exploration shared across millennia. This journey, captivating in its essence, elevates space exploration to a sacred place where fossils, once cradled by the Earth’s soil, now dance among the stars. Just as with pivotal moments in space history, it is also a compelling cue to states that are currently lagging in this race to timely embrace the possibilities of this frontier. Countries, like Pakistan, should draw inspiration from such milestones to fervently chart their own celestial courses.

Upon their return to South Africa, the relics would be displayed in museums and other institutions, offering a chance to the public to view them and draw inspiration. As we witness the rise of commercial space travel, this unique journey provides glimpses of the multifaceted nature of space exploration – one that prompts us to reflect on our past, engage actively with the present and anticipate the future that awaits us. Something Pakistan’s national poet Allama Iqbal eloquently captured in one his verses, translated as: I see my tomorrow (future) in the mirror of my yesterday (past).

Continue Reading

Science & Technology

Artificial Intelligence and Advances in Chemistry (I)

Avatar photo

Published

on

With the advent of Artificial Intelligence technology in the field of chemistry, traditional methods based on experiments and physical models are gradually being supplemented with data-driven machine learning paradigms. Ever more data representations are developed for computer processing, which are constantly being adapted to statistical models that are primarily generative.

Although engineering, finance and business will greatly benefit from the new algorithms, the advantages do not stem only from algorithms. Large-scale computing has been an integral part of physical science tools for decades, and some recent advances in Artificial Intelligence have begun to change the way scientific discoveries are made.

There is great enthusiasm for the outstanding achievements in physical sciences, such as the use of machine learning to reproduce images of black holes or the contribution of AlphaFold, an AI programme developed by DeepMind (Alphabet/Google) to predict the 3D structure of proteins.

One of the main goals of chemistry is to understand matter, its properties and the changes it can undergo. For example, when looking for new superconductors, vaccines or any other material with the properties we desire, we turn to chemistry.

We traditionally think chemistry as being practised in laboratories with test tubes, Erlenmeyer flasks (generally graduated containers with a flat bottom, a conical body and a cylindrical neck) and gas burners. In recent years, however, it has also benefited from developments in the fields of computer science and quantum mechanics, both of which became important in the mid-20th century. Early applications included the use of computers to solve calculations of formulas based on physics, or simulations of chemical systems (albeit far from perfect) by combining theoretical chemistry with computer programming. That work eventually developed into the subgroup now known as computational chemistry. This field began to develop in the 1970s, and Nobel Prizes in chemistry were awarded in 1998 to Britain’s John A. Pople (for his development of computational methods in quantum chemistry: the Pariser-Parr-Pople method), and in 2013 to Austria’s Martin Karplus, South Africa’s Michael Levitt, and Israel’s Arieh Warshel for the development of multiscale models for complex chemical systems.

Indeed, although computational chemistry has gained increasing recognition in recent decades, it is far less important than laboratory experiments, which are the cornerstone of discovery.

Nevertheless, considering the current advances in Artificial Intelligence, data-centred technologies and ever-increasing amounts of data, we may be witnessing a shift whereby computational methods are used not only to assist laboratory experiments, but also to guide and orient them.

Hence how does Artificial Intelligence achieve this transformation? A particular development is the application of machine learning to materials discovery and molecular design, which are two fundamental problems in chemistry.

In traditional methods the design of molecules is roughly divided into several stages. It is important to note that each stage can take several years and many resources, and success is by no means guaranteed. The phases of chemical discovery are the following: synthesis, isolation and testing, validation, approval, commercialisation and marketing.

The discovery phase is based on theoretical frameworks developed over centuries to guide and orient molecular design. However, when looking for “useful” materials (e.g. petroleum gel [Vaseline], polytetrafluoroethylene [Teflon], penicillin, etc.), we must remember that many of them come from compounds commonly found in nature. Moreover, the usefulness of these compounds is often discovered only at a later stage. In contrast, targeted research is a more time-consuming and resource-intensive undertaking (and even in this case it may be necessary to use known “useful” compounds as a starting point). Just to give you an idea, the pharmacologically active chemical space (i.e. the number of molecules) has been estimated at 1060! Even before the testing and sizing phases, manual research in such a space can be time-consuming and resource-intensive. Hence how can Artificial Intelligence get into this and speed up the discovery of the chemical substance?

First of all, machine learning improves the existing methods of simulating chemical environments. We have already mentioned that computational chemistry enables to partially avoid laboratory experiments. Nevertheless, computational chemistry calculations simulating quantum-mechanical processes are poor in terms of both computational cost and accuracy of chemical simulations.

A central problem in computational chemistry is solving the 1926 equation of physicist Erwin Schrödinger’s (1887-1961). The scientist described the behaviour of an electron orbiting the nucleus as that of a standing wave. He therefore proposed an equation, called the wave equation, with which to represent the wave associated with the electron. In this respect, the equation is for complex molecules, i.e. given the positions of a set of nuclei and the total number of electrons, the properties of interest must be calculated. Exact solutions are only possible for single-electron systems, while for other systems we must rely on “good enough” approximations. Furthermore, many common methods for approximating the Schrödinger equation scale exponentially, thus making forced solutions difficult to solve. Over time, many methods have been developed to speed up calculations without sacrificing precision too much. However, even some “cheaper” methods can cause computational bottlenecks.

A way in which Artificial Intelligence can accelerate these calculations is by combining them with machine learning. Another approach fully ignores the modelling of physical processes by directly mapping molecular representations onto desired properties. Both methods enable chemists to more efficiently examine databases for various properties, such as nuclear charge, ionisation energy, etc.

While faster calculations are an improvement, they do not solve the issue that we are still confined to known compounds, which account for only a small part of the active chemical space. We still have to manually specify the molecules we want to analyse. How can we reverse this paradigm and design an algorithm to search the chemical space and find suitable candidate substances? The answer may lie in applying generative models to molecular discovery problems.

But before addressing this topic, it is worth talking about how to represent chemical structures numerically (and what can be used for generative modelling). Many representations have been developed in recent decades, most of which fall into one of the four following categories: strings, text files, matrices and graphs.

Chemical structures can obviously be represented as matrices. Matrix representations of molecules were initially used to facilitate searches in chemical databases. In the early 2000s, however, a new matrix representation called Extended Connectivity Fingerprint (ECFP) was introduced. In computer science, the fingerprint or fingerprint of a file is an alphanumeric sequence or string of bits of a fixed length that identifies that file with the intrinsic characteristics of the file itself. The ECFP was specifically designed to capture features related to molecular activity and is often considered one of the first characterisations in the attempts to predict molecular properties.

Chemical structure information can also be transferred into a text file, a common output of quantum chemistry calculations. These text files can contain very rich information, but are generally not very useful as input for machine learning models. On the other hand, the string representation encodes a lot of information in its syntax. This makes them particularly suitable for generative modelling, just like text generation. Finally, the graph-based representation is more natural. It not only enables us to encode specific properties of the atom in the node embeddings, but also captures chemical bonds in the edge embeddings. Furthermore, when combined with message exchange, graph-based representation enables us to interpret (and configure) the influence of one node on another node by its neighbours, which reflects the way atoms in a chemical structure interact with each other. These properties make graph-based representations the preferred type of input representation for deep learning models. (1. continued)

Continue Reading

Trending