This article is a reprinted article, the original text: "Here's where AI will advance in 2021"

The content is the result of Google translation. Although there are some problems, it does not affect the overall understanding.

Artificial intelligence continues to develop rapidly.Even in 2020, when there is no lack of eye-catching news, the advancement of artificial intelligence has repeatedly attracted mainstream attention.Especially OpenAIGPT-3, Demonstrating the novel and surprising ways in which we may soon see AI permeate daily life.Such rapid progress has made it difficult to predict the future of AI, but breakthroughs seem to have been made in some areas.The following are some of the areas of AI that we are particularly optimistic about in 2021.


Two of the biggest AI achievements of 2020 have quietly shared the same underlying AI structure. Both OpenAI's GPT-3 and DeepMind's AlphaFold are based onTransformThe sequence processing model.althoughTransformThe structure of has existed since 2017, but GPT-3 and Alphafold proved that Transformer's extraordinary ability is deeper than the previous generation sequence model, learns faster, and can handle problems outside of natural language processing well.

Compared with previous sequence modeling structures (such as recurrent neural networks andLSTM) Different, Transformers breaks away from the paradigm of processing data sequentially.They use calledattentionThe mechanism of processing the entire input sequence at once to understand which parts of the input are related to other parts.This allows Transformers to easily correlate distant parts of the input sequence, which is a recursive modelHard to overcomeTask.It also allows important parts of training to be carried out in parallel, thereby making better use of the massively parallel hardware that has been available in recent years and greatly reducing training time.There is no doubt that researchers will look for new places to apply this promising structure in 2021, and there are good reasons to expect positive results.In fact, OpenAI has modified GPT-2021 in 3 toAccording to text descriptiongenerateimage.The transformer looks ready to rule 2021.

Graph neural network

Data in many fields naturally has a chart structure: computer networks, social networks, molecules/proteins and transportation routes are just a few examples.Graph neural network(GNN) allows deep learning to be applied to graph structured data. We hope that GNN will become an increasingly important AI method in the future.More specifically, we expect that by 2021, methodological advances in some key areas will promote the widespread adoption of GNN.

Dynamic graphs are the first important area.So far, most GNN studies have assumed a static, unchanging graph, but the above situation will inevitably change over time: for example, in a social network, members join (new nodes) and friendships change (different edges) .In 2020, we have seen some efforts to model the time evolution diagram as a series of snapshots, but in 2021, this new research direction will be expanded, focusing onMethod of modeling dynamic graphs as continuous time series.In addition to the usual topological structure, this continuous modeling should also enable GNN to discover and learn from the temporal structure in the graph.

The improvement of the messaging paradigm will be another step forward.Message passing is a common method for implementing graph neural networks. It is a method of gathering information about nodes by "passing" information along the edges connecting neighbors.Although it is very intuitive, it is difficult to capture the effect of the need for information to spread over long distances on the graph.Next year, we hope to make a breakthrough in this paradigm, such as learning which information dissemination paths are the most relevant through iterative learning, or even learning a new cause and effect diagram on relational data sets.

Application Field

last yearMany headlinesNews CapitalEmphasizes the emerging advances in AI in practical applications, and is expected to take advantage of these advances in 2021.In particular, applications that rely on natural language understanding may make progress as access to the GPT-3 API becomes more available.This API allows users to access the functions of GPT-3 without them having to train their own AI, which is inherently expensive.After Microsoft purchases the GPT-3 license, we may also see the technology appearing in Microsoft products.

In 2021, other application fields may also benefit greatly from AI technology. Artificial intelligence and machine learning (ML) have gradually entered the field of network security, but in 2021 it has shown the potential to push the trajectory steeper.Just likeSolarWinds vulnerabilityAs highlighted, the company is facing imminent threats from cybercriminals and state actors, as well as the threat of evolving malware and ransomware configurations.In 2021, we expect to actively promote advanced behavior analysis AI to enhance cyber defense systems.Artificial intelligence and behavioral analysis are essential to help identify new threats, including variants of earlier threats.

We also expect that by 2021, the number of applications that will run machine learning models on edge devices by default will increase.With the advancement of processing power and quantitative technology, such asGoogle's CoralEtc. has an onboard tensor processing unit (TPU) The equipment is bound to become more and more popular. Edge AI eliminates the need to send data to the cloud for inference, saves bandwidth and reduces execution time, both of which are critical in areas such as healthcare.Edge computing can also open new applications in other areas that require privacy, security, and low latency, as well as areas in the world where high-speed Internet cannot be accessed.

Final Thoughts

Artificial intelligence technology continues to proliferate in the actual field, and advances in Transformer structure and GNN are likely to stimulate advances in fields that have not easily adapted to existing AI technologies and algorithms.We have focused on several areas that seem to be improved this year, but as this year approaches, there will undoubtedly be surprises.As the saying goes, it is difficult to predict the future, especially the future, but right or wrong, 2021 seems to be an exciting year for the AI ​​field.