I understand meta-learning in a text

Meta-learning is a relatively new direction in the field of artificial intelligence and is considered to be the key to realizing general artificial intelligence.

Why is he so important? How to quickly and easily understand the essence of meta-learning? This article will introduce you to the meta-learning in detail.


Why is meta-learning important?

The core of meta-learning isself-study ability. Then why do you need self-learning ability?

Machine learning: Simple machine learning is difficult to deal with complex problems;

Deep learning: Deep learning is better than machine learning in terms of complex problems, but it is impossible to deal with interactive problems or continuous problems, intelligently handling one-to-one mapping problems;

Reinforcement: Rely on massive training and require precise rewards. The cost is higher and more complicated.

Meta-learning: Self-learning ability to make full use of past experience to guide future tasks. Is considered to be realizedUniversal artificial intelligencekey.


What is meta-learning?

The idea of ​​meta-learning is to learn the process of learning (training).

Meta-learning has several implementation methods, but the two methods of "learning the learning process" mentioned in this article are similar to the methods described above. In our training process, specifically, you can learn two things:

Meta-learning mainly learns 2 points: neural network initial parameters, optimizer parameters

  1. The initial parameters of the neural network (blue in the figure);
  2. The parameters of the optimizer (pink ★).

I will introduce the combination of these two points, but each of the points here is also very interesting, and can be simplified, accelerated and some good theoretical results. Now, we have two parts to train:

  1. Using the term "model (M)" to refer to our previous neural network, we can now also understand it as a low-level network. Sometimes people also use "optimizee" or "learner" (learNer)" to call it. The weight of the model is indicated by ■ in the figure.
  2. Use "optimizer (O)" or "meta learner" to refer to the advanced model used to update the weight of the low-level network (that is, the above model). The weight of the optimizer is indicated by ★ in the figure.

Read More



Wikipedia version

Meta-learning was originally described by Donald B. Maudsley (1979) as "the process by which learners realize and increasingly control the perception, inquiry, learning, and growth habits they have internalized." Maudsely sets the conceptual basis of his theory to be synthesized under the headings of Hypothesis, Structure, Transformation Process and Promotion. Five principles are elaborated to promote meta-learning. The learner must:

  1. There is a theory, no matter how primitive;
  2. Working in a safe supportive social and physical environment;
  3. Discover its rules and assumptions;
  4. Reconnect with the real information in the environment;
  5. Reorganize yourself by changing its rules/hypotheses.

John Biggs (1985) later used the concept of meta-learning to describe the state of "knowing and controlling your own learning." You can define meta-learning as the understanding and understanding of the phenomenon of learning itself, not the subject knowledge. This definition implies the learner's perception of the learning environment, including what the expectations of the discipline are, and, more simply, the requirements for a particular learning task.

In this context, meta-learning depends on the learner's learning concept, epistemological beliefs, learning process and academic skills, and is summarized here as a learning method. A student with a high level of meta-learning can assess the effectiveness of her/his learning methods and manage them according to the requirements of the learning task. Conversely, students with low meta-learning will not be able to reflect on the nature of her/his learning methods or learning task sets. Therefore, when learning becomes more difficult and demanding, he/she will not be able to adapt successfully.

Read More