This article is reproduced from the front line of the public number AI.Original address

What kind of deep learning framework is suitable for developers of different learning stages and different positions? To this end, the AI ​​frontline invited a deep learning framework expert from BAT to answer this question.

Here are a few common questions when choosing a deep learning framework:

At present, what are the mainstream deep learning frameworks at home and abroad? What are the advantages and disadvantages of each?

TensorFlow:Produced by Google Dachang, the community is rich in resources and the user community is extensive. If your business is to achieve a more mature application scenario, choose TensorFlow is a good choice, after all, most people have stepped on it. If you are still a beginner and want to implement a completely new algorithm, you should choose TensorFlow carefully. Because of its long development time and huge code base, it is a big challenge to really understand the code.

PaddlePaddle:Produced by Baidu, similar to TF, PaddlePaddle is also an industrial framework for practical application within Baidu. It is worth mentioning that PaddlePaddle provides official support for a large number of industrial-level models; large-scale distributed training technology has strong capabilities, supports parallel training of large-scale dense parameters and sparse parameter scenarios; based on its Chinese document friendly, simple and easy to use features, Community activity continues to increase. Local developers have very convenient Chinese communication.

PaddlePaddle has been upgraded to a comprehensive deep learning development kit. In addition to the core framework, it also opens a complete set of deep learning tool components and service platforms such as VisualDL, PARL, AutoDL, EasyDL, and AI Studio to better meet different levels of deep learning development. Development needs. In 18, Baidu has increased its ecological investment. There are many cases in industrial manufacturing and agriculture and forestry. PaddlePaddle may be a good choice for scenarios where you want to use it, especially for Chinese language processing and large-scale recommendation.

PyTorch+Caffe2:Facebook is produced. PyTorch was formerly known as Torch (a deep learning library based on the Lua language). PyTorch flexibility is greatly enhanced compared to TensorFlow, which supports dynamic composition, and developers can change the structure of the neural network at any time. PyTorch has become more and more popular in the academic world this year because of its outstanding ease of use.

If you are a beginner and want to reproduce the leading edge algorithms, PyTorch is recommended. If you are an industrial scene developer, it is best to search for a successful case of using PyTorch. Because of the computational efficiency and industrial deployment, PyTorch is still not mature. Designed for production environments, Caffe2 provides runtime on a variety of platforms, including mobile devices. Caffe2 is intended to make up for Pytorch's lack of production environment, and has not seen too much developer feedback on Caffe2's industrial performance, which requires more time to verify.

MxNet:It has been selected by Amazon as a deep learning framework on AWS that supports dynamic graph calculations. The advantage is that it is efficient and compact, and is more suitable for companies and researchers with high customization requirements. If your work requires modifications to the framework, it is much more efficient to change MxNet than to change TF. MXNet users are not too many compared to other frameworks, with a small number of supporters. MXNet is often criticized for its ease of document quality, and novices need to work more together.

Keras:It is a high-level neural network library for quickly building deep learning prototypes. Keras is written in pure Python and is based on Tensorflow, Theano, and CNTK backends. Keras hides most of the internal operations and is very friendly to novices. If you want to get started quickly, it is recommended to choose Keras. But if you really want to be a deep learning master, you still have to manually write a neural network code from scratch. Otherwise, it is easy to rely on the ease of use of Keras and ignore the underlying principle.

XDL:Ali produced. X-Deep Learning (XDL) is a solution to deep learning problems for specific industrial-grade scenarios, such as advertising, rather than the underlying framework alongside other frameworks. In fact, XDL supports the use of TensorFlow and MXNet as a single-node compute backend in a bridged manner. For developers who already use TensorFlow and MXNet, if you need to choose XDL to try in large-scale sparse scenarios, XDL relies on Ali to provide a specific deployment environment, which is not very user-friendly for non-Ali users.

What deep learning framework should be chosen for developers in different learning stages and different positions in response to different business scenarios? What are the criteria for selection and measurement?

There are some differences in the criteria for selection and measurement for different types of users. Selection frameworks typically consider issues such as ease of use, performance, community, platform support, and more. Beginners should consider a framework that is easy to use. Research-oriented developers generally choose a framework that is easy to use and has a leading model baseline. Developers of partial industrial applications may consider using a framework with high stability and good performance. For higher-level users, it is often necessary to master multiple frameworks at the same time.

The latest open source algorithms may be based on a framework, and multiple frameworks can reproduce cutting-edge outcomes faster. At the same time, different frameworks have performance differences on different models. Mastering multiple frameworks can also choose the most suitable framework in a certain scenario.

What experiences can be shared when choosing a deep learning framework? What factors need to be considered?

Developers often use deep learning frameworks to build networks that address specific issues in real-world enterprise-level business, such as lexical analysis and machine translation. When choosing a framework, you usually need to pay attention to the following factors:

  1. Whether the framework has officially supported the current best algorithms, such as Chinese lexical analysis tasks.
  2. Whether the framework is efficient. The same algorithm is implemented in different frameworks, training faster, means less offline resources, faster iteration speed; faster prediction, which means that the actual deployment can save the enterprise a lot of machine resources, but also have more High response speed.
  3. Whether the interface of the framework is easy to use.

Mastering a deep learning framework, what preparations do developers need to make?

Under the premise that developers have already learned the basics of Python program development and deep learning, mastering a deep learning framework, developers need to understand the basic principles of the framework, learn and master the basic concepts and usage of the framework.

  1. You can usually start with the official documentation and have a general understanding of the basic concepts and usage of the framework by looking at the official documentation.
  2. In addition, you can install and practice using it on your own machine in conjunction with the official quick start example.
  3. Then you can select some algorithms from the official model library for learning reference.
  4. Combine your own specific tasks to build a network, view API interface documentation, and solve practical tasks.

Can a deep learning framework “cure all diseases”? Is it suitable for all application scenarios?

What needs to be reminded is that the deep learning framework can't be covered with all kinds of diseases. The deep learning framework can solve the application scenarios suitable for using deep learning technology.

Deep Learning (DL) is a branch of Machine Learning (ML) and the most popular machine learning method in the world. Although in recent years, in the application direction of image, voice, natural language processing, deep learning technology has made breakthrough progress. However, we still should not study myths deeply and think that deep learning is omnipotent.

Tasks suitable for deep learning should have such characteristics:

  1. A large amount of sample data. If the sample data is difficult to obtain or the quantity is too small, we think it is not suitable for deep learning technology.
  2. The sample data is sufficiently complete for coverage of the scene. The effect of the deep learning model is completely dependent on the performance of the sample data. If there is a situation other than the sample data, the generalization of the model will be worse.
  3. The results are not very demanding on interpretability. If the application scenario not only requires the machine to perform a certain task, but also has a clear interpretability to the completion process, such a scenario is less suitable for deep learning.

What are the characteristics and innovations of China's deep learning framework that are different from the mainstream deep learning and development frameworks in Europe and the United States?

From the characteristics of the deep learning framework, the distinction between China and Europe and the United States is not necessarily a suitable classification. Many of China's deep learning frameworks are developed by Chinese engineers and also grown up in Chinese companies. Naturally, they are more accurate in grasping the actual application characteristics in China, and they are unique in Chinese documents, Chinese tutorials and Chinese model matching. .

In the future, what trends will occur in the development of the framework?

First, it is still in the midst of a period of intense learning and application, and the demand for deep learning frameworks and platforms in the research and industry continues to grow. The market share of deep learning frameworks and platforms in the industry has not yet entered a stable period. Of course, in the future, there should not be a monopoly of monopoly.

On the one hand, the overall market demand is very large and diverse. On the other hand, different frameworks naturally have some different adaptations in terms of characteristics and power points. With the extensive application of deep learning in all walks of life, the framework of service capabilities and closer to the needs of users will have greater opportunities for improvement.

In the past, the flexibility and ease of use of dynamic graphs have shown great appeal to researchers. More frameworks have begun to support dynamic graphs, but further optimization of performance optimization problems under dynamic graphs is needed. Because in industry, training efficiency and scale are always the most important. Therefore, the convergence of each frame is obvious, and the front-end ease of use, computational efficiency, and frame stability become the direction of continuous optimization. Of course, the characteristics of the innate genes of different frameworks are still very obvious.

In addition, it should be noted that the AI ​​proprietary chip is also developing rapidly, and the integrated research and development of the hard and soft combination of the framework and the underlying chip is also attracting more and more attention. The AI ​​application deployment scenario is rapidly expanding, and the framework becomes very important for rapid deployment and efficient reasoning of various hardware platforms. Overall, there is a need to support efficient computing with multiple hardware. This also makes the internal architecture of the framework tend to be better abstraction layering, maintaining the integrity of the overall logic and the efficiency of heterogeneous hardware execution.

On the other hand, the connotation of the deep learning framework is also expanding. The emergence of the deep learning framework itself is for the convenience of deep learning research, experimentation and application, saving developers a lot of programming costs. But for deep learning technology, network design is also a very important part. The basic deep learning framework cannot solve this problem. Therefore, the automatic design of the current neural network structure AutoDL technology has become a new hot spot, and is also more closely integrated with the deep learning framework.

From the perspective of facilitating deep learning technology applications, it is not enough to rely solely on the open source framework. It requires a combination of scenario components, modules, platforms and products. The platform and supporting components of the framework form a deep learning full-process solution to the actual Application is very critical. If the deep learning framework is to play a greater role in practical applications, it needs a more comprehensive comprehensive service supporting construction.

How do you view the next framework battle?

With the popularity of deep learning technology, the maturity of various frameworks, the threshold of deep learning technology will be lower and lower. In the research scenario, easy to use, flexible and easy to expand is the most important consideration. From the current trend, PyTorch has certain advantages in ease of use. PaddlePaddle In the actual production scenario, the ease of use, stability, and richness of ecological resources will replace the leading edge of technology as a more important consideration for developers.

In the future, a framework with complete ecological resources and support for practical application scenarios will compete for more and more enterprise developer users. In foreign countries, relying on Google Eco-TensorFlow, MXNet, which relies on the AWS cloud ecosystem, has certain opportunities in the future.

What are the necessary conditions to create your own product ecology?

To form your own product ecology, you need to have four conditions:

  1. Master the core technology that delivers great benefits to end users. Key technologies such as heterogeneous computing, parallel training, multi-channel communication, multi-end deployment, and core algorithms involved in the deep learning domain framework. Vendors need to have deep learning core technology and form their own characteristics in order to have an ecological appeal.
  2. Core products or services can be applied and implemented on a large scale. In the process of ecological expansion, vendors have the ability to precipitate enough common solutions through platforms and resources.
  3. Provide a comprehensive experience for end users. The application threshold of deep learning framework is relatively high, and it is really widely used in enterprises. There is still a lot of work to be done, such as deep customization optimization combined with enterprise requirements, deep learning supporting tools construction, meticulous enterprise services, etc., which requires manufacturers to effectively integrate. Each partner's capabilities in the ecosystem provide end users with a more comprehensive product experience.
  4. Have the ability to continue to innovate. Vendors should have sufficient funds and incentives to continue to innovate core technologies and products, enabling ecosystem users to believe that core business can always benefit them.