With the widespread landing of AI, deep learning technology has entered every corner of our side. From photo recognition to beauty cameras, from speech translation to text prediction, a wide range of applications go from computers to mobile phones, from the cloud to mobile devices. How toMobile endBetter deployment and use of deep learning models, making full use of limited computing power to achieve fast and accurate prediction is a key issue that mobile terminals need to solve.

In recent years, various manufacturers at home and abroad have launched a series of deep learning architectures, in addition to optimization of conventional operations such as network compression, pruning, and quantization, and also for memory,GPUThe /CPU model has been specifically upgraded, and even in-depth optimization at the assembly level of the instruction set architecture has made the mobile framework increasingly sophisticated. Let's take a look at the widely used ones below.10 deep learning mobile framework(Recommended for collection and reading) to better perform in future study workDevelopment and deployment of mobile AI projects.

1. TensorFlow Lite

As one of the largest frameworks in the field of deep learning,TensorFlowTargetedly in 2017Introduced the lite framework system for the mobile sideTo provide users with a seamless migration of training models to the mobile side.


When using TensorFlow Lite, mainly throughPackaging, transformation, deployment, and optimizationFour steps are implemented. The Lite converter converts the trained model into a compressed format, and the optimization part converts the floating point number of the 32 bit into an 8 bit integer, or deploys it to the mobile GPU to perform calculations to increase the calculation speed.

Frame address:https://www.tensorflow.org/lite

2. Core ML

appleAs one of the largest mobile phone manufacturers, the machine learning on the mobile side has been laid out for a long time. Core ML is a set ofIntegrate the machine learning model into the appToolchain system.


When using, first convert the trained model to the mlmodel format supported by Apple. Then use Core ML to load into the app to run. Convenient converters are available for a variety of frameworks.


The entire framework is built on top of Apple's underlying acceleration components, including basic neural network modules and graphics modules and accelerators optimized for GPUs. On this basis, mainstream visual, natural language processing and various machine learning tasks can be supported. Because Apple's own hardware design, there is more room for optimization of software, achieving a lot of equipment operation and memory optimization.

Frame address: https://developer.apple.com/documentation/coreml

3. Caffe2

这 是FacebookThe lightweight, modular, and scalable architecture can be applied to multi-platform deep learning applications, adapting to large-scale cloud computing and mobile devices at the same time, and providing a series of pre-training models for developers to use or Further training. Currently Caffe2 and PyTorch have been merged to get through more effectivelyComplete process from training to deployment.

馃敆嗘嗘灦鍦板潃锛 https://caffe2.ai/

4. Bender

Bender isXmartlabsAn open source set of machine learning frameworks based onMetalConstruct.

Bender can easily define and run a neural network in an iOS app. It is more convenient and easy to use than Core ML. It can support more layers of models and completely release GPU performance through the MPS API, and can easily increase preprocessing and The post-processing layer handles the model input and output.

Frame address:https://xmartlabs.github.io/Bender/

5. MXNet

AmazonThe officially recommended deep learning framework is a flexible and efficient deep learning architecture with multiple language interfaces. For mobile, MXNet provides a lightweight interface and a single file dependency (MXNet Amalgamation) to minimize developer configuration and deployment. All interfaces are fused together into a single .cc file and rely only on the BLAS library. Even there is no dependence on BLASMini versionGreatly facilitates the development of mobile terminals.

Frame address:https://mxnet.incubator.apache.org/versions/master/faq/smart_device.html#deployment-environments

Domestic Internet giants and mobile phone manufacturers have also launched a series of self-developed open source frameworks in the development of AI on the mobile side, includingBaidu's MDL, Ali's MNN, Tencent's NCNNAnd Xiaomi's MACEI have accepted the test of the market on my own app.

6. NCNN

TencentThe mobile phone forward forward reasoning framework is optimized for mobile phone CPUs, does not depend on any third-party libraries, and does not depend on the basic computing framework package. NCNN supports a variety of complex network structures and operations, which can be extended, support for quantification and semi-precision, and facilitate the migration of many common models.


In addition, Tencent has also opened up the FeatherCNN reasoning library and the PockerFlow model compression framework for mobile AI.

Frame address:
https://github.com/Tencent/PocketFlow
https://github.com/Tencent/FeatherCN

7. paddle-mobile (MDL)

belongBaiduThe mobile terminal project under the open source framework is optimized for a variety of GPU hardware, and is optimized for embedded devices in addition to the mobile terminal.FPGARaspberry Pi can be run. Currently, it supports the flying model of the flying paddle, but the converter can convert the model of other formats to the Fluid model.

Paddle-mobile mainly includes load, program, execution, operation, core and variable tensor management modules. The detailed design architecture diagram is included in the documentation.

Frame address:https://github.com/PaddlePaddle/paddle-mobile/blob/develop/doc/design_doc.md

8. MNN

AlibabaOpen source lightweight mobile inference engine, mainly responsible for running on the mobileModel prediction. It optimizes multiple operations for device depth, independent of other packages and libraries, and static and dynamic libraries are on the order of a few megabytes to hundreds of kilobytes. Model format that supports the mainstream deep learning framework.


The framework mainly contains the converter part and the interpreter part. The front end and graph optimization process are included in the converter, and the interpreter contains the load model, the engine for calculating the graph schedule, and the backend component responsible for memory allocation and Op implementation.

Frame address:https://github.com/alibaba/MNN/blob/master/README_CN.md

9. MACE

MilletThe open source deep learning framework optimizes neural network computing for heterogeneous platforms, optimizes the instruction set for convolution operations, and supports multiple high- and low-end CPUs, GPU chips, and multiple model formats such as TensorFlow and Caffe.

Frame address: https://github.com/XiaoMi/mace

10. MindSpore

HuaweiThe newly released mobile phone is supported by a series of AIs, especially computer technology, which has a mobile framework called MindSpore. The frame design is friendly, the training execution is efficient, and the adaptability of different devices in different scenarios is achieved.

In addition, each company has a series of mobile end frameworks that do not have open source due to their own products, includingShangtang's PPL (Parrots), Ali's xNN, Qualcomm's SNPEand so on.

It is believed that with the accelerated landing of AI and the demand for AI for more mobile and edge computing, more models with better performance, more accuracy and faster will appear in the future, which will facilitate our lives in all aspects.

Reference materials:
https://zhuanlan.zhihu.com/p/60896625

https://blog.csdn.net/zeusee/article/details/89601634
https://blog.csdn.net/u010333076/article/details/87896734
https://blog.csdn.net/zchang81/article/details/74280019
https://www.zhihu.com/people/nihui-2/activities
https://www.zhihu.com/question/62871439
MindSporehttps://www.huawei.com/en/about-huawei/publications/communicate/86/driving-ai-to-new-horizons
Coral and Nano: https://www.leiphone.com/news/201903/HAmHSBKkpd0rrveF.html
https://www.zhihu.com/question/295045153
https://blog.csdn.net/u010333076/article/details/87896734
https://github.com/XiaoMi/mobile-ai-bench/blob/master/README_zh.md

This article is transferred from the public number to the door venture,Original address