Updated time: 2021-11-15 15:44:17
05A-1081018
2020-01-03
Patent pending
To execute simultaneously multi-purpose deep learning models are needed in many applications. When deploying multiple well-trained CNNs on a system, running them separately could be inefficient. Hence, we would need to merge those models for executing efficiently in a resource-limited device. To deploy two or more well-trained deep-learning models for inference, we design an approach that unifies them into a compact model. The approach consists of three phases: Filter Alignment, Shared-weight Initialization, and Model Calibration. It can merge well-trained feed-forward neural networks of the same architecture into a single network to reduce online storage and inference time. Our approach can improve both run-time memory compression ratios and speedup of the merged well-trained CNN with a good performance.
- Can effectively merge multiple heterogeneous CNN models into a unified one model.
- Can avoid trial-and-error process in training by leveraging well-trained models.
- Only few calibration data is needed, thus greatly reducing training time.
- Deep Learning edge devices. (Smart phone AI applications, smart home device, digital camera)
- Intelligent robot, Human-computer interaction.
- Smart city, security surveillance

Fig1. (a) Original well-trained CNN models for two task. (b) Our “zippering process” can generate merged model, layer by layer from the first layer to the consequent layers. A series of generated models can handle both tasks. Users can choose which model by accuracy or speed by their requirement.

Fig2. In the merged layer, we will find the best permutation for each filter to be merged. The initial shared weights are the average of the weights of filter from the selected pair.
Chu-Song Chen
If you are interested in this technology, please fill in the information on the webpage under the manufacturer option (Request for Technology Licensing, Material Transfer, or Sponsored Research) on the website of this office

Merging Well-Trained Deep CNN Models for Efficient and Versatile Inference