Collaborative on-device CNN inference : design and optimization of communication and computation

Kilcioglu, Emre [UCL]

Deep learning (DL) models have experienced a notable increase in adoption across various latency-critical artificial intelligence (AI) applications. However, the computational demands of these DL models pose significant challenges, especially when deploying them on resource-constrained devices (RCDs), which are characterized by limited computing power, battery life, and memory. The solution to the computational demands of DL-driven applications is to leverage the concept of edge intelligence (EI). EI aims to bring the computation-heavy DL execution of AI applications to edge networks. Within the EI framework, this thesis introduces several schemes for collaborative on-device deep learning inference, where multiple RCDs collaboratively execute DL models without reliance on centralized servers. To achieve collaborative DL inference, this thesis explores various parallelism techniques, including data parallelism, model parallelism, and hybrid parallelism, with a specific focus on convolutional neural networks (CNNs). Data parallelism distributes input data among multiple RCDs, while model parallelism partitions DL models. Hybrid parallelism combines elements of both approaches to tackle challenges related to large input data sizes for model parallelism and the execution of multiple filters for data parallelism. In the context of hybrid parallelism, relay-assisted communication is proposed to facilitate efficient collaboration among devices. These strategies aim to distribute the computational load efficiently and reduce communication overhead to optimize the collaborative deep learning inference for specific scenarios.