Recent breakthroughs in computer vision unlock a number of applications previously unavailable to users. Discriminative tasks, such as detection, segmentation, pose & depth estimation, and others, reach incredible accuracy. Generative applications of adversarial networks, autoencoders, diffusion models, image-to-image translation, video synthesis, and animation methods show high fidelity of their inputs, making it hard to be distinguished from the real by even a human observer. Neural Radiance Fields promise democratized object reconstruction and rendering for various applications.

These breakthroughs, however, come with the cost of high computational requirements of such models. For example, transformer models with quadratic complexity according to the token length, image-to-image translation comes with large FLOPs, and neural rendering approaches require sophisticated rendering pipelines. This translates into the need to run such models on a server-side, elevated service cost, and suboptimal user experience due to the latency incurred for transmitting the data and receiving the result.

Bringing such methods to edge devices is notoriously difficult. Existing works are targeting the compression of large models that do not necessarily lead to on-device acceleration, as it requires not only neural network compression background but also domain-specific background. For instance, to run neural rendering on a device, one needs to be experienced in graphics; to run generative models, one needs to know how to reduce the size without sacrificing the quality of the output. Speeding up neural networks for server-side inference will not necessarily improve edge inference, as deploying them on edge devices requires considerable model size reductions, sometimes at a couple of orders of magnitude.

On the other hand, the benefits of edge inference are obvious: 1. The cost of service is greatly reduced. 2. User privacy is elevated by design, as all the processing runs on the device without data transmission between the server and the device. 3. It facilitates user interaction, as the absence of transmission reduces the latency. 4. Reduces the cycle time to build many novel CV-based applications as expensive infrastructure is not required.

This tutorial will introduce effective methodologies for re-designing algorithms for efficient content understanding, image generation, and neural rendering. Most importantly, we show how the algorithms can be efficiently deployed on mobile devices, eventually achieving real-time interaction between users and mobile devices.