AI-powered applications increasingly adopt Deep Neural Networks (DNNs) for solving many prediction tasks, leading to more than one DNNs running on resource-constrained devices. Supporting many models simultaneously on a device is challenging due to the linearly increased computation, energy, and storage costs. An effective approach to address the problem is multi-task learning (MTL) where a set of tasks are learned jointly to allow some parameter sharing among tasks. MTL creates multi-task models based on common DNN architectures and has shown significantly reduced inference costs and improved generalization performance in many machine learning applications. In this talk, we will introduce our recent efforts on leveraging MTL to improve accuracy and efficiency for edge computing. The talk will introduce multi-task architecture design systems that can automatically identify resource-efficient multi-task models with low inference costs and high task accuracy.