Liang, K; Wang, Y; Li, Z; Zheng, G; Wong, KK; Chae, CB; (2025) Digital Twin Assisted Deep Reinforcement Learning for Computation Offloading in UAV Systems. IEEE Transactions on Vehicular Technology 10.1109/TVT.2025.3526198. (In press).

Preview

Text DT-paper.pdf - Accepted Version Download (2MB) | Preview

Abstract

The emerging digital twin (DT) technology involves creating a digital clone of the physical world through techniques such as computer simulation and machine learning. This facilitates the management and decision-making of the physical world. In this correspondence, we propose a DT-assisted deep reinforcement learning (DRL) method to optimize the decision of computation offloading in an unmanned aerial vehicle (UAV)-based communication system. Our focus is on minimizing task processing delay by jointly optimizing task offloading and the UAV's flight path. This problem can be effectively addressed using the DRL method. However, it necessitates frequent interactions with the environment, which may lead to increased resource consumption and risks associated with some "bad"actions. To mitigate these challenges, our proposed DT comprises a predictive model to anticipate the reward and next stage of the physical environment. Additionally, a generative model is introduced to enhance sample efficiency, with these two models realized by a fully connected neural network and a variational auto-encoder, respectively. Subsequently, we generate a hybrid experience replay buffer to facilitate DRL training, resulting in faster convergence, improved performance, and fewer environmental interactions. Numerical results demonstrate that our proposed method achieves superior delay performance and training efficiency compared to state-of-the-art methods.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as CSVJSONXML

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item