Deep Reinforcement Learning for Caching Placement and Content Delivery in UAV NOMA Networks

Abstract

The cache-enabling unmanned aerial vehicle (UAV) cellular network is investigated in this article, where the massive access capability is enhanced by applying non-orthogonal multiple access (NOMA). More particularly, a mobile UAV base station, which caches the popular contents to release the pressure on wireless backhaul links, is deployed to assist the delivery of large volume multimedia contents for ground users. The dynamic UAV cellular network with the dynamic UAV locations and content requests in practical scenario is considered in this paper. A long-term caching placement and content delivery joint optimization problem for content delivery delay minimization is formulated as a Markov decision process (MDP) to cope with the dynamic environment. A deep reinforcement learning (DRL) based caching placement and content delivery algorithm is proposed to tackle the MDP with large action space. Finally, it is demonstrated by the numerical results that: 1) a low content delivery delay is achieved by the studied cache-enabling UAV NOMA networks; 2) a good performance is provided by the proposed algorithm.