Victoria Manfredi

         Home Research Publications Teaching CV         

NSF Award #2154190

Hierarchical Deep Reinforcement Learning for Routing in Mobile Wireless Networks. This project is a joint collaboration between Wesleyan University and the University of Connecticut.

Project Overview

The use of multi-hop routing in mobile wireless networks is becoming more prevalent, just as these networks are becoming more dense, dynamic, and heterogeneous. Designing a universal multi-hop routing strategy for mobile wireless networks is challenging, however, due to the need to seamlessly adapt routing behavior to spatially diverse and temporally changing network conditions. An alternative to using hand-crafted routing strategies is to use Reinforcement Learning (RL) to learn adaptive multi-hop routing strategies automatically. RL focuses on the design of intelligent agents: an RL agent interacts with its environment to learn a policy, i.e., which actions to take in different environmental states. By using function approximation like deep neural networks (DNNs) as in deep reinforcement learning (DeepRL) to approximate the policy, the RL agent can learn to generalize from its training experience to unseen network conditions and scale the learned routing strategy to larger networks.

The goal of this project is to use DeepRL to develop a universal multi-hop routing strategy for mobile wireless networks that is scalable, generalizable, and adaptive. Specifically, this project will build a novel routing framework that uses hierarchical DeepRL to design an option hierarchy, comprised of multiple layers of routing decisions working together to achieve the overall goals of the network. To enable the same routing strategy to be used at different devices and in unseen network scenarios, the framework will use relational features combined with novel neural network models to handle mobility and perform feature estimation. To further enhance generalizability, the framework will use continual learning to ensure that the routing behaviors learned for more recently seen network scenarios do not dominate the learned routing policy. The developed routing strategies will be thoroughly evaluated using both simulation and experimental testbeds. Through the use of hierarchical DeepRL, this project will provide a significant step forward in developing RL-based routing strategies, and will facilitate development of adaptive strategies for a wide range of mobile wireless networks.

Associated Personnel

Related Publications
  • Learning to Route in Mobile Wireless Networks
    Victoria Manfredi, Alicia Wolfe, Xiaolan Zhang, Bing Wang
    2023, arXiv:2207.11386

  • Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency
    Victoria Manfredi, Alicia Wolfe, Xiaolan Zhang, Bing Wang
    2022, Reinforcement Learning for Real Life (RL4RealLife) Workshop in the 36th Conference on Neural Information Processing Systems (NeurIPS)
    Keywords: multi-hop routing, mobile wireless networks, deep neural networks, reinforcement learning

  • Relational Deep Reinforcement Learning for Routing in Wireless Networks
    Victoria Manfredi, Alicia Wolfe, Bing Wang, Xiaolan Zhang
    2021, International Symposium on a World of Wireless, Mobile, and Multimedia Networks (WoWMoM)
    [pdf, slides]