This project uses reinforcement learning to train the beacon to forward the packet containing the proximity sensing information in a harsh location to the remote server for remote monitoring. Figure below illustrates the major problem in realizing the remote proximity sensing, i.e., it is very hard to access the Internet in the harsh location. While we can use the overlay mesh network to forward the packet over the existing beacon network, each beacon should be able to decide if they should participate in packet forwarding so that the packet can be deliver with low latency at the same time it will not affect much on the primary advertising activity.

Research Challenge

Figure below depicts the network architecture of our overlay mesh network, it manipulates the same advertising channel used by the existing underlying beacon to handle the packet forwarding request. Note that the primary purpose of the underlying beacon is to broadcast a short advertising packet. One of the usage of this advertising packet is to infer the proximity between a mobile receiver to the beacon. On the other hand, the goal of the overlay mesh network is to forward the packet to the receiver located in an area where there is Internet accessibility. Hence, there exists two research challenges:

  • 1. deliver the packet through the overlay mesh network with minimum latency
  • 2. ensure the advertising utilization of the underlying beacon network

Reinforcement Learning

We applied the reinforcement learning, in particular Q-learning, to deal with the above problem. Q-learning can be implemented either through the tabular approach or the approximation approach. However, both approaches have their own limitation and drawback. Check out our paper for further discussion. The source codes for both approaches are available in our Github repository.

Overlay Mesh Network Simulation

The package of our simulation environment is available here.