“…s k ← f (s k , a k ) by (5) for each IoT device n = 1, 2, · · · , N dõ s n,k ← f n (s n,k , a k ) by (15) end for for each IoT device n = 1, 2, · · · , N do Update the per-node value function V n,k (s n,k−1 ) by (29), (30), (31) Update the f-weight vector w fs n,k−1 ,k by (32),(33) Update the c-weight vector w cs n,k−1 ,k by (34),(35) Update the average reward θ n,k by (36),(37), (38) end for Generate the next event e k+1 for each IoT device n = 1, 2, · · · , N do s n,k+1 ← h n (s n,k , e k+1 ) according to (16) end for end for…”