Even video telephony services have been pervasively applied, it is still a challenge to provide satisfactory quality of experience to users. Multipath transmission is a promising solution to improve video transmission quality by providing aggregated bandwidth. Due to varied traffic load, the default paths may fall into long lasting congestion. Sender has no choice but to reduce video bitrate to adapt such change, even in multipath transmission scenario. Insisting previous rate may lead the congestion in bottleneck deteriorate further. Deploying relay servers in current Internet infrastructure provides users with alternatives to bypass path failure and congestion. For such transmission scenario for video telephony service, an arising question is how to select paths dynamically to gain maximum profit. The path selection is mapped to the multi‐armed bandit problem. Tradeoff is made between exploration and exploitation to make decision under uncertain environment. The estimated bandwidth provided by congestion controller is used for decision‐making. Further, to make multiple flows share resource fairly and avoid congestion, an algorithm adapted from BBR is implemented for video telephony service. To maintain throughput stability and fairness, a smaller probe up gain value is used and the cycle length in bandwidth probe phase is randomized. To reduce delay, the inflight packets will be drained to match with bandwidth delay product in the probe down phase. The effectiveness of the proposed solution is verify on simulation platform.