Real-time bidding (RTB) is an important mechanism in online display advertising, where a proper bid for each page view plays an essential role for good marketing results. Budget constrained bidding is a typical scenario in RTB where the advertisers hope to maximize the total value of the winning impressions under a pre-set budget constraint. However, the optimal bidding strategy is hard to be derived due to the complexity and volatility of the auction environment. To address these challenges, in this paper, we formulate budget constrained bidding as a Markov Decision Process and propose a model-free reinforcement learning framework to resolve the optimization problem. Our analysis shows that the immediate reward from environment is misleading under a critical resource constraint. Therefore, we innovate a reward function design methodology for the reinforcement learning problems with constraints. Based on the new reward design, we employ a deep neural network to learn the appropriate reward so that the optimal policy can be learned effectively. Different from the prior model-based work, which suffers from the scalability problem, our framework is easy to be deployed in large-scale industrial applications. The experimental evaluations demonstrate the effectiveness of our framework on large-scale real datasets.

BackgroundTo evaluate the efficacy and outcomes of hypofractionated stereotactic radiotherapy (HSRT) for brain metastases > 3 cm.MethodsFrom March 2003 to October 2009, 40 patients with brain metastases larger than 3 cm were treated by HSRT. HSRT was applied in 29 patients for primary treatment and in 11 patients for rescue. Single brain metastasis was detected in 21 patients. Whole brain radiotherapy was incorporated into HSRT in 10 patients for primary treatment. HSRT boosts were applied in 23 patients. The diameters of the brain metastases ranged from 3.1 to 5.5 cm (median, 4.1 cm). The median prescribed dose (not including HSRT boosts) was 40 Gy (range, 20-53 Gy) with a median of 10 fractions (range, 4-15 fractions) to the 90% isodose line. The median dose of the boost was 20 Gy (range, 10-35 Gy) in 4 fractions (range, 2-10 fractions).ResultThe median overall survival time was 15 months. The overall survival and local control rate at 12 months was 55.3% and 94.2%, respectively. Four patients experienced local progression of large brain metastases. Nine patients died of intracranial disease progression. One patient died of radiation necrosis with brain edema.ConclusionHSRT was a safe and effective treatment for patients with brain metastases ranged from 3.1 to 5.5 cm. Dose escalation of HSRT boost may improve local control with an acceptable toxicity.

No abstract

Our data suggest that use of SRT as the initial treatment while reserving WBRT as the salvage therapy in case of distant intracranial recurrence made about 50 % of the patients avoid WBRT throughout the course of their lives and may be another optional treatment modality for multiple brain metastases.

No abstract