No abstract

Real-time bidding (RTB) is an important mechanism in online display advertising, where a proper bid for each page view plays an essential role for good marketing results. Budget constrained bidding is a typical scenario in RTB where the advertisers hope to maximize the total value of the winning impressions under a pre-set budget constraint. However, the optimal bidding strategy is hard to be derived due to the complexity and volatility of the auction environment. To address these challenges, in this paper, we formulate budget constrained bidding as a Markov Decision Process and propose a model-free reinforcement learning framework to resolve the optimization problem. Our analysis shows that the immediate reward from environment is misleading under a critical resource constraint. Therefore, we innovate a reward function design methodology for the reinforcement learning problems with constraints. Based on the new reward design, we employ a deep neural network to learn the appropriate reward so that the optimal policy can be learned effectively. Different from the prior model-based work, which suffers from the scalability problem, our framework is easy to be deployed in large-scale industrial applications. The experimental evaluations demonstrate the effectiveness of our framework on large-scale real datasets.

No abstract

No abstract

No abstract

Indoor airborne culturable fungi exposure has been closely linked to occupants' health. However, conventional measurement of indoor airborne fungal concentration is complicated and usually requires around one week for fungi incubation in laboratory. To provide an ultra-fast solution, here, for the first time, a knowledge-based machine learning model is developed with the inputs of indoor air quality data for estimating the concentration of indoor airborne culturable fungi. To construct a database for statistical analysis and model training, 249 data groups of air quality indicators (concentration of indoor airborne culturable fungi, indoor/outdoor PM and PM concentrations, indoor temperature, indoor relative humidity, and indoor CO concentration) were measured from 85 residential buildings of Baoding (China) during the period of 2016.11.15-2017.03.15. Our results show that artificial neural network (ANN) with one hidden layer has good prediction performances, compared to a support vector machine (SVM). With the tolerance of ± 30%, the prediction accuracy of the ANN model with ten hidden nodes can at highest reach 83.33% in the testing set. Most importantly, we here provide a quick method for estimating the concentration of indoor airborne fungi that can be applied to real-time evaluation.

It is of great significance to achieve the prediction of building energy consumption. However, machine learning, as a promising technique for many practical applications, was rarely utilized in this field. The most important reason is that the predictive structure with best performance is difficult to be determined. To fill the gap, this paper offers one in-depth review, which focuses on the accuracy analyses and model comparisons. Specifically, the accuracy analyses were conducted based on different types of buildings (e.g. residential building, commercial building, government building or educational building), different type of temporal granularity (e.g. sub-hourly, hourly, daily or annual), as well as input/output variables and historical data collections. Further, artificial neural network (ANN) and support vector machine (SVM), as the epidemic models, were compared in terms of their complexity of prediction processes, accuracies of results, the amounts of required historical data, the numbers of inputs, etc. Then the hybrid and single machine learning methods were outlined and compared in terms of their strengths and weaknesses. In addition, several vital defects and further research directions are presented from a multivariate perspective. We hope

No abstract