Reinforcement learning (RL) applications require a huge effort to become established in real-world environments, due to the injury and break down risks during interactions between the RL agent and the environment, in the online training process. In addition, the RL platform tools (e.g., Python OpenAI’s Gym, Unity ML-Agents, PyBullet, DART, MoJoCo, RaiSim, Isaac, and AirSim), that are required to reduce the real-world challenges, suffer from drawbacks (e.g., the limited number of examples and applications, and difficulties in implementation of the RL algorithms, due to difficulties with the programing language). This paper presents an integrated RL framework, based on Python–Unity interaction, to demonstrate the ability to create a new RL platform tool, based on making a stable user datagram protocol (UDP) communication between the RL agent algorithm (developed using the Python programing language as a server), and the simulation environment (created using the Unity simulation software as a client). This Python–Unity integration process, increases the advantage of the overall RL platform (i.e., flexibility, scalability, and robustness), with the ability to create different environment specifications. The challenge of RL algorithms’ implementation and development is also achieved. The proposed framework is validated by applying two popular deep RL algorithms (i.e., Vanilla Policy Gradient (VPG) and Actor-Critic (A2C)), on an elevation control challenge for a quadcopter drone. The validation results for these experimental tests, prove the innovation of the proposed framework, to be used in RL applications, because both implemented algorithms achieve high stability, by achieving convergence to the required performance through the semi-online training process.

The violence detection in surveillance videos is a complicated task, due to the requirements of extracting the spatio-temporal features in different videos environment, and various videos prospective cases. Hereby, in this paper, different architectures are proposed to perform this task in high performance, by using the UBI-Fights dataset as a comprehensive case study. The proposed architectures are based on involving the Convolutional Block Attention Modules (CBAM) with other simple layers (e.g., ConvLSTM2D or Conv2D&LSTM). In addition, using the Categorical Focal Loss (CFL) as loss function during architectures training to increase the focus on the most important features. To evaluate the proposed architectures, the performance metrics like are Area Under the Curve (AUC), and Equal Error Rate (EER); are mainly used, to declare the architecture ability of identifying the violence correctly, with low interaction value between classes. The performance results declare the ability of the proposed architectures, to achieve higher results that the state of art techniques. For example, the Conv2D&LSTM based architecture, get AUC value of 0.9493, and EER value of 0.0507; that outperform the most of the other proposed ones, and the state of art performance.

This paper aims to prove that the artificial neural network (ANN) is a powerful tool in prediction of buildings energy consumption, this target is achieved by comparing the accuracy of ANN prediction with the output of simple linear regression algorithm and previous work. First of all, the flowchart depends on four main steps: 1) Data selection, 2) Data preparation, 3) Model training and tuning, and 4) Evaluate results. The Commercial Buildings Energy Consumption Survey (CBECS) is selected as a data set to apply ANN on it by choosing the most effective features that have the main influence on the energy consumption. Data preparation process is done by replacing missing values and outliers' values with median value of each feature. The model's hyper-parameters are tuned by manual method depending on the author expeience of ANN algorithm and the evaluation step done by using mean absolute error (MAE), mean square error (MSE), root mean square error (RMSE) and r-squared value as a metric for performance. The results showed that the proposed ANN algorithm achives high performance comparing to simple linear regression algorithm and previous work on the same data.This section represents the steps that needed to make energy prediction in building field by applying the flowchart (Figure 2) on the Commercial Buildings Energy Consumption Survey (CBECS). Figure 2: Artificial neural network flowchart to make building energy prediction [9, 10]. Data Set Selection StepCBECS is an open source data set contain information on the stock of U.S. commercial buildings with size of 6720 building. CBECS includes building types such as schools, hospitals, correctional institutions, and buildings used for religious worship, in addition to traditional commercial buildings such as stores, restaurants, warehouses, and office buildings. In the Commercial Buildings Energy Consumption Survey (CBECS), buildings are classified according to principal activity, which is the primary business, commerce, or function carried on within each building [11,12]. Data Set Preparation StepsAfter choosing suitable data set to test the artificial neural network algorithm, data preprocessing step comes here. The numerical features are selected in addition to Using Test Data to Evaluate The Trained Model by Applying MAE, MSE and RMSE Values Evaluate Results

One of the biggest problems in applying machine learning (ML) in the energy and buildings field is the lack of experience of ML users in implementing each ML algorithm in real-life applications the right way, because each algorithm has prerequisites to be used and specific problems or applications to be implemented. Hence, this paper introduces a generic pipeline to the ML users in the specified field to guide them to select the best-fitting algorithm based on their particular applications and to help them to implement the selected algorithm correctly to achieve the best performance. The introduced pipeline is built on (1) reviewing the most popular trails to put ML pipelines for the energy and building, with a declaration for each trial drawbacks to avoid it in the proposed pipeline; (2) reviewing the most popular ML algorithms in the energy and buildings field and linking them with possible applications in the energy and buildings field in one layout; (3) a full description of the proposed pipeline by explaining the way of implementing it and its environmental impacts in improving energy management systems for different countries; and (4) implementing the pipeline on real data (CBECS) to prove its applicability.