Big Data Processing for Commercial Buildings and Assessing Flexibility in the Context of Citizen Energy Communities

Oprea, Simona-Vasilica; Bârã, Adela; Dıaconıța, Vlad; Ceaparu, Catalin; Ducman, Anca Alexandra

doi:10.1109/access.2021.3137352

Cited by 7 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the scraping pipeline is operated, Scraping A and Scraping B will run a program to read website elements from page A and Page B. Selenium is a tool used to carry out testing on web-based applications [15]. Using a combination of Selenium and BeautifulSoup libraries is the best approach because it can process program execution faster [16]. The creation of a scraping pipeline is not just about creating a smooth flow of data extraction, but also opens up opportunities to gain deep insights from the acquired data.…”

Section: Creation Of Big Data Pipeline Infrastructurementioning

confidence: 99%

Big Data Pipeline Infrastructure Design in Msme E-Commerce Systems With a Focus on Data Source Processing Using Orchestration Tools

Wibowo,

Sani,

Dewi

et al. 2024

Transmisi J.Ilm. Tek. Elekt.

View full text Add to dashboard Cite

In the digital era, Micro, Small and Medium Enterprises (MSMEs) need to utilize data to improve their business performance, such as increasing customer targets, product development and pricing strategies. Apache Airflow is a powerful tool for building data scraping pipelines that are scalable, flexible, and easy to monitor. One of them is the Central Java MSME data scraping pipeline, which collects business registration information, business type, location, contacts, products, and financial information from various websites, including the Central Java Provincial Government website, basic goods price comparison tables, and specialized news sites. The captured data is stored in a data warehouse for further analysis by the Central Java souvenir entrepreneurs association (ASPOO) in the region. Apache Airflow is used to manage the scraping pipeline in the Central Java MSME E-Commerce system and ensure it runs smoothly. Apache Airflow also has a built-in dashboard for monitoring pipelines and troubleshooting issues. Overall, scraping pipeline in the Central Java MSME e-commerce system is a valuable tool for collecting and analyzing data on the MSME sector in Central Java. This pipeline is scalable, flexible and easy to use, and can be adapted to different user needs and can be integrated with various systems.

show abstract

Section: Creation Of Big Data Pipeline Infrastructurementioning

confidence: 99%

Big Data Pipeline Infrastructure Design in Msme E-Commerce Systems With a Focus on Data Source Processing Using Orchestration Tools

Wibowo,

Sani,

Dewi

et al. 2024

Transmisi J.Ilm. Tek. Elekt.

View full text Add to dashboard Cite

show abstract

“…The work of [23] shows the potential of these kind of models. Machine learning algorithms have been also adopted in order to forecast changes in energy consumer pattern, due to event as the past pandemic one, and to support the energy suppliers, grid operators, and traders to better calculate the required operational flexibility [30].…”

Section: State Of the Art A Demand Forecastingmentioning

confidence: 99%

An Efficient Artificial Intelligence Energy Management System for Urban Building Integrating Photovoltaic and Storage

et al. 2023

View full text Add to dashboard Cite

The emerging leading role of green energy in our society pushes the investigation of new economic and technological solutions. Green energies and smart communities increase efficiency with the use of digital solutions for the benefits of inhabitants and companies. The paper focuses on the development of a methodology for the energy management, combining photovoltaics and storage systems, considering as the main case study a multi-story building characterized by a high density of households, used to generate data which allow feasibility foresights. The physical model of the algorithm is composed by two main elements: the photovoltaics modules and the battery energy storage system. In addition, to gain information about the real-time consumption a machine learning module is included in our approach to generate predictions about the near future demand. The benefits provided by the method are evaluated with an economic analysis, which computes the return of the investment using the real consumptions of a Boarding School, located in Turin (Italy). The case study analyzed in this article showed an increase in purchased energy at the minimum price from 25% to 91% and a 55% reduction in the electricity bill compared to most solutions on the market, with no additional costs and a stabilizing effect on the grid. Finally, the economic analysis shows that the proposed method is a profitable investment, with a breakeven point of thirteen years, due to the very simple implementation and the zero additional cost requested.INDEX TERMS Deep learning, energy management systems, energy storage, environmental economics, renewable energy sources. FRANCESCO GRIMACCIA (Senior Member, IEEE) received the M.S. and Ph.D. degrees (cum laude) in electrical engineering from the Politecnico di Milano, Milan, Italy, in 2003 and 2007, respectively. He is currently an Associate Professor with the Energy Department, Politecnico di Milano. His main research interest includes softcomputing technique development and applications in different electric engineering fields, such as unmanned systems, photovoltaics, and other energy-harvesting devices. He is a member of the IEEE Computational Intelligence Society and the Vice-President of the Associazione Italiana di Elettrotecnica, Elettronica and the Automazione, Informatica e Telecomunicazioni, Milan Section.

show abstract

“…As a result of the overall processing, the SQL plan is obtained (producing the set of operations covering the data retrieval process via index or direct data block loading [21,22]), followed by the execution process itself. There are two basic Oracle architectures -single and multitenant architecture [23]. Single-tenant was introduced in Oracle version 6 released in 1988 and was used until 2012.…”

Section: Database Infrastructurementioning

confidence: 99%

“…Although the container does interconnect between the physical database and instance, if any instance fails, the workload manager automatically routes the survivors ensuring availability and reliability of the whole solution model. From the performance perspective, the relevant model is also sharded database introduced in 2017 [25], providing linear scalability, fault tolerance, and geographic data distribution by using horizontal fragmentation across multiple regions [23,26,27]. It is shown in Fig.…”

Section: Database Infrastructurementioning

confidence: 99%

The Complexity of the Data Retrieval Process Using the Proposed Index Extension

Kvet

Papán²

2022

IEEE Access

View full text Add to dashboard Cite

Data retrieval, access, and tuple identification are inevitable in database processing, ensuring performance. The entities and relationships form a relational database. Data themselves are not specifically formatted, requiring sequential data block scanning if the particular index is unavailable. This paper summarizes existing indexing principles focusing on the B+tree, which forms the default structure for data access based on the index key. Such design is reliable and prone to an increase in the amount of data. However, it cannot manage undefined values properly, whereas they cannot be mathematically compared. Secondly, migrated rows can be present due to the size demand extension after the update operation. Thirdly, the index is always balanced, resulting in additional demands of the transaction. All these factors are covered by the proposed paper, discussing the limitations, opportunities, and own solutions to improve the performance. Several architectures are discussed, maintained, and computationally studied, focusing on the size demands, processing time, and costs. By using proposed techniques, significant improvements can be reached. A pointer list is introduced for migrated row management to reference the index set from the data block and store reference path or using a data reflector. When dealing with transaction support, index management and rebalancing are shifted to the separate autonomous transaction. Thanks to that, the main transaction can be approved sooner with no reliability issues. Finally, the proposed paper introduces the NULL value management structure in the instance memory for the index node reference.

show abstract

Big Data Processing for Commercial Buildings and Assessing Flexibility in the Context of Citizen Energy Communities

Cited by 7 publications

References 35 publications

Big Data Pipeline Infrastructure Design in Msme E-Commerce Systems With a Focus on Data Source Processing Using Orchestration Tools

Big Data Pipeline Infrastructure Design in Msme E-Commerce Systems With a Focus on Data Source Processing Using Orchestration Tools

An Efficient Artificial Intelligence Energy Management System for Urban Building Integrating Photovoltaic and Storage

The Complexity of the Data Retrieval Process Using the Proposed Index Extension

Contact Info

Product

Resources

About