Classification of diversified web crawler accesses inspired by biological adaptation

Kuze, Naomi; Ishikura, Shu; Yagi, T.; Chiba, Daiki; Murata, Masayuki

doi:10.1504/ijbic.2021.114877

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Issues such as cloud computing capacity limitations, data security concerns, and data silos are increasingly recognized as barriers to fostering trust, gathering private user data, and facilitating federated training [20]. Consequently, to mitigate these challenges, there arises a critical need for effective solutions in privacy-sensitive scenarios.…”

Section: Federated Learningmentioning

confidence: 99%

TabFedSL: A Self-Supervised Approach to Labeling Tabular Data in Federated Learning Environments

Wang,

Hu,

Chen

et al. 2024

Mathematics

View full text Add to dashboard Cite

Currently, self-supervised learning has shown effectiveness in solving data labeling issues. Its success mainly depends on having access to large, high-quality datasets with diverse features. It also relies on utilizing the spatial, temporal, and semantic structures present in the data. However, domains such as finance, healthcare, and insurance primarily utilize tabular data formats. This presents challenges for traditional data augmentation methods aimed at improving data quality. Furthermore, the privacy-sensitive nature of these domains complicates the acquisition of the extensive, high-quality datasets necessary for training effective self-supervised models. To tackle these challenges, our proposal introduces a novel framework that combines self-supervised learning with Federated Learning (FL). This approach aims to solve the problem of data-distributed training while ensuring training quality. Our framework improves upon the conventional self-supervised learning data augmentation paradigm by incorporating data labeling through the segmentation of data into subsets. Our framework adds noise by splitting subsets of data and can achieve the same level of centralized learning in a distributed environment. Moreover, we conduct experiments on various public tabular datasets to evaluate our approach. The experimental results showcase the effectiveness and generalizability of our proposed method in scenarios involving unlabeled data and distributed settings.

show abstract

Section: Federated Learningmentioning

confidence: 99%

TabFedSL: A Self-Supervised Approach to Labeling Tabular Data in Federated Learning Environments

Wang,

Hu,

Chen

et al. 2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…Before the FL was put forward, machine learning privacy protection problem has been always been a hot research topic [8]. As an innovative technology, the research direction of FL is also closely related to previous privacy protection techniques.…”

Section: Privacy and Security Challenges In Federated Learningmentioning

confidence: 99%

“…For machines to truly approach the level of human thought, they need to be trained with vast amounts of real data [6,7]. However, cloud computing power, data security, data silos and other risks will inevitably become constraints for AI to win user trust, collect private data, and achieve largescale implementation [8]. Therefore, it is an urgent need for a practical and effective technique to alleviate the above problems and make the AI full of vitality again.…”

mentioning

confidence: 99%

A survey on federated learning: challenges and applications

Zhang

Lan

et al. 2022

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

y impact on human life [1][2][3]. As great progress has been made in AI technology in recent years, various application fields have stepped into intelligence [4,5]. On the road of AI development, models, computing power, chip performance, and other technical issues have been the focus of academic research, so that AI technology can continue to evolve. For machines to truly approach the level of human thought, they need to be trained with vast amounts of real data [6,7]. However, cloud computing power, data security, data silos and other risks will inevitably become constraints for AI to win user trust, collect private data, and achieve largescale implementation [8]. Therefore, it is an urgent need for a practical and effective technique to alleviate the above problems and make the AI full of vitality again. Under this background, the concept of "federated learning (FL) " came into being.The notion of FL is first proposed by Google in 2016, mainly to make android mobile phone users update models locally without revealing private personal data [9]. After then, Google implemented an application-oriented FL system. The designed FL system, which focused on running federated average (FedAvg) algorithms on mobile phones, can perform federated analytics and be applied to monitor statistics for large-scale cluster equipment without recording raw device data to the cloud server. FL is one of the most Zhihua Cui

show abstract

“…The focused crawler uses the initial URLs to predict the priorities of unvisited URLs through the text similarity model. The focused crawler continuously crawls web pages related to a given topic from the Internet according to these priorities in descending order [5,6]. Firstly, the focused crawler selects web pages related to a given topic through the manual screening or machine recognition, and construct the term vector representing the given topic through the text processing technology [7,8].…”

Section: Introductionmentioning

confidence: 99%

A focused crawler based on semantic disambiguation vector space model

Liu

et al. 2022

Complex Intell. Syst.

View full text Add to dashboard Cite

The focused crawler grabs continuously web pages related to the given topic according to priorities of unvisited hyperlinks. In many previous studies, the focused crawlers predict priorities of unvisited hyperlinks based on the text similarity models. However, the representation terms of the web page ignore the phenomenon of polysemy, and the topic similarity of the text cannot combine the cosine similarity and the semantic similarity effectively. To address these problems, this paper proposes a focused crawler based on semantic disambiguation vector space model (SDVSM). The SDVSM method combines the semantic disambiguation graph (SDG) and the semantic vector space model (SVSM). The SDG is used to remove the ambiguation terms irrelevant to the given topic from representation terms of retrieved web pages. The SVSM is used to calculate the topic similarity of the text by constructing text and topic semantic vectors based on TF × IDF weights of terms and semantic similarities between terms. The experiment results indicate that the SDVSM method can improve the performance of the focused crawler by comparing different evaluation indicators for four focused crawlers. In conclusion, the proposed method can make the focused crawler grab the higher quality and more quantity web pages related to the given topic from the Internet.

show abstract

Classification of diversified web crawler accesses inspired by biological adaptation

Cited by 3 publications

References 0 publications

TabFedSL: A Self-Supervised Approach to Labeling Tabular Data in Federated Learning Environments

TabFedSL: A Self-Supervised Approach to Labeling Tabular Data in Federated Learning Environments

A survey on federated learning: challenges and applications

A focused crawler based on semantic disambiguation vector space model

Contact Info

Product

Resources

About