Application Research of Crawler and Data Analysis Based on Python

Wu, Hejing; Liu, Fang; Zhao, Long; Shao, Yabin; Cui, Ran

doi:10.21307/ijanmc-2020-018

Cited by 12 publications

(4 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, a web crawler computing application was implemented to gather firsthand textual data by accessing the webpage data, through the designed code of computer programming languages R and Python, to obtain useful data from massive data [ 26 , 27 ]. The process was implemented using Python programming, as shown in Figure 1 .…”

Section: Methodsmentioning

confidence: 99%

Psychological Well-Being of Left-Behind Children in China: Text Mining of the Social Media Website Zhihu

Lyu

Chow

Hwang

et al. 2022

IJERPH

View full text Add to dashboard Cite

China’s migrant population has significantly contributed to its economic growth; however, the impact on the well-being of left-behind children (LBC) has become a serious public health problem. Text mining is an effective tool for identifying people’s mental state, and is therefore beneficial in exploring the psychological mindset of LBC. Traditional data collection methods, which use questionnaires and standardized scales, are limited by their sample sizes. In this study, we created a computational application to quantitively collect personal narrative texts posted by LBC on Zhihu, which is a Chinese question-and-answer online community website; 1475 personal narrative texts posted by LBC were gathered. We used four types of words, i.e., first-person singular pronouns, negative words, past tense verbs, and death-related words, all of which have been associated with depression and suicidal ideations in the Chinese Linguistic Inquiry Word Count (CLIWC) dictionary. We conducted vocabulary statistics on the personal narrative texts of LBC, and bilateral t-tests, with a control group, to analyze the psychological well-being of LBC. The results showed that the proportion of words related to depression and suicidal ideations in the texts of LBC was significantly higher than in the control group. The differences, with respect to the four word types (i.e., first-person singular pronouns, negative words, past tense verbs, and death-related words), were 5.37, 2.99, 2.65, and 2.00 times, respectively, suggesting that LBC are at a higher risk of depression and suicide than their counterparts. By sorting the texts of LBC, this research also found that child neglect is a main contributing factor to psychological difficulties of LBC. Furthermore, mental health problems and the risk of suicide in vulnerable groups, such as LBC, is a global public health issue, as well as an important research topic in the era of digital public health. Through a linguistic analysis, the results of this study confirmed that the experiences of left-behind children negatively impact their mental health. The present findings suggest that it is vital for the public and nonprofit sectors to establish online suicide prevention and intervention systems to improve the well-being of LBC through digital technology.

show abstract

Section: Methodsmentioning

confidence: 99%

Psychological Well-Being of Left-Behind Children in China: Text Mining of the Social Media Website Zhihu

Lyu

Chow

Hwang

et al. 2022

IJERPH

View full text Add to dashboard Cite

show abstract

“…Using Python software, through the way of crawling, using the URL address of the crawl data, to this URL, and then get the response data from this URL server, in accordance with the requirements of Web crawler standards, in a specific order and principles to get the web page information into the URL parts to be grabbed, the process has been repeated until the information captured to meet the requirements of the system, before stopping the abovementioned action, then the data information obtained by the crawl will be filtered and summarized, while building a search engine [7] , put it into a database or folder, and finally according to the specific query needs [8] , from the database or folder to retrieve the required data information, and the data information through the text of the way presented, and then download, so as to get different shopping data [9] . Then you need to crawl the data to clean, and finally get the project can use the data, such as Table 1, to facilitate the construction of the subsequent database.…”

Section: Crawler-based Data Collectionmentioning

confidence: 99%

3D data and stereo projection system combined with “Yuan” shopping applet

Song,

Wei,

Zhai

2024

Proceedings of the 5th International Conference on E-Commerce and Internet Technology, ECIT 2024, March 15–17, 2024, Changsha,

View full text Add to dashboard Cite

This project emerged in the context of the metaverse. We combine 3D data and stereoscopic projection to collect data on traditional shopping and people's views on online shopping today. We launch the "Yuan" shopping plan and are committed to creating a WeChat mini program that combines 3D data and stereoscopic projection. Digitize the products, utilize data, and use stereo projection technology to allow users to visually see the products, thereby bringing people a wonderful and brand new shopping experience, allowing consumers to better understand the adaptability of the products and improve shopping efficiency.

show abstract

“…which significantly accelerate development. By separating concerns with the MTV pattern and providing rich reusable components, Django reduces complexity and speeds up development and maintenance of scalable database-driven web applications [10].…”

Section: Application Architecture Designmentioning

confidence: 99%

Analysis of Python web development applications based on the Django framework

Pang

2024

Third International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024)

View full text Add to dashboard Cite

This article begins by analyzing the application characteristics, component structure, and advantages of the Django framework. With Python's widespread use in web development, choosing an efficient web development framework is crucial. Django, as an open-source framework, has become the preferred choice for Python web development due to its simplicity, comprehensive components, and high development efficiency. This article delves into the methods of application design and implementation based on Django, including the application architecture design using the MTV pattern, model and database design, and views and template design, to guide the practical application of the Django framework in web development. Finally, it summarizes the efficiency and advantages of developing web applications with the Django framework, providing a theoretical basis for framework selection.

show abstract

Application Research of Crawler and Data Analysis Based on Python

Cited by 12 publications

References 2 publications

Psychological Well-Being of Left-Behind Children in China: Text Mining of the Social Media Website Zhihu

Psychological Well-Being of Left-Behind Children in China: Text Mining of the Social Media Website Zhihu

3D data and stereo projection system combined with “Yuan” shopping applet

Analysis of Python web development applications based on the Django framework

Contact Info

Product

Resources

About