SASSCAL WebSAPI: A Web Scraping Application Programming Interface to Support Access to SASSCAL’s Weather Data

Thapelo, Tsaone Swaabow; Namoshe, Molaletsa; Matsebe, Oduetse; Motshegwa, Tshiamo; Bopape, Mary-Jane M.

doi:10.5334/dsj-2021-024

Cited by 2 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Парсинг передбачає використання спеціальних алгоритмів для екстракції та структурування даних з різноманітних неструктурованих та напівструктурованих джерел, у той час як веб-скрейпінг зосереджений на автоматизованому зборі даних із вебсторінок. Наприклад, парсинг HTML дає можливість ефективно вилучати інформацію з вебсторінок за допомогою регулярних виразів, описаних в офіційній бібліотеці мови Python [12]. Найпростіший вираз може мати вигляд: regex = r<tag>(.*?…”

Section: використання великих даних для інформаційного забезпечення р...unclassified

Usage of Big Data for Information Support of the Labor Market

ROZBYTSKYI

2024

Demogr. soc. ekon.

View full text Add to dashboard Cite

The challenges of collecting current labor market data amidst global changes and geopolitical fluctuations increasingly necessitate new approaches and alternative methods of data collection and analysis. This highlights the relevance of research aimed at developing and adapting labor market information provision methods to contemporary challenges. A promising approach in this context is the use of Big Data for labor market assessment, which involves collecting information from sources such as online job search and vacancy portals. This method allows for a deeper analysis of market trends and provides a more accurate and timely assessment of labor market needs and opportunities. The aim of this article is to discuss approaches to developing labor market information systems using Big Data, particularly online data from job vacancy websites. It examines the use of Big Data in labor market analysis based on a database containing over four million job vacancies posted on Ukrainian job search portals over the last five years, provided by the European Training Foundation (ETF). The effectiveness of these approaches in facilitating job search for all interested parties is evaluated, particularly through providing insights into the dynamics of supply and demand in the labor market based on data from these portals. The opportunities and limitations of using Big Data in this context are analyzed, including their impact on employment policy development and labor market planning. The potential benefits of Big Data in providing deeper and more accurate market condition analyses are outlined, along with technical aspects and challenges associated with their processing and interpretation. The article examines methodological approaches to data collection and analytical processing in the context of accelerated transformations, volatility, and limited access to traditional information resources. The scientific novelty of the article lies in the substantiation of the feasibility and appropriateness of using open data from online job portals for labor market information provision under current conditions. In conducting the research, methods of analysis, synthesis, and generalization were applied to identify the main contemporary issues of labor market information provision in Ukraine. The effectiveness of data collection methods based on web scraping and parsing techniques was evaluated, as well as the use of the integrated Snowflake platform to identify key trends and patterns in the labor market. The conclusions summarize the main points and substantiate directions for further research, highlighting the significance of Big Data in developing employment strategies and optimizing the labor market.

show abstract

Section: використання великих даних для інформаційного забезпечення р...unclassified

Usage of Big Data for Information Support of the Labor Market

ROZBYTSKYI

2024

Demogr. soc. ekon.

View full text Add to dashboard Cite

show abstract

“…Essa técnica é comumente usada por empresas e pesquisadores para coletar dados sobre as condições do clima e realizar análise de tendências. Para concluir, a raspagem da web também pode ser usada para análise e visualização de dados, o que pode ajudar as os pesquisadores e órgãos públicos a tomar decisões mais assertivas (Thapelo et al, 2021).…”

Section: Web Scrapping Dos Dadosunclassified

Abordagem de árvore de regressão para análise de dados de distribuição pluviométrica na cidade de São Carlos – SP

Teixeira

2023

Anais Da XI Escola Regional De Computação Do Ceará, Maranhão E Piauí (ERCEMAPI 2023)

View full text Add to dashboard Cite

O advento da inteligência artificial nos dias atuais tem possibilitado associação com modelos de predição de desastres naturais com foco em chuvas intensas na prevenção de fenômenos naturais e extremos. Este trabalho propõe uma abordagem de análise de dados pluviométricos da cidade de São Carlos – SP. A abordagem de árvore de regressão apresentada visa contribuir com uma determinação assertiva acerca de dados pluviométricos futuros. Para tanto, foi realizada uma estimativa de dados volumétricos de precipitação no intervalo de 2016 a 2023, a qual se aproximou dos valos reais divulgados, com valores de RMSE de 1,274, de MSE de 1,624 e MAE de 0,305. Com vistas ao desenvolvimento futuro, espera-se aplicar o método em diferentes contextos.

show abstract

SASSCAL WebSAPI: A Web Scraping Application Programming Interface to Support Access to SASSCAL’s Weather Data

Cited by 2 publications

References 24 publications

Usage of Big Data for Information Support of the Labor Market

Usage of Big Data for Information Support of the Labor Market

Abordagem de árvore de regressão para análise de dados de distribuição pluviométrica na cidade de São Carlos – SP

Contact Info

Product

Resources

About