Social Media Analysis using Natural Language Processing Techniques

Singh, Jyotika

doi:10.25080/majora-1b6fd038-01a

Cited by 4 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vectorization approaches are popular because they can be highly performant out-of-the-box for a wide range of datasets. For the purposes of this study, we used the ClinSpacy framework [ 25 ]. ClinSpacy is the R implementation of the Cui2Vec concept model [ 22 ].…”

Section: Methodsmentioning

confidence: 99%

An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records

Graham,

Shifflet,

Amjad

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

The goal of this study is to develop and validate a lightweight, interpretable machine learning (ML) classifier to identify opioid overdoses in emergency medical services (EMS) records. We conducted a comparative assessment of three feature engineering approaches designed for use with unstructured narrative data. Opioid overdose annotations were provided by two harm reduction paramedics and two supporting annotators trained to reliably match expert annotations. Candidate feature engineering techniques included term frequency-inverse document frequency (TF-IDF), a highly performant approach to concept vectorization, and a custom approach based on the count of empirically-identified keywords. Each feature set was trained using four model architectures: generalized linear model (GLM), Naïve Bayes, neural network, and Extreme Gradient Boost (XGBoost). Ensembles of trained models were also evaluated. The custom feature models were also assessed for variable importance to aid interpretation. Models trained using TF-IDF feature engineering ranged from AUROC = 0.59 (95% CI: 0.53–0.66) for the Naïve Bayes to AUROC = 0.76 (95% CI: 0.71–0.81) for the neural network. Models trained using concept vectorization features ranged from AUROC = 0.83 (95% 0.78–0.88)for the Naïve Bayes to AUROC = 0.89 (95% CI: 0.85–0.94) for the ensemble. Models trained using custom features were the most performant, with benchmarks ranging from AUROC = 0.92 (95% CI: 0.88–0.95) with the GLM to 0.93 (95% CI: 0.90–0.96) for the ensemble. The custom features model achieved positive predictive values (PPV) ranging for 80 to 100%, which represent substantial improvements over previously published EMS encounter opioid overdose classifiers. The application of this approach to county EMS data can productively inform local and targeted harm reduction initiatives.

show abstract

Section: Methodsmentioning

confidence: 99%

An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records

Graham,

Shifflet,

Amjad

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…14 Medical terms and phrases were extracted from primary care notes using named-entity recognition based on the scispaCy biomedical language model through the clinspacy R interface. 15 Negated terms were removed using the medspaCy package. Lemmatization was used to normalize across different forms of the same word (for example, “eyes” was transformed to “eye”).…”

Section: Methodsmentioning

confidence: 99%

Development and Validation of a Model to Predict Anterior Segment Vision-Threatening Eye Disease Using Primary Care Clinical Notes

Singh

Thibodeau

Niziol

et al. 2021

Cornea

View full text Add to dashboard Cite

show abstract

“…For example, in natural language processing, word embedding is a term used for the representation of words for text analysis, typically in the form of a real-valued numerical vector that encodes the meaning of the word such that the words that are closer in the vector space are expected to be similar in meaning [Wik22b]. Word embeddings work great for many applications surrounding textual data [JS21]. However, passing numbers, an audio signal, or an image through a word embeddings generation method is not likely to return any meaningful numerical representation that can be used to train machine learning models.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the 21st Python in Science Conference

2022

Proceedings of the Python in Science Conference

View full text Add to dashboard Cite

We report on progress in developing and extending the new (ASDF) format we have developed for the data from the James Webb and Nancy Grace Roman Space Telescopes since we reported on it at a previous Scipy. While the format was developed as a replacement for the long-standard FITS format used in astronomy, it is quite generic and not restricted to use with astronomical data. We will briefly review the format, and extensions and changes made to the standard itself, as well as to the reference Python implementation we have developed to support it. The standard itself has been clarified in a number of respects. Recent improvements to the Python implementation include an improved framework for conversion between complex Python objects and ASDF, better control of the configuration of extensions supported and versioning of extensions, tools for display and searching of the structured metadata, better developer documentation, tutorials, and a more maintainable and flexible schema system. This has included a reorganization of the components to make the standard free from astronomical assumptions. A important motivator for the format was the ability to support serializing functional transforms in multiple dimensions as well as expressions built out of such transforms, which has now been implemented. More generalized compression schemes are now enabled. We are currently working on adding chunking support and will discuss our plan for further enhancements.

show abstract

Social Media Analysis using Natural Language Processing Techniques

Abstract: Natural language on social media consists of free form text; no rules around grammar, capitalization, abbreviation, or writing style apply. Human language is ever evolving as new and popular abbreviations, topics and terms develop.

Cited by 4 publications

References 0 publications

An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records

An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records

Development and Validation of a Model to Predict Anterior Segment Vision-Threatening Eye Disease Using Primary Care Clinical Notes

Proceedings of the 21st Python in Science Conference

Contact Info

Product

Resources

About