One Size Doesn’t Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research

Knekta, Eva; Runyon, Christopher; Eddy, Sarah L.

doi:10.1187/cbe.18-04-0064

Cited by 346 publications

(350 citation statements)

References 50 publications

Supporting

Mentioning

303

Contrasting

Unclassified

Order By: Relevance

“…The EFA and CFA were recommended to be performed in independent samples ( Knekta et al., 2019 ), particularly the EFA was undertaken with half of the sample and the CFA was followed with the other half of the sample. This is because the results from the EFA should be confirmed with the CFA on different sample; otherwise the CFA will repeat the relationships derived from the EFA results.…”

Section: Research Methodsology and Data Collection Proceduresmentioning

confidence: 99%

Developing and validating five-construct model of customer satisfaction in beauty and cosmetic E-commerce

Nguyen¹

2020

Heliyon

View full text Add to dashboard Cite

Grounded on the American Customer Satisfaction Framework (ACSF) of Bryant (1995) , the European Customer Satisfaction Framework Model (ECSF) of Anderson and Fornell (2000) , and Eleven-factor Customer Satisfaction Model of Hokanson (1995) , this study develops reliable and valid five constructs of customer satisfaction theoretical model and build up a questionnaire in the unique context of Beauty and Cosmetic Online Shopping in the Vietnamese market. More specifically, we identified five main constructs including 5 implementation constructs (i.e., online shopping experience, customer service, external incentives, security/privacy, and personal characteristics), and one outcome construct (i.e., customer satisfaction). A detailed questionnaire was then developed with items for these constructs along with the questions on the personal characteristics of the respondents. The questionnaire was sent to randomly consumers via online channels including Facebook. The target participants are Vietnamese people who have experience in shopping online products like beauty and cosmetics. Based on 334 full responses have been received, we use Exploratory Factor Analysis (EFA) to test for 167 observations to ensure that all items in each scale reflected sufficiently the scope of each construct. This technique helps to identify the underlying dimensions of consumer satisfaction. Furthermore, we also employ Confirmatory Factor Analysis (CFA) to validate the constructs and items in questionnaire. To do this, we have used the sample of 167 responses. Finally, together with results from SEM models, our study contributes to providing a reliable and valid questionnaire which fully reflect for our self-constructed theoretical model of five constructs including Online shopping experience (OSE), External incentives (EI), Seller service (SS), and Security/privacy (SP), as well as personal characteristics (PC).

show abstract

Section: Research Methodsology and Data Collection Proceduresmentioning

confidence: 99%

Developing and validating five-construct model of customer satisfaction in beauty and cosmetic E-commerce

Nguyen¹

2020

Heliyon

View full text Add to dashboard Cite

show abstract

“…The second limitation pertains to the items in which we measured general anxiety among GTAs. As we mentioned in the Methods, because our sample size did not allow for any of our measures to undergo confirmatory factor analysis [37], we were unable to even initially test whether the general anxiety measures we used formed a true “general anxiety” factor. Thus, although they were useful in exploring potential relationships between different types of anxiety, these results should be treated with caution.…”

Section: Discussionmentioning

confidence: 99%

“…We calculated measures of reliability and validity to determine whether the anxiety, self-efficacy, and coping instruments accurately measured the identified variables for the GTA population. Reliability measures consistency when a testing procedure is repeated [37]; while validity is a measure of its accuracy in drawing correct inferences from survey scores [38]. Two forms of evidence were used to assure reliability and validity of the three surveys.…”

Section: Methodsmentioning

confidence: 99%

“…Second, content validity of the questions were checked based on professional judgment by experts (one psychology faculty and 3 biology faculty) as to the appropriateness of the instrument for the Biology GTA population [38]. Though confirmatory factor analysis (CFA) is commonly used to validate the use of an instrument with a new population, it requires a much larger data set than we had available for this project, so we were not able to conduct this analysis [37,39].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

The Ph.D. Panic: Examining the relationships among teaching anxiety, teaching self-efficacy, and coping in Biology graduate teaching assistants (GTAs)

Musgrove

Schussler

2020

Preprint

View full text Add to dashboard Cite

Anxiety among graduate students in the United States has increased over the last several decades, affecting not only their overall mental health but also reducing retention in graduate programs. Teachers with high teaching anxiety can negatively impact student learning, yet the impacts of teaching anxiety on graduate teaching assistants (GTAs) is not well studied. Biology GTAs teach most introductory Biology labs and discussions nationally, thus broadly influencing the quality of undergraduate education. In Fall 2016, we investigated Biology GTA teaching anxiety at a large research-intensive southeastern university by (1) measuring teaching anxiety of Biology GTAs, and (2) exploring the relationships among teaching anxiety, self-efficacy, and coping. Using multiple linear regressions, we found that greater teaching self-efficacy is related to lower teaching anxiety in Biology GTAs (R2adi =0.65, p<0.001). Coping strategies and frequencies did not significantly contribute to teaching anxiety in our models. We found similar levels of teaching anxiety across genders, ethnicities, student citizenship status (domestic vs. international) and teaching experience level. However, there were significant differences among student subgroups in teaching self-efficacy and coping strategies. Effective coping may contribute to the lack of anxiety differences among some of the student subgroups. These results can inform teaching professional development for GTAs, and encourage greater awareness and dialogue about the impacts of mental health issues in academia.

show abstract

“…There are many ways to test hypotheses in DBER-regression analysis, discussed here, is one. In contrast, hypotheses regarding social interactions between groups of students can be tested with network analysis [18]; hypotheses about how items on assessments relate to each other, differentiate respondents, and show bias can be tested with factor analysis [19], Rasch analysis [20], and differential item functioning [21], and some hypotheses are best developed and tested with mixed-methods research [22]. All of these methods are beyond the scope of this paper but are not to be overlooked when considering the best method to analytically test hypotheses in DBER.…”

Section: Variancementioning

confidence: 99%

Beyond linear regression: A reference for analyzing common data types in discipline based education research

Theobald

Aikens

Eddy

et al. 2019

Phys. Rev. Phys. Educ. Res.

Self Cite

View full text Add to dashboard Cite

This paper is part of the Focused Collection on Quantitative Methods in PER: A Critical Examination.] A common goal in discipline-based education research (DBER) is to determine how to improve student outcomes. Linear regression is a common technique used to test hypotheses about the effects of interventions on continuous outcomes (such as exam score) as well as control for student nonequivalence in quasirandom experimental designs. (In quasirandom designs, subjects are not randomly assigned to treatments. For example, when treatment is assigned by classroom, and observations are made on students, the design is quasirandom because treatment is assigned to classroom, not subject (students).) However, many types of outcome data cannot be appropriately analyzed with linear regression. In these instances, researchers must move beyond linear regression and implement alternative regression techniques. For example, student outcomes can be measured on binary scales (e.g., pass or fail), tightly bound scales (e.g., strongly agree to strongly disagree), or nominal scales (i.e., different discrete choices for example multiple tracks within a physics major), each necessitating alternative regression techniques. Here, we review extensions of linear modeling-generalized linear models (glms)-and specifically compare five glms that are useful for analyzing DBER data: logistic, binomial, proportional odds (also called ordinal; including censored regression), multinomial, and Poisson (including negative binomial, hurdle, and zero-inflated) regression. We introduce a diagnostic tool to facilitate a researcher's identification of the most appropriate glm for their own data. For each model type, we explain when, why, and how to implement the regression approach. When: we provide examples of the types of research questions and outcome data that would motivate this regression approach, including citations to articles in the DBER literature. Why: we name which linear regression assumption is violated by the data type. How: we detail implementation and interpretation of this modeling approach in R, including R syntax and code, and how to discuss the regression output in research papers. Code accompanying each analysis can be found in the online github repository that is associated with this paper (https://github.com/ejtheobald/BeyondLinearRegression). This paper is not an exhaustive review of regression techniques, nor does it review nonregression-based analyses. Rather, it aims to compile and summarize regression techniques useful for the most common types of DBER data and provide examples, citations, and heavily annotated R code so that researchers can easily implement the technique in their work.

show abstract

One Size Doesn’t Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research

Cited by 346 publications

References 50 publications

Developing and validating five-construct model of customer satisfaction in beauty and cosmetic E-commerce

Developing and validating five-construct model of customer satisfaction in beauty and cosmetic E-commerce

The Ph.D. Panic: Examining the relationships among teaching anxiety, teaching self-efficacy, and coping in Biology graduate teaching assistants (GTAs)

Beyond linear regression: A reference for analyzing common data types in discipline based education research

Contact Info

Product

Resources

About