Chelsea Song and Chen Tang for their helpful comments on an earlier version of this paper.Thank you to the many members of the Well-being and Measurement Lab and the Laboratory for Understanding Careers and Individual Differences who assisted with data collection and rating the interviewees in the study.

In the age of big data, substantial research is now moving toward using digital footprints like social media text data to assess personality. Nevertheless, there are concerns and questions regarding the psychometric and validity evidence of such approaches. We seek to address this issue by focusing on social media text data and (i) conducting a review of psychometric validation efforts in social media text mining (SMTM) for personality assessment and discussing additional work that needs to be done; (ii) considering additional validity issues from the standpoint of reference (i.e. ‘ground truth’) and causality (i.e. how personality determines variations in scores derived from SMTM); and (iii) discussing the unique issues of generalizability when validating SMTM for personality assessment across different social media platforms and populations. In doing so, we explicate the key validity and validation issues that need to be considered as a field to advance SMTM for personality assessment, and, more generally, machine learning personality assessment methods. © 2020 European Association of Personality Psychology

Recent advances in text mining have provided new methods for capitalizing on the voluminous natural language text data created by organizations, their employees, and their customers. Although often overlooked, decisions made during text preprocessing affect whether the content and/or style of language are captured, the statistical power of subsequent analyses, and the validity of insights derived from text mining. Past methodological articles have described the general process of obtaining and analyzing text data, but recommendations for preprocessing text data were inconsistent. Furthermore, primary studies use and report different preprocessing techniques. To address this, we conduct two complementary reviews of computational linguistics and organizational text mining research to provide empirically grounded text preprocessing decision-making recommendations that account for the type of text mining conducted (i.e., open or closed vocabulary), the research question under investigation, and the data set’s characteristics (i.e., corpus size and average document length). Notably, deviations from these recommendations will be appropriate and, at times, necessary due to the unique characteristics of one’s text data. We also provide recommendations for reporting text mining to promote transparency and reproducibility.

Technological advances have led to the development of automated methods for personnel assessment that are purported to augment or outperform human judgment. However, empirical research providing validity evidence for such techniques in the selection context remains scarce. In addressing this void, this study focuses on language-based personality assessments using an off-the-shelf, commercially available product (i.e., IBM Watson Personality Insights) in the context of video-based interviews. The scores derived from the language-based assessment were compared to self and observer ratings of personality to examine convergent and discriminant relationships. The language-based assessment scores showed low convergence with self-ratings for openness, and with self- and observer ratings for agreeableness. No validity evidence was found for extraversion and conscientiousness. For neuroticism, the patterns of correlations were in the opposite of what was theoretically expected, which raised a significant concern. We suggest more validation work is needed to further improve emerging assessment techniques and to understand when and how such approaches can appropriately be applied in personnel assessment and selection.

Purpose Effective leadership has been the focus of much research in recent years, but leadership development is still understudied. Information technology (IT) continues to grow in importance due to the industries IT creates, the industries IT disrupts, and the potential IT holds for all companies. Because leadership is highly context bound, the purpose of this paper is to examine leadership development in the IT context to take first steps toward establishing best practices for IT leadership development. Design/methodology/approach This conceptual paper reviews leadership in general before performing an integrated literature review of leadership as it has been studied in the IT context. Then the paper presents three propositions regarding effective IT leadership development. Findings IT leadership development should involve formal mentoring, robust feedback that is integrated into the development plan, and should be treated as a core process for long-term success. Emergent and transformational leadership are important for IT. Practical implications It behooves IT departments to implement the leadership development programs proposed here because leadership has been identified as one of the most difficult skills to find in IT employees. The findings can inform training professionals exploring ways to improve the leadership capacity and leadership development in the IT units of their organizations. Originality/value The paper’s literature review uncovered no quantitative peer-reviewed research on leadership development in the IT context, suggesting an area of need for further empirical studies. Researchers and practitioners alike will benefit from a greater understanding of leadership development in IT.

We introduce the psychometric concepts of bias and fairness in a multimodal machine learning context assessing individuals' hireability from prerecorded video interviews. We collected interviews from 733 participants and hireability ratings from a panel of trained annotators in a simulated hiring study, and then trained interpretable machine learning models on verbal, paraverbal, and visual features extracted from the videos to investigate unimodal versus multimodal bias and fairness. Our results demonstrate that, in the absence of any bias mitigation strategy, combining multiple modalities only marginally improves prediction accuracy at the cost of increasing bias and reducing fairness compared to the least biased and most fair unimodal predictor set (verbal). We further show that gendernorming predictors only reduces gender predictability for paraverbal and visual modalities, while removing gender-biased features can achieve gender blindness, minimal bias, and fairness (for all modalities except for visual) at the cost of some prediction accuracy. Overall, the reduced-feature approach using predictors from all modalities achieved the best balance between accuracy, bias, and fairness, with the verbal modality alone performing almost as well. Our analysis highlights how optimizing model prediction accuracy in isolation and in a multimodal context may cause bias, disparate impact, and potential social harm, while a more holistic optimization approach based on accuracy, bias, and fairness can avoid these pitfalls. CCS CONCEPTS• Applied computing → Law, social and behavioral sciences;• Information systems → Multimedia and multimodal retrieval; Content analysis and feature selection; • Computing methodologies → Artificial intelligence.

The Problem. Employers view today's science, technology, engineering, and math (STEM) program graduates as deficient in interpersonal skills that are essential for team and organizational performance. However, STEM programs continue to effectively engage in interpersonal skills development in college level, instead placing the responsibility of such development on employers. The Solution. A competency modeling framework should inform the design of such education programs, and this article describes such a framework and an educational program that used the framework to identify needed interpersonal skills and successfully develop them in STEM students. The framework will help HRD (human resource development) professionals take an active role in identifying the competencies needed for STEM program graduate success. The Stakeholders. The article provides HRD academics with a framework to identify competencies needed for workplace success in a given academic field. As education programs such as described here are developed, STEM organizations may receive workers who are more productive and less in need of leadership development expenditure.

Given significant concerns about fairness and bias in the use of artificial intelligence (AI) and machine learning (ML) for assessing psychological constructs, we provide a conceptual framework for investigating and mitigating machine learning measurement bias (MLMB) from a psychometric perspective. MLMB is defined as differential functioning of the trained ML model between subgroups. MLMB can empirically manifest when a trained ML model produces different predicted score levels for individuals belonging to different subgroups (e.g., race, gender) despite them having the same ground truth level for the underlying construct of interest (e.g., personality), and/or when the model yields differential predictive accuracies across the subgroups. Because the development of ML models involves both data and algorithms, both biased data and algorithm training bias are potential sources of MLMB. Data bias can occur in the form of nonequivalence between subgroups in the ground truth, platform-based construct, behavioral expression, and/or feature computing. Algorithm training bias can occur when algorithms are developed with nonequivalence in the relation between extracted features and ground truth (i.e., algorithm features are differentially used, weighted, or transformed between subgroups). We explain how these potential sources of bias may manifest during ML model development and share initial ideas on how to mitigate them, recognizing that the development of new statistical and algorithmic procedures will need to follow. We also discuss how this framework brings clarity to MLMB but does not reduce the complexity of the issue.