Bias is neither new nor unique to AI and it is not possible to achieve zero risk of bias in an AI system. NIST intends to develop methods for increasing assurance, GOVERNANCE and practice improvements for identifying, understanding, measuring, managing, and reducing bias. To reach this goal, techniques are needed that are flexible, can be applied across contexts regardless of industry, and are easily communicated to different stakeholder groups. To contribute to the growth of this burgeoning topic area, NIST will continue its work in measuring and evaluating computational biases, and seeks to create a hub for evaluating socio-technical factors. This will include development of formal guidance and standards, supporting standards development activities such as workshops and public comment periods for draft documents, and ongoing discussion of these topics with the stakeholder community.

The effects of ingesting ethanol have been shown to be somewhat variable in humans. To date, there appear to be but few universals. Yet, the question often arises: is it possible to determine if a person is intoxicated by observing them in some manner? A closely related question is: can speech be used for this purpose and, if so, can the degree of intoxication be determined? One of the many issues associated with these questions involves the relationships between a person's paralinguistic characteristics and the presence and level of inebriation. To this end, young, healthy speakers of both sexes were carefully selected and sorted into roughly equal groups of light, moderate, and heavy drinkers. They were asked to produce four types of utterances during a learning phase, when sober and at four strictly controlled levels of intoxication (three ascending and one descending). The primary motor speech measures employed were speaking fundamental frequency, speech intensity, speaking rate and nonfluencies. Several statistically significant changes were found for increasing intoxication; the primary ones included rises in F0, in task duration and for nonfluencies. Minor gender differences were found but they lacked statistical significance. So did the small differences among the drinking category subgroups and the subject groupings related to levels of perceived intoxication. Finally, although it may be concluded that certain changes in speech suprasegmentals will occur as a function of increasing intoxication, these patterns cannot be viewed as universal since a few subjects (about 20%) exhibited no (or negative) changes.

The degree to which speech and/or speech samples are noncontemporary is considered important to the speaker identification process. There are two dimensions to the problem; the first relates to the listener and, especially, to earwitness lineups. Here, the subject or witness is asked to make identifications at various times after having heard (but not having seen, of course) the speaker. It has been found that a person's memory for a voice decays over time. In the second case, it is the samples of the speaker's utterances which are temporally displaced. The prevailing opinion here has been that the use of noncontemporary speech samples poses just as difficult a challenge to the speaker identification process as does the decaying memory of a witness. Accordingly, research was carried out to test this possibility (Hollien and Schwartz, in press); it was found that the overall drop in correct identification over latencies from four weeks to six years was only about 15-25 per cent. It was not until the greatest of the time separations was studied (i.e., twenty years) that a substantial drop occurred (to 31 per cent). At this juncture, a number of questions arose; and three of them have been investigated. First, is listener gender important to the process; second, are the identification levels affected by the type of listeners employed and, finally, can external factors serve to differentially degrade listener performance? It was found that the first question could be answered in the negative and the second two in the affirmative. These findings should aid in clarifying some of the relationship between sample latency and identification accuracy.

The noncontemporariness of speech is important to both of the two general approaches to speaker identification. Earwitness identification is one of them; in that instance, the time at which the identification is made is noncontemporary. A substantial amount of research has been carried out on this relationship and it now is well established that an auditor's memory for a voice decays sharply over time. It is the second approach to speaker identification which is of present interest. In this case, samples of a speaker's utterances are obtained at different points in time. For example, a threat call will be recorded and then sometime later (often very much later), a suspect's exemplar recording will be obtained. In this instance, it is the speech samples that are noncontemporary and they are the materials that are subjected to some form of speaker identification. Prevailing opinion is that noncontemporary speech itself poses just as difficult a challenge to the identification process as does the listener's memory decay in earwitness identification. Accordingly, series of aural-perceptual speaker identification projects were carried out on noncontemporary speech: first, two with latencies of 4 and 8 weeks followed by 4 and 32 weeks plus two more with the pairs separated by 6 and 20 years. Mean correct noncontemporary identification initially dropped to 75–80% at week 4 and this general level was sustained for up to six years. It was only after 20 years had elapsed that a significant drop (to 33%) was noted. It can be concluded that a listener's competency in identifying noncontemporary speech samples will show only modest decay over rather substantial periods of time and, hence, this factor should have only a minimal negative effect on the speaker identification process.

No abstract

NIST contributes to the research, standards, evaluation, and data required to advance the development and use of trustworthy artificial intelligence (AI) to address economic, social, and national security challenges and opportunities. Working with the AI community, NIST has identified the following technical characteristics needed to cultivate trust in AI systems: accuracy, explainability and interpretability, privacy, reliability, robustness, safety, and security (resilience) -and that harmful biases are mitigated. Mitigation of risk derived from bias in AIbased products and systems is a critical but still insufficiently defined building block of trustworthiness. This report proposes a strategy for managing AI bias, and describes types of bias that may be found in AI technologies and systems. The proposal is intended as a step towards consensus standards and a risk-based framework for trustworthy and responsible AI. The document, which also contains an alphabetical glossary that defines commonly occurring biases in AI, contributes to a fuller description and understanding of the challenge of harmful bias and ways to manage its presence in AI systems. Key wordsbias, trustworthiness, AI safety, AI lifecycle, AI development provide their valuable feedback. AudienceThe main audience for this document is researchers and practitioners in the field of trustworthy and responsible artificial intelligence. Researchers will find this document useful for understanding a view of the challenge of bias in AI, and as an initial step toward the development of standards and a risk framework for building and using trustworthy AI systems. Practitioners will benefit by gaining an understanding about bias in the use of AI systems. Trademark InformationAll trademarks and registered trademarks belong to their respective organizations. Note to ReviewersAs described throughout this report, one goal for NIST's work in trustworthy AI is the development of a risk management framework and accompanying standards. To make the necessary progress towards that goal, NIST intends to carry out a variety of activities in 2021 and 2022 in each area of the core building blocks of trustworthy AI (accuracy, explainability and interpretability, privacy, reliability, robustness, safety, and security (resilience), and mitigation of harmful bias). This will require a concerted effort, drawing upon experts from within NIST and external stakeholders. NIST seeks additional collaborative feedback from members of the research, industry, and practitioner community throughout this process. All interested parties are encouraged to please submit comments about this draft report, and the types of activities and events which would be helpful, via the public comment process described on page 3 of this document. There will also be opportunities for engaging in discussions about and contributing to development of key practices and tools to manage Bias in AI. Please look for announcements for webinars, call for position papers, and request for comment on NIST document(s).

No abstract

The United States Secret Service (USSS) teamed with MIT Lincoln Laboratory (MIT/LL) in the US National Institute of Standards and Technology's 2010 Speaker Recognition Evaluation of Human Assisted Speaker Recognition (HASR). We describe our qualitative and automatic speaker comparison processes and our fusion of these processes, which are adapted from USSS casework. The USSS-MIT/LL 2010 HASR results are presented. We also present post-evaluation results. The results are encouraging within the resolving power of the evaluation, which was limited to enable reasonable levels of human effort. Future ideas and efforts are discussed, including new features and capitalizing on naïve listeners.