JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.
Program evaluations often are based on data drawn from samples of observation units that are not simple random samples. In particular, samples often may be clustered to maximize the number of observations that can be obtained, given project cost constraints. However, clustering gives rise to well-known problems in estimating the variance of sample estimates. The most common procedure is to apply a scalar "design effect" multiplier more or less judgmentally to inflate sample variances. An alternative procedure, as reported in this article, is to construct balanced repeated replication (BRR) estimates of sample variances. BRR estimates of variance are constructed from orthogonally-weighted subsamples of the data, and can provide better estimates of variances from complex samples than other methods.
Systems of Systems (SoS) require organisational components to have staff with abilities appropriate to the challenges within SoS. Large organisations frequently use competency models as a tool to ensure that their staff have the proper skills to deal with these challenges. Competency models have been used for 40 years to select, train, promote or separate employees. While competency models represent an improvement on their primary predecessor, intelligence tests, weaknesses have been noted in the literature and challenges have been made to improve them. Most improvement efforts have been evolutionary adjustments or insertion of more elements. This research presents a completely new way to create, assess or transform a competency model. This framework can be used to design, assess or transform an existing competency model. This paper represents a significant expansion of a conference paper presented at Biographical notes: Joseph M. Bradley is the President of Leading Change, LLC, a small consulting firm focused on assisting clients with complex governance problems, challenges in systems of systems engineering and competency models. His research focus is in decision making, action science and performance measurement. He is retired from the US Navy and has served Resit Unal is a Professor of Engineering Management and Systems Engineering at Old Dominion University, Norfolk, VA. He holds a BS in Electrical Engineering and MS and PhD in Engineering Management. His research interests include robust design, response surface methods, multidisciplinary design optimisation and probabilistic risk analysis. He has published in multiple journals including, International Journal of Critical Infrastructures, IEEE Transactions in Engineering Management, Reliability Engineering and Systems Safety, Quality Engineering and in the Journal of Spacecraft and Rockets. He is a Fellow of the American Society of Engineering Management. C. Ariel Pinto is an Associate Professor of Engineering Management and Systems Engineering at Old Dominion University. He earned his degrees from the University of the Philippines and the University of Virginia.His works focus on multi-disciplinary approaches to risk management in engineered systems and systems engineering, including the effects security and non-security related disruptions in the continuity of operation of organisations and information systems, as well as sustainable risk management.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.