This note describes a scoring scheme for the coreference task in MUC6. It improves o n the original approach l by: (1) grounding the scoring scheme in terms of a model ; (2) producing more intuitive recall and precision scores ; and (3) not requiring explici t computation of the transitive closure of coreference. The principal conceptual differenc e is that we have moved from a syntactic scoring model based on following coreferenc e links to an approach defined by the model theory of those links .

We analyze the development of 49 local bond markets. Our main finding is that policies and laws matter: Countries with stable inflation rates and strong creditor rights have more developed local bond markets and rely less on foreign-currency-denominated bonds. The results suggest that "original sin" is a misnomer. Emerging economies are not inherently dependent upon foreign-currency debt. Rather, by improving policy performance and strengthening institutions they may develop local currency bond markets, reduce their currency mismatch, and lessen the likelihood of future crises.

We analyze the development of, and foreign participation in, 49 local bond markets. Countries with stable inflation rates and strong creditor rights have more developed local bond markets and rely less on foreign-currency-denominated bonds. Less developed bond markets have returns characterized by high variance and negative skewness, factors eschewed by U.S. investors. Results based on a three-moment CAPM indicate, however, that it is diversifiable idiosyncratic risk that U.S. investors appear to shun. Taken as a whole our results hint at a virtuous cycle of bond market development. Creditor friendly policies and laws can spark local bond market development and result in lower variance and more right-skewed returns, characteristics that attract foreign participation. In turn, the ability to borrow internationally in the local currency helps avoid the pitfalls of a currency mismatch and thus may further stabilize macroeconomic performance.

We assess the development of local currency bond markets in emerging market economies (EMEs). Supported by policies and laws that helped to improve macroeconomic stability and creditor rights, many local currency EME bond markets have grown substantially over the past decade and have also provided USD-based investors with attractive returns. U.S. investors have responded by increasing their holdings of EME local currency bonds from less than $2 billion in 2001 to over $27 billion by end-2008. While the increase in U.S. investment spanned many EMEs, empirical tests suggest that relatively more went to those with identifiable investorfriendly institutions and policies.

Despite methodology difficulties noted in this pilot study, there was statistical improvement in intervention group versus controls.

Motivated by the high cost of human curation of biological databases, there is an increasing interest in using computational approaches to assist human curators and accelerate the manual curation process. Towards the goal of cataloging drug indications from FDA drug labels, we recently developed LabeledIn, a human-curated drug indication resource for 250 clinical drugs. Its development required over 40 h of human effort across 20 weeks, despite using well-defined annotation guidelines. In this study, we aim to investigate the feasibility of scaling drug indication annotation through a crowdsourcing technique where an unknown network of workers can be recruited through the technical environment of Amazon Mechanical Turk (MTurk). To translate the expert-curation task of cataloging indications into human intelligence tasks (HITs) suitable for the average workers on MTurk, we first simplify the complex task such that each HIT only involves a worker making a binary judgment of whether a highlighted disease, in context of a given drug label, is an indication. In addition, this study is novel in the crowdsourcing interface design where the annotation guidelines are encoded into user options. For evaluation, we assess the ability of our proposed method to achieve high-quality annotations in a time-efficient and cost-effective manner. We posted over 3000 HITs drawn from 706 drug labels on MTurk. Within 8 h of posting, we collected 18 775 judgments from 74 workers, and achieved an aggregated accuracy of 96% on 450 control HITs (where gold-standard answers are known), at a cost of $1.75 per drug label. On the basis of these results, we conclude that our crowdsourcing approach not only results in significant cost and time saving, but also leads to accuracy comparable to that of domain experts.Database URL: ftp://ftp.ncbi.nlm.nih.gov/pub/lu/LabeledIn/Crowdsourcing/.

Background: This article describes capture of biological information using a hybrid approach that combines natural language processing to extract biological entities and crowdsourcing with annotators recruited via Amazon Mechanical Turk to judge correctness of candidate biological relations. These techniques were applied to extract gene– mutation relations from biomedical abstracts with the goal of supporting production scale capture of gene–mutation–disease findings as an open source resource for personalized medicine. Results: The hybrid system could be configured to provide good performance for gene–mutation extraction (precision ∼82%; recall ∼70% against an expert-generated gold standard) at a cost of $0.76 per abstract. This demonstrates that crowd labor platforms such as Amazon Mechanical Turk can be used to recruit quality annotators, even in an application requiring subject matter expertise; aggregated Turker judgments for gene–mutation relations exceeded 90% accuracy. Over half of the precision errors were due to mismatches against the gold standard hidden from annotator view (e.g. incorrect EntrezGene identifier or incorrect mutation position extracted), or incomplete task instructions (e.g. the need to exclude nonhuman mutations). Conclusions: The hybrid curation model provides a readily scalable cost-effective approach to curation, particularly if coupled with expert human review to filter precision errors. We plan to generalize the framework and make it available as open source software.Database URL: http://www.mitre.org/publications/technical-papers/hybrid-curation-of-gene-mutation-relations-combining-automated

We analyze the development of, and foreign participation in, 49 local bond markets. Countries with stable inflation rates and strong creditor rights have more developed local bond markets and rely less on foreign-currency-denominated bonds. Less developed bond markets have returns characterized by high variance and negative skewness, factors eschewed by U.S. investors. Results based on a three-moment CAPM indicate, however, that it is diversifiable idiosyncratic risk that U.S. investors appear to shun. Taken as a whole our results hint at a virtuous cycle of bond market development. Creditor friendly policies and laws can spark local bond market development and result in lower variance and more right-skewed returns, characteristics that attract foreign participation. In turn, the ability to borrow internationally in the local currency helps avoid the pitfalls of a currency mismatch and thus may further stabilize macroeconomic performance.