Various publications claim that medical AI systems perform as well, or better, than clinical experts.However, there have been very few controlled trials and the quality of existing studies has been calledinto question. There is growing concern that existing studies overestimate the clinical benefits of AIsystems. This has led to calls for more, and higher-quality, randomized controlled trials of medicalAI systems. While this a welcome development, AI RCTs raise novel methodological challenges thathave seen little discussion. We discuss some of the challenges arising in the context of AI RCTs andmake some suggestions for how to meet them.
Topological models of empirical and formal inquiry are increasingly prevalent. They have emerged in such diverse fields as domain theory [1,16], formal learning theory [18], epistemology and philosophy of science [10,15,8,9, 2], statistics [6,7] and modal logic [17,4]. In those applications, open sets are typically interpreted as hypotheses deductively verifiable by true propositional information that rules out relevant possibilities. However, in statistical data analysis, one routinely receives random samples logically compatible with every statistical hypothesis. We bridge the gap between propositional and statistical data by solving for the unique topology on probability measures in which the open sets are exactly the statistically verifiable hypotheses. Furthermore, we extend that result to a topological characterization of learnability in the limit from statistical data. Verifiability from Propositional InformationThe results in this section appear in [5], [8], and [2], but we restate them here to clarify the intended analogy between propositional and statistical verifiability. Let W be a set of possible worlds, or possibilities one takes seriously, consistent with the background assumptions of inquiry. A proposition is identified with the set of worlds in which it is true, so propositions are subsets of W . Let P, Q be arbitrary propositions. Logical operations correspond to set-theoretic operations in the usual way: P ∩ Q is conjunction, P ∪ Q is disjunction, P c = W \ P is negation, and P ⊆ Q is deductive entailment of Q by P. Finally, P is deductively valid iff P = W and is deductively contradictory iff P = ∅.In the propositional information setting, information states are propositions that rule out relevant possibilities. For every w in W , let I w be the set of all information states true in w. It is assumed that I w is non-empty (at worst, one receives the trivial information W ). Furthermore, it is assumed that for each E, F in I w , there exists G in I w such that G ⊆ E ∩ F. The underlying idea is that a sufficiently diligent inquirer in w eventually receives information as strong as an arbitrary information state E in I w . Since that is true of both E and F, there must be true information as strong as E ∩ F.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.