We present StreamPipes, a semantics-based approach aiming to provide a description and management layer to define and execute stream processing pipelines consisting of multiple, potentially heterogeneous runtime implementations. StreamPipes consists of i) an ontology-based model to describe requirements and capabilities of stream processing services, (ii) a software development kit to describe and publish existing event patterns according to the StreamPipes protocol, (iii) a matchmaking engine to compute matchings between streams, event processing agents and actuators and (iv) a web-based authoring tool to model pipelines in a drag-and-drop style. We implemented the solution to the 2015 DEBS Grand Challenge with StreamPipes using two underlying open source frameworks as runtime implementations. Our approach decouples event pattern descriptions from their specific implementations and therefore facilitates reuse of implemented stream processing elements in multiple pipelines without any further development effort.
Despite excellent average-case performance of many image classifiers, their performance can substantially deteriorate on semantically coherent subgroups of the data that were under-represented in the training data. These systematic errors can impact both fairness for demographic minority groups as well as robustness and safety under domain shift. A major challenge is to identify such subgroups with subpar performance when the subgroups are not annotated and their occurrence is very rare. We leverage recent advances in text-to-image models and search in the space of textual descriptions of subgroups ("prompts") for subgroups where the target model has low performance on the prompt-conditioned synthesized data. To tackle the exponentially growing number of subgroups, we employ combinatorial testing. We denote this procedure as PROMPTAT-TACK as it can be interpreted as an adversarial attack in a prompt space. We study subgroup coverage and identifiability with PROMPTATTACK in a controlled setting and find that it identifies systematic errors with high accuracy. Thereupon, we apply PROMPTATTACK to ImageNet classifiers and identify novel systematic errors on rare subgroups.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.