Yes we care!-Certification for machine learning methods through the care label framework

Abstract

Machine learning applications have become ubiquitous. Their applications range from embedded control in production machines over process optimization in diverse areas (e.g., traffic, finance, sciences) to direct user interactions like advertising and recommendations. This has led to an increased effort of making machine learning trustworthy. Explainable and fair AI have already matured. They address the knowledgeable user and the application engineer. However, there are users that want to deploy a learned model in a similar way as their washing machine. These stakeholders do not want to spend time in understanding the model, but want to rely on guaranteed properties. What are the relevant properties? How can they be expressed to the stake- holder without presupposing machine learning knowledge? How can they be guaranteed for a certain implementation of a machine learning model? These questions move far beyond the current state of the art and we want to address them here. We propose a unified framework that certifies learning methods via care labels. They are easy to understand and draw inspiration from well-known certificates like textile labels or property cards of electronic devices. Our framework considers both, the machine learning theory and a given implementation. We test the implementation's compliance with theoretical properties and bounds.

Stress-Testing USB Accelerators for Efficient Edge Inference

“…B0, the smallest variant of this family, is nearly 0.6 times smaller and eight times more compute efficient than DenseNet121 [37], so this statement is clearly true, isn't it? We argue that such a statement is more nuanced and highly depends on the context, or in other words, execution environment [5]. As we will discuss in section 4.2, we found that inference with Densenet121 on a Raspberry Pi 4 CPU is nearly four times slower compared to EfficientNetB0.…”

“…In this context, it is important to remember that any model's performance is not only contingent upon its inherent algorithms. Instead, we find performance also being intricately affected by the environment that the model is executed in [5]. Any study thus needs to consider the question whether it really compares algorithms, implementations, or possibly both [6].…”

Acknowledging Sustainability in the Framework of Ethical Certification for AI

“…Nuanced assessments already exist in the field of environmental sustainability, for example in assessing the energy performance of household appliances, and there is already at least one attempt to produce a similar "Care Label" certification suite for Machine Learning, labeling not only energy consumption but also other features such as runtime, memory usage, expressivity, usability, and the reliability of the AI software [54,55]. This kind of assessment would be an excellent tool to audit the mentioned features separately and optimize AI systems to achieve a better balance of the performances in the different audit areas.…”