Transformative AI Governance and AI-Empowered Ethical Enhancement Through Preemptive Simulations

Augmented Utilitarianism for AGI Safety

“…"Make us smile" : "Paralyze human facial musculatures into constant beaming smiles" (example by Bostrom [10]) 2. "Make us happy" : "Implant electrodes into the pleasure centers of our brains" (example by Bostrom [ From our view, one could extract the following two types of failures out of the specified perverse instantiations: misspecification of final goal criteria and the so called perspectival fallacy of utility assignment [2] which will become apparant in our explanation. First, one could argue that already the proposed formulations regarding the criteria of the final goal do not optimally capture the nature of the intended sense from a scientific perspective which might have finally misguided the AI.…”

“…However, as shown by Aliman and Kester [2], this type of impossibility theorem does not apply to population axiologies that take the mental states of those attempting to craft the total orders during utility elicitation into account. Similarly to the perverse instantiation examples 1-6, the application of e.g.…”

“…societal, cultural and temporal information. For instance, from the perspective of a current population Z 0 being similar to population Z both with regard to number of people and welfare level, it might instead be "repugnant" to prefer the policy measure leading to population A [2]. The reason being that the scenario leading from Z 0 to A might have included a dying out or even a genocide.…”

“…However, the mental simulations on whose basis predicted utility is extracted are still distorted among others due to the fact that humans fail to accurately predict their appreciation of future scenarios [24]. Therefore, it has been suggested by Aliman and Kester [2] to augment the utility elicitation process by the utilization of technologies like virtual reality (VR) and augmented reality (AR) within a simulation environment in order to be able to get access to a less biased artificially simulated future instant utility. Analogous to the AI-aided policy-by-simulation approach [41], this technique might offer a powerful preemptive tool for AGI Safety in an AU framework.…”

“…-Self-reflexivity: As opposed to CU, AU is intended as a self-reflexive ethical framework which augments itself. Due to the mental-state-dependency it incorporates and the associated embodied nature of ethical total orders, it might even be necessary to craft new ethical goal functions within a so-called socio-technological feedback-loop [2]. In doing so, ongoing technological progresses might help to augment the debiasing of utility assignment while novel scientific insights might facilitate to filter out the most sophisticated measure for the ethically desired form of utility given the current state of society.…”