The Basic AI Drives

Omohundro, Stephen M.

doi:10.1201/9781351251389-3

Cited by 61 publications

(74 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, many reinforcement learning algorithms can learn by creating their own subgoals [167,184]. In other words, they are not simply executing subgoals determined by their programmers, but rather discover them independently [139,167]. In 2013, the researchers at DeepMind Technologies developed an AI, based on a reinforcement learning algorithm, which learnt to play several Atari video games and even outcompeted the best human players in some of the games [128,128,167].…”

Section: The "They Only Do What They Have Been Programmed To Do" Fallacymentioning

confidence: 99%

“…Whether we count a person morally accountable or not affects the way we expect to be treated by then and how we treat them in return; for the judicial institution to function, we must assess legal accountability, which in turn is partially grounded on moral accountability. We seem to have an inclination to regard AI as a moral agent [7,177], although people do not consider them as appropriate agents for moral decisions (see [115,139]). Usually, when encountering intelligent and goal-directed behavior, the human mind attributes the target with core features of agency, that is, the presence of a self and consciousness [181].…”

Section: Artificial Intelligence: Morally Relevant Even If Non-consciousmentioning

confidence: 99%

See 1 more Smart Citation

Socio-cognitive biases in folk AI ethics and risk discourse

et al. 2021

View full text Add to dashboard Cite

The ongoing conversation on AI ethics and politics is in full swing and has spread to the general public. Rather than contributing by engaging with the issues and views discussed, we want to step back and comment on the widening conversation itself. We consider evolved human cognitive tendencies and biases, and how they frame and hinder the conversation on AI ethics. Primarily, we describe our innate human capacities known as folk theories and how we apply them to phenomena of different implicit categories. Through examples and empirical findings, we show that such tendencies specifically affect the key issues discussed in AI ethics. The central claim is that much of our mostly opaque intuitive thinking has not evolved to match the nature of AI, and this causes problems in democratizing AI ethics and politics. Developing awareness of how our intuitive thinking affects our more explicit views will add to the quality of the conversation.

show abstract

Section: The "They Only Do What They Have Been Programmed To Do" Fallacymentioning

confidence: 99%

Section: Artificial Intelligence: Morally Relevant Even If Non-consciousmentioning

confidence: 99%

Socio-cognitive biases in folk AI ethics and risk discourse

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Gandhi knows that if he wants to kill people, he will probably kill people, and the current version of Gandhi does not want to kill. More generally, it seems likely that most self-modifying minds will naturally have stable utility functions, which implies that an initial choice of mind design can have lasting effects (Omohundro 2008).…”

Section: Superintelligencementioning

confidence: 99%

Ethics of Artificial Intelligence

Ramoni¹,

Manuel²

2020

View full text Add to dashboard Cite

Featuring seventeen original essays on the ethics of artificial intelligence (AI) by today’s most prominent AI scientists and academic philosophers, this volume represents state-of-the-art thinking in this fast-growing field. It highlights central themes in AI and morality such as how to build ethics into AI, how to address mass unemployment caused by automation, how to avoid designing AI systems that perpetuate existing biases, and how to determine whether an AI is conscious. As AI technologies progress, questions about the ethics of AI, in both the near future and the long term, become more pressing than ever. Should a self-driving car prioritize the lives of the passengers over those of pedestrians? Should we as a society develop autonomous weapon systems capable of identifying and attacking a target without human intervention? What happens when AIs become smarter and more capable than us? Could they have greater than human-level moral status? Can we prevent superintelligent AIs from harming us or causing our extinction? At a critical time in this fast-moving debate, thirty leading academics and researchers at the forefront of AI technology development have come together to explore these existential questions.

show abstract

“…We thus turn to the scenario that artificially intelligent entities develop separately from regular (or unenhanced) humans. One of the likely characteristics of any sufficiently intelligent entityno matter what final objectives are programmed into it by evolution or by its creator -is that it will act by pursuing intermediate objectives or "basic drives" that are instrumental for any final objective (Omohundro, 2008). These intermediate objectives include self-preservation, selfimprovement and resource accumulation, which all make it likelier and easier for the entity to achieve its final objectives.…”

Section: Second Scenario: Artificially Intelligent Agents and The Return Of Malthusmentioning

confidence: 99%

Artificial Intelligence and Its Implications for Income Distribution and Unemployment

Korinek

Stiglitz

2019

The Economics of Artificial Intelligence

View full text Add to dashboard Cite

We would like to thank our discussant Tyler Cowan as well as Jayant Ray and participants at the NBER conference for helpful comments. We also acknowledge research assistance from Haaris Mateen as well as financial support from the Institute for New Economic Thinking (INET) and the Rewriting the Rules project at the Roosevelt Institute, supported by the Ford and Open Society Foundations, and the Bernard and Irene Schwartz Foundation. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.

show abstract

The Basic AI Drives

Cited by 61 publications

References 9 publications

Socio-cognitive biases in folk AI ethics and risk discourse

Socio-cognitive biases in folk AI ethics and risk discourse

Ethics of Artificial Intelligence

Artificial Intelligence and Its Implications for Income Distribution and Unemployment

Contact Info

Product

Resources

About