The Data Engineer

Tamir, Mike; Miller, Steven; Gagliardi, Alessandro

doi:10.2139/ssrn.2762013

Cited by 3 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the number of jobs for Data Engineering is increasing, there is very little up-to-date peerreviewed research available that defines the job responsibilities and/or technical skills of a Data Engineer (Saltz, Yilmazel, & Yilmazel, 2016). Below is a list of the job responsibilities for a Data Engineer by Tamir, Miller, and Gagliardi (2015):…”

Section: What Are Th E Job Responsibilities And/or Technical Skills Needed By Data Engineers?mentioning

confidence: 99%

Changing Paradigms of Technical Skills for Data Engineers

Mason

2018

IISIT

View full text Add to dashboard Cite

Aim/Purpose: This paper investigates the changing paradigms for technical skills that are needed by Data Engineers in 2018. Background: A decade ago, data engineers needed technical skills for Relational Database Management Systems (RDBMS), such as Oracle and Microsoft SQL Server. With the advent of Hadoop and NoSQL Databases in recent years, Data Engineers require new skills to support the large distributed datastores (Big Data) that currently exist. Job demand for Data Scientists and Data Engineers has increased over the last five years. Methodology: This research methodology leveraged the Pig programming language that used MapReduce software located on the Amazon Web Services (AWS) Cloud. Data was collected from 100 Indeed.com job advertisements during July of 2017 and then was uploaded to the AWS Cloud. Using MapReduce, phrases/words were counted and then sorted. The sorted phrase / word counts were then leveraged to create the list of the 20 top skills needed by a Data Engineer based on the job advertisements. This list was compared to the 20 top skills for a Data Engineer presented by Stitch that surveyed 6,500 Data Engineers in 2016. Contribution: This paper presents a list of the 20 top technical skills required by a Data Engineer.

show abstract

Section: What Are Th E Job Responsibilities And/or Technical Skills Needed By Data Engineers?mentioning

confidence: 99%

Changing Paradigms of Technical Skills for Data Engineers

Mason

2018

IISIT

View full text Add to dashboard Cite

show abstract

Creating meaningful work in the age of AI: explainable AI, explainability, and why it matters to organizational designers

Wulff

Finnestrand

2023

AI & Soc

View full text Add to dashboard Cite

In this paper, we contribute to research on enterprise artificial intelligence (AI), specifically to organizations improving the customer experiences and their internal processes through using the type of AI called machine learning (ML). Many organizations are struggling to get enough value from their AI efforts, and part of this is related to the area of explainability. The need for explainability is especially high in what is called black-box ML models, where decisions are made without anyone understanding how an AI reached a particular decision. This opaqueness creates a user need for explanations. Therefore, researchers and designers create different versions of so-called eXplainable AI (XAI). However, the demands for XAI can reduce the accuracy of the predictions the AI makes, which can reduce the perceived usefulness of the AI solution, which, in turn, reduces the interest in designing the organizational task structure to benefit from the AI solution. Therefore, it is important to ensure that the need for XAI is as low as possible. In this paper, we demonstrate how to achieve this by optimizing the task structure according to sociotechnical systems design principles. Our theoretical contribution is to the underexplored field of the intersection of AI design and organizational design. We find that explainability goals can be divided into two groups, pattern goals and experience goals, and that this division is helpful when defining the design process and the task structure that the AI solution will be used in. Our practical contribution is for AI designers who include organizational designers in their teams, and for organizational designers who answer that challenge.

show abstract