Controlled experimentation in continuous experimentation: Knowledge and challenges

Ros, Rasmus; Kaltenbrunner, Lukas; Runeson, Per; Felderer, Michael

doi:10.1016/j.infsof.2021.106551

Cited by 17 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Historically used by medical scientists to figure out if a drug works or not by comparing it to a placebo or other medicine, A/B testing, or controlled experimentation (CE), is "the test of an idea or a hypothesis in which variables are systematically changed to isolate the effects" (Auer, Ros, Kaltenbrunner, Runeson & Felderer, 2021), and is already common practice for big tech (Gallo, 2017). From perfecting online website design to creating new products, the method is useful for deriving information about the effects of a change.…”

Section: Controlled Experimentationmentioning

confidence: 99%

See 1 more Smart Citation

Redesigning tax debt collection process in Brazil: process automation & nudging for tax compliance

Labarrère,

Ribeiro,

Lima

et al. 2024

DyO

View full text Add to dashboard Cite

Propósito: Este estudio tiene como objetivo contribuir al conocimiento sobre el problema del cumplimiento tributario en Brasil al probar si Business Process Management (BPM) y el Nudge pueden ayudar a mejorar la recaudación de la deuda tributaria. El objetivo es evaluar si este enfoque mixto es efectivo y si puede implementar una transformación duradera en la administración tributaria brasileña.Diseño/metodología/enfoque: Este artículo presenta un estudio de caso orientado a la práctica que utiliza la metodología Business Process Transformation (BPT) para rediseñar y automatizar un proceso de cobro de deudas tributarias aplicando los conocimientos de la teoría del Nudge y luego prueba su efectividad usando experimentación controlada o prueba A/B.Hallazgos: al personalizar los parámetros para la mensajería automatizada con los contribuyentes en función del diseño de comportamiento y las pruebas A/B de los grupos, los autores pudieron identificar un aumento del comportamiento de cumplimiento del 2,90 % junto con un aumento de ingresos estimado de U$ 4 millones (P = 0,01). La evaluación posterior mostró que las nuevas prácticas eran duraderas, lo que indicaba el éxito de los métodos adoptados.Originalidad/valor: este estudio agrega nuevos conocimientos sobre la efectividad de BPT para la administración tributaria brasileña, reafirma cómo el Nudge puede apoyar el cumplimiento tributario en este contexto y ofrece información sobre la combinación de estos diferentes campos de conocimiento.

show abstract

Section: Controlled Experimentationmentioning

confidence: 99%

“…This study adopted a framework for controlled experimentation as proposed by Auer et al (2021), which consists of five main phases: ideation, experiment design, implementation, execution and analysis.…”

Section: Controlled Experimentationmentioning

confidence: 99%

Redesigning tax debt collection process in Brazil: process automation & nudging for tax compliance

Labarrère,

Ribeiro,

Lima

et al. 2024

DyO

View full text Add to dashboard Cite

show abstract

“…Two of them were designed for specific software contexts [Eklund and Bosch 2012] [Mattos et al 2020b, and the others are for general purposes. Table 5 shows these eleven processes classified according to the five phases of experimentation presented by [Auer et al 2021], identifying whenever a phase is cited superficially, detailed, or not addressed in the model. We consider that a stage is detailed if more than one step in that phase or if the one-step has associated characteristics.…”

Section: Continuous Experimentation Processesmentioning

confidence: 99%

“…It has been widely implemented by several big companies, mainly in web and mobile software systems. Large organizations such as Facebook, Google, Microsoft, LinkedIn, and Netflix have reported their experience using CE to evolve their software products [Auer et al 2021].…”

Section: Introductionmentioning

confidence: 99%

“…Beyond CE, different expressions have been proposed to refer to this practice in the technical literature, such as Data-Driven Development (DDD) [Bosch and Olsson 2017], Innovation Experiment System (IES) [Bosch 2012], Online Controlled Experiments (OCE) [Kohavi et al 2013], among others. As a result, the studies report countless processes and strategies from the vast array of expressions [Auer et al, 2021]. Therefore, it is not easy to observe a common understanding regarding continuous experimentation in software development.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Literature Study to Characterize Continuous Experimentation in Software Engineering

Erthal

Souza²,

Santos³

et al. 2022

Anais Do XXV Congresso Ibero-Americano Em Engenharia De Software (CIbSE 2022)

View full text Add to dashboard Cite

Continuous Experimentation (CE) has become increasingly popular across industry and academic communities. Given its rapid evolution in software engineering (SE), the lack of a common understanding of CE can jeopardize new implementations and justify research efforts. Therefore, this literature study characterizes CE in SE based on its definitions, processes, and strategies for experimentation available in the technical literature. Seventy-six sources of information provided many different definitions, processes, and experimental procedures used to describe CE in SE. Despite the increasing use of CE in SE, it is impossible to observe a common terminology yet to support its characterization and use.

show abstract

Reference Architecture for Running Large Scale Data Integration Experiments

Bodziony

Wrembel

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Controlled experimentation in continuous experimentation: Knowledge and challenges

Cited by 17 publications

References 32 publications

Redesigning tax debt collection process in Brazil: process automation & nudging for tax compliance

Redesigning tax debt collection process in Brazil: process automation & nudging for tax compliance

A Literature Study to Characterize Continuous Experimentation in Software Engineering

Reference Architecture for Running Large Scale Data Integration Experiments

Contact Info

Product

Resources

About