Challenges in Deploying Machine Learning: A Survey of Case Studies

Paleyes, Andrei; Urma, Raoul-Gabriel; Lawrence, Neil D.

doi:10.1145/3533378

Cited by 199 publications

(128 citation statements)

References 100 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While numerous studies have explored stock market applications of DL, we focused on those that demonstrate evidence of research methodology consistent with the domain and thus more likely to be considered by industry practitioners (Paleyes et al 2020;The Institute for Ethical AI & Machine Learning 2020;Gundersen et al 2018). In following this approach, it was hoped that this survey might serve as a basis for future research answering similar questions.…”

Section: Discussionmentioning

confidence: 99%

“…However, more work is needed to ensure domain-specific metrics and considerations are used to assess applicability and usability across diverse ML domains. Paleyes et al (2020) suggest practical consideration in deploying ML for production use: "The ability to interpret the output of a model into understandable business domain terms often plays a critical role in model selection, and can even outweigh performance consideration." For example, Nascita et al (2021) fully embraces XAI paradigms of trustworthiness and interpretability to classify data generated by mobile devices using DL approaches.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning in the stock market—a systematic survey of practice, backtesting, and applications

Olorunnimbe

Viktor

2022

Artif Intell Rev

View full text Add to dashboard Cite

The widespread usage of machine learning in different mainstream contexts has made deep learning the technique of choice in various domains, including finance. This systematic survey explores various scenarios employing deep learning in financial markets, especially the stock market. A key requirement for our methodology is its focus on research papers involving backtesting. That is, we consider whether the experimentation mode is sufficient for market practitioners to consider the work in a real-world use case. Works meeting this requirement are distributed across seven distinct specializations. Most studies focus on trade strategy, price prediction, and portfolio management, with a limited number considering market simulation, stock selection, hedging strategy, and risk management. We also recognize that domain-specific metrics such as “returns” and “volatility” appear most important for accurately representing model performance across specializations. Our study demonstrates that, although there have been some improvements in reproducibility, substantial work remains to be done regarding model explainability. Accordingly, we suggest several future directions, such as improving trust by creating reproducible, explainable, and accountable models and emphasizing prediction of longer-term horizons—potentially via the utilization of supplementary data—which continues to represent a significant unresolved challenge.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep learning in the stock market—a systematic survey of practice, backtesting, and applications

Olorunnimbe

Viktor

2022

Artif Intell Rev

View full text Add to dashboard Cite

show abstract

“…This leads to many business process artifacts that need to be tracked. The cyclicality of analytical systems development processes is related to the large number of experiments required to build an accurate model [6,10]. During the development process, the data scientist may test the applicability of multiple data processing algorithms and must use many input features to produce a result.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, the large number of experiments generates even more artifacts and metadata of the business process. Once an analytical system is built, it goes through verification and implementation phases [10], which can also lead to changes in business processes. Meanwhile, an important part of the life cycle of ML models is monitoring and modification of models in case deviations are identified [10], which leads the model to return to the previous phases within the business processes.…”

Section: Related Workmentioning

confidence: 99%

Actor-oriented approach for business-process management of analytical system development

Pribylev¹,

Zaytsev²,

Vikentieva³

2022

Proceedings of ISP RAS

View full text Add to dashboard Cite

This paper aims at investigating the feasibility of an actor-oriented approach for modelling analytical systems development business processes. The study analyzes existing management challenges of analytical systems development processes, identifies key business process modeling approaches, and proposes a modeling approach based on actor-oriented approach with high flexibility and enhanced control over business artifacts. The article also describes examples of possible applications of this approach in a business process management tool.

show abstract

“…An established problem in practical applications of machine learning, often referred to as 'high variance' data[18].…”

mentioning

confidence: 99%

A sampling-based approach for information-theoretic inspection management

et al. 2022

View full text Add to dashboard Cite

A partially supervised approach to Structural Health Monitoring is proposed, to manage the cost associated with expert inspections and maximize the value of monitoring regimes. Unlike conventional data-driven procedures, the monitoring classifier is learnt online while making predictions—negating the requirement for complete data before a system is in operation (which are rarely available). Most critically, periodic inspections are replaced (or enhanced) by an automatic inspection regime, which only queries measurements that appear informative to the evolving model of the damage-sensitive features. The result is a partially supervised Dirichlet process clustering that manages expert inspections online given incremental data. The method is verified on a simulated example and demonstrated on in situ bridge monitoring data.

show abstract

Challenges in Deploying Machine Learning: A Survey of Case Studies

Cited by 199 publications

References 100 publications

Deep learning in the stock market—a systematic survey of practice, backtesting, and applications

Deep learning in the stock market—a systematic survey of practice, backtesting, and applications

Actor-oriented approach for business-process management of analytical system development

A sampling-based approach for information-theoretic inspection management

Contact Info

Product

Resources

About