A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting

Abstract

Efficient exploration of design spaces is highly sought after in engineering applications. A spectrum of tools has been proposed to deal with the computational difficulties associated with such problems. In the context of our case study, these tools can be broadly classified into optimization and supervised learning approaches. Optimization approaches, while successful, are inherently data inefficient, with evolutionary optimization-based methods being a good example. This inefficiency stems from data not being reused from previous design explorations. Alternately, supervised learning-based design paradigms are data efficient. However, the quality of ensuing solutions depends heavily on the quality of data available. Furthermore, it is difficult to incorporate physics models and domain knowledge aspects of design exploration into pure-learning-based methods. In this work, we formulate a reinforcement learning (RL)-based design framework that mitigates disadvantages of both approaches. Our framework simultaneously finds solutions that are more efficient compared with supervised learning approaches while using data more efficiently compared with genetic algorithm (GA)-based optimization approaches. We illustrate our framework on a problem of microfluidic device design for flow sculpting, and our results show that a single generic RL agent is capable of exploring the solution space to achieve multiple design objectives. Additionally, we demonstrate that the RL agent can be used to solve more complex problems using a targeted refinement step. Thus, we address the data efficiency limitation of optimization-based methods and the limited data problem of supervised learning-based methods. The versatility of our framework is illustrated by utilizing it to gain domain insights and to incorporate domain knowledge. We envision such RL frameworks to have an impact on design science.

Design Optimization of a Pneumatic Soft Robotic Actuator Using Model-Based Optimization and Deep Reinforcement Learning

“…Among those, in Viquerat et al (2021), proximal policy optimization (PPO) was used in shape optimization, where indirect supervision from a generic reward signal is used as a non-linear optimizer. Deep Q-Network (DQN) was shown to be successful in optimizing the design of the angle of attack of airfoils (Yonekura and Hattori, 2019) and similarly, double-DQN with hindsight experience replay (HER) demonstrated good performance in design optimization of microfluidic devices for flow sculpting (Lee et al, 2019). However, utilizing machine learning for soft robot design has not been fully explored and our work complements the existing research in this field.…”

A Data-Driven Methodology to Improve Tolerance Allocation Using Product Usage Data

“…But the above two assembly variation analysis methods cannot establish the analytical relationship between deviations. Machine learning method could be a useful supplement for engineering design [27,28]. Since parts in the assembly have a relationship with each other and the essence of a graph is the relationship, the machine learning methods on the graph could be used to predict the unmeasured deviation in the assembly.…”

Explainable deep convolutional learning for intuitive model development by non–machine learning domain experts

“…Therefore, it is assumed that a highdimensional building design space can be represented in a low-dimensional space; exploring the low-dimensional representations of the DL model should then highlight interpretations that can be linked to the design space and energy distribution. Methods for dimensionality reduction for model explainability include principal components analysis (Lee et al, 2019;Singaravel, Suykens & Geyer 2019), or t-Distribution Stochastic Neighbour Embedding (t-SNE) (van der Maaten & Hinton 2008). In this paper, low-dimensional representations are obtained through t-SNE.…”