Temporal Fusion Transformer for forecasting vital sign trajectories in intensive care patients

Phetrittikun, Ratchakit; Suvirat, Kerdkiat; Pattalung, Thanakron Na; Kongkamol, Chanon; Ingviya, Thammasin; Chaichulee, Sitthichok

doi:10.1109/bmeicon53485.2021.9745215

Cited by 5 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We employed five multi-step, multivariate baselines, ranging from a simple baseline to state-of-the-art forecasting models. Specifically, we used a naïve model that repeats the last observed value, a linear regression model, a time series LightGBM model, a Temporal Fusion Transformer [10,28] as well as a TiDE model [29]. These models were selected due to being able to handle future covariates and have been shown to achieve state-of-the-art results in both medical and standard time series forecasting [30,31].…”

Section: Baseline Modelsmentioning

confidence: 99%

“…Generative artificial intelligence (AI) holds promises for creating digital twins due to its potential to produce synthetic yet realistic data, but this area of application is still in its infancy [4]. Generative AI methods for predicting patient trajectories include recurrent neural networks [5,6,7,8], transformers [9,10] and stable diffusion [11]. These often fall short in terms of handling missing data, interpretability and performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Large Language Models forecast Patient Health Trajectories enabling Digital Twins

Makarov,

Bordukova,

Rodriguez-Esteban

et al. 2024

Preprint

View full text Add to dashboard Cite

BackgroundGenerative artificial intelligence (AI) facilitates the development of digital twins, which enable virtual representations of real patients to explore, predict and simulate patient health trajectories, ultimately aiding treatment selection and clinical trial design, among other applications. Recent advances in forecasting utilizing generative AI, in particular large language models (LLMs), highlights untapped potential to overcome real-world data (RWD) challenges such as missingness, noise and limited sample sizes, thus empowering the next generation of AI algorithms in healthcare.MethodsWe developed the Digital Twin - Generative Pretrained Transformer (DT-GPT) model, which leverages biomedical LLMs using rich electronic health record (EHR) data. Our method eliminates the need for data imputation and normalization, enables forecasting of clinical variables, and prediction exploration via a chatbot interface. We analyzed the method’s performance on RWD from both a long-term US nationwide non-small cell lung cancer (NSCLC) dataset and a short-term intensive care unit (MIMIC-IV) dataset.FindingsDT-GPT surpassed state-of-the-art machine learning methods in patient trajectory forecasting on mean absolute error (MAE) for both the long-term (3.4% MAE improvement) and the short-term (1.3% MAE improvement) datasets. Additionally, DT-GPT was capable of preserving cross-correlations of clinical variables (average R2of 0.98), and handling data missingness as well as noise. Finally, we discovered the ability of DT-GPT both to provide insights into a forecast’s rationale and to perform zero-shot forecasting on variables not used during the fine-tuning, outperforming even fully trained, leading task-specific machine learning models on 14 clinical variables.InterpretationDT-GPT demonstrates that LLMs can serve as a robust medical forecasting platform, empowering digital twins that are able to virtually replicate patient characteristics beyond their training data. We envision that LLM-based digital twins will enable a variety of use cases, including clinical trial simulations, treatment selection and adverse event mitigation.

show abstract

Section: Baseline Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Large Language Models forecast Patient Health Trajectories enabling Digital Twins

Makarov,

Bordukova,

Rodriguez-Esteban

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…This new method is an attention based network. TFT is already used in a number of areas for time series forecasting, like meteorology (Wu et al, 2022), medicine (Phetrittikun et al, 2021) and the stock market (Hu, 2021). While there will be some commonalities in input data with forecasting in meteorology, right now there is no research about the performance of TFT for energy demand forecasting.…”

Section: Related Workmentioning

confidence: 99%

Temporal Fusion Transformer for thermal load prediction in district heating and cooling networks

Behrens¹,

Leiprecht²,

Brantl³

et al. 2022

Linköping Electronic Conference Proceedings

View full text Add to dashboard Cite

Accurate forecasting of thermal loads is a critical factor for operating district heating and cooling networks economically, efficiently and with minimized emissions. If thermal loads are known with high accuracy in advance, use of renewable energies can be maximized, and fossil generation, in particular in peaking units, can be avoided. Machine learning has already proven to be an efficient tool for time series forecasting in this context. One recent advancement in machine learning is the "Temporal Fusion Transformer" (TFT), which shows especially good results in the area of time series forecasting. This paper examines the performance of TFT in the concrete context of thermal load forecasting for district heating and cooling networks. First, a brief summary of differences between TFT and other machine learning methods is given. Secondly, it is described how the method can be adopted to train a machine learning model for thermal load forecasting. The data to train and evaluate the neural network is based on 8 years of hourly operating data made available from the district heating network of the city of Ulm in Germany. The presented technique is used to produce 72 hours of heating load forecasts for three different district heating grids in the city of Ulm. The results are compared to forecasts of other machine learning methods that have been previously made as part of the publicly funded research project "deepDHC", in order to evaluate if TFT is an improvement to further reduce forecasting uncertainties.

show abstract

“…However, despite the promising developments in stress detection, the exploration of forecasting future states remains limited. Some research has been conducted on forecasting vital signs in intensive care patients [9], postoperative complications [10], or in health monitoring [11]. In [11], the authors compared different models, evaluating their accuracy in univariate forecasts of pulse, oxygen level percentage (SpO2), and blood pressure.…”

Section: Introductionmentioning

confidence: 99%

Forecasting Vital Signs in Human–Robot Collaboration Using Sequence-to-Sequence Models with Bidirectional LSTM: A Comparative Analysis of Uni- and Multi-Variate Approaches

Chojnowski,

Luipers,

Neef

et al. 2023

Ecsa 2023

View full text Add to dashboard Cite

show abstract

Temporal Fusion Transformer for forecasting vital sign trajectories in intensive care patients

Cited by 5 publications

References 6 publications

Large Language Models forecast Patient Health Trajectories enabling Digital Twins

Large Language Models forecast Patient Health Trajectories enabling Digital Twins

Temporal Fusion Transformer for thermal load prediction in district heating and cooling networks

Forecasting Vital Signs in Human–Robot Collaboration Using Sequence-to-Sequence Models with Bidirectional LSTM: A Comparative Analysis of Uni- and Multi-Variate Approaches

Contact Info

Product

Resources

About