TimeCLR: A self-supervised contrastive learning framework for univariate time series representation

Yang, Xinyu; Zhen-guo, Zhang; Cui, Rongyi

doi:10.1016/j.knosys.2022.108606

Cited by 35 publications

(16 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, in time series pre-training, many designs of positive and negative pairs have been proposed by utilizing the invariant properties of time series. Concretely, to make the representation learning seamlessly related to temporal variations, TimCLR (Yang et al, 2022) adopts the DTW (Mueen & Keogh, 2016) to generate phase-shift and amplitudechange augmentations, which is more suitable for time series. TS2Vec (Yue et al, 2022) splits multiple time series into several patches and further defines the contrastive loss in both instance-wise and patch-wise aspects.…”

Section: Related Workmentioning

confidence: 99%

SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling

Dong¹,

Wu²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Time series analysis is widely used in extensive areas. Recently, to reduce labeling expenses and benefit various tasks, self-supervised pre-training has attracted immense interest. One mainstream paradigm is masked modeling, which successfully pre-trains deep models by learning to reconstruct the masked content based on the unmasked part. However, since the semantic information of time series is mainly contained in temporal variations, the standard way of randomly masking a portion of time points will ruin vital temporal variations of time series seriously, making the reconstruction task too difficult to guide representation learning. We thus present SimMTM, a Simple pre-training framework for Masked Time-series Modeling. By relating masked modeling to manifold learning, SimMTM proposes to recover masked time points by the weighted aggregation of multiple neighbors outside the manifold, which eases the reconstruction task by assembling ruined but complementary temporal variations from multiple masked series. SimMTM further learns to uncover the local structure of the manifold helpful for masked modeling. Experimentally, SimMTM achieves state-of-theart fine-tuning performance in two canonical time series analysis tasks: forecasting and classification, covering both in-and cross-domain settings.

show abstract

Section: Related Workmentioning

confidence: 99%

SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling

Dong¹,

Wu²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…CoST (Woo et al, 2022) utilizes both time domain and frequency domain contrastive losses to learn disentangled seasonal-trend representations of TS. TimeCLR (Yang et al, 2022) introduces phase-shift and amplitude change augmentations, which are data augmentation methods based on DTW. TF-C (Zhang et al, 2022) learns both time-and frequencybased representations of TS and proposes a novel time-frequency consistency architecture.…”

Section: Related Workmentioning

confidence: 99%

“…We conduct experiments on transfer learning for classification in in-domain and cross-domain settings which are used in previous works (Zhang et al, 2022;Eldele et al, 2021;Dong et al, 2023), by adopting our SoftCLT to TS-TCC and CA-TCC. As baseline methods, we consider TS-SD (Shi et al, 2021), TS2Vec (Yue et al, 2022), Mixing-Up (Wickstrøm et al, 2022), CLOCS (Kiyasseh et al, 2021), CoST (Woo et al, 2022), LaST (Wang et al, 2022), TF-C (Zhang et al, 2022), TS-TCC (Eldele et al, 2021), TST (Zerveas et al, 2021) and SimMTM (Dong et al, 2023).…”

Section: Transfer Learningmentioning

confidence: 99%

“…(1) SleepEEG (Kemp et al, 2000) dataset contains EEG recordings of 153 whole-night sleep sessions from 82 healthy individuals. We segmented the EEG signals using a non-overlapping approach, following the same preprocessing method as (Zhang et al, 2022), to obtain 371,055 univariate brainwaves, each sampled at 100 Hz and categorized into one of five sleep stages: Wake, Non-rapid eye movement (3 sub-states), and Rapid Eye Movement. When using SleepEEG dataset as a source dataset in transfer learning task, we use cosine similarity instead of DTW due to the property of EEG datasets (Li et al, 2022).…”

Section: Ethics Statementmentioning

confidence: 99%

See 1 more Smart Citation

Effect of a Boarding Restriction Protocol on Emergency Department Crowding

et al. 2022

View full text Add to dashboard Cite

Purpose Access block due to the lack of hospital beds causes crowding of emergency departments (ED). We initiated the “boarding restriction protocol” that limits the time of stay in the ED for patients awaiting hospitalization to 24 hours from arrival. The purpose of this study was to determine the effect of the boarding restriction protocol on ED crowding. Materials and Methods The primary outcome was ED occupancy rate, which was calculated as the ratio of the number of occupying patients to the total number of ED beds. Time factors, such as length of stay (LOS), treatment time, and boarding time, were investigated. Results The mean of the ED occupancy rate decreased from 1.532±0.432 prior to implementation of the protocol to 1.273±0.353 after ( p <0.001). According to time series analysis, the absolute effect caused by the protocol was -0.189 (-0.277 to -0.110) ( p =0.001). The proportion of patients with LOS exceeding 24 hours decreased from 7.6% to 4.0% ( p <0.001). Among admitted patients, ED LOS decreased from 770.7 (421.4–1587.1) minutes to 630.2 (398.0–1156.8) minutes ( p <0.001); treatment time increased from 319.6 (198.5–482.8) minutes to 344.7 (213.4–519.5) minutes ( p <0.001); and boarding time decreased from 298.9 (109.5–1149.0) minutes to 204.1 (98.7–545.7) minutes ( p <0.001). In pre-protocol period, boarding patients accumulated in the ED during the weekdays and resolved on Friday, but this pattern was alleviated in post-period. Conclusion The boarding restriction protocol was effective in alleviating ED crowding by reducing the accumulation of boarding patients in the ED during the weekdays.

show abstract

“…The existing methods can be summarized in the below groups. JPEG compression artifacts, 12,13 edge inconsistencies, 14 color consistency, 15 visual similarity, 9,16 EXIF inconsistency, 17 camera model 18,19 and noise-pattern. [20][21][22] Specifically, most current methods focus on mining contextual features in all kinds of ways between images, ignoring the potential context relation information within images.…”

Section: Introductionmentioning

confidence: 99%

Center region aware cross former for image forgery localization

Wu,

Gong,

Zhang

et al. 2024

Fifteenth International Conference on Signal Processing Systems (ICSPS 2023)

View full text Add to dashboard Cite

Image forgery has been a serious issue in real life during this boosting big data era. There have been many methods depending on detecting footprints such as edge inconsistency, camera noise, JPEG artifacts, etc., proposed. Unlike the existing methods, we propose a more general method, CRAC-formerNet to capture more stable manipulation traces, not only including RGB domain but also from high-frequency features generated from Steganalysis Rich Model (SRM) 1 and District Cosine transformer in the frequency domain. A shared query is generated from features from both the RGB and frequency domains. The keys and values are generated from each domain respectively. Moreover, features of manipulated and manipulated regions should belong to different distributions, especially features in edge regions. We use a center contrastive loss to learn fine-grained features. A manipulated region proposal module is proposed for improving computing efficiency when calculating the loss. We empirically demonstrate that our method achieves competitive results on four benchmark image manipulation datasets.

show abstract

TimeCLR: A self-supervised contrastive learning framework for univariate time series representation

Cited by 35 publications

References 22 publications

SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling

SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling

Effect of a Boarding Restriction Protocol on Emergency Department Crowding

Center region aware cross former for image forgery localization

Contact Info

Product

Resources

About