Objective: This study proposes a novel technique for atrial fibrillatory waves (f-waves) extraction and investigates the performance of the proposed method comparing with different f-wave extraction methods. Approach: We propose a novel technique combining a periodic component analysis (PiCA) and echo state network (ESN) for f-waves extraction, denoted PiCA-ESN. PiCA-ESN benefits from the advantages of using both source separation and nonlinear adaptive filtering. PiCA-ESN is evaluated by comparing with other state-of-the-art approaches, which include template subtraction technique based on principal component analysis, spatiotemporal cancellation, nonlinear adaptive filtering using an echo state neural network, and a source separation technique based on PiCA. Quality assessment is performed on a recently published reference database including a large number of simulated ECG signals in atrial fibrillation (AF). The performance of the f-wave extraction methods is evaluated in terms of signal quality metrics (SNR, ΔSNR) and robustness of f-wave features. Main results: The proposed method offers the best signal quality performance, with a ΔSNR of approximately 22 dB across all 8 sets of the reference database, as well as the most robust extraction of f-wave features, with 75% of all estimates of dominant atrial frequency well below 1 Hz.