To develop robust algorithms for automated diagnosis of medical conditions such as cardiac arrhythmias, researchers require large collections of data with human expert annotations. Currently, there is a lack of accessible, open-source platforms for human experts to collaboratively develop these annotated datasets through a web interface. In this work, we developed a flexible, generalizable, web-based framework to enable multiple users to create and share annotations on multi-channel physiological waveforms. Using the developed annotation platform, we carried out a pilot study to assess the validity of ventricular tachycardia (VT) alarms from multiple commercial monitors. Thus far, four clinical experts have used this annotation tool to annotate a total of 5,658 VT alarm events, among which approximately 44% (N=2,468) have been labeled by two independent annotators.