TEI and Git in ParlaMint: Collaborative Development of Language Resources

Abstract

This paper discusses the encoding, validation and development of language resources of the completed ParlaMint I and on-going ParlaMint II CLARIN projects, which centre on the collaborative development of a large set of interoperable corpora of parliamentary proceedings. It focuses on the ParlaMint encoding and the GitHub development platform and the evaluation of their use by project partners. We introduce the use of TEI ODD for the encoding guidelines and validation schemas. We motivate and explain using Git to develop and maintain the encoding schemas, validation and conversion scripts and valid samples of the corpora. Apart from revision control, issues, and publishing documentation, the GitHub platform also supports integrating code execution with pull requests, which significantly automates the data submission process. The paper also presents the results of a survey on the use of TEI and Git in the ParlaMint projects among the project participants. Overall, participants were mostly positive about their experiencewith TEI and Git, although some difficulties were reported. Some partners also expressed doubts about whether the current scheme is flexible enough to support encoding the unique features of sometimes drastically different parliamentary systems. On the other hand, reactions to Git were very positive in terms of information and feedback received via GitHub Issues, effectiveness in the communication process, and plans to use Git in the future. However, there was less agreement on whether the requirements and workflows were adequately explained.The reported difficulties will serve as a basis for further Git workflow optimisation in ParlaMint.