A usual target of automatic assessment in CS education has been a computer program or some other software engineering product. In this paper, we describe the 'PARSI' tool that is used for the automatic assessment of stylistic and technical correctness of office documents and some basic IT skills such as email netiquette and email lists usage. The PARSI tool enables us to run a highly populated course with only one teacher, and investigate how we could make such a course a flexible learning experience where learners take an active role.We contextualize the Parsi tool and our goals in Section 2. Section 3 outlines the course context in which we use the tool. The functional idea and the usage of the Parsi tool are described in Section 4, and our experiences with the tool are summarized in Section 5. Concluding remarks and suggestions for future work are presented in Section 6.