Whilst widely accepted as an important facet of Open Source technologies and their application, the scientific evaluation of such technologies and systems is often underexplored in research. This work presents an integrated approach of developing a prototype Environmental Monitoring System (EMS) based on open source hardware & software, and the system's reliability in terms of data accuracy. The system is able to measure nine environmental parameters: rainfall, temperature, barometric pressure, humidity, wind speed, wind direction, solar radiation, soil moisture and water level. Opposing to sophisticated and conventional environmental data collection systems such as automatic and manual weather stations belong to Meteorological and Irrigation department of Sri Lanka, this research has shown a promising way of establishing a dense coverage to monitor the environmental phenomena in a more cost effective manner.