It is very difficult to measure the effectiveness of social media marketing, and such measurement has not been done in any systematic way. This study shows how to develop a measurement instrument to evaluate consumer comments made on a social media community site to test their influence on consumer purchasing decisions. Comments on a Samsung Mobile Thailand Facebook fan page were categorized and recorded on a daily basis for a period of 5 months. A model was developed and tested using confirmatory factor analysis to determine how comments representing four variables affected purchasing decisions. The study found that the most influential comments focused on the following: usage experience issues, information requests, business practice issues and comments about product launches and developments. These four variables can be indicator variables for consumer purchasing decisions in social media marketing. Marketers can apply these four key indicator variables to evaluate social media marketing.