Classification green coffee beans is one of the main tasks during the quality grading process. This evaluation is normally carried out by specialist doing a visual inspection or using traditional instruments which have some limitations. This work is focused on the implementation of a computer vision system combining a hardware prototype and a software module. The hardware was made to guarantee the controlled conditions to capture the images of green coffee beans, the software is based on computer vision algorithms in order to detect defects of the coffee beans. The novelty of our proposal is the combination of algorithms to enhance the accuracy and the high number of defects detected. We applied a White Patch algorithm as an image enhancement procedure, color histograms as feature extractor and Support Vector Machine (SVM) for the classification task. It was constituted an image beans database of 1930 instances, and it was extracted 768 features, finally, the model was applied over 13 categories of defects described by the Specialty Coffee Association of America (SCAA). Results of classification achieved a 98.8% of overall accuracy detection, therefore the proposed system proved to be effective in classifying physical defects of green coffee beans. With this work we showed that the grading green coffee process can be automatized, adding a new paradigm in quality evaluation task to enhance the coffee industry.