Chokanan Mango Sweetness Determination using HSB Color Space

Siti Khairunniza-Bejo Department of Biological and Agricultural Engineering

Faculty of Engineeing Universiti Putra Malaysia

43400 Serdang Selangor, Malaysia Email:skbejo@ieee.org

Syahidah Kamarudin Department of Biological and Agricultural Engineering

Faculty of Engineeing Universiti Putra Malaysia

43400 Serdang Selangor, Malaysia Email:ai_berg8@yahoo.com

Abstract—The objective of this research is to determine Chokanan mango sweetness by using color properties. Keyence machine vision system was used to capture mango images in HSB color space. Meanwhile, Digital AR2008 Abbe refractometer was used to obtain the value of sweetness. The process was divided into two major stages, the training stage and the testing stage. One hundred and eighty Chokanan mango images were used in this research. Fifty percent of the images were used during training stage and the other 50% were used during testing stage. In the training stage, the hue, saturation and brightness bands were analysed by using linear regression analysis. Based on the result, it has been shown that hue gave the highest value of correlation (-0.92). It also gave the lowest value of standard deviation when compared to the other bands. Therefore, it has been used as a model to determine the sweetness of Chokanan mango images. The results showed that the developed model can determine sweetness at Index 1 and Index 2 with 100% of accuracy and Index 3 with 87% of accuracy. In average, it can be used to determine the sweetness of Chokanan mango with 95.67% of accuracy.

Keywords-component; Chokanan; sweetness; HSB; Hue;

I. INTRODUCTION Mango (Mangifera indica) is one of the most popular

tropical fruits natural to South-East Asia. It has been cultivated for over 4000 years during which it has spread to other tropical and sub-tropical countries. Mangoes are full packed with vitamins, minerals and anti-oxidants and contain like all fruits very few proteins, fats and calories. They are perfect to replenish salts, vitamins and energy after physical exercise. There are some mango varieties which are popular in Malaysia such as MA 162 (Galek), MA 165 (Maha 65), MA 204 (Masmuda), MA 128 (Harumanis) and MA 125 (Chokanan). However, only Harumanis and Chokanan have high commercial potentials. Compared to Harumanis, Chokanan has changed in color while the ripening process. According to Federal Agriculture Marketing Authority (FAMA), Malaysia, there are six maturities levels of mangoes. The most common method used to classify the Chokanan mango maturity is by visual inspection based on its color. However, we don’t have any idea on its index of sweetness.

Fruits and vegetables are important sources of vitamins, minerals, dietary fiber and antioxidants. The relative contribution of each commodity to human health and wellness depends upon its nutritive value and per capita consumption [1]. Consumers nowadays are highly concerned about the fruits and vegetables quality. Their appearance has a major influence on the perceived quality. Color is one of the most important quality parameters in consumers’ preferences. When selecting fruits, consumers normally squeezing the fruits and look at its external appearance and texture. Although the health promotions have been initiated recommending eating at least five portions of fruits and vegetables per day to improve the nutritional quality of diet, it was unsuccessful when the products to be promoted did not meet sensory qualities and consumer expectations. This would mean over the long term, that the consumer will choose fruits and vegetables beyond the mere health-driven argumentation [2]. The work on nondestructive techniques in measuring fruits quality using near-infrared systems has led to commercial use in a packing line situation. However, it is costly and did not practical to be used in the local supermarket.

In this research, the capability of HSB color space to determine Chokanan mango sweetness will be presented. Section II presents literature review of the nondestructive technique of fruit quality evaluation. Section III presents overview of the proposed Chokanan mango sweetness determination. Chapter IV presents methodology in details. Chapter V presents results and discussion, and finally the conclusion will be presented in Section VI.

II. LITERATURE REVIEW The color aspect of visual appearance of the skin can be

measured nondestructively using three types of sensors: colorimeters, spectrophotometers and color machine vision systems [3]. Colorimeters are instruments designed to quantify color in terms of human perception. Colorimeters are broadband instruments that generally divide the information in the visible spectrum into three components similar to the red, green and blue cone cells in the human eye. Spectrophotometers are designed to provide more detailed information about the optical properties of the sample, typically dividing the information in the visible

2011 Third International Conference on Computational Intelligence, Modelling & Simulation

978-0-7695-4562-2/11 $26.00 © 2011 IEEE

DOI 10.1109/CIMSim.2011.45

216

spectrum into fifteen or more components. Colorimeters and spectrophotometers are designed to give a single average reading over a spot on the sample typically ranging in size from 5 to 25 mm in diameter. Although colorimeter and spectrophotometer has been used in measuring maturity of mangoes [4, 5, 6, 7], however, both approaches have limited color sensing capabilities due to low spatial resolution. The instruments also had problems in dealing with fruits and vegetables with non-homogeneous colors. In contrast, color machine vision offers a significantly higher spatial resolution that creates new opportunities for color quality control. It allows color measurement of non-uniform shapes and colors [8, 9]. It can measure color of any sample(s) that fits into the view area of the camera.

Color machine vision involves a camera connected to a computer, controlled lighting and the software to control camera settings, image acquisition, and processing. A charge-coupled device (CCD) has been used as the sensor in the camera. It converts photons to electrical signals. Once an image is captured, it can be immediately evaluated or stored for future analysis and comparisons. Although the used of machine vision to determine fruit sweetness especially in mango is still limited, however the color machine vision has been used to maintain quality and price of products such as apples [10, 11], fresh market peaches [12,13,14], oranges [15], lemon [16], red grapefruit juice [17], peppers [18], cucumbers [19], potatoes [20], tomatoes [21], dates [22], palm oil fresh fruit bunches [23] and beef [24] based on their surface colors.

Color images are represented either by the primary colors red, green and blue (the RGB color model), or by the main factors of human color sensation namely hue, saturation, intensity (the HSI color model). In most color machine vision grading system, the image is first captured in a red, green, and blue (RGB) color components. It is then converted to a hue, saturation, intensity (HSI) representation to improve the consistency of the results. Alternative names of HSI include HSV (value), HSB (brightness), HSL (lightness) etc. Hue is a color attribute that describes a pure color. Saturation gives a measure of the degree to which a pure color is diluted by a white light. A highly saturated color has a low white content. The intensity of a color image corresponds to the gray level version of the image. It has been experimentally found that HSI model is most suitable for finding out the ripeness of fruits and vegetables [20, 25]. It has been used to sort Chokun oranges [15], grading [20, 26] and discriminate russet [27] in ‘Golden Delicious’ apples. The Hue attribute is also independent on intensity changes.

III. THE PROPOSED CHOKANAN MANGO SWEETNESS DETERMINATION

The color of fruit is frequently used as an index of maturity or ripeness [3]. Although external fruit maturity indices can provide approximate information on the internal quality characteristics [28], however the used of this

approach for internal quality evaluation is still limited. The changes in carotenoids, anthocyanins and other flavonoids, betalains, and chlorophylls pigments as fruit develops affect the perception of fruit color [29, 30]. Chokanan has a change of color while the ripening process. Unripe Chokanan appear dull and green peel. Ripe Chokanan appear all peel in yellow color. Furthermore, ripe Chokanan mangoes have a high content of sugar [31]. Based on the previous work [32], the maturity level of Chokanan mango can be determined using its external color. Therefore, it is believed that the sweetness of Chokanan mango at different level of maturity can be determined using color. In this research, the capability of HSB color model to determine Chokanan mango sweetness has been investigated. The performance was evaluated based on the standard deviation, Root Mean Square Error (RMSE) and the value of Pearson’s correlation, together with its significance. A sweetness model was then developed and tested using another set of Chokanan mango images. Detail method will be presented in Section IV.

IV. METHODOLOGY

A. Image Acquisition Basically, Chokanan mangoes are classified into six

grades based on its maturity level. In this research, the mangoes were grouped into three different sweetness indexes as shown in Table 1. Each sweetness index contained two different maturity levels. Keyence machine vision was used to acquire mango images. It consists of CCD camera, LCD, microcontroller and joystick. The experiment was conducted in a room that has a proper lighting condition. The camera was set up horizontally and facing towards the mango with a constant distance of 40 cm as shown in Figure 1.

Figure 1. Image Acquisition.

217

TABLE I. SWEETNESS INDEX NUMBER WITH MATURITY LEVEL

B. Obtaining sweetness value AR2008 Abbe refractometer was used to obtain the

sweetness of mango. The AR2008 has automatic temperature compensation, thermostat connection of prisms and a built-in light source for the measuring prism. The refractive index or Brix value and temperature will be shown on a LCD display. The experiment was conducted on the same day after the mango image being captured under the temperature room of 24.60C. In order to measure sweetness of mango, the mango was cut into slices and put into contact plate of digital refractometer. After that, the knob was closed and locked up tightly to make sure that the sample was on the plate. Figure 2 shows the process of obtaining sweetness value using AR2008 Abbe refractometer.

Figure 2. Obtaining sweetness value using AR2008 Abbe refractometer.

C. Sweetness level estimation Average pixel value taken from 200cm x 200cm window

has been used to represent the mango image. It is calculated by using equation 1:

where is the average pixel value of mango image, xi is

pixel value and n is the total number of pixel inside the window. Linear regression analysis was then used to determine the relationship between average pixel values and the value of sweetness obtained from the refractometer.

D. Performance evaluation The performance of HSB was evaluated based on the

standard deviation, Root Mean Square Error (RMSE) and value of Pearson’s correlation and its significance. Standard deviation of the data in every level of sweetness index was calculated using equation 2:

where S is a standard deviation of the average pixel values in every sweetness index, xi is average pixel values in n sample and is the average value of sample n. A low standard deviation indicates that the data points tend to be very close to the mean.

The RMSE of the determined sweetness was calculated using equation 3:

where yi is the value of sweetness calculated using a developed model and xi is the value of sweetness obtained from a refractometer. A low RMSE indicates high accuracy of the model.

Index No.

Maturity Level

Description Images

1

1

Unripe fruit. Dull and green peel.

2

Mature fruit. Shinny and green peel.

2

3

Green yellowish peel.

4

Yellowish green peel.

3

5

Ripe fruit. All peel in yellow color.

6

Overripe fruit. Yellowish with orange color peel.

218

The Pearson's correlation is used to find a correlation between index of sweetness and image properties. The value for a Pearson's can fall between 0.00 (no correlation) and 1.00 (perfect correlation). Generally, correlations above 0.80 are considered pretty high. In this research, the value of Pearson’s correlation and its significance was calculated using SPSS software.

V. RESULTS AND DISCUSSION Hundred and eighty mango images had been used in this

research (60 mango images at each index of sweetness). This is to make sure that the model is well trained and tested properly. Fifty percent of the images were used as the training dataset and another 50% were used as testing dataset. The average pixel value for HSB mango image inside 200cm x 200cm window was appeared on the camera screen as shown in Figure 3. Meanwhile, the value of mango sweetness has been obtained using AR2008 Abbe refractometer on the same day of the image being captured and has been grouped into three different indexes as shown in Table II.

Statistical analysis of the pixels in hue, saturation and brightness taken from three different indexes of sweetness is tabulated in Table III. Standard deviation was used for variability or diversity measurement. It shows how much variation there is from the mean. In Table III, the lowest value of standard deviation and RMSE in each level are highlighted in grey color. Hue gave the lowest values in all indexes (2.73 at index 1, 6.31 at index 2 and 2.44 at index 3), which indicates that the data points of hue are tend to be very close to the mean. Meanwhile, brightness gave highest value in all indexes (35.15 at index 1, 18.09 at index 2 and 22.79 at index 3), indicating that the data of brightness are spread out over a large range of values. RMSE is a frequently-used measure of the differences between values predicted by a model and the values actually observed from the thing being modeled. Results tabulated in Table III shows that hue gave the lowest values of RMSE at index 2 (0.22) and index 3 (0.03). Although it gave higher value of RMSE at index 1 (0.06) compared to brightness (0.04), however it only gave 0.02 of difference, and at the same time hue gave the smallest value of standard deviation which is 2.73, compared to brightness, which is 35.15. From this table, it can be concluded that hue gave the best performance among the other bands.

Pearson’s correlation has been used to find the correlation between hue, saturation and brightness with the sweetness index of Chokanan mango. Results of Pearson’s correlation as tabulated in Table IV has shown that the entire band gave significant correlation at 0.01 level (2-tailed) i.e. -0.92 (hue), 0.85 (saturation) and 0.66 (brightness). Hue gave negative correlation, while saturation and brightness gave positive correlation. These significance correlations gave good indicator on the possibility of HSB color space to be used to determine Chokanan mango of sweetness.

Figure 4-6 show the linear regression analysis of Chokanan mango of sweetness in hue, saturation and brightness based on 90 mango images. The graph clearly shown that, the value of hue was decreased when the value of sweetness was increased. Meanwhile, the value of saturation and brightness were increased when the value of sweetness increased. Since hue gave the highest value of correlation (-0.92) and the lowest value of standard deviation compared to the other bands, therefore it has been used to model the Chokanan mango of sweetness index determination. The developed model is as follows:

Sweetestimate=-0.19H+20.06 (4) where H is the average hue calculated within 200cm x 200cm windowed image.

Ninety mango images had been used to test the model. The result is shown in Figure 7. From this figure, it has been shown that the model can accurately predict sweetness at index 1 (100%) and index 2 (100%). Although it can’t predict all of the sweetness at index 3, however it still gave higher percentage of accuracy which is 87%.

Figure 3. Captured mango image.

TABLE II. RANGE OF SWEETNESS IN EVERY INDEX

To date, research on fruit surface color for sweetness determination is still limited. Therefore, our method will be compared with the other research works in a related field of study. Based on the work being done by Lin et al. [10], hue gave promising result in sorting maturity of lemon fruits. It can sort lemon into 3 grades with the percentage accuracy of

Index no. Range (Brix) 1 4.0o- 8.0o 2 8.0o-13.0o 3 13.0o-17.0o

219

95.45% (Grade 1), 100% (Grade 2) and 86.67% (Grade 3) by using average hue and volume. The results are quite similar with our results.

TABLE III. STATISTICAL ANALYSIS OF THE PIXELS IN HUE, SATURATION AND BRIGHTNESS

Index No.

Color

Standard Deviation

(Pixel Value)

Root Mean Square Error

1

Hue 2.73 0.06Saturation 6.93 0.07 Brightness 35.15 0.04

2

Hue 6.31 0.02 Saturation 11.32 0.04 Brightness 18.09 0.08

3

Hue 2.44 0.03 Saturation 12.91 0.07 Brightness 22.79 0.06

TABLE IV. PEARSON’S CORRELATION

Color Pearson’s Correlation Hue -0.92**

Saturation 0.85** Brightness 0.66**

** Correlation is significant at the 0.01 level (2-tailed).

Figure 4. Relationship between sweetness and hue.

Figure 5. Relationship between sweetness and saturation.

Figure 6. Relationship between sweetness and brightness.

Figure 7. Percentage accuracy of the measured sweetness and predicted

sweetness.

In average, our proposed method can determine the sweetness of Chokanan mango with the percentage accuracy of 97.7%. This result is quite similar with the result obtained by [15] to sort Chokun oranges in three classes i.e. raw, ripe and overripe. Hue also has been widely used to grade ‘Golden Delicious’ apples. Heinemann et al. [27] used the average hue on the apples to discriminate russet in apples. A discriminant function sorted the apples as accepted or rejected. The accuracy reached 82.5%, which is poor compared with European standards [27]. Other studies involving ‘Golden Delicious’ apples were performed for the purpose of yellow or green group classification [20]. The results showed that an accuracy of over 90% was achieved for the 120 samples tested. Kavdir and Guyerb [26] also got 90% of accuracy when grading ‘Golden Delicious’ apples based on the size of the red and the Hue with the help of the neural network. From here, it can be shown that our method gave acceptable results. Hue performed better in grading mango and oranges compared to apples.

VI. CONCLUSION From this research, it has been shown that hue, saturation

and brightness have significant relationship with the sweetness of Chokanan mango. Hue gave the highest relationship among the other bands with the value of Pearson’s correlation of -0.92 at the 0.01 level (2-tailed). The capability of the model has been tested using 90 samples of mango images. From the result, it has been shown that the model can be used to determine sweetness at index 1 and index 2 with 100% accuracy and index 3 with 87% accuracy.

220

When compared with the other related work available to date, it has been shown that our proposed method gave acceptable results. Therefore, from this study, it can be concluded that hue can be used to determine the sweetness of Chokanan mango.

REFERENCES [1] A. Kader, “Perspective Flavor Quality of Fruits and Vegetables,”

Journal of the Science of Food and Agriculture, vol. 88, 2008, pp. 1863–1868.

[2] S. Péneau, “Freshness of Fruits and Vegetables: Concept and Perception”, PhD Thesis, Swiss Federal Institute Of Technology Zurich, 2005.

[3] D. C. Slaughter, “Nondestructive Maturity Assessment Methods for Mango: A Review of Literature and Identification of Future Research Needs,” National Mango Board, Orlando, FL, USA, 2009.

[4] Y. Malevski, L. Gomez-Brito, M. Peleg, and M. Silberg,”External Color as Maturity Index of Mango,” J. Food Sci., vol. 42, 1977, pp. 1316-1318.

[5] S.N. Jha, S. Chopra, and A.R.P Kingsly, “Determination of Sweetness of Intact Mango Using Visual Spectral Analysis,” Biosystems Engineering , vol. 91(2), 2005, pp. 157-161.

[6] S. N. Jha, A.R.P Kingsly, and S. Chopra, “Non-destructive Determination of Firmness and Yellowness of Mango During Growth and Storage Using Visual Spectroscopy,” Biosystems Engineering, vol. 94(3), 2006, pp. 397-402.

[7] S. N. Jha, S. Chopra, and A.R.P Kingsly, “Modeling of Color Values for Nondestructive Evaluation of Maturity of Mango,” Food Engineering, vol. 78, 2007, pp. 22-26.

[8] F. Korel, D. A. Luzuriaga, and M. O. Balaban, “ Quality Evaluation of Raw and Cooked Catfish Fillets: Use of Electronic Nose and Machine Vision,” J. Aquatic Food Prod. Technol., vol. 10(1), 2001, pp. 3-18.

[9] S. E. Papadakis, S. Abdul-Malek, R. E. Kamdem, and K. L. Yam, “A Versatile and Inexpensive Technique for Measuring Color of Foods,” Food Technol., vol. 54(12), 2000, pp. 48-50.

[10] Z. Varghese, C.T. Morrow, P.H. Heinemann, H.J. Sommer, Y. Tao, and R.W. Crassweller, “Automated Inspection of Golden Delicious Apples using Color Computer Vision”, American Society of Agricultural Engineers, vol. 91(7002), 1991, pp. 16.

[11] Y.C Hung, K. Morita, R. Shewfelt, A. Resurreccion, and S. Prussia, “Color Evaluation of Apples,” American Society of Agricultural Engineers 93(6541), 1993, pp. 15.

[12] S. Nimesh, M. J. Delwiche, and R. S. Johnson, “Image Analysis Methods for Real-Time Color Grading of Stonefruit,” Computers and Electronics in Agriculture, vol. 9(1), 1993, pp. 71– 84.

[13] B. K. Miller and M. J. Delwiche, “Automatic Grading of Fresh Peaches using Color Computer Vision,” Acta-Horticulturae, vol. 251, 1989, pp. 161–166.

[14] N. Singh, M. J. Delwiche, R. S. Johnson, and J. Thompson, “Peach Maturity Grading with Color Computer Vision,” American Society of Agricultural Engineers, vol. 92(3029), 1992, pp. 23.

[15] Y. Sirisathitkul, N. Thumpen, and W. Puangtong, “Automated Chokun Orange Maturity Sorting by Color Grading,” Walailak Journal of Science and Technology vol. 3(2), 2006, pp. 195-205.

[16] M. Khojastehnazhand, M. Omid, and A. Tabatabaeefar, 2010. “Development of a Lemon Sorting System Based on Color and Size,” African Journal of Plant Science, vol. 4(4), 2010, pp.122-127.

[17] H. S. Lee, “Objective Measurement Of Red Grapefruit Juice Color,” Journal of Agricultural and Food Chemistry, vol. 48(5), 2000, pp. 1507–1511.

[18] S. A. Shearer and F. A. Payne, “Color and Defect Sorting of Bell Peppers using Machine Vision,” Transactions of the ASAE vol. 33(6), 1990, pp. 2045-2050.

[19] W. C. Lin, J. W. Hall, and A. Klieber, “Video Imaging for Quantifying Cucumber Fruit Color,” HortTechnology, vol. 3(4), 1993, pp. 436–439.

[20] Y. Tao, P. H. Heinemann, Z. Varghese, C. T. Morrow, and H. J. Sommer, “Machine Vision for Color Inspection of Potatoes and Apples,” Transactions of the ASAE, vol. 38(5), 1995, pp. 1555–1561.

[21] K. H. Choi, G. H. Lee, Y. J. Han, and J. M. Bunn, “Tomato Maturity Evaluation using Color Image Analysis,” Transactions of the ASAE vol. 38(1), 1995, pp. 171–176.

[22] A. Janobi “Color Line Scan System for Grading Date Fruits,” In ASAE Annual International Meeting, Orlando, FL, USA, 12–16 July, 1998.

[23] A. Jamil, A. Mohamed, and S. Abdullah, “Automated Grading of Palm Oil Fresh Fruit Bunches (FFB) using Neuro-Fuzzy Technique,” In Proc. Int. Conf. Soft Comput. Pattern Recognit., 2009, pp. 245–249.

[24] X. Sun, H. J. Gong, F. Zhang, and K. J. Chen, “A Digital Image Method for Measuring and Analyzing Color Characteristics of Various Color Scores of Beef,” In Proc. Int. Conf. Image and Signal Process., 2009, pp. 1–6.

[25] P. H. Heinemann, R. Hughes, C. T. Morrow, H. J. Sommer, III, R. B. Beelman, and P. J. Wuest, “Grading of Mushrooms using a Machine Vision System,” Transactions of the ASAE, vol. 37(5), 1944, pp. 1671–1677.

[26] I. Kavdır and D. E Guyerb, “Evaluation of Different Pattern Recognition Techniques for Apple Sorting,” Biosystems Engineering, 2008, vol. 99, pp. 211-219.

[27] P. H. Heinemann, Z. A.Varghese, C. T. Morrow, H. J. III. Sommer, and R. M. Crassweller, “Machine Vision Inspection of Golden Delicious Apples,” Applied Engineering in Agriculture Transactions of the ASAE, vol. 11 (6), 1995, pp. 901-906.

[28] T. Thangaraj and I. Irulappan, “Studies on The Maturity Standards for Mango Fruit,” South Indian Hort., vol. 37, 1989, Pp. 341-342.

[29] J. Gross, “Pigments in Fruits,” London: Academic Press, 1987. [30] G. Mazza and E. Miniati, “Anthocyanins in Fruits, Vegetables, and

Grains,” Boca Raton, FL, USA:CRC Press, 1993. [31] X. Li, B. Yu, P. Curran And S. Q. Liu, “Chemical and Volatile

Composition of Mango Wines Fermented with Different Saccharomyces Cerevisiae Yeast Strains,” S. Afr. J. Enol. Vitic. , vol. 32(1), 2011, pp. 117-128.

[32] S. Khairunniza-Bejo and S. Hip, “Image Processing for Automatic Grading of Mango,” International Advanced of Technology Congress (ATCi) 2009, Kuala Lumpur, Malaysia, 2009, pp.119.

221