8
Laser Vision-Based Plant Geometries Computation in Greenhouses Yu Zhang 1, 2 and Xiaochan Wang 1, 2* 1. College of Engineering, Nanjing Agricultural University, Nanjing, 210031, China; 2. Jiangsu Province Engineering Laboratory for Modern Facilities Agricultural Technology and Equipment, Nanjing, 210031, China *Corresponding author, Email, [email protected], [email protected] AbstractPlant growth statuses are important parameters in the greenhouse environment control system. It is time- consumed and less accuracy that measuring the plant geometries manually in greenhouses. To find a portable method to measure the growth parameters of plants portably and automatically, a laser vision-based measurement system was developed in this paper, consisting of a camera and a laser sheet that scanned the plant vertically. All equipments were mounted on a metal shelf in size of 30cm*40cm*100cm. The 3D point cloud was obtained with the laser sheet scanning the plant vertically, while the camera videoing the laser lines which projected on the plant. The calibration was conducted by a two solid boards standing together in an angle of 90. The camera’s internal and external parameters were calibrated by Image toolbox in MatLab®. It is useful to take a reference image without laser light and to use difference images to obtain the laser line. Laser line centers were extracted by improved centroid method. Thus, we obtained the 3D point cloud structure of the sample plant. For leaf length measurement, iteration method for point clouds was used to extract the axis of the leaf point cloud set. Start point was selected at the end of the leaf point cloud set as the first point of the leaf axis. The points in a radian of certain distance around the start point were chosen as the subset. The centroid of the subset of points was calculated and taken as the next axis point. Iteration was continued until all points in the leaf point cloud set were selected. Leaf length was calculated by curve fitting on these axis points. In order to increase the accuracy of curve fitting, bi-directional start point selection was useful. For leaf area estimation, exponential regression model was used to describe the grown leaves for sampled plant (water spinach) in this paper. To evaluate the method in a sample of 18 water spinaches, planted in the greenhouse (length 16 meter and width 8 meter) on the roof of library building in Nanjing Agricultural University, the lengths of 200 leaves and were measured manually and plotted versus their automatically measured counterparts. The accuracy of leaf lengths is 95.39% respectively. For the leaf length measurement, the average error is less than 5mm which is in the boundary of error. A few laser measurement results take on larger errors, because that some leaves were shaded from the others which make an effect on the curve fitting. In additional, the results are affected by the over-wide laser line. Manual measurement gave result information an accuracy of millimeter level, while laser measurement improved the measurement precision. For the leaf area estimation, the accuracy of modeling trained by 200 sampled leaves, is 81.9% with 50 testing sampled leaves. For the plant growth properties, plant height and canopy width are obtained by point clouds reconstruction of plant. The test experiment proved that laser vision-based method could be used on plants geometry measurement and growth monitoring in greenhouses. This method will improve visualization and digitalization of plants in greenhouses, and make a progress on greenhouse environment control system. Index TermsLaser Vision; 3D Point Cloud; Plants Geometry; Leaf Length; Leaf Area; Plant Height; Canopy Width I. INTRODUCTION A facility farming is growing fast in recent years. Many scientists and experts are working hard on the artificial control of greenhouse environment [11, 14, 25, 27, 30, 31]. But the target of greenhouse environment control is providing appropriate environment for the crops in the greenhouse. Results of the control system were displayed on the status of crop growth. So it is of more importance to detect the crop response at different growth status [25]. Remote sensing [17] is widely used in field farm crops, while in greenhouses it is not adaptive due to the block of glass. So many experts made endeavor on the geometric crop properties detection in greenhouses to obtain the crop growth response for the plant speaking- based greenhouse environmental control system. In conventional greenhouse, crop growing conditions are obtained via human observations or presetting environmental parameters, instead of the plants’ specific needs at a different status. Contact sensing is typically used to determine a plant’s physical characteristics, a process that is cumbersome, labor-intensive, and often destructive. Computer vision is a good choice for geometric crop properties detection in greenhouses, because of non-contact sensing, non-destructive and time- saving. David Story [4] used machine vision-guided plant sensing and monitoring system to detect calcium deficiency in lettuce crops grown in greenhouse conditions using temporal, color and morphological changes of the plant. The machine vision system extracted plant features to determine overall plant growth and health status, including top projected canopy area(TPCA)as a morphological feature; redgreenblue(RGB) and huesaturationluminance(HSL)values as color features; and entropy, energy, contrast, and 534 JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014 © 2014 ACADEMY PUBLISHER doi:10.4304/jmm.9.4.534-541

Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

Laser Vision-Based Plant Geometries

Computation in Greenhouses

Yu Zhang 1, 2

and Xiaochan Wang 1, 2*

1. College of Engineering, Nanjing Agricultural University, Nanjing, 210031, China;

2. Jiangsu Province Engineering Laboratory for Modern Facilities Agricultural Technology and Equipment, Nanjing,

210031, China

*Corresponding author, Email, [email protected], [email protected]

Abstract—Plant growth statuses are important parameters

in the greenhouse environment control system. It is time-

consumed and less accuracy that measuring the plant

geometries manually in greenhouses. To find a portable

method to measure the growth parameters of plants

portably and automatically, a laser vision-based

measurement system was developed in this paper, consisting

of a camera and a laser sheet that scanned the plant

vertically. All equipments were mounted on a metal shelf in

size of 30cm*40cm*100cm. The 3D point cloud was obtained

with the laser sheet scanning the plant vertically, while the

camera videoing the laser lines which projected on the plant.

The calibration was conducted by a two solid boards

standing together in an angle of 90. The camera’s internal

and external parameters were calibrated by Image toolbox

in MatLab®. It is useful to take a reference image without

laser light and to use difference images to obtain the laser

line. Laser line centers were extracted by improved centroid

method. Thus, we obtained the 3D point cloud structure of

the sample plant. For leaf length measurement, iteration

method for point clouds was used to extract the axis of the

leaf point cloud set. Start point was selected at the end of the

leaf point cloud set as the first point of the leaf axis. The

points in a radian of certain distance around the start point

were chosen as the subset. The centroid of the subset of

points was calculated and taken as the next axis point.

Iteration was continued until all points in the leaf point

cloud set were selected. Leaf length was calculated by curve

fitting on these axis points. In order to increase the accuracy

of curve fitting, bi-directional start point selection was

useful. For leaf area estimation, exponential regression

model was used to describe the grown leaves for sampled

plant (water spinach) in this paper. To evaluate the method

in a sample of 18 water spinaches, planted in the greenhouse

(length 16 meter and width 8 meter) on the roof of library

building in Nanjing Agricultural University, the lengths of

200 leaves and were measured manually and plotted versus

their automatically measured counterparts. The accuracy of

leaf lengths is 95.39% respectively. For the leaf length

measurement, the average error is less than 5mm which is in

the boundary of error. A few laser measurement results take

on larger errors, because that some leaves were shaded from

the others which make an effect on the curve fitting. In

additional, the results are affected by the over-wide laser

line. Manual measurement gave result information an

accuracy of millimeter level, while laser measurement

improved the measurement precision. For the leaf area

estimation, the accuracy of modeling trained by 200

sampled leaves, is 81.9% with 50 testing sampled leaves. For

the plant growth properties, plant height and canopy width

are obtained by point clouds reconstruction of plant. The

test experiment proved that laser vision-based method could

be used on plants geometry measurement and growth

monitoring in greenhouses. This method will improve

visualization and digitalization of plants in greenhouses, and

make a progress on greenhouse environment control system.

Index Terms—Laser Vision; 3D Point Cloud; Plants

Geometry; Leaf Length; Leaf Area; Plant Height; Canopy

Width

I. INTRODUCTION

A facility farming is growing fast in recent years.

Many scientists and experts are working hard on the

artificial control of greenhouse environment [11, 14, 25,

27, 30, 31]. But the target of greenhouse environment control is providing appropriate environment for the

crops in the greenhouse. Results of the control system

were displayed on the status of crop growth. So it is of

more importance to detect the crop response at different

growth status [25]. Remote sensing [17] is widely used in

field farm crops, while in greenhouses it is not adaptive

due to the block of glass. So many experts made endeavor

on the geometric crop properties detection in greenhouses

to obtain the crop growth response for the plant speaking-

based greenhouse environmental control system.

In conventional greenhouse, crop growing conditions are obtained via human observations or presetting

environmental parameters, instead of the plants’ specific

needs at a different status. Contact sensing is typically

used to determine a plant’s physical characteristics, a

process that is cumbersome, labor-intensive, and often

destructive. Computer vision is a good choice for

geometric crop properties detection in greenhouses,

because of non-contact sensing, non-destructive and time-

saving. David Story [4] used machine vision-guided plant

sensing and monitoring system to detect calcium

deficiency in lettuce crops grown in greenhouse

conditions using temporal, color and morphological changes of the plant. The machine vision system

extracted plant features to determine overall plant growth

and health status, including top projected canopy

area(TPCA)as a morphological feature; red–green–

blue(RGB) and hue–saturation–luminance(HSL)values as

color features; and entropy, energy, contrast, and

534 JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014

© 2014 ACADEMY PUBLISHERdoi:10.4304/jmm.9.4.534-541

Page 2: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

TABLE I. NOTATION

α Coefficient of leaf area regression model ID Inner diameter of donut point on calibration board (Fig. 3)

β Coefficient of leaf area regression model OD Outer diameter of donut point on calibration board (Fig. 3)

dcrit

Predefined distance in iteration for leaf axis points extraction Oc Center point of camera pinhole model (Fig. 2)

Leaf-i The target leaf points subset OcZc Camera optic axis (Fig. 2)

f Focal length (Fig. 2) L L LO X Y A planar coordinates on the laser light plain(Fig. 2)

l Leaf length, cm (Fig. 11,Tab3) 0P ,

1P , ∙∙∙ Basal point in iteration for leaf axis points extraction

L Laser line sheet (ui,vi) Image pixel coordinates (Fig. 2)

LA Estimated leaf area, cm2 (Fig. 11,Tab3) (xwi,ywi,zwi) World coordinates (Fig. 2)

homogeneity as textural features. The methodology

developed was capable of identifying calcium-deficient lettuce plants 1 day prior to visual stress detection by

human vision. Ling et al [12] used spectral and

morphological characteristics of lettuce leaves to detect

nutrient deficiency. This study showed that the

reflectance of wave bands between 415nm and 720 nm

could be used as a signal wave band for machine vision

implementation. A wave band closer to the visible was

recommended because it gives a better signal strength.

The study also suggested the possibility of using multiple

signals based on water and nutrient stress detection in

plants, using machine vision systems that could determine deviations from “well-grown” as an indicator

of deficiencies and plant status. Meyer et al [16] used a

machine vision system to detect single leaves and

poinsettia foliage, and reported that a normalized

difference index provided the best method of

discriminating nitrogen-deficient from healthy plants.

Low-nitrogen plants grown in greenhouses and growth

chambers showed similar increases in red reflectance, but

had different levels of near-infrared reflectance due to

differing amounts of plant canopy cover. E.V.Henten [18]

used an indirect non-destructive measurement by means

of image processing to detect crop growth of lettuce and build 3D models of the relationship between relative soil

coverage by the crop canopy and by dry weight.

Geometric crop properties measurements are a broad

research topic in agriculture. With the development of

computer vision, 3D measurement shows a great potential

in vegetation growth parameters measurement [1-3, 10].

Hans J. Andersen [7] investigated the potential of using

area-based binocular stereo vision for three-dimensional

analysis of single plants and estimation of geometric

attributes such as height and total leaf area. Dominik

Seidel [5] measured total above-ground biomass (stems, twigs, leaves), the biomass of axes (stems and twigs),of

leaves biomass and the leaf area of 63 experimental trees

by means of terrestrial 3D laser scanner. Hosoi and

Omasa [8] used a portable 3D laser scanner to calculate

canopy leaf area density profiles for deciduous trees. His

studies focused on measurements of the biomass, and

mostly concentrated on forest trees resting upon

relationships among parameters which were measured

with terrestrial laser scanners, e.g. diameter at breast

height or total tree height, and the trees biomass [28]. The

application of terrestrial laser scanning (TLS) and mobile or vehicle based laser scanning (MLS) enables fast 3D

data acquisition. They provide a direct capture of tree

geometry generating high resolution point measurements

representing the geometrical characteristics of target

objects in 3D space [13]. Generally, Terrestrial LiDAR (Light Detection and Ranging) Scanners (TLSs) have

recently turned into potential tools for 3D vegetation

structure measurement. It provides accurate measures of

distances to objects using a large number of sampling

laser beams within the instrument field of view, thus

creating a cloud of points positioned in 3D space. It is

widely used in field farms and forests. But in greenhouses,

TLSs do not show the superiorities because of the space

limitation.

In our studies, we are aiming to find a portable and low

cost vegetation geometric properties measurement system basing on laser vision. The main parameters of plant

geometric properties include the leaf length, leaf area,

plant height and canopy width of a single crop in

greenhouses. This laser vision-based measurement

system will have great prospects on both improving the

visualization and digitization for plants growth

monitoring and analysis and providing multi-aspect data

for the greenhouse environmental control system.

II. MATERIALS AND METHODS

A. Crops and Acquisition

A growth experiment with water spinaches in

greenhouses which was built on the roof of laboratory

building at College of Engineering, Nanjing Agricultural

University served as the study object to test the

applicability of 3D-laser scanning as a non-destructive

method for growth parameters detection in greenhouses,

which is a three ridge-Venlo greenhouse size 8m* 16m.

The measurement principles, as shown in Fig. 1, is

comprised of a CCD web-camera (HikVision® DS-

2CD3332D-I) that recorded the video of laser scanning

line, projected onto the crop stems by a laser sheet with a downward slope angle of 45 degree. The laser sheet was

created by mounting a horizontal raster on a 80 mW, 520

nm (green) laser diode. And it had an approximate

thickness of 2 mm and an aperture angle of 60 degree.

The laser sheet was mounted on a Pan-Tilt (PTZ®W3300)

with horizontal rotation angle of 355 degree and vertical

lift angle of 60 degree. All the components were mounted

on a mental shelf with 30cm by 40cm by 100cm. The

laser sheet scanned the crop up and down with a

horizontal laser line. A PC machine was used to control

the Pan-Tilt through 485-cable and the camera through a

network, respectively. The video resolution was 1080*1270 pixels with the frame of 30 fps, and the frame

image resolution was 640*480 pixels by programming in

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014 535

© 2014 ACADEMY PUBLISHER

Page 3: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

MatLab®. The camera was calibrated using a standard

procedure contained in a MatLab® Toolbox based on

Tsai’s method, which enabled distortion correction of the

imagery.

Laser sheet

PTZ Pan-tilt

Camera

Metal shelf

Figure 1. 3D laser sheet measurement system

B. Calibration of 3D Measurement System

3D measurement system consists of camera perspective transformation model and light plain equation

model [32, 34] which is used for building the relation

between 2D coordinates and 3D coordinates of target

points, shown in Fig. 2. Oc is the center point of camera

pinhole model. OcZc is the camera optic axis, which is

perpendicular to CCD plain, with point O as the

intersection. The distance of O-Oc is the focal length, f.

X-axis and Y-axis in retinal coordinates is parallel to u-

axis and v-axis in image coordinates, respectively.

yL

xL

Laser

source

OL

X

Y

Zc

xc

yc

u

v

O

Oc

P(u,v)

O(u0,v0)

Zw

Xw

YwOw

Laser light plane

Object

CCD

Screen

Figure 2. Sketch of 3d measurement principle of line plane equation

In the process of measurement, laser line sheet (L)

projects a curve on the surface of the object that contains

3D spatial information of the object. In the world coordinates, laser light plain equation in the camera

coordinates can be described as:

acXc + bcYc + ccZc + dc = 0 (1)

On the laser light plain, planar coordinates L L LO X Y

was built. The relation between the laser light plain and

the retinal plain is

31 2

4 5 6

7 8 91

L

x

L

y

L

z

Xr tr rx

Yy r r r t

Zz r r r t

(2)

where 0Lz , * , *x y

X f Y fz z

were substituted

into Eq. (2).

1 2

4 5

7 8

1 2 3

4 5 6

7 8

1 1

*

1 1

* *

1

x L

y L

z

L

Z L

L

Z L

X fr fr fT X

Y fr fr fT Y

r r T

a a a X

T a a a Y

a a

X

T A Y

(3)

Penalty function was used to solve the optimal

equation, as followed,

1 2 3

7 8

4 5 6

7 8

*( 1)

*( 1)

xi Li Li

i Li Li

yi Li Li

i Li Li

f a X a Y a

X a X a Y

f a X a Y a

Y a X a Y

(4)

where i=1,2,3,…N, and the constrains of the penalty

function is

2 2 2

1 1 1 4 7

2 2 2

2 2 2 4 8

3 3 1 2 4 5 7 8

( 1)

( 1)

( )

p

p

p

f M a a a

f M a a a

f M a a a a a a

, corresponding

optimal target function 2 2 2 2 2

1 2 8 1 2 31( , ,..., ) ( ) min

n

xi yi p p piI a a a f f f f f

. Gaussian-Newton method was used to solve the parameters.

Integrating traditional camera calibration plane targets

and light plain calibration [27, 35] 3D targets, a vertical

target was used in this paper, consisted of two rectangular

boards standing at an angle of 90 degree. A certain

number of solid dots 10mm in diameter and donut dots

with OD 10mm and ID 6mm were spread on each of the

board target evenly. The distance between centers is

50mm. Two targets, in the size of 210mm width and

297mm height, were mounted at a corner at an angle of

90 degree. The intersection of the two boards was defined as Y-axis, defined the mid-point as the original point of

the world coordinates. OX and OY were defined on the

target board in the right-hand rule, respectively. An

image without laser line was taken. The image

coordinates of all the solid dots and donut dots and that in

the world coordinates were known. And considering the

radial constrains in the camera, we can obtain

1 2 3 4

5 6

w d w d w d w d

w d w d d y

x Y r y Y r z Y r x X r

y Y r z X r X t

(5)

Substitute the known point coordinates

, , , ,wi wi wi i ix y z u v into equation (5), we can obtain

,* dA B x (6)

where the row vector , , ,

w d w d w d d w d w d w dA x y y y z y y x x y x z x

536 JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014

© 2014 ACADEMY PUBLISHER

Page 4: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

is known, and the column vector

1 2 3 4 5 6

T

y y y x y y y yB r s t r s t r s t t s t r t r t r t

is unknown. All the parameters can be solved when the

known point coordinates were substituted into Eq. (6). An

image with laser line on it was taken. The points were

both on the laser line and the plain of XY and YZ.

Similarly, the known point coordinates were substituted

into Eq.(1), and the equation of laser light plain in the world coordinates was

1 1 1

2 2 2

0

0

0

w w w w w w w

w w w w w w w

w wn w wn w wn w

a x b y c z d

a x b y c z d

a x b y c z d

(7)

Least-square method was used to solve the equation set,

and the parameters of laser light plain equation in the

world coordinates were obtained. Since the vector

x y zt t t , 1 4 7 2 5 8r r r and r r r are

transformation vector and the unit direction vectors of

LX -axis and LY -axis in camera coordinates respectively,

we can obtain

4 8 5 7

2 7 1 8

1 5 2 4

c

c

c

c c x c y c z

a r r r r

b r r r r

c r r r r

d a t b t c t

(8)

Parameters 3r ,

6r , 9r can be calculated by the

transformation of camera coordinates and world

coordinates.

○ ○

○ ○○

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

○○

Y

XZ

○ ○

○●

● ●

●●●

●●

○●

○●●

●●●

● ●

●●

●●

●●

●●

○○

○ ○

XZ

Y

Figure 3. Sketch of calibration

C. Images Processing and Laser Line Centroid Method

1) Images Processing The first step was background removing to reduce the

noise disturbances after extracting the frame images.

Difference imaging was used to remove the background.

No laser line projected video was recorded at the beginning. Afterwards, laser sheet was opened to scan on

the crop. So images with laser line projected and without

laser line projected were extracted. The laser line stripe

was shown in Fig. 4.

Figure 4. Background removing result

2) Laser Line Centroid Extraction and Point Clouds

Reconstruction The second step was extracting the laser line centers.

The noise of the sunlight, reflection of the surface of the

target crop and other disturbances would make an

influence on the laser line centers extraction. So the noise

filtering was a necessary process before proceeding to the

extraction. Ideal distribution of laser light is Gauss

Distribution, so Gauss filter was chosen to remove the

noises.

Conventional center extraction methods were

extremum method and threshold method, while the two

methods were affected by the noises severely. In this

paper, centroid method [19, 20] based on the center of gravity computation of rigid-body, was chosen to obtain

the centroid of the pixels gray distribution on every cross-

section of the laser line stripe. Fig. 5 showed result of

laser line centers on one image. The pixel coordinate of

the center points were easily obtained by programming in

MatLab®. Thus, 3D point clouds construction [9, 17, 18,

21, 22, 26] of target plant was plotted in Fig. 6.

Figure 5. Laser line centers

D. Leaf Length Calculation

In this paper, we proposed an iteration which fit with

the features of plant point clouds. Axis points of a leaf

were extracted and fitted by curve fitness. The length of

axis curve was the leaf length. Details were as followed:

Firstly, rotate the main plot of the plant, select the target

leaf, and save the target leaf points as leaf subset, named

Leaf-i. A basal point 0P was chosen by mouse click,

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014 537

© 2014 ACADEMY PUBLISHER

Page 5: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

which is usually at the end of the leaf point subset in 3D

plot in the first iteration step(i=1). This basal point 0P

became the first point in the axis point set. Secondly, a

known distance dcrit

was predefined. All points in Leaf-i

with a distance to 0P less than d

crit are selected by going

through all over the points in Leaf-i, namely subset i.

Calculation of the centroid of subset i is conducted,

marked as 1P ,which is the basal point in the next iteration

step. And all the points selected in the first iteration step

are deleted. The next iteration step stats from the reduced

leaf subset. Iteration continues until all the points in the

subset of Leaf-i are deleted. At last, all centroid points are

saved in point set named Ta. The length of the curve

made of these centroid points is the length of the leaf. Fig.

7 shows the result of the leaf axis extraction. Sometimes, bi-directional iteration is necessary to avoid the missing

point at the end of the point set. The red points and the

blue points are two axis extraction results basing on two

start points at two ends of the point set. All the two axis

points are used to fit the curve. This method shows robust

in case that there are some missing points in the leaf point

set because of laser sheet scanning angle.

Figure 6. Point clouds of crops

5060

7080

90100

40

50

60

70

80

50

55

60

65

70

75

80

85

90

Points Cloud

Figure 7. Leaf axis extraction

E. Leaf Area Estimation

Leaf area takes on a high correlation to leaf length,

which can be described by mathematical model. Different

plants show different correlation models on leaf length

and leaf area. In this paper, Pearson Correlation Analysis

was conducted at a sample of 200 water spinach leaves

obtaining regression model [33].

ln( ) *LA l (9)

III. RESULTS

A. Leaf Length Calculation

In the process of leaf length calculation, the number of

leaf axis points affected the result of leaf axis curve

fitting. Technically, the value of dcrit

should be larger

than half of the width of leaf point set. The value of dcrit

is in the inverse proportional relationship to the number

of axis points. If dcrit

was assigned a large value, the

number of axis points was too rare to finish the curve

fitting. Otherwise, if dcrit

was assigned a small value, the

number of axis points was too redundant to ensure the

accuracy of curve length calculation. In this paper, the

value of dcrit

ranged from 1 to 20, producing a set of axis

point number and fitted leaf length couples. In Fif.8,a

singular point valued 50 of the axis point number fit with

d crit valued 1. In this figure, the value of d crit can be

inversely computed from the average number of axis

point 7. Finally, the length of leaf can be obtained.

dcirt Number of axis point Leaf length

0

10

20

30

40

50

Figure 8. Iteration step result

La

ser

mea

sure

men

t/cm

Manual measurement/cm

Figure 9. Comparison of laser measurement and manual measurement

of leaf length

538 JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014

© 2014 ACADEMY PUBLISHER

Page 6: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

TABLE II. RESULTS OF ANOVA ANALYSIS OF LEAF AREA ESTIMATION MODEL

Sum of Squares df Mean Square F Sig.

Regression 17.383 1 17.383 267.97 .000

Residual 12.455 192 0.065

Total 29.838 193

The independent variable is length

TABLE III. COEFFICIENTS AND ANALYSIS OF LEAF AREA ESTIMATION MODEL

Unstandardized Coefficients Standardized Coefficients

t Sig. B Std.Error Beta

l (length) 0.164 0.010 0.763 16.370 .000

(Constant) 1.923 0.174 5.315 11.021 .000

The independent variable is l(length)

The dependent variable is ln(LA)

To evaluate the performance of the laser scanning

measurement system, a sample of 180 water spinach

leaves were measured using manual measurements and

laser scanning measurement system. Fig. 7. shows the result of leaf axis extraction. Fig. 9. shows the calculated

leaf length from laser scanning measurement system,

versus the manually measured leaf length. A straight line

was fitted, yielding a slope of 0.998, an intercept of 0,

and a standard error of 0.909 and a coefficient

determination of 0.915. For the leaf length measurement,

the average error is less than 5mm with accuracy of leaf

lengths is 95.39%, which is in the boundary of error. A

few laser measurement results take on larger errors,

because that some leaves were shaded from the others

which make an effect on the curve fitting.

B. Leaf Area Estimation

To establish the regression model of leaf length and

leaf area, a destructive experiment of leaf sampling was

conducted. All the sampled leaves were listed on a piece

of A4 paper, which is marked with a calibration square

with area of 1*1cm². Real areas of leaves were computed by pixel counting, comparing with the calibrated square.

The regression model was established with 200 training

leaves and 50 testing leaves. Fig. 10 shows the original

image and segmented image of sampled leaves.

Figure 10. RGB image and segmentation image of leaves

Exponential regression model was used to evaluate the

leaf area, and the result was derived, with R 2

adj of 0.9864

and confidence interval of 95%. The result of the leaf

area model is shown in Fig. 11.

ln( ) 0.164 1.923LA l (10)

Results of ANOVA analysis of leaf area estimation

model are shown in Table 2 and Table 3 with the model

significance of 0 and F statistical testing of 267.97. The

coefficients of the leaf area model are shown in Table 3

with standard error of 0.763 and 5.315. The accuracy of

the model is 81.9%.

To test the leaf area model, the area of 50 sampled

leaves were measured by pixels calibration and estimated by the model. Relationship of leaf area estimation and

pixels calibration measured was shown in Fig. 12. A

straight line was fitted, yielding a slope of 0.947, an

intercept of 0.434.

Leaf length/cm

Lea

f are

a/c

Figure 11. Regression model of leaf length and leaf area

Manual measurement/cm²

La

ser

mea

sure

men

t/cm

²

Figure 12. Relationship of leaf area estimation and observed

measurement

C. Growth Trend

Laser scanning the whole plant profile can be used to

record and monitor plant growth with leaf area, plant

height and canopy width. A sampled water spinach

growth conditions (plant height, canopy width) was

monitored within 10 days after field planting. The height

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014 539

© 2014 ACADEMY PUBLISHER

Page 7: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

growth of sampled plant was shown in Fig. 13(a), and the

canopy width growth of sampled plant was shown in Fig.

13(b). Fig. 14 shows the leaf area growth of six sampled

plants during the period of June twenty-seventh and July

twentieth in 2013.

(a)

(b)

Figure 13. Growth of heights and canopy area(Field planting Day 1,

Field planting Day 4, Field planting Day 7, Field planting Day 10)

Figure 14. Total leaf area growth trend

IV. DISCUSSIONS

Plant geometries measurement system based on laser vision performed well in measuring leaf length, plant

height and canopy width. Our leaf axis point extraction

has been based mainly on iteration fitting to the

properties of plant 3D point clouds. This method

addressed the situation of missing points in the leaf point

set. And bi-directional iteration increased the accuracy of

axis point extraction (shown in Fig. 7). Exponential

modeling was applied to estimate the leaf area using leaf

length avoiding destructive sampling in leaf area

measurement. To record and monitor the whole process

of plant growth, leaf areas models of different growth

periods are necessary. In this paper, we modeled the leaf

area with leaf length to estimate the leaf area, but in

additional, comparison a mound the leaf area estimation

using TLS [15] and the traditional LAI estimation is of

value.

For the leaf length measurement, the average error is

less than 5mm with accuracy of leaf lengths is 95.39%,

which is in the boundary of error. A few laser

measurement results take on larger errors, because that some leaves were shaded from the others which make an

effect on the curve fitting. In additional, the results are

affected by the over-wide laser line. Manual

measurement gave result information an accuracy of

millimeter level, while laser vision-based measurement

system improved the measurement precision. For the leaf

area estimation, the accuracy of modeling trained by 200

sampled leaves is 81.9% with 50 testing sampled leaves.

For the plant growth properties, plant height and canopy

width are obtained by point clouds reconstruction of plant.

The test experiment proved that laser vision-based method could be used on plants geometry measurement

and growth monitoring in greenhouses. This method will

improve visualization and digitalization of plants in

greenhouses, and make a progress on greenhouse

environment control system.

Although we have concentrated on laser vision-based

plant geometries measuring, we would also expect, more

generally, that 3D point clouds information would be

capable of measuring more phenotypes [3] of plants in

greenhouses. The laser vision sensors could also be easily

combined with additional sensors to monitor automatically for stress, deficiency [4], spectral

information et al at the same time. An overall phenotypes

database of plant would be established to provide

information for greenhouse environment control system.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the funding of

their projects of ‘Research of Multi-scale Greenhouse

Environmental Control System Based on Crop

Information Fusion’ supported by National Natural

Science Foundation of China.

REFERENCES

[1] Alexander M S, Keith L E, Randy W, etc. “Measuring Canopy Coverage with Digital Imaging,” Communications in Soil Science and Plant Analysis, 2007, 38 pp. 7-8, 895-902.

[2] Carlos C, Garcia MI, Daza C, etc. “Study of a Non-destructive Method for Estimating the Leaf Area Index in Vegetable Crops Using Digital Images,” HORTSCIENCE, 2010, 45(10) pp. 1459–1463.

[3] Christopher J. K, John M. N, Stith T. G. “Measurements of leaf orientation, light distribution and sunlit leaf area in a boreal aspen forest,” Agricultural and Forest Meteorology, 1998, 91 pp. 127-148.

[4] David S, Murat K, Chieri K, etc. “Lettuce calcium deficiency detection with machine vision computed plant

540 JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014

© 2014 ACADEMY PUBLISHER

Page 8: Laser Vision-Based Plant Geometries Computation in Greenhouses · 2019-09-09 · Laser Vision-Based Plant Geometries Computation in Greenhouses . Yu Zhang 1, ... The 3D point cloud

features in controlled environments,” Computers and Electronics in Agriculture, 2010, 74 pp. 238–243.

[5] Dominik S, Friderike B, Dietrich H, etc. “3D-laser scanning: A non-destructive method for studying above-ground biomass and growth of juvenile trees,” Agricultural and Forest Meteorology, 2011, 151 pp. 1305–1311.

[6] Fontana M, Rizzi C, Cugini U. “3D virtual apparel design for industrial applications,” Computer­Aided Design, 2005, 37(6) pp. 609­622.

[7] Hans J A, Lars R, Kristian K. “Geometric plant properties

by relaxed stereo vision using simulated annealing,” Computers and Electronics in Agriculture, 2005, 49 pp. 219–232.

[8] Hosoi, F., Omasa, K., “Factors contributing to accuracy in the estimation of the woody canopy leaf area density profile using 3D portable lidar imaging,” J. Exp. Bot., 2007, 58, 3463–3473.

[9] Huang J, Menq C H. “Combinatorial manifold mesh

reconstruction and optimization from unorganized points with arbitrary topology,” Comput. Aided Des, 2002, 34, 149–165.

[10] Jiangui Liu, Elizabeth P, Stuart A. “Assessment of in situ crop LAI measurement using unidirectional view digital Photography,” Agricultural and Forest Meteorology, 2013, 169 pp. 25–34.

[11] JoséBoaventura C, Moura Oliveira de J P. “Optimal

Management of Greenhouse Environments,” EFITA 2003 Conference, 2003(7), 5-9.

[12] Ling P P, Giacomelli G A, “Russell T P. Monitoring of plant development in controlled environment with machine vision,” Advances in Space Research, 1996, 18(4–5) pp. 101–112.

[13] Livny, Y., Feilong, Y., Olson, M., Chen, B., Zhang, H., El-Sana, J.. “Automatic reconstruction of tree skeletal

structures from point clouds,” ACM Transactions on Graphics, 2010, 29(6), 1511–1518.

[14] Mario T P, Gilberto H R, Jose R M, etc. “Greenhouse Energy Consumption Prediction using Neural Network Models,” International Journal of Agriculture and Biology, 2009, 11(1) pp. 1-6.

[15] Martin B, Jean-Luc W, Richard A F, etc. “Estimating leaf area distribution in savanna trees from terrestrial LiDAR Measurements,” Agricultural and Forest Meteorology,

2011, 151 pp. 1252–1266. [16] Meyer G E., Troyer W W, “Fitzgerald J B, etc. Leaf

nitrogen analysis of poinsettia (Euphorbia Pulcherrima Will D.) using spectral properties in natural and controlled lighting,” Applied Engineering in Agriculture, 1992, 8(5) pp. 715–722.

[17] Pachidis T P, Lygouras J N. Pseudo. “Stereo-vision system: a monocular stereo-vision system as a sensor for real-time

robot applications,” IEEE Transactions on Instrumentation and Measurement, 2007, 56(6), 2547-2560.

[18] Pagàs J, Salvi J, Garcäa R, Matabosch C. “Over view of coded light projection techniques for automatic 3D profiling,” IEEE International Conference on Robotics and Automation, ICRA 2003, pp. 133-138, Taipei, Taiwan, 14-19th September 2003.

[19] Parsons N R, Edmondson R N, Song Y. “Image analysis

and statistical modelling for measurement and quality assessment of ornamental horticulture crops in glasshouses,” Biosystems engineering, 2009, 104 pp. 161–168.

[20] Remondino F. “3­D reconstruction of static human body shape from image sequence,” Computer Vision and Image Understanding, 2004, 93(1) pp. 65­85.

[21] Saxena A, Chung S H, Ng A Y. “3-D depth reconstruction from a single still image,” International Journal of

Computer Vision, 2007, 76(1) pp. 53-69. [22] Surmann H, Nuchter A, Hertzberg J. “An autonomous

mobile robot with a 3D laser range finder for 3D exploration and digitalization of indoor environments,” International Journal of Robotics and Autonomous Systems, 2003, 45(2) pp. 181-198.

[23] Tao Li, Sun Changku, He Li, Ye Shenghua. “Research on the color 3D profilometry and parameter calibration

technology,” Proceedings of SPIE-The International Society for Optical Engineering, v5638, Optical Design and Testing, 2005, II pp. 464-472.

[24] Tetsuo M, Yasushi H. “An intelligent control for greenhouse automation, oriented by the concepts of SPA and SFA—an application to a post-harvest process,” Computers and Electronics in Agriculture, 2009, 29 pp. 3-20.

[25] Van Henten E J, Bontsema J. “Non-destructive Crop Measurements by Image Processing for Crop Growth,” J. agric. Engng Res, 1995, 61 pp. 97-105.

[26] Wang Dingcheng, Qiao Xiaojun, Wang Chunxiu, et al. “Simulation of greenhouse environment SVMR control based on plant response,” Transactions of the CSAE, 2010, 26(Supp. 2) pp. 290-293. (in Chinese with English abstract).

[27] Watanabe T, Hanan J S, Room P M, etc. “Rice morphogenesis and plant architecture: measurement, specification and the reconstruction of structural development by 3D architectural modeling,” Ann. Bot, 2005, 95, 1131–1143.

[28] Watt, P. J., Donoghue, D. N. M., Dunford, R. W., 2003. “Forest parameter extraction using terrestrial laser scanning. In: Proceeding of the ScandLaser Scientific

Workshop on Airborne Laser Scanning of Forests,” in Umea, Sweden, 2–4 September 2003, pp. 237–244.

[29] Weixing Zhu, Hanping Mao, Pingping Li, etc. “A Fuzzy Controller Based on Genetic Optimization and Its Application to a Distributed Control System of Greenhouse,” Transactions of the Chinese Society for Agricultural Machinery, 2002, 33(3) pp. 76-78.

[30] Wilczkowiak M, Boyer E, Sturm P. “Camera calibration and 3D reconstruction from single images using

parallelepipeds,” IEEE International Conference on Computer Vision, 2001(1) pp. 142-148.

[31] Xiaochan Wang, Weimin Ding, Weihong Luo. “Design Experiment to Measure Microclimate and Study Transpiration in Summer in Greenhouse,” Transactions of the Chinese Society for Agricultural Machinery, 2003, 34(2) pp. 86-89.

[32] Xiaochan Wang, Weimin Ding. “Simulation and analysis

of micro-climate of gutter connected Venlo greenhouse during rainy season in Jianghuai region of China using BP neural network,” Transactions of the CSAE, 2004, 20(2) pp. 235-238.

[33] Xiaojin Gong, Ying Lin, Jilin Liu. “Extrinsic calibration of a 3D LIDAR and a camera using a trihedron,” Optics and Lasers in Engineering 51(2013) pp. 394–401.

[34] Yin Xinyou, van L, H. H.. “Crop Systems Dynamics: An

eco-physiological Simulation Model for Genotype-by-Environment Interactions,” Wageningen Academic Publishers, 2005.

[35] Zhiqin Xu, Li Tao, Changku Sun, YiZhong Zheng. “A Rapid Calibration Method for the Laser Scanning 3D Measurement System,” Proceeding of ISMTI’2003. Hong Kong, China, 2003, 151 Key Engineering Materials, 2003, 295-296 pp. 735-740.

JOURNAL OF MULTIMEDIA, VOL. 9, NO. 4, APRIL 2014 541

© 2014 ACADEMY PUBLISHER