9

Click here to load reader

Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

Embed Size (px)

Citation preview

Page 1: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

Journal of Invertebrate Pathology 95 (2007) 101–109

www.elsevier.com/locate/yjipa

Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri

Kuwayama (Hemiptera: Psyllidae), in Florida

Jason M. Meyer ¤, Marjorie A. Hoy, Drion G. Boucias

Department of Entomology and Nematology, University of Florida, Institute of Food and Agricultural Sciences, Building 970, P.O. Box 110620, Gainesville, FL 32611-0620, USA

Received 6 September 2006; accepted 26 January 2007Available online 3 February 2007

Abstract

The Asian citrus psyllid, Diaphorina citri Kuwayama, is an invasive pest that vectors citrus greening disease, which recently wasdetected in Florida. Mycosed adult D. citri were collected at four sites in central Florida between September 2005 and February 2006.Observation of the cadavers using scanning electron microscopy revealed that the pathogen had branched synnemata supporting mono-philadic conidiogenous cells. A high-Wdelity polymerase chain reaction (PCR) assay was used to amplify the 18S rRNA, 28S rRNA and �-tubulin genes of the pathogen for phylogenetic analysis. The morphological and genetic data indicated that the pathogen was a novel iso-late related to Hirsutella citriformis Speare. PCR assays using isolate-speciWc primers designed from the unique putative intron region ofthe �-tubulin sequence distinguished the psyllid pathogen from Wve related Hirsutella species. The pathogen was maintained in vivo byexposing healthy D. citri to the synnemata borne on Weld-collected cadavers. Infected psyllids had an abundance of septate hyphal bodiesin their hemolymph and exhibited behavioral symptoms of disease. In vitro cultures of the pathogen were slow-growing and producedsynnemata similar to those found on mycosed D. citri. In laboratory bioassays, high levels of mortality were observed in D. citri that wereexposed to the conidia-bearing synnemata produced in vivo and in vitro.© 2007 Elsevier Inc. All rights reserved.

Keywords: Diaphorina; Hirsutella; Huanglongbing; Asian citrus psyllid; Biological control; Fungal pathogen; Liberibacter

1. Introduction

The Asian citrus psyllid, Diaphorina citri Kuwayama(Hemiptera: Psyllidae), vectors the Gram-negative bacte-rium Candidatus Liberibacter asiaticus, the causal agent ofcitrus greening disease or Huanglongbing (HLB) (Garnieret al., 2000). Citrus trees infected with L. asiaticus are oftenasymptomatic for years, but they ultimately show signs ofdisease, including yellowing of leaf veins, leaf mottling, andmisshapen and poor-tasting fruit before dying as a result ofthe infection (da Graça, 1991). D. citri rapidly colonized allcitrus-growing regions in Florida after it was discovered in

* Corresponding author. Fax: +1 352 392 0190.E-mail address: [email protected] (J.M. Meyer).

0022-2011/$ - see front matter © 2007 Elsevier Inc. All rights reserved.doi:10.1016/j.jip.2007.01.005

1998 (Halbert, 1998; Knapp et al., 1998; Halbert et al.,2000). In 2005, citrus trees infected with HLB were foundnear Homestead, FL, and more infected trees have sincebeen detected in multiple counties (Halbert, 2005; BouVard,2006; http://www.doacs.state.X.us/pi/chrp/greening/maps/cgsit_map.pdf). Integrated pest management tactics areneeded to slow the spread of HLB to uninfected citrusgroves and nurseries.

Currently, suppression of D. citri populations isachieved by chemical applications and biological control(Rae et al., 1997; Browning et al., 2006). Foliar insecticidesare most eVective when the density of D. citri is high, partic-ularly during the early phase of each Xush (tender newgrowth) cycle (Browning et al., 2006; Stansly and Rogers,2006). Natural enemies of D. citri in Florida include nativelady beetles, lacewings, spiders (Michaud, 2004) and the

Page 2: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

102 J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109

specialist parasitoid, Tamarixia radiata (Waterston)(Hymenoptera: Eulophidae), which was released in a classi-cal biological control program (Hoy et al., 1999; Hoy andNguyen, 2000; Skelley and Hoy, 2004).

Several species of entomopathogenic fungi, includingPaecilomyces fumosoroseus (Wize) A.H.S. Brown and G.Smith (Samson, 1974; Subandiyah et al., 2000), Hirsutellacitriformis Speare (Rivero-Aragon and Grillo-Ravelo,2000; Subandiyah et al., 2000; Étienne et al., 2001), Cepha-losphorium lecanii Zimm (Verticillium lecanii) (Rivero-Ara-gon and Grillo-Ravelo, 2000; Xie et al., 1988), Beauveriabassiana (Bals.) Vuill. (Rivero-Aragon and Grillo-Ravelo,2000), Cladosporium sp. nr. oxysporum Berk. and M.A. Cur-tis (Aubert, 1987) and Capnodium citri Berk. and Desm.(Aubert, 1987) have been found to infect D. citri worldwide.In most cases, mycosed D. citri have been observed duringperiods of high relative humidity. In Florida, Halbert andManjunath (2004) noted the occurrence of an unidentiWedfungal pathogen that attacked D. citri, but no informationon the pathogen was provided. This research was initiatedfollowing the Weld collection of mycosed D. citri betweenSeptember 2005 and February 2006 in citrus groves in Flor-ida. We report the morphology and biology of this fungalpathogen associated with D. citri populations in Florida.

2. Materials and methods

2.1. Insect colony

D. citri were reared according to a method modiWedfrom Skelley and Hoy (2004). Small citrus trees, approxi-mately 30–50 cm tall and grown in 15.2 cm diameter pots,were used to maintain D. citri in a greenhouse at 20–32 °Cwith a 16L:8D photoperiod. Twenty trees were pruned eachweek, fertilized with Peter’s 20–20–20 (N–P–K) water-solu-ble fertilizer (United Industries, St. Louis, MO), andwatered as necessary. Approximately 2 weeks after pruning,the trees produced Xush and were placed inside wooden-framed mesh cages (0.76 m£ 0.91 m£ 1.11 m) where adultD. citri females were allowed to oviposit. Upon emergence,adult D. citri were aspirated and transferred to anothercage to initiate the next generation.

2.2. Collection, maintenance, and cultivation of the D. citri pathogen

Between September 2005 and February 2006, mycosedD. citri were collected at four sites in three counties of Flor-ida on orange and grapefruit trees (Hendry county:26°20.307�N, 80°54.597�W; Marion county: 28°58.943�N,81°51.098�W; Polk county: 28°03.656�N, 81°34.937�W;28°06.295�N, 81°42.895�W). Mycosed psyllids were foundduring each trip to the Weld, and they were collected oncefrom Hendry county, twice from Marion county, and fourtimes from Polk county. The cadavers were transported tothe laboratory, and the pathogen collected in Polk countyand Marion county was maintained by in vivo passage con-

ducted at weekly intervals, as follows. Ten to twentyhealthy adult D. citri were collected from the laboratorycolony in a sterile 50-mL centrifuge tube and placed on icefor 10–15 min. The immobilized psyllids, held with Wne-tipped forceps, were touched to conidial-bearing synne-mata present on fresh cadavers of mycosed D. citri. Treatedpsyllids were placed individually in 50-mL centrifuge tubescontaining a single mature citrus leaf with a water-soakedcotton ball placed under the cap to maintain a relativehumidity (RH) of approximately 100% and held at 24–25 °C with a 16L:8D photoperiod.

Cadavers of adult D. citri collected in Polk county andMarion county were used to initiate in vitro cultures. Singleconidia were inoculated on 6-cm diameter plates containingquarter-strength Sabouraud dextrose agar + 1% yeastextract (SDAY). Sub-cultures of the pathogen were main-tained by transferring a 1-cm square section from a 2- to 3-week-old culture to a fresh SDAY plate. In order to pro-duce sporulating cultures of the fungus, additional in vitrocultures were initiated by inoculating approximately 4 g ofautoclaved boiled rice with mycelia produced on the SDAYplates (Sosa Gomez, 1991). Hemolymph samples from sur-face-sterilized infected D. citri were harvested and used toinoculate TNM-FH insect tissue culture media (Sigma, St.Louis, MO) + gentamycin (50 �g/mL) + 5% fetal calf serumto cultivate the hyphal body phenotype. All in vitro cultureswere maintained in a growth chamber at 26 °C withoutlight. A subculture of the fungus, initiated from an adult D.citri cadaver collected in Polk county, was deposited in theUSDA-ARS Collection of Entomopathogenic Fungal Cul-tures (ARSEF 8315).

2.3. Microscopy

The fungus on the insect host was examined with a dis-secting microscope and photographed using the Auto-Montage Pro system using software ver. 5.02 (Synoptics,Frederick, MD). Hemolymph samples from infected adultD. citri were examined with diVerential interference con-trast (DIC) microscopy at 360–1000£. For SEM, cadaversof mycosed D. citri collected in Polk county were Wxed inOsO4 vapors for 48 h, dehydrated in an ethanol series, criti-cal-point dried using a Bal-Tec 030 critical point dryer,sputter coated with Au/Pt alloy and examined on a Hitachi4000 FE-SEM operating at 4–6 kV (Quattlebaum and Car-ner, 1980). Measurements of all digitally captured subjectswere made using SPOT software 3.4.3 (Diagnostic Instru-ments, Sterling Heights, MI).

2.4. Bioassays

Qualitative bioassays were conducted to test the infectiv-ity of the D. citri pathogen against a laboratory colony ofD. citri. Three replicates over time were conducted usingadults, each including 20 healthy D. citri that were exposedto mycosed psyllids as described above or not treated (con-trol). Third-instar nymphs were also assayed with the fun-

Page 3: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109 103

gal pathogen. Tender Xush from sour orange trees was usedto support nymphal development. Two replicates of ten D.citri nymphs were exposed to the mycosed adult psyllids ornot treated (control). Mortality due to fungal pathogenesiswas monitored daily in the bioassays, which were con-ducted at 25 °C with a 16L:8D photoperiod. To test theeVect of treatment on mean percentage mortality the datawere subjected to a one-way analysis of variance withPROC GLM, and the least squares means were separatedusing a probability of a signiWcant divergence of P < 0.05(SAS Institute, 1996).

To determine if the pathogen requires a speciWc site toinitiate infection, the head, thorax or abdomen of Wve adultD. citri was exposed to the synnemata on mycosed adultpsyllids and held as described above. Five psyllids were nottreated and maintained as the control. Ten adult D. citriwere exposed to the in vitro cultures propagated on rice ornot treated (control) and held as described above.

2.5. Molecular analyses

DNA was isolated from synnemata arising from D. citricadavers and also from in vitro cultures initiated from thepathogen collected in Polk county and Marion county.Fungal tissue was homogenized and processed usingPUREGENE reagents according to the manufacturer’sinstructions (Gentra Systems, Minneapolis, MN). DNApellets were air dried for 30 min to remove any remainingEtOH, re-suspended in 25–50 �l sterile water and stored at¡70 °C. A high-Wdelity polymerase chain reaction (PCR)was used to amplify the 18S small ribosomal subunit (SSU)with primers NS1 (5�-GTAGTCATATGCTTGTCTC-3�)and FS2 (5�-TAGGNATTCCTCGTTGAAGA-3�) (Nikohand Fukatsu, 2000), the 5� variable region of the 28S largeribosomal subunit (LSU) with primers LS1 (5�-AGTACCCGCTGAACTTAAG-3�) and LR5 (5�-CCT-GAGGGAAACTTCG-3�) (Hausner et al., 1993; Rehnerand Samuels, 1995), and the �-tubulin gene with degenerateprimers betatubF (5�-TGGGCYAARGGYCACTACACYGA-3�) and betatubR (5�-TCAGTGAACTCCATCTCRT CCAT-3�) (Tartar et al., 2002). The high Wdelity PCR(50 �l) included 50 mM Tris, pH 9.2, 16 mM ammoniumsulfate, 1.75 mM MgCl2, 350 mM dNTPs, 800 pmol of prim-ers, 1 U Pwo DNA polymerase and 5 U of Taq DNA Poly-

merase (Roche Molecular Biochemicals, Indianapolis, IN)(Barnes, 1994; Hoy et al., 2001). The PCR cycling parame-ters included three linked temperature proWles: (i) 1 cycleconsisting of denaturation at 94 °C for 2 min; (ii) 10 cycles,each consisting of denaturation at 94 °C for 10 s, annealingat 50 °C for 30 s, and elongation at 68 °C for 1 min; and (iii)25 cycles, each consisting of denaturation at 94 °C for 10 s,annealing at 50 °C for 30 s, and extension at 68 °C for 1 minplus an additional 20 s for each consecutive cycle (Hoy andJeyaprakash, 2005). PCR products were separated on 1%agarose TAE gels, stained with ethidium bromide, and visu-alized with ultraviolet light. PCR products were puriWedusing QIAquick PCR PuriWcation Kit (Qiagen, Valencia,CA) and cloned into the pCR2.1 TOPO plasmid (Invitro-gen, Carlsbad, CA). Plasmid DNA was isolated from over-night cultures of randomly picked Escherichia coli coloniesusing Qiagen Plasmid Mini columns (Valencia, CA), andthe size of the inserts was conWrmed with EcoRI digestionfollowed by gel electrophoresis. Three clones of each genewere bidirectionally sequenced using an ABI Prism DNASequencer at the Interdisciplinary Center for Biotechnol-ogy Research Core Facility at the University of Florida,Gainesville, FL. DNA sequences were compared to thosedeposited in GenBank with BLAST (blastn) using thedefault settings. The deduced amino acid sequences ofDNA sequences were obtained using the translate tool ofthe Expert Protein Analysis System (ExPASy) provided onthe Proteomics Server (Swiss Institute of Bioinformatics).

2.6. Phylogenetic analysis

The LSU and �-tubulin sequences from the D. citri path-ogen were used for phylogenetic analysis. Sequences fromseven Hirsutella species that had both the LSU and �-tubu-lin genes available in GenBank were included in the analysis(Table 1). Beauveria bassiana, which belongs to the phylumAscomycota, class Sordariomycetes, order Hypocreales andfamily Clavicipitaceae, was used as the outgroup taxon inthe analysis. The DNA sequences were aligned with CLUS-TAL X v. 1.83 (Thompson et al., 1997), and the LSU and �-tubulin sequences were combined according to species usingMacClade 4.0 software (Maddison and Maddison, 2000).The putative intron region of the �-tubulin gene wasremoved from the data matrix for each taxon because the

Table 1Collection data and GenBank Accession Nos. for taxa used in phylogenetic analysis

Taxon Collection data GenBank Accession No.

Host Origin LSU �-Tubulin

B. bassiana Orthoptera: Gryllotalpidae Brazil DQ075680 DQ079603H. citriformis Hemiptera: Delphacidae Indonesia DQ075678 DQ079601H. guyana Hemiptera: Cicadellidae Philippines DQ075676 DQ079598H. homalodiscae Hemiptera: Cicadellidae FL, USA DQ075674 DQ079600H. kirchneri Acari: Eriophyidae UK AY518382.1 DQ079597H. nodulosa Lepidoptera: Pyralidae MI, USA DQ075675 DQ079596H. repens Hemiptera: Delphacidae Korea DQ075679 DQ079602H. thompsonii Acari: Eriophyidae FL, USA DQ075673 DQ079595Florida Hirsutella isolate from D. citri Hemiptera: Psyllidae FL, USA EF363707 EF363706

Page 4: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

104 J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109

high degree of variability in that region prevented an accept-able DNA sequence alignment. The Wnal data matrix wascomposed of 1128 total characters (505 bp of LSU, 623 bp of�-tubulin). The incongruence length diVerence (ILD) testwas used to measure incongruence between the LSU and �-tubulin datasets (Farris et al., 1995). Maximum likelihood(ML) and maximum parsimony (MP) analyses were con-ducted using heuristic searches implemented in PAUP¤

4.0b4a. For ML, the MODELTEST 3.7 program (Posadaand Crandall, 1998) was executed on the data matrix toselect the best-Wt nucleotide substitution model for the align-ment. The substitution rate-matrix parameters and shapeparameter (�) were estimated via ML. Support for eachbranch in the ML tree was generated by the bootstrapmethod (100 replicates) in PAUP¤. For MP, the number ofparsimony-informative characters, tree length, consistencyindex and retention index were obtained in PAUP¤.

2.7. Isolate-speciWc PCR

Isolate-speciWc PCR primers were designed based on theunique putative intron region of the �-tubulin sequence ofthe Florida Hirsutella isolate from D. citri. The forwardprimer (betatub_intF: 5�-GGCTTCCAGATCACCC-ACTC-3�) was designed at the 5� portion of the �-tubulinsequence (5�Dposition 71), and the reverse primer (beta-tub_intR: 5�-TATCCACCTTCGTCAGCACA-3�) wasnested within the unique putative intron region(5�Dposition 707) (Table 2). The speciWcity of the primerswas tested on DNA isolated from in vivo and in vitro cul-tures of the Florida Hirsutella isolate from D. citri and onDNA samples from in vitro cultures of Wve related Hirsu-tella species (H. citriformis ARSEF 2346, H. guyanaARSEF 878, H. homalodiscae, H. nodulosa ARSEF 5473,and H. thompsonii). A 0.9–1.0 kb portion of the �-tubulingene was ampliWed from each DNA sample using degener-ate primers (see above) to control for template quality.PCRs (25 �l) included 5 units of Taq DNA Polymerase, 1£PCR buVer (Bioline USA, Inc., Randolph, MA), 350 mMdNTPs, and 800 pmol of primers. Standard PCR cyclingparameters included an initial denaturation at 94 °C for2 min, followed by 35 cycles of denaturation at 94 °C for

30 s, primer annealing at 58 °C for 30 s, and extension at68 °C for 1 min. A Wnal extension of 5 min at 68 °C con-cluded the assay. PCR products were analyzed by gel elec-trophoresis as described above.

3. Results

3.1. Collection, maintenance and cultivation of the D. citri pathogen

Between September 2005 and January 2006, a total of365 mycosed adult D. citri were collected from orange andgrapefruit trees at four sites in central Florida. No mycosedD. citri nymphs were detected during the Weld collection.Mycosed adult D. citri (Fig. 1A) were attached to theundersurface of leaves, the stems of leaves, or on the under-surface of branches by a brown-colored mycelial matunderlying the ventral portion of the head and thorax.

The pathogen was propagated in vivo by exposinghealthy adult and immature psyllids to the synnemataborne on the Weld-collected cadavers. One week post-inocu-lation at 24–25 °C, topically treated adult psyllids displayedchanges in behavior. Disease symptoms included tremblinglegs and antennae, wing Xicking, reduced Xight and a dark-ening of the cuticle on the head and thoracic regions. Aftersuccumbing to the pathogen, adult D. citri had pale-whitehyphae emanating from the intersegmental regions of thelegs, thorax, head, and from the tip of the abdomen. This“early” phenotype was not observed in the Weld-collectedcadavers. Mycosed adult psyllids were situated in a feedingposition with their head down and secured to the leaf orside of the centrifuge tube by a mycelial mat. Synnematawere Wrst observed protruding from the head and abdomenof D. citri cadavers 14 and 16 days after inoculation, respec-tively, when held at 24–25 °C. The distal regions of the syn-nemata were covered with small drops of mucus. Cadaversof D. citri remained infectious for at least 10 weeks in thelaboratory when held at 24–25 °C in the centrifuge tube, butinfectivity after 10 weeks was not assessed. Mycosed imma-ture D. citri (Fig. 1B) had white hyphae that emerged ini-tially from the intersegmental regions of the legs and thendeveloped to cover the entire dorsal surface of the insect.

Table 2Putative intron sequences for the Florida Hirsutella isolate from D. citri ARSEF 8315 (GenBank Accession EF363706) and H. citriformis ARSEF 2346(GenBank Accession DQ079601)

The 5�-GTA and AG-3� eukaryotic consensus boundaries are underlined at each end, and the putative stop codon in the H. citriformis ARSEF 2346sequence is in boldface (sequence position number 584-586). The isolate-speciWc reverse PCR primer (reverse complement shown in boldface: sequenceposition number 688-707) was nested in the unique putative intron sequence of the Florida Hirsutella isolate from D. citri and used to discriminate it fromWve other Hirsutella species.

Species Size (bp)

�-tubulin intron sequence

Florida Hirsutella isolate from D. citri ARSEF 8315

628 70881 GTACGTGCCGCGGGCCCCTCGAACCGGCCTCTCCATAGCCCCTCCGCTCCCCCTCGAGCCTGTGCTGACGAAGGTGGATAG

H. citriformis ARSEF 2346

571 65383 GTACGTGACGCCATGATGGCTCGAGGCCGTCGCCTCCCATTGCCCCCTTGCTCGAGCTCCTACTGACGATGGCCCACCCACAG

Page 5: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109 105

In vitro cultures propagated on SDAY media were slowgrowing at 26 °C, with a diameter measuring 1.3§0.1 (stan-dard deviation) cm after 7 days, 2.3§0.1 cm after 14 daysand 3.5§0.1 cm after 21 days (ND3). These culturesappeared white to light-gray 2 weeks post-inoculation anddid not sporulate during the 5 weeks that they were main-tained. An in vitro culture grown on rice produced multiplesynnemata on a single rice grain 6 weeks post-inoculation.The morphology and number of these synnemata was simi-lar to the synnemata produced on a single adult D. citri.

3.2. Microscopy

A total of 9§ 3 (mean§ standard deviation) synnematawere observed extending from the body of Weld-collectedmycosed D. citri (ND 10). The synnemata, measuring0.8§0.6 mm in length (ND90) and 50.0§10.0 �m in diam-eter (ND90), were simple or possessed short lateralbranches (Fig. 1A). Monophiladic conidiogenous cellsarose laterally and terminally from the distal portions ofthe synnemata (Fig. 1C–D). Conidiogenous cells measured

Fig. 1. Light and electron micrographs of the Florida isolate of Hirsutella from D. citri. (A) synnemata borne on mycosed adult D. citri, (B) deceasedimmature D. citri bearing fungal hyphae, (C) apex of synnemata showing mononematious philades, (D) high magniWcation of the philade and conidia, and

(E) septate hyphal body isolated from hemolymph of infected adult D. citri. Scale bars: (A) 0.5 mm, (B) 0.4 mm, (C) 15.0 �m, (D) 3.0 �m, and (E) 5.0 �m.
Page 6: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

106 J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109

17.5§1.9 �m in length (ND 16) and had an ellipsoid basewith a diameter of 2.6§ 0.5 �m that tapered to 0.6§0.1 �mat the conidial apex (ND 15). Each phialide produced a sin-gle, smooth-walled, aseptate, fusiform or ellipsoidal conid-ium averaging 5.9§ 0.8 �m (ND 17) in length and2.6§0.3 �m in diameter (ND16) (Fig. 1C–D).

One week post-inoculation, microscopic examination ofthe hemolymph isolated from diseased adult D. citrirevealed an abundance of septate hyphal bodies measuring26.7§6.2 �m long and 4.8§0.4 �m in diameter (ND20)(Fig. 1E). Observations using DIC microscopy conWrmedthat the in vitro cell phenotype derived from hemolymphsamples closely resembled the structure of the hyphal bod-ies isolated from the hemolymph of infected D. citri.

3.3. Bioassays

No sign of disease was observed in the control psyllidsthroughout the bioassays, and no mortality was observed inimmature and adult psyllids exposed to the pathogen untildays 5 and 6, respectively. The mean time after inoculationuntil death was 7.4§0.7 (standard deviation) days in adults(ND60) and 5.3§ 0.6 days in immatures (ND 20) at 25 °Cand approximately 100% RH. After 6 days, there was 8.3%mortality in adult psyllids exposed to the pathogen, whichwas signiWcantly greater than the 1.7% mortality in the con-trols (FD 8.0; dfD 1, 4; P < 0.05). After 9 days, 100% mortal-ity was observed in adult psyllids exposed to the pathogen,which was signiWcantly greater than the 1.7% mortality inthe controls (FD 3481; dfD1, 4; P < 0.0001).

Inoculations targeting speciWc tagma (head, thorax orabdomen) of adult D. citri all resulted in infections thatcaused mortality, while no mortality was observed in thecontrols. The adult D. citri exposed to synnemata producedin vitro on rice all succumbed to the pathogen between 9and 10 days after they were inoculated at 25 °C and approx-imately 100% RH. These psyllids exhibited the same behav-ioral symptoms of disease as previously described, and nomortality was observed in the controls.

There was 75% mortality in immature psyllids exposedto the pathogen after 5 days, which was signiWcantly greaterthan the 5% mortality in the controls (FD98; dfD 1, 2;PD 0.01). After 7 days, 100% mortality was observed inimmature psyllids, which was signiWcantly greater than the5% mortality in the controls (FD 361; dfD 1, 2; PD0.003).The synnemata structure found commonly on adultsformed on only one of the twenty immature D. citriexposed to the pathogen in the bioassays.

3.4. Molecular analyses

DNA prepared from the synnemata of a D. citri cadavercollected in Polk county and from three asporogenousin vitro cultures was subjected to PCR ampliWcation withgene-speciWc primers. The three in vitro cultures used in theanalysis included two cultures initiated from mycosed adultD. citri from Polk county (propagated on SDAY media)

and one culture initiated from hyphal bodies found in thehemolymph of infected adult D. citri inoculated by expo-sure to a cadaver from Polk county (grown in TNM-FHinsect tissue culture media). Sequence data from the fourDNA preparations were obtained for the SSU (1521 bp,GenBank Accession No. EF363708), LSU (896 bp, Gen-Bank Accession No. EF363707) and �-tubulin (964 bp,GenBank Accession No. EF363706) genes, and thesesequences were 100% identical for each gene analyzed ineach DNA preparation. This indicated that the in vitro iso-lates were identical to the isolate that infected D. citri. Anadditional �-tubulin gene sequence was ampliWed fromDNA extracted from an in vitro culture (grown on SDAY)initiated from a D. citri cadaver collected in Marion county.This sequence was 100% identical to the �-tubulin sequenceof the pathogen collected in Polk county, which suggestedthat the same isolate was infecting D. citri at each location.

A BLAST search of the SSU, �-tubulin, and LSU genesequences produced signiWcant alignments to thesesequences from species in the phylum Ascomycota andclass Sordariomycetes. The most signiWcant alignment forboth the SSU and �-tubulin genes was to sequences fromH. citriformis Speare, in the order Hypocreales and familyClavicipitaceae. The next most signiWcant alignments forthe SSU gene were to SSU sequences of various Paecilomy-ces and Cordyceps species, all classiWed in the Clavicipita-ceae. The next most signiWcant alignments for the �-tubulingene were to �-tubulin sequences in the family Chaetosph-aeriales. The most signiWcant alignments of the LSU genewere to LSU sequences from various Cordyceps species inthe Clavicipitaceae.

The SSU sequence was 99% identical (1516/1521 bp) tothe SSU sequence of H. citriformis isolated from D. citri inIndonesia (GenBank Accession No. AB032476) (Subandi-yah et al., 2000). The LSU sequences were 97% identical tothe available LSU sequences of the H. citriformis isolatesARSEF 532 (750/766 bp: GenBank Accession No.AY518376) and ARSEF 2346 (585/600 bp: GenBankAccession No. DQ075678), which were 100% identical toeach other in the 600 bp available for comparison. Therewere no sequences available in GenBank for the SSU of H.citriformis isolates ARSEF 532 or ARSEF 2346.

The most similar �-tubulin gene sequence deposited inGenBank was a 915-bp sequence from H. citriformis isolateARSEF 2346 (GenBank Accession No. DQ079601). How-ever, when these two sequences were aligned, the �-tubulinsequence for the Florida Hirsutella isolate from D. citri was57 nucleotides longer at the 5� end, and the H. citriformissequence had an additional 6 nucleotides at the 3� end. The�-tubulin sequences were 95% identical (508/534 bp)between bases 94 and 627 (bases 37–570 of the H. citrifor-mis isolate ARSEF 2346) and 95% identical (242/256 bp)between bases 709 and 964 (bases 654–909 of ARSEF2346). These sequences were diVerent between bases 58 and93 (bases 1–36 of the H. citriformis isolate ARSEF 2346),and 628 and 708 (bases 571–653 of the H. citriformis isolateARSEF 2346). There were no sequences available in Gen-

Page 7: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109 107

Bank for the �-tubulin gene of H. citriformis isolateARSEF 532 or the H. citriformis isolate collected on D.citri in Indonesia (Subandiyah et al., 2000).

The �-tubulin gene sequence of the Florida Hirsutellaisolate from D. citri is unique because it does not contain astop codon within a putative intron, whereas all other avail-able �-tubulin sequences from Hirsutella isolates contain astop codon in this region (Boucias et al., 2006). The 5�-GTAand AG-3� eukaryotic consensus boundaries were detectedat position 628 and 708 of the �-tubulin sequences from theFlorida Hirsutella isolate from D. citri and at 571–653 of H.citriformis isolate ARSEF 2346, respectively, that Xankedputative introns (Table 2). The putative intron sequences ofthe Florida Hirsutella isolate from D. citri (81 bp) and H.citriformis (83 bp) are larger than the putative intronsequences of other Hirsutella species, which range from 53–59 bp (Boucias et al., 2006). When the putative introns andthe unique sequences at the 5� region were removed fromthe �-tubulin sequences of the Florida Hirsutella isolatefrom D. citri and from H. citriformis isolate ARSEF 2346,the deduced amino acid sequences were 99% identical (261/263 amino acids). However, it has not been demonstratedthat the putative introns of the �-tubulin mRNA are actu-ally spliced out, so the intron-free deduced amino acidsequences may not reXect the actual amino acids of the �-tubulin proteins in vivo.

3.5. Phylogenetic analyses

The �-tubulin and LSU DNA sequences from the D.citri pathogen, seven related Hirsutella species, and the out-group Beauveria bassiana were used to compile a datamatrix for ML and MP analysis (Table 1). The GTR + Gnucleotide substitution model (Yang et al., 1994) wasselected by MODELTEST for the data matrix and used forML analysis. The topologies of the ML and MP (154 parsi-mony-informative characters, tree lengthD 487, consistencyindexD0.6920, retention indexD0.4643) trees were identi-cal. The consensus tree shown in Fig. 2 was rooted with theoutgroup B. bassiana. However, the bootstrap method didnot support separation of H. thompsonii from the outgroup.The remaining Hirsutella species were clustered in a mono-phyletic clade supported by the bootstrap method (66%)that consisted of two groups. Group I had Wve taxa includ-ing a clade that paired H. guyana and H. homalodiscae(bootstrap 100%), a second clade linking H. repens with H.kirchneri (bootstrap 99%), and a third clade including H.nodulosa. Group II consisted of a single clade linking H.citriformis isolate ARSEF 2346 with the Florida Hirsutellaisolate from D. citri, and the pairing was strongly sup-ported by the bootstrap method (99%). The ILD test indi-cated that the data partitions for the intron-free �-tubulinand LSU sequences were heterogeneous (PD 0.01), so thedatasets were analyzed separately by MP. In these analyses,the clades that paired H. guyana with H. homalodiscae andthe D. citri pathogen with H. citriformis remained as sistertaxa, respectively, but rearrangements in the topology were

observed for some of the other Hirsutella species, as waspreviously reported by Boucias et al., 2006.

3.6. Isolate-speciWc PCR

PCR assays conducted using the isolate-speciWc PCRprimers designed based on the �-tubulin gene produced a0.6-kb product with template DNA extracted from bothin vivo and in vitro cultures of the Florida Hirsutella isolatefrom D. citri. No ampliWcation products were detected inPCR assays that included DNA extracted from a healthyadult psyllid or DNA prepared from in vitro cultures of Wveother related Hirsutella isolates. However, ampliWcation ofa 0.9–1.0 kb portion of the �-tubulin gene using degenerateprimers was observed in all samples, demonstrating thateach DNA template was adequate for the PCR and con-tained the �-tubulin gene sequence.

4. Discussion

This research resulted in the morphological and molecu-lar characterization of a Hirsutella isolate related to H. citr-iformis found infecting D. citri in Florida citrus grovesduring 2005 and 2006. The structure of the synnemata, phi-lades, and conidia formed on mycosed D. citri were similarto the characteristics of Hirsutella citriformis described onthe brown planthopper Nilaparvata lugens Stål (Hemiptera:Delphacidae) (Brady, 1979). The measurements of theconidia were also consistent with this description, but therewere distinct diVerences in length of the philades (30–40 �m) and diameter of the synnemata (200–300 �m) pro-duced on N. lugens (Brady, 1979) in comparison to thosestructures found on mycosed D. citri (philadelengthD17.5§1.9 �m, synnemata diameterD 50.0§10.0 �m). The �-tubulin gene sequence of the Florida Hirsu-tella isolate from D. citri was novel and diVerent from the

Fig. 2. Consensus phylogenetic tree using ML and MP for the FloridaHirsutella isolate from D. citri and related fungal species.

Page 8: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

108 J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109

�-tubulin gene sequence from H. citriformis isolate ARSEF2346. Boucias et al. (2006) also reported that size and nucle-otide diVerences were found in the putative intron region ofthe �-tubulin gene among other Hirsutella species. The iso-late-speciWc PCR primers that were based on the uniqueputative intron region of the �-tubulin sequence distin-guished the Florida Hirsutella isolate from D. citri from Wveother related Hirsutella species, and will be useful to iden-tify the pathogen on D. citri cadavers in future studiesaddressing its frequency and distribution.

The origin of the Florida Hirsutella isolate from D. citriis unknown, but it may have accompanied D. citri to Flor-ida. This Hirsutella isolate is not the same as the Indonesianisolate of H. citriformis from D. citri due to diVerences inthe SSU sequence (Subandiyah et al., 2000). However, thispathogen could represent an as-of-yet unidentiWed AsianH. citriformis. Alternatively, an indigenous isolate of Hirsu-tella may have adapted to D. citri after it became estab-lished in Florida. Hirsutella citriformis was previouslyidentiWed in Florida (Mains, 1951), and it is known toattack D. citri in Cuba and Guadeloupe (Rivero-Aragonand Grillo-Ravelo, 2000; Étienne et al., 2001) and a varietyof other hemipteran insects including the leucaena psyllid(Mains, 1951; Rombach and Roberts, 1987; Sajap, 1993;Hywel-Jones, 1997; Villacarlos and Robin, 1999).

The studies of the pathogen–host interaction were madepossible using cultures of the pathogen maintained in thelaboratory. Conidia produced in vivo and in vitro werepathogenic to psyllids and produced a phenotype similar tothat observed in Weld-collected cadavers, thus fulWllingKoch’s postulates. We do not yet know if the pathogenoverwhelmed D. citri during its vegetative stage, killed thepsyllids with toxins such as hirsutellins, hirsutides and hir-sutatins (Mazet and Vey, 1995; Liu et al., 1995; Isaka et al.,2005; Lang et al., 2005), or used a combination of theseeVects. The susceptibility of immature psyllids to the patho-gen was consistent with that reported by Étienne et al.(2001), in which immature D. citri were killed by H. citrifor-mis in Guadeloupe.

In the Weld, survival and transmission of this pathogenwere likely enhanced by the production of synnemata,which emanated laterally from the cadavers near the ven-tral portion of the head and thorax and from the tip of theabdomen. The synnemata could provide a slow release ofinfectious conidia because only a limited number of phi-lades produced conidia simultaneously, while the vastmajority of philades remained undiVerentiated. The patho-gen could have been protected from exposure to sunlightbecause the cadavers were usually found secured to theunderside of citrus foliage. Similarly, cadavers of the sugar-cane leaf hopper killed by H. citriformis were found on theundersurface of sugarcane leaves (Singaravelu et al., 2003).Further investigation addressing the transmission mecha-nisms of the pathogen is needed to characterize the spatialand temporal patterns of the pathogen-host interaction.

Additional research also is needed to evaluate the poten-tial for using the Florida Hirsutella isolate from D. citri as

part of an integrated pest management program to sup-press D. citri populations. The fastidious nature of the fun-gal pathogen, coupled with the slow development ofcultures in vitro, may limit the likelihood of developing it asa microbial insecticide. However, studies are needed toinvestigate additional culturing methods aimed at increas-ing sporulation of the pathogen in vitro and to develop andtest a formulation of fragmented mycelia, as was conductedwith H. thompsonii to control citrus rust mite (McCoy et al.,1971). An augmentative biological control approach, inwhich live infected D. citri are released or mycosed psyllidsare manually dispersed, might also be used to increase theprevalence of the pathogen in D. citri populations in Flor-ida, or elsewhere.

Acknowledgments

The authors thank Raguwinder Singh for assistance inthe Weld collection and Verena Bläske for SEM, conductedat the University of Florida Electron Microscopy CoreFacility. We thank Reginald Wilcox for maintenance andpruning of citrus trees. Lyle Buss, Jane Medley and MikeSanford are acknowledged for their contributions to pho-tography and Wgure construction. We thank Jennifer Zas-pel for assistance with the phylogenetic analysis andHeather McAuslane for statistical advice. DNA sequencingwas conducted at the Interdisciplinary Center for Biotech-nology Research Core Facility at the University of Florida.This research was funded by the Davies, Fischer and EckesEndowment in Biological Control to M.A. Hoy.

References

Aubert, B., 1987. Trioza erytreae Del Guercio and Diaphorina citri Kuway-ama (Homoptera: Psylloidea), the two vectors of citrus greening dis-ease: biological aspects and possible control strategies. Fruits 42, 149–162.

Barnes, W.M., 1994. PCR ampliWcation of up to 35-kb DNA with highWdelity and high yield from � bacteriophage templates. Proc.Natl.Acad. Sci. USA 91, 2216–2220.

BouVard, K., 2006. Greening found in 10 counties. Citrus Ind. 87 (1), 5–26.Boucias, D.G., Scharf, D.W., Breaux, S.E., Purcell, D.H., Mizell, R.E.,

2006. Studies on the fungi associated with the glassy-winged sharp-shooter Homalodiscae coagulata with emphasis on a new speciesHirsutella homalodiscae nom prov. BioControl. doi:10.1007/S10526-006-9019-3.

Brady, B.L.K., 1979. Hirsutella citriformis [Descriptions of fungi and bac-teria]. IMI Descriptions of Fungi and Bacteria 61, 607.

Browning, H.W., Childers, C.C., Stansly, P.A., Peña, J., Rogers, M.E., 2006.Florida Citrus Pest Management Guide: Soft-Bodied Insects Attack-ing Foliage and Fruit: University of Florida IFAS Extension. <http://edis.ifas.uX.edu/BODY_CG004/>.

Étienne, J., Quilici, S., Marival, D., Franck, A., 2001. Biological control ofDiaphorina citri (Hemiptera: Psyllidae) in Guadeloupe by importedTamarixia radiata (Hymenoptera: Eulophidae). Fruits 56, 307–315.

Farris, J.S., Källersjö, M., Kluge, A.G., Bult, C., 1995. Testing signiWcanceof incongruence. Cladistics 10, 315–319.

Garnier, M., Jagoueix, E.S., Cronje, P.R., Le Roux, H.F., Bove, J.M., 2000.Genomic characterization of a Liberibacter present in an ornamentalrutaceous tree, Calodendrum capense, in the Western Cape province ofSouth Africa. Proposal of ’Candidatus Liberibacter africanus subsp.capensis’. Int. J. System. Evol. Microbiol. 50, 2119–2125.

Page 9: Morphological and molecular characterization of a Hirsutella species infecting the Asian citrus psyllid, Diaphorina citri Kuwayama (Hemiptera: Psyllidae), in Florida

J.M. Meyer et al. / Journal of Invertebrate Pathology 95 (2007) 101–109 109

da Graça, J.V., 1991. Citrus greening disease. Annu. Rev. Phtopathol. 19,109–136.

Halbert, S.E., 1998. Asian citrus psyllid—a serious potential exotic pest ofFlorida citrus. <http://www.ifas.uX.edu/~entweb/DCITRI.htm/>.

Halbert, S.E. 2005. Pest Alert: Citrus Greening/Huanglongbing. <http://www.doacs.state.X.us/pi/chrp/greening/citrusgreeningalert.html/>.

Halbert, S.E., Manjunath, K.L., 2004. Asian Citrus Psyllids (Sternorrhyn-cha: Psyllidae) and greening disease of citrus: a literature review andassessment of risk in Florida. Florida Entomol. 87, 330–353.

Halbert, S.E., Sun, X., Dixon, W.N., 2000. Asian citrus psyllid and citrusgreening disease. Citrus Ind. 91 (5), 22–24.

Hausner, G., Reid, J., Klassen, G.R., 1993. On the subdivision of Ceratocystiss.l., based on partial ribosomal DNA sequences. Can. J. Bot. 71, 52–63.

Hoy, M.A., Nguyen, R., 2000. Classical biological control of Asian citruspsylla. Citrus Ind. 81 (12), 48–50.

Hoy, M.A., Jeyaprakash, A., Nguyen, R., 2001. Long PCR is a sensitivemethod for detecting Liberobacter asiaticum in parasitoids undergoingrisk assessment in quarantine. Biol. Control 22, 278–287.

Hoy, M.A., Nguyen, R., Jeyaprakash, A., 1999. Classical biological controlof Asian citrus psylla. Citrus Ind. 80 (9), 20–22.

Hywel-Jones, N., 1997. Hirsutella species associated with hoppers(Homoptera) in Thailand. Mycol. Res. 101, 1202–1206.

Isaka, M., Palasarn, S., Sriklung, K., Kocharin, K., 2005. Cyclohexadepsi-peptides from the insect pathogenic fungus Hirsutella nivea BCC 2594.J. Nat. Prod. 68, 1680–1682.

Knapp, J.L., Halbert, S., Lee, R., Hoy, M., Clark, R., Kesinger, M., 1998. TheAsian citrus psyllid and citrus greening disease. Citrus Ind. 79 (10), 28–29.

Lang, G., Blunt, J.W., Cummings, N.J., Cole, A.L., Munro, M.H., 2005.Hirsutide, a cyclic tetrapeptide from a spider-derived entomopatho-genic fungus, Hirsutella sp. J. Nat. Prod. 68, 1303–1305.

Liu, W.Z., Boucias, D.G., McCoy, C.W., 1995. Extraction and character-ization of the insecticidal toxin Hirsutellin A produced by Hirsutellathompsonii var. thompsonii. Exp. Mycol. 19, 254–262.

Maddison, W.P., Maddison, D.K., 2000. MacClade: Analysis of Phylogenyand Character Evolution. Version 4.0. Sinauer Associates. MD, USA.

Mains, E.B., 1951. Entomogenous species of Hirsutella, Tilachlidium andSynnematium. Mycologia 43, 691–718.

Mazet, I., Vey, A., 1995. Hirsutellin A, a toxic protein produced in vitro byHirsutella thompsonii. Microbiology 141, 1343–1348.

McCoy, C.W., Selhime, A.G., Kanavel, R.F., Hill, A.J., 1971. Suppressionof citrus rust mite populations with application of fragmented myceliaof Hirsutella thompsonii. J. Invertebr. Pathol. 17, 270–276.

Michaud, J.P., 2004. Natural mortality of Asian citrus psyllid (Homoptera:Psyllida) in central Florida. Biol. Control 29, 260–269.

Nikoh, N., Fukatsu, T., 2000. Interkingdom host jumping underground:phylogenetic analysis of entomoparasitic fungi of the genus Cordyceps.Mol. Biol. Evol. 17, 629–638.

Posada, D., Crandall, K.A., 1998. MODELTEST: testing the model ofDNA substitution. Bioinformatics 14, 817–818.

Quattlebaum, E.C., Carner, G.R., 1980. A technique for preparing Beauve-ria spp. for scanning electron microscopy. Can. J. Bot. 58, 1700–1703.

Rae, D.J., Liang, W.G., Watson, D.M., Beattie, G.A.C., Huang, M.D., 1997.Evaluation of petroleum spray oils for control for the Asian citruspsylla, Diaphorina citri (Kuwayama) (Hemiptera: Psyllidae), in China.Intl. J. Pest Man. 43, 71–75.

Rehner, S.A., Samuels, G.J., 1995. Molecular systematics of the Hypocre-ales: a telomorph gene phylogeny and the status of their anamorphs.Can. J. Bot. 73, S816–S823.

Rivero-Aragon, A., Grillo-Ravelo, H., 2000. Natural enemies of Diapho-rina citri Kuwayama (Homoptera: Psyllidae) in the central region ofCuba. Centro-Agricola 27 (3), 87–88 (Abstract only).

Rombach, M.C., Roberts, D.W., 1987. Calcarisporium ovalisporum, symbi-otic with the insect pathogen Hirsutella citriformis. Mycologia 79, 153–155.

Sajap, A.S., 1993. Prevalence of an entomopathogenic fungus, Hirsutellacitriformis on leucaena psyllid, Heteropsylla cubana, in Malaysia. Per-tanika J. Trop. Agric. Sci. 16, 95–99.

SAS Institute, 1996. SAS/STAT software; Changes and Enhancementsthrough Release 6.11.1996. SAS Institute, Cary, NC, USA.

Samson, R.A., 1974. Paecilomyces and some allied Hyphomycetes. Stud.Mycol. 6, 1–119.

Singaravelu, B., Easwaramoorthy, S., Premachandran, M.N., 2003. Epizo-otics of a fungal pathogen, Hirsutella citriformis Speare, on sugarcaneleaf hopper, Pyrilla perpusilla (Walker). Insect Env. 9, 62–63.

Skelley, L.H., Hoy, M.A., 2004. A synchronous rearing method for theAsian citrus psyllid and its parasitoids in quarantine. Biol. Control 29,14–23.

Sosa Gomez, D.R., 1991. Production of three Hirsutella thompsonii varie-ties on semi-solid media and diVerential eVects of two fungicides. Ann.Soc. Entomol. Brasil 20 (1), 155–163.

Stansly, P.A., Rogers, M.E., 2006. Managing Asian citrus psyllid popula-tions. Citrus Ind. 87 (3), 17–19.

Subandiyah, S., Nikoh, N., Sato, H., Wagiman, F., Tsuyumyu, S., Fukatsu,T., 2000. Isolation and characterization of two entomopathogenicfungi attacking Diaphorina citri (Homoptera, Psylloidea) in Indonesia.Mycoscience 41, 509–513.

Tartar, A., Boucias, D., Adams, B., Becnel, J., 2002. Phylogenetic analysisidentiWes the invertebrate pathogen Helicosporidium sp. as a greenalgae (Chlorophyta). Int. J. System. Evol. Microbiol. 52, 273–279.

Thompson, J.D., Gibson, T.J., Plewniak, F., Jeanmougin, F., Higgins, D.G.,1997. The CLUSTAL_X windows interface: Xexible strategies for mul-tiple sequence alignment aided by quality analysis tools. Nucleic AcidsRes. 25 (24), 4876–4882.

Villacarlos, L.T., Robin, R.P., 1999. Entomogenous fungi infecting Het-eropsylla cubana Crawford (Homoptera: Psyllidae) in Leyte, Philip-pines. Trop. Pest Man. 35, 120–122.

Xie, P.H., Su, C., Lin, Z.G., 1988. A preliminary study on an entomogenousfungus [Verticillium lecanii] of Diaphorina citri Kuwayama (Hom.: Psy-llidae). Chin. J. Biol. Control 4 (2), 92.

Yang, Z., Goldman, N., Friday, A., 1994. Comparison of models for nucle-otide substitution used in maximum-likelihood phylogenetic estima-tion. Mol. Biol. Evol. 11, 316–324.