Re-stating the obvious: To solve structures from powder diffraction, first they must be indexed! Lachlan M. D. Cranswick CCP14 (Collaborative Computation

$Page 1: Re-stating the obvious: To solve structures from powder diffraction, first they must be indexed! Lachlan M. D. Cranswick CCP14 (Collaborative Computation$
Re-stating the obvious: To solve structures from powder diffraction, first they must be indexed!

Lachlan M. D. Cranswick CCP14 (Collaborative Computation Project No 14 for

Single Crystal and Powder Diffraction)

Department of Crystallography;

Birkbeck College, University of London,

Malet Street, Bloomsbury, London, WC1E 7HX, UK.

E-mail: [email protected]

WWW: http://www.ccp14.ac.uk

Based on a co-authored presentation at the ACA 2002 conference in San-Antonio, Texas, USA

Lachlan M. D. Cranswick CCP14 (Collaborative Computation Project No 14 for

Single Crystal and Powder Diffraction)

Department of Crystallography;

Birkbeck College, University of London,

Malet Street, Bloomsbury, London, WC1E 7HX, UK.


WWW: http://www.ccp14.ac.uk

Robin ShirleySchool of Human Sciences

University of Surrey

Guildford, Surrey, GU2 7XH, U.K.

Tel: (+44) 01483 686864

Fax: (+44) 01483 259553


Lachlan M. D. Cranswick ([email protected]) http://www.ccp14.ac.ukSlide 3

Talk Aims• Some history of Indexing of Powder Diffraction Data• Peak finding and peak profiling software• Fundamental Parameters Peak Fitting• Available powder indexing software and indexing suites• Crysfire in action• Chekcell in action• MCMaille by Armel Le Bail (released Sep 2002)• Possibilities for the future and conclusion


Some History of Powder Indexing

•First published manual powder indexing method:

– Runge, C. (1917), "Die Bestimmung eines Kristallsystems durch Rontgenstrahlen", Physik. Z., 18, 509-515.

•Candidates for first published computer based powder indexing

– Haendler, H. M. & Cooney, W. A. (1963), "Computer Determination of Unit-Cell from Powder-Diffraction Data", Acta Cryst., 16, 1243-1248.

– Lefker, R. (1964), "Indexing of tetragonal and hexagonal X-ray powder photographs with the aid of a small computer", Anal. Chem., 36, 332-334.


Some History of Powder Indexing - 2

•Candidates for first published publicly available powder-indexing program

– Haendler, H. M. & Cooney, W. A. (1963), "Computer Determination of Unit-Cell from Powder-Diffraction Data", Acta Cryst., 16, 1243-1248.

– Lefker, R. (1964), "Indexing of tetragonal and hexagonal X-ray powder photographs with the aid of a small computer", Anal. Chem., 36, 332-334.

– Werner, P.-E. (1964), "Trial and error computer methods for the indexing of unknown powder patterns", Z. Krist., 120, 375-387.

– Hoff, W. D. & Kitchingman, W. J. (1966), "Computer indexing of x-ray powder patterns from crystals of unknown structures", J. Sci. Instrum., 43, 952-953.


Some History of Powder Indexing - 3

•Candidates for first publically available general indexing program

– Ishida, T & Watanabe, Y. (1967), "Probability Computer Method of Determining the Lattice Parameters from Powder Diffraction Data", J. Phys. Soc. Japan, 23, 556-565.

– Roof, R. B. (1968), "INDX: A Computer Program to Aid in the Indexing of X-Ray Powder Patterns of Crystal Structures of Unknown Symmetry", Los Alamos Laboratory, University of California, Report LA-3920.

– Taupin, D. (1968), "Une Methode Generale pour l'Indexation des Diagrammes de Poudres", J. Appl. Cryst., 1, 178-181.

– Visser, J. W. (1969), "A Fully Automatic Program for Finding the Unit Cell from Powder Data", J. Appl. Cryst., 2, 89-95.


Checking Sample Purity

• Powder indexing requires line-position data of the highest available quality

• An important aspect of this is purity, since indexing programs don’t like to struggle with impurity lines, and several will not accept them at all.

• Some suggestions from Robin for checking sample purity:

– Optical (polarising) microscope

– Density analysis in a gradient column

– Electron microscopy and diffraction of selected crystallites

– And, if time permits, see whether properties are changing with time


Peak Profiling Software

•For Overall Summary of available peak profiling software refer to:

http://www.ccp14.ac.uk/solution/peakprofiling/

•Some of the available freeware includes: – CMPR

– DRXWin

– EFLECH

– GPLSFT

– pearson.xls

– Rawplot (with GSAS)

– SHADOW

– Powder v2.00

– PowderX

– Winfit

– Winplotr (with Fullprof)

– XFIT


Empirical peak fitting : Some tricks may be required for stability and reasonable results

• Peaks over small ranges may have to be linked to have the same shape and width

• Usually fit small ranges of data - groups of peaks

• Might have to fit large and medium peaks first, followed by trace peaks


Peak Profiling: high accuracy peak positions using Fundamental Parameters peak profiling

•Example of Fundamental parameters (convoluting in the geometry elements of the diffractometer) that can provide accurate peak positions as though your sample was being run on an “ideal” diffractometer.

•Tutorial at:

– http://www.ccp14.ac.uk/tutorial/xfit-95/fun1.htm

•Available Fundamental Parameters Peak Profiling and Rietveld software:

•XFIT (no longer maintained)

– http://www.ccp14.ac.uk/tutorial/xfit-95/xfit.htm

•Topas (Commercial - sequel to XFIT)

– http://www.bruker.com

•BGMN (Commercial - academic demonstration version is freely downloadable)

– http://www.bgmn.de

•EFLECH/Index freeware from BGMN website

– http://www.bgmn.de/related.html


XFIT Fundamental Parameters Peak Profiling mode : fit to a low angle peak on a Bragg-Brentano diffractometer


Powder Indexing FOM Results XFIT Fundamental Parameters peak profiling compared to

empirical Peak Fitting


Powder Indexing - indexing programs• Overall Summary of available powder indexing software refer to:

http://www.ccp14.ac.uk/solution/indexing/

• Powder Indexing: – Classic (widely used): Ito, Dicvol, Treor

– High symmetry (exhaustive): Taup (=Powder), Dicvol

– Other powerful (general symmetry): Lzon, Kohl (=TMO), Fjzn

– Dominant-zone cases: Lzon, Losh, Mmap

– Semi-automatic (via SIW basis set): Losh, Mmap

– Fundamental parameters & covariance matrix: EFLECH/Index

– Incommensurate, etc: Supercell

– Monte Carlo / Random search: EFLECH/Index, McMaille

– Others (better with some user guidance): Scanix, Autox


Powder Indexing - indexing suites• Overall Summary of available powder indexing software refer to:

http://www.ccp14.ac.uk/solution/indexing/

• Integrated Suites: – Crysfire

– Powder v2.00

– PowderX

– PROSZKI

– WinPlotr

• and after primary indexing:– Chekcell


Powder Indexing - specialist programs• Supercel is a specialised indexing program by Juan Rodriguez-Carvajal for

tackling incommensurate cells and super/sub-cell relationships. (available within Winplotr/Fullprof)

Web: http://www-llb.cea.fr/winplotr/winplotr.htm

FTP: ftp://bali.saclay.cea.fr/pub/divers/fullprof.2k/


Powder Indexing - specialist programs• McMaille reconstitutes a peak listing into a full profile and performs

monte carlo based searching for good cells. Because it is Monte Carlo based it can be very slow, but due to it being whole profile, can be insensitive to trace impurity peaks.

Web: http://www.cristal.org/McMaille/


Powder Indexing - Using PowderX–by Cheng Dong– http://www.ccp14.ac.uk/tutorial/powderx/

• Peak finding• Links to Treor


Powder Indexing - Using Winplotr–by Juan Rodriguez-Carvajal and T. Roisnel

http://www-llb.cea.fr/fullweb/powder.htm

• Peak finding / profiling• Links to

– WinTreor,

– Windicvol,

– Winito and

– supercell


Powder Indexing - the Crysfire suite (1)

–by Robin Shirley (+ many contributing authors)– http://www.ccp14.ac.uk/tutorial/crys/

• Intelligent defaults for control-parameters when launching each indexing program

• Data rescaling for tackling high-volume cells from macromolecular samples and zeolites

• Integrated under a common interface for use by non-specialists



• Principal Contributors:

• Franz Kohlbeck (Kohl [=TMO])• Daniel Louër (Dicvol, Lzon, Losh)• Robin Shirley (Overall + Mmap, Lzon, Losh, Fjzn, etc)• Ton Spek & A.Meetsma (Lepage)• Daniel Taupin (Taup [=Powder])• Jan Visser (Ito, Fjzn, Lzon, etc)• Per-Eric Werner (Treor)



Crysfire 2000: 8 different indexing programs

(Ito, Dicvol, Treor, Taup, Kohl, Lzon, Fjzn, Losh)

• It’s good to have a wide variety of methods available:– to cater for different types of problem

– to give a feel for the range of solutions out there

– and see which ones turn up repeatedly using different methods, either identically or as derivative cells


New Release - Crysfire 2002

August 2002

Now 9 indexing programs:

Ito, Dicvol, Treor, Taup, Kohl, Lzon, Fjzn, Losh, Mmap

• New features:• Mmap for exploring solution-space and evaluating trial cells• Le Page/Niggli reduced-cell summary• Greater ease of use, especially for first-time users• Better resilience, especially under the Windows/NT family

(master program totally rewritten)


Why bother running more than one powder indexing program?

• Maximize the possibility of indexing unknowns by taking advantage of different algorithms and different program implementations.


Powder Indexing - Routine Crysfire ExampleRun crysfire (giving the following screen)


Powder Indexing - Routine Crysfire Example Importing an XFIT peak file using the IM (Import command)


Powder Indexing - Routine Crysfire Example“lo” to load a CDT file

estimate of expected unit cell volume is given


Powder Indexing - Routine Crysfire Example “in” to target an indexing program


Powder Indexing - Routine Crysfire Example target ITO with defaults


Powder Indexing - Routine Crysfire Example ITO runs


Powder Indexing - Routine Crysfire Example After Ito completes, press enter to see output file


Powder Indexing - Routine Crysfire Example Then “enter” again to see “one solution per line” summary


Powder Indexing - Routine Crysfire Example “Enter” again to see Le Page summary of reduced cells


Powder Indexing - Routine Crysfire Example•Repeat the sequence to run the remaining relevant indexing programs. Recommended sequence is actually:

– Taup, Dicvol high sym, Ito, FJZN, Treor, Kohl, Dicvol low sym, Lzon

– in to select and launch an indexing program

– look at the output file

– look at the summary file

– look at the Le Page summary file of reduced cells

– repeat


Powder Indexing - Routine Crysfire Example Finally: the complete summary file - 3332 trial solutions


Powder Indexing - Routine Crysfire Example Complete Le Page summary file of reduced cells - 3332 trial solutions


Crysfire Self CalibrationBragg-Bretano Data

• Sometimes it’s hard to be sure how far the beam is penetrating into the sample, giving an unknown sample-displacement error

• This can be addressed by trying self-calibration (SC)– If indexing the uncorrected data has failed, try SC

– Select T (for specimen-displacement correction ratio)

– A list of candidates for T will be displayed, based on 1st/2nd-order line pairs

– If a particular ratio comes up several times and looks plausible, adopt it

– Save it (SA) in Crysfire format under a new name, so that it can be reloaded

– Do another cycle of indexing runs, this time with the “corrected” data


Crysfire Self Calibration Example• Self calibration output of an organometallic sample in histogram format


Crysfire Rescaling

• Most indexing program have evolved using a range of data with solutions between 500 and 1500

• If you have cells outside this range, many indexing programs can fail

• A possible solution to still index large cells it to rescale the data by changing the wavelength


Crysfire Re-scalingProtein Data (from Bob von Dreele)

Input for RAWPLOT:

(1.15006 Å)1.168100

1.656041

1.852174

1.918506

2.089703

2.345176

2.536523

2.623267

2.916919

2.992132

3.144491

3.319995

3.419907

3.458655

3.522905

3.564255

3.660448

3.712916

3.838720

3.933115

4.098415

4.151839

4.187484

4.234265

Lysozyme – single peak fits for 24 reflections : Rescale factor = 0.1

Rescaled cell = a of ~7.9Å and c of ~3.8ÅUnscaled cell = a of ~79Å and c of ~38Å


Crysfire Rescaling example Inorganic - on opening : Crysfire suggests rescaling


Ignoring the rescaling suggestion No obvious solutions


RS is the Rescale Command Crysfire will automatically change the filename and title


After running the indexing programs:MS, then U to unscale the summary file


Indexing using the suggested rescaling of 0.5 An obvious cubic (23.923 A) solution found by Taup


Chekcell: Interpreting Crysfire Summary Files: Powder Indexing and Spacegroup Assignment

•Crysfire interlinks with Chekcell for Windows (part of the LMGP suite for Windows by Jean Laugier and Bernard Bochu). Chekcell provides a graphical interface for manually and automatically suggesting a best cell/spacegroup combination using both FOM and algorithms relating to parsimony of superfluous HKLs.

http://www.ccp14.ac.uk/tutorial/lmgp/


Chekcell : easy to see non-matching or impurity peaks


Chekcell : automatic cell and spacegroup searching

can trudge through a single selected unitcell; or over 1000s of trial cells looking for the best cell and spacegroup combination based on parsimony of extra reflections criteria.


Chekcell: “integration” of Ton Spek and A. Meetsma’s Le Page(Crysfire 2001 now also provides a reduced-cell summary)

•Obtaining the Reduced Cell – which in the past many powder indexing

programs have not determined either reliably or at all

– Refer: "'Reduced Cells', M.J. Buerger, (Zeitschift fur Kristallographie, BD 109, S. 42-60 (1957)”

– Crysfire 2001 gives a LePage overview Chekcell displays a fuller analysis

•Efficient Sub-cell and super-cell searching, then easy reviewing of newly derived cells within the Chekcell interface


Chekcell: GUI Cell transformation•Easily transform cells and test them withing Chekcell •Knows about common transformations•Can manually look at sub-cells and super-cells


Chekcell: Density / Z/ Mol. Vol explorer• Easily explore values of Z, density and estimated molecular volume -

comparing with your found trial cells.


Chekcell: example of it runningLoad the raw diffraction data


Chekcell: example of it runningLoad the peak position file


Chekcell: example of it runningLoad the Crysfire summary file


Chekcell: example of it runningUse the tools to evaluate the trial cells


Crysfire / Chekcell: indexing powder Protein data

Using the “Lepage” feature in Chekcell

Can find the correct rhombohedral cell as published in:

•R. B. Von Dreele, P. W. Stephens, G. D. Smith and R. H. Blessing, "The first protein crystal structure determined from high-resolution X-ray powder diffraction data: a variant of T3R3 human insulin-zinc complex produced by grinding", Acta Cryst. (2000). D56, 1549-1553.


Crysfire / Chekcell: indexing powder Protein dataEven on rescaled data, if Crysfire and the various indexing programs do not find the “true cell”, Chekcell and possibly can find a derivative cell which LePage (combined with Chekcell’s “parsimony and superflous reflections” criteria), can find the true cell. In the case of reindexing of the protein data:

– Crysfire and Dicvol finds • 73.2645 70.5256 40.7430 90.000 90.000 90.000 Orthorhomic

– P222 or PMM2 or PMMM : 157 hkls to match 41 reflections

– LePage followed by Chekcell “parsimony check” then finds:• 81.449 81.449 73.265 90 .000 90.000 120.00 Hexagonal setting

• 52.988 52.988 52.988 100.45 100.45 100.45 Rhombohedral– R-3 or R3 etc : 60 hkls to match 41 reflections


MMAP• Mmap explores sections of solution-space (e.g. sections with varying alpha* and

beta*, holding the other 4 parameters constant)

• The results are displayed as a map, with the high-ground colored

• It lists all the peaks in the map and refines them - each is a trial solution

• So it can act as an indexing program in its own right (taking the basis set containing the first 4 constants from Lzon’s output)

• This offers another way to search for trial cells

• Use the new “LC” (Load a trial cell from the Crysfire summary file) to then run MMAP (using the MM command) on it.


MMAP - what does the output look like?Example MMap thumbnail output from a high FOM trial cell


MMAP - what does the output look like?Example MMap output from a high FOM trial cell


MCMaile by Armel Le Bail

Developed in September 2002 (on from the ESPOIR code) GPL’d (free and you get the source code) Whole profile Monte Carlo and Grid Search methods using a

pattern reconstituted from the peak list Columnar peak shape - can increase/decrease width Can make “important” trace peaks more likely to be part of a

solution by increasing their intensities. Insensitive to impurity peaks 20,000 tests per second - but can still take many hours Thus a method of last resort


A MCMaile ASCII input file


MCMaile is running


MCMaile output files Chekcell *.ckm summary file of all trial cells and Fullprof PRF

profile file of top solution: viewable in Winplotr


MCMaile : on large cubic Finds cubic cell easily - even with trace “impurity” peaks Via outputted Chekcell *.ckm summary file. Top solution is not correct solution (possible effect of impurity peaks)


MCMaile : on large cubic Finds cubic cell easily - even with trace “impurity” peaks Via outputted Chekcell *.ckm summary file. Second set of solutions is correct


SDPDRR-II (Structure Determination by Powder Diffractometry Round Robin - 2 )

WWW: http://www.cristal.org/sdpdrr2/ Started: Monday 9th September Part 1 (powder indexing):

Start: Monday 9th September Finish: Sunday 13th October

8 samples 1 - 3 : Powder indexing and structure solution 4 - 8 : Powder Indexing only

> 100 downloads of the data (samples 1 to 3)


SDPDRR-II - Indexing Results WWW: http://sdpd.univ-lemans.fr/sdpdrr2/results/ 6 returns/100 downloads = 6% return rate


SDPDRR-II - Sample 1 WWW: http://sdpd.univ-lemans.fr/sdpdrr2/results/ Organisers: 10.323 7.395 8.535 90 91.29 90

Reduced cell: 7.395 8.535 10.323 91.29 90.00 90.00

P1: 10.318 7.393 8.527 90 91.3 90 P2: 10.321 7.392 13.236 90 139.90 90 P3: 10.322 14.787 8.529 90 91.35 90

Reduced cell: 8.529 10.322 14.787 90.00 90.00 91.30

P4: No solution P5: 8.523 10.327 7.397 90 90 91.34 P6: 8.525 7.394 13.230 90 128.748 90


SDPDRR-II - Sample 2 WWW: http://sdpd.univ-lemans.fr/sdpdrr2/results/ Organisers: 11.239 8.194 19.943 90 106.727 90

Reduced cell: 8.194 11.239 19.875 106.06 90.00 90.00 P1: 19.880 8.195 11.242 90 106.06 90 P2: 11.242 8.195 19.948 90 106.72 90 P3: 19.878 8.193 11.240 90 106.06 90 P4: 19.880 8.194 11.243 90 106.06 90 P5: 11.244 19.882 8.196 90 90 106.06 P6: 11.243 8.195 19.949 90 106.72 90


SDPDRR-II - Sample 3 WWW: http://sdpd.univ-lemans.fr/sdpdrr2/results/ Organisers: 18.881 18.881 18.881 90 90 90 (vol

6734Å3) P1: 13.349 13.349 9.439 90 90 90 (Tet -

1638Å3) P2: 18.878 18.878 18.878 90 90 90 P3: 13.354 13.354 9.442 90 90 90 (Tet -

1638Å3) P4: No Solution P5: 18.878 18.878 18.878 90 90 90 P6: 18.88 18.88 18.88 90 90 90


SDPDRR-II - Sample 3 Tetragonal or Cubic?

2 participants (1 and 3) obtained the same tetragonal cell instead of the correct cubic. Possibly due to default volume restrictions in the programs

used(?)

What happens if you run LePage (via Chekcell) followed by a “parsimony of extra reflections” check on the tetragonal solution


SDPDRR-II - Sample 3: Tetragonal: 1 peak unmatched - 46 hkl’s (P42MC et al) to match 25 reflections


SDPDRR-II - Sample 3: Tetragonal: Running LePage shows a cubic cell (18.885Å)


SDPDRR-II - Sample 3: Visual inspection of cubic cell implies “bad”peak could still be an impurity


SDPDRR-II - Sample 3: Cubic: 37 hkl’s to match 25 reflections (I23 et al) (vs 46 hkl’s in tetragonal)


SDPDRR-II - Samples 4 to 8 WWW: http://sdpd.univ-lemans.fr/sdpdrr2/results/ Only one response - using the EFLECH/Index software of

Joerg Bergmann (http://www.bgmn.de/related.html)

S4: none suggested S5: 6.011 16.937 18.229 90 90 92.19 S6: none suggested S7: 4.417 11.479 17.126 77.87 85.09 82.67 S8: 3.796 9.368 28.915 90 90 90

Summary: Powder indexing is not easy and not routine!


Be on the lookout for bugs in the Software!Found in LePage - within Chekcell


The Future

The message has got through that indexing has become a bottleneck for solving structures from powder diffraction data.

New Indexing programs are being developed. MCMaille (Free GPL’d - get the source code as well) Bruker SVD Indexing (commercial) Accerlys Indexing program (commercial) EFLECH/Index (new developments due to the SDPDRR-2)


Conclusion Don’t give up if your lab’s favorite indexing program doesn’t

work on a particular dataset Suites like Crysfire offer a wide range of indexing tools Follow on with Chekcell to help pin down the correct physical

cell If that does not work - whole profile methods such as

MCMaille (still using Chekcell to evaluate trial solutions)• Relevant web links:

• CCP14 Summary Page: http://www.ccp14.ac.uk/solution/indexing/

– Crysfire: http://www.ccp14.ac.uk/tutorial/crys/– Chekcell: http://www.ccp14.ac.uk/tutorial/lmgp/– McMaille: http://www.cristal.org/McMaille/

Documents

Re-stating the obvious: To solve structures from powder diffraction, first they must be indexed! Lachlan M. D. Cranswick CCP14 (Collaborative Computation