Tensor-structured methods in electronic structure calculations

Overview Tensor-structured (TS) methods in Rd Accurate TS computation of integral operators in R

d , d ≥ 3 TS numerical solution

Tensor-structured methods in electronic structure

calculations

Venera KhoromskaiaGAMM 2010, Karlsruhe, March 22-26, 2010.

Max-Planck Institute for Mathematics in the Sciences, Leipzig

Venera Khoromskaia GAMM 2010, Karlsruhe, March 22-26, 2010.TS methods in electronic structure calculations


d , d ≥ 3 TS numerical solution

Outline of the talk

1 Tensor-structured (TS) methods in Rd

Basic tensor formatsTensor product operationsBest Tucker approximationMultigrid accelerated BTA

2 Accurate TS computation of integral operators in Rd , d ≥ 3

The Hartree-Fock equationTS Computation of the Coulomb matrixComputation of the exchange matrix

3 TS numerical solution of the Hartree-Fock equation3D nonlinear EVP solver by TS methods



d , d ≥ 3 TS numerical solutionBasic tensor formats Tensor product operations Best Tucker

Rank-structured representation of higher order tensors

A tensor of order d , A = [ai1...id : iℓ ∈ Iℓ] ∈ RI1×...×Id ,

Iℓ = {1, ..., nℓ}, ℓ = 1, . . . , d ,for nℓ = n, N = nd .

tensor product of vectors u(ℓ) = {u(ℓ)iℓ

}niℓ=1 ∈ R

Iℓ forms thecanonical rank-1 tensor

A(1) ≡ [ui]i∈I = u(1) ⊗ . . . ⊗ u(d) with entries ui = u(1)i1

· · · u(d)id

,

storage: dn << nd .the canonical format, CR,n

A ≈ A(R) =∑R

k=1cku

(1)k ⊗ . . . ⊗ u

(d)k , ck ∈ R, (1)




Tucker tensor format

The rank-(r1, . . . , rd) orthogonal Tucker approximation[Tucker, De Lathauwer].

A ≈ A(r) =∑r1

ν1=1. . .∑rd

νd=1βν1,...,νd

v (1)ν1

⊗ . . . ⊗ v (d)νd

. (2)

A(r) = β ×1 V (1) ×2 V (2) ×3 . . . ×d V (d).

Vectors in V (ℓ) = [v(ℓ)1 . . . v

(ℓ)rℓ ] form the orthonormal basis.

The core tensor β = [βν1,...,νd] ∈ R

r1×...×rd .r = max

ℓ{rℓ} is the (maximal) Tucker rank.

Storage: rd + drn ≪ nd .




Mixed TC tensor format

A(r) =

(

R∑

k=1

bku(1)k ⊗ . . . ⊗ u

(d)k

)

×1 V (1) ×2 V (2) ×3 . . . ×d V (d).

A

r3

I3

I2

r2

(3)

(2)

I1

I

I

I

2

3

1r2

r3

r1

+ . . . +1(3) (3)

r

1

1(2)

(1)r2

r3

r1

+ d r nr1 rdnd

(1)

u

u

u u

b1 bR (2)

ru

u(1)r

2R < = rV

V

V β

β β

Advantages of the Tucker tensor decomposition:

1. Robust algorithm for decomposing full format tensors of size nd .2. Rank reduction of the rank-R canonical tensors with large R.

3. Very efficient for 3D tensors since r3 is small.




Multilinear operations in CR,n tensor format

A1 =

R1∑

k=1

cku(1)k ⊗ . . . ⊗ u

(d)k , A2 =

R2∑

m=1

bmv (1)m ⊗ . . . ⊗ v (d)

m .

Euclidean scalar product (complexity O(dR1R2n) ≪ nd ),

〈A1, A2〉 :=

R1∑

k=1

R2∑

m=1

ckbm

d∏

ℓ=1

⟨

u(ℓ)k , v (ℓ)

m

⟩

.

Hadamard product of A1, A2

A1 ⊙ A2 :=

R1∑

k=1

R2∑

m=1

ckbm

(

u(1)k ⊙ v (1)

m

)

⊗ . . . ⊗(

u(d)k ⊙ v (d)

m

)

.

Convolution, for d = 3 (O(R1R2n log n) ≪ n3 log n (3D FFT))

A1 ∗ A2 =

R1∑

k=1

R2∑

m=1

ckbm(u(1)m ∗ v

(1)k ) ⊗ (u(2)

m ∗ v(2)k ) ⊗ (u(3)

m ∗ v(3)k )




Tucker tensor approximation

A0 ∈ Vn : f (A) := ‖A − A0‖2 → min over A ∈ T r. (3)

The minimisation problem [De Lathauwer et al., 2000] (3) is equivalentto the maximisation problem

g(V (1), ..., V (d)) :=∥

∥

∥A0 ×1 V (1)T × ... ×d V (d)T

∥

∥

∥

2

→ max (4)

over the set of orthogonal matrices V (ℓ) ∈ Rnℓ×rℓ .

For given V (ℓ), the core β that minimises (3) is

β = A0 ×1 V (1)T × ... ×d V (d)T ∈ Rr1×...×rd . (5)

[De Lathauwer et al, 2000]: Algorithm BTA (Vn → T r,n).Complexity for d = 3: WF→T = O(n4 + n3r + n2r2 + nr3).




BTA of function related tensors

2 4 6 8 10 12 14

10−10

10−5

100

Tucker rank

erro

r

Slater function, AR=10, n = 64

EFN

EFE

EC 0

5

10

05

10

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10

−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

Orthogonal vectors, r=6

y−axis

Slater function g(x) = exp(−α‖x‖) with x = (x1, x2, x3)T ∈ R

3

EFN =‖A0 − A(r)‖

||A0||EFE =

‖A0‖ − ‖A(r)‖||A0||

, EC :=maxi∈I |a0,i − ar,i |

maxi∈I |a0,i |.

[Khoromskij,VKH. ’07]




MGA BTA for 3D periodic structures

2 4 6 8 10 1210

−15

10−10

10−5

100

Tucker rank

Multicentered Slater funct.with 1000 samp.

EFN

, n=129

EFE

, n=129

−5

0

5

−5

0

50

0.2

0.4

0.6

The “multi-centered Slater potential“ obtained by displacing a singleSlater potential with respect to the m × m × m spatial grid of sizeH > 0, (here m = 10)

g(x) =

m∑

i=1

m∑

j=1

m∑

k=1

e−α√

(x1−iH)2+(x2−jH)2+(x3−kH)2.




Canonical to Tucker Approximation

[B.N. Khoromskij, VKH ’09]

Theorem

Let A ∈ CR,n. Then the minimisation problem

A ∈ CR,n ⊂ Vn : A(r) = argminV∈Tr,n‖A − V ‖Vr , (6)

[Z (1), ..., Z (d)] = argmaxV (ℓ)∈Mℓ

∥

∥

∥

∥

∥

R∑

ν=1

cν

(

V (1)T u(1)ν

)

⊗ ... ⊗(

V (d)T u(d)ν

)

∥

∥

∥

∥

∥

2

Vr

,

Init. guess: RHOSVD Z(ℓ)0 ⇒ the truncated SVD of U(ℓ) = [u

(ℓ)1 ...u

(ℓ)R

].Error bounds for RHOSVD

‖A − A0(r)‖ ≤ ‖c‖

dX

ℓ=1

(

min(n,R)X

k=rℓ+1

σ2ℓ,k )

1/2, where ‖c‖

2=

RX

ν=1

c2ν .




Canonical to Tucker Approximation

The maximizer Z (ℓ) = [z(ℓ)1 ...z

(ℓ)rℓ ] ∈ R

n×rℓ is obtained by ALS of thematrix unfolding

B(q) ∈ Rn×r̄q , r̄q = r1...rq−1rq+1...rd . (7)

(c) The minimiser in (6) is then

A(r) = β ×1 Z (1) ×2 Z (2) ×3 . . . ×d Z (d).

β =

R∑

ν=1

cν(Z (1)T u(1)ν ) ⊗ ... ⊗ (Z (d)T u(d)

ν ) ∈ CR,r.

Complexity for d = 3, n ≥ R : WC→T = O(nR2 + nr4) .

C2T ⇒ T2C: R → R ′ ≤ r2 ≪ R




MGA Tucker approximation for CR,n tensors

1 Sequence of nonlinear appr. problems for A = An ,n = nm := n02m,

m = 0, 1, . . . , M , on a sequence of refined grids ω3,nm.

2 Z(q)0 on grid ω3,nm

→ by linear interp. of Z (q) ∈ Rnm−1×rq from

ω3,nm−1 .

3 The restricted ALS iteration, is based on “most important fibers”(MIFs) of unfolding matrices.Positions of MIFs are extracted at the coarsest grid

β(q) = Z (q)TB(q) ∈ Rrq×rq . (8)

Location of MIFs corresponds to pr columns in β(q) with maximalEuclidean norms (maximal energy principle).

4 Complexity for d = 3: O(RrnM + p2r2nM).(Linear w.r.t. n and R !)




MGA C2T (CR,n)[B.N. Khoromskij, VKH ’08]

B(q) → unfolding of (B[q]), β(q) = Z (q)TB(q); n0 = 64, nM = 16384

+ ... +

n1 n nn

1 11

n

n3

2

rr

r12r r2

r33 r3 3r

r2 23

+ ... +

n3

n2

n1find MIFs

core

B[q]

[q]β

number 1

2 4 6 8 10 12 14

2

4

6

8

10

12

14

Number 2

2 4 6 8 10 12 14

2

4

6

8

10

12

14

Number 3

2 4 6 8 10 12 14

2

4

6

8

10

12

14

MIFs for H2O




MGA Tucker vs. single grid

200 400 600 800 1000 12000

10

20

30

40

50

60

70

80

90

rank

seco

nds

n=2048n=1024n=512n=256

1000 2000 3000 4000 5000 6000 7000 80000

20

40

60

80

100

120

univariate grid size

min

utes

Single grid vs. multigrid times for C2 H

6

C2T single gridC2T multigrid

Linear scaling in R and in n (left) for C2T algorithm.

Multigrid vs. single grid times for the C2T transform in computations of

VH for C2H6 (right).




MGA BTA for full format tensors

MGA best Tucker decomposition for full format tensors,

[B. Khoromskij,VKH’09], [VKH ’10, in progress]

WF→T = O(n3) for MGA BTA, instead of O(n4) for BTA.

2 4 6 8 10 1210

−14

10−12

10−10

10−8

10−6

10−4

10−2

100

Tucker rank

Slater function, MGA Tucker, EFN

(solid), EEN

(dashed)

n=512

n=256

n=128

n=64

n=512

n=256

n=128

n=64

2 4 6 8 10 120

0.5

1

1.5

2

2.5

3

3.5

Tucker rank

min

ute

s

Times for MGA Tucker, Slater function

n=512

n=256

n=128

Full size tensor with number of entries 5123 , times for Matlab.




MGA BTA for full format tensors (Aluminium Cluster)

[T. Blesgen, V. Gavini and VKH, ’09]

−10

0

10

−15

−10

−5

0

5

10

15

0

0.05

−10−5

05

10

−10

−5

0

5

100

0.02

0.04

0.06

cluster1/sdv4 , initial ρ for Al, z=0

−10−5

05

10

−10

0

100

1

2

3

4

x 10−4

cluster1/subdiv4, abs. diff. for ρ, rT=20

−8 −6 −4 −2 0 2 4 6 8−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

u1(1)

u2(1)

u3(1)

u4(1)

u5(1)

u6(1)

5 10 15 20 2510

−5

10−4

10−3

10−2

10−1

100

Tucker rank

ρ for Al, Cluster1, Subdib3

Een

, n=201

EF, n=201

EF, n=101

Een

, n=101

FE (coarse graining) ⇒ computed on large supercomputer clusters∗ .

FE data∗ ⇒ 3D Cartesian grid ⇒ Tucker decomposition

∗[V. Gavini, J. Knap, K. Bhattacharya, and M. Ortiz,’07]



d , d ≥ 3 TS numerical solutionThe Hartree-Fock equation TS Computation of the Coulomb matrix

The Hartree-Fock equation

The Hartree-Fock equation

Fϕi (x) = λi ϕi (x), i = 1, ..., Norb, x ∈ R3

F := −1

2∆ − Vc + VH −K,

the Hartree potential

VH(x) :=

∫

R3

ρ(y)

|x − y | dy , ρ(y) = 2

Norb∑

i=1

(ϕi (y))2

where ϕi =R0∑

k=1

ci ,kgk(x).

The exchange operator

(Kϕ) (x) :=

∫

R3

τ(x , y)

|x − y | ϕ(y)dy , τ(x , y) =

Norb∑

i=1

ϕi (x)ϕi (y).




O(n) computation of the Hartree potential on huge 3D grids

VH is computed on large n × n × n Cartesian grids, with n ∼ 103 ÷ 104

ρ(x) = 2

Norb∑

i=1

(ϕi )2, ϕi =

R0∑

k=1

ci ,kgk(x), x = (x1, x2, x3) ∈ R3

gk(x) = gk,1(x1) gk,2(x2) gk,3(x3),

gk,ℓ = (xℓ − Ak,ℓ)βk,ℓe−λk (xℓ−Ak,ℓ)

2

,

gk,ℓ → u(ℓ)k ; xℓ ∈ [−b, b],

b = 6 ÷ 10◦

A.

H

H

H

H

H

H

C

C

z

y

x

R0(CH4) = 55, R0(C2H6) = 96, R0(H2O) = 41, Rρ = R0(R0+1)2 .

ρ ≈ Fρ =∑Rρ

k=1cku

(1)k ⊗ u

(2)k ⊗ u

(3)k , ck ∈ R, u

(ℓ)k ∈ R

n.

⇒ ρ is presented in the computation box [−b, b]3 with Rρ ∼ 103 ÷ 104.




O(n) computation of the Hartree potential on huge 3D grids

Rρ(CH4) = 1540, Rρ(C2H6) = 4656, Rρ(H2O) = 861

Next : MGA C2T + T2C transformations to reduce the rank:

Rρr∼ 102.

RN ∼ 20 ÷ 30 canonical tensor G for the Newton kernel 1‖x‖

(using sinc-quadratures) [C. Bertoglio, B. Khoromskij ’08]).

The rank-Rρrcanonical tensor F for ρ:

VH = F ∗ G =

Rρr∑

m=1

RN∑

k=1

ckbm

(

uk1 ∗ vm

1

)

⊗(

uk2 ∗ vm

2

)

⊗(

uk3 ∗ vm

3

)

Coulomb matrix: tensor inner products in GTO basis {gk}R0

k=1

Jkm :=

∫

R3

gk(x)gm(x)VH(x)dx , k , m = 1, . . .R0.




Tensor-product convolution vs. 3D FFT

The cost of computation of VH by tensor product convolution (1D FFT):

NC∗C = O(RρrRNn log n)

instead of O(n3 log n) for 3D FFT.

n3 1283 2563 5123 10243 20483 40963 81923 163843

FFT3 4.3 55.4 582.8 ∼ 6000 – – – ∼ 2 yearsC ∗ C 0.2 0.9 1.5 8.8 20.0 61.0 157.5 299.2C2T 4.2 4.7 5.6 6.9 10.9 20.0 37.9 86.0

CPU time (in sec) for the computation of VH for H2O.

(3D FFT time for n ≥ 1024 is obtained by extrapolation).




High accuracy computations of VH

(Calculation over the number of entries N ≈ 4.3 · 1012).

−6 −4 −2 0 2 4 6

10−4

10−3

hart

ree

Abs. approx. error, VH

for H2O

a) atomic units

n=4096n=8192Ri−4096−8192

2000 4000 6000 8000 10000 12000 14000 160000

1

2

3

4

5

6

univariate grid size n

min

utes

H2O , r

T=20

C−2−T time3D conv. time




High accuracy computations of the Coulomb matrix for H2O

(for calculation over the number of entries N ≈ 4.3 · 1012). Approx. error O(h3), h = 2b/n (h ≈ 0.0008◦

A).




High accuracy computations of the Coulomb matrix for C2H6

a) CPU times for VH : O(n log n) on n × n × n grids with max.:n = 8192.b) Absolute approximation error of the tensor-product computation ofthe Coulomb matrix for C2H6 molecule.

2000 4000 6000 80000

2

4

6

8

10

12

14

b) univariate grid size

min

utes

C−2−T time3D conv. time

−4 −2 0 2 4 6

10−6

10−4

10−2

100

a) atomic units

hart

ree

C2H

6, abs. appr. error for V

H

n=4096n=8192Ri−4096−8192




TS computation of the exchange matrix

[VKH ’09]

Kk,m :=

∫

R3

∫

R3

gk(x)τ(x , y)

|x − y |gm(y)dxdy , k , m = 1, . . .R0

1. Compute the discrete tensor product convolution

Wa,m(x) =

∫

R3

ϕa(y)gm(y)

|x − y | dy , cost:O(RNR0n log n) per element (×R0N)

2. Entries of the Exchange matrix for every orbital,

Vkm,a =

∫

R3

ϕa(x)gk(x)Wa,m(x)dx

with the cost O(RNR40nf ).

3. Entries of the Exchange matrix

Kk,m =

Norb∑

a=1

Vkm,a




TS computation of the exchange matrix on huge 3D grids

Abs. error for the Exchange matrices Kex forCH4 (Richardson extrapolation for n = 4096)

and H2O (Rich. for n = 16384).



d , d ≥ 3 TS numerical solution3D nonlinear EVP solver by TS methods

Solution of the Hartree-Fock on a sequence of 3D grids

[Flad, Khoromskij, VKH ’09]

Discretized GTO basis, ϕi (x) ≈R0∑

k=1

ckigk(x), x ∈ R3, i = 1, ..., Norb

Fϕi (x) = λi ϕi (x), ⇒ FU = λSU

F = H0 + J(C ) − K (C ), C = {cki} = [U1U2 . . . UNorb] ∈ R

R0×Norb

H0 is the stiffness matrix of the core Hamiltonian − 12∆ + Vc

Our iterative scheme

Initial guess for m = 0: J = 0, K = 0 ⇒ F0 = H0.

On a sequence of refined grids n = n0, 2n0, . . . , 2pn0,

solve EVP [H0 + Jm(C ) − Km(C )]U = λSU .

Fast update of Jm(C ) and Km(C ).

Grid dependent termination criteria εp = ε0 · 4−p.

Use DIIS [Pulay ’80] to provide fast convergence.




HF for the pseudopotential case of CH4

Left: multilevel convergence of iterations for the pseudopotential case ofCH4.Right: convergence in effective iterations for finest grid.

2 4 6 8 1010

−4

10−3

10−2

10−1

100

abs. error, | λ − λn,it

|, CH4, pseudo

n=128, 115 s/it

n=64, 72 s/it

n=256, 260 s/it

n=512, 488 s/it

total time: 33 min.

0 1 2 3 410

−4

10−3

10−2

10−1

100

conv.in eff.iterations, CH4, pseudo, n=512




Convergence in HF energy

Left: convergence in HF energy.Right: linear scaling of the iteration time in the univariate grid size.

200 400 600 800 100010

−5

10−4

10−3

10−2

10−1

100

univariate grid size

Abs. err. for energy, CH4, pseudo

O(h2)

O(h3)

200 400 600 800 10000

5

10

15

20

25

30

univariate grid sizem

inu

tes

time per SCF iteration

The total energy is calculated by

EHF = 2

NorbX

i=1

λi −

NorbX

i=1

“eJi − eKi

”

with eJi = (ϕi , VH ϕi )L2 and eKi = (ϕi ,Vexϕi )L2 (i = 1, ..., Norb ) the Coulomb and exchange integrals,

computed with respect to the orbitals ϕi .




Iterations of the nonlinear 3D EVP solver

Left: all electron case of H2O.Right: the pseudopotential case of CH3OH .

5 10 1510

−6

10−4

10−2

100

iterations

residual in HF EVP, all electron case H2O

5 10 15 20 2510

−5

10−4

10−3

10−2

10−1

100

CH3OH, residual

n=64

n=128

n=256

n=512




Thank you for attention

The talk is based on

1 H.-J. Flad, V. Khoromskaia, B. Khoromskij.Solution of the Hartree-Fock Equation by the Tensor-Structured Methods.

Preprint MPI MIS 44/2009.

2 V. Khoromskaia.Computation of the Hartree-Fock Exchange by the Tensor-structured Methods.

Preprint MPI MIS 25/2009

3 T. Blesgen, V. Gavini and V. Khoromskaia.Approximation of the Electron Density of Aluminium Clusters in Tensor-product Format,

Preprint MPI MIS 66/2009

4 V. Khoromskaia and B. N. Khoromskij.Multigrid Accelerated Tensor Approximation of Function Related Multi-dimensional Arrays.

SIAM Journal on Scientific Computing, vol. 31, No. 4, pp. 3002-3026, 2009.

5 B. N. Khoromskij, V. Khoromskaia, S.R. Chinnamsetty and H.-J. Flad.Tensor Decomposition in Electronic Structure Calculations on 3D Cartesian Grids.

Journal of Computational Physics, 228(2009), pp. 5749-5762, 2009.

6 B. N. Khoromskij and V. Khoromskaia.Low Rank Tucker-Type Tensor Approximation to Classical Potentials.

Central European Journal of Mathematics v.5, N.3, 2007, pp.523-550.

http://personal-homepages.mis.mpg.de/vekh/


Documents

Tensor-structured methods in electronic structure calculations