1

Mathematical Algorithms

1. Arithmetic, (Pseudo) Random Numbers and all that2. Evaluation & Multiplication of Polynomials I3. Multiplication of Large Integers4. Strassen's algorithm for large Matrices5. Zeros of Polynomials: Bisection vs Newton's method6. Evaluation & Multiplication of Polynomials II7. Fast Fourier Transforms 8. Interpolation/splines etc.9. SHOULD BE A SEPARATE COURSE!!!

Ari’s class: PHO 211 – 4/13/06

2

Adding, Multiplying and Evaluating Polynomials

Why are polynomials so interesting? PN(x) = a0 + a1 x + a2 x2 aN xN

Fit data di = PN(xi) by picking ai’s

Integer arithmetic base B is polynomial with x = B: e.g Integer decimal base (B=10)

PN-1 (10) = a0 + a1 10 + a2 102 aN-1 10N-1

Fourier transform is PN-1(xk) with xk=kN= exp[i k 2 /N]

yk ´ l eik n 2 /N an

ETC

3

Evaluation of p(x) = a0 + a1 x + ... + aN xN

Method 1: Compute x, x2, x3, ...xN with N-1-mults plus N mults and N adds

or 2N-1 mults and N adds

Horner’s Method 2: a0 + x(a1 + x (a2 + x (a3 + .a4 x) ) ) for N = 4

N adds and N multiplies! Evaluating it at N points is O(N2)

Special case: p(x) = xN do in Log(N) steps! Consider N in binary base eg.100111001101.

Cal x, x2, x4, x8.... by repeated squaring and then multiply

to get xN in O(Log2(N)) steps.

4

Polynomial Multiplication P(x) = a0 + a1 x + ... + aN xN and Q(x) = b0 + b1 x + ... + b

N xN

P(x) Q(x) = a0 b0 + (a0 b1 + a1 b0)x + (a0 b2 + a1 b1 + a2 b0)x2 + ...+ aN bN x2N

This is O(N2): 1 + 2 + ...N-1 to get to xN and similarly for xN+1 to N2N

Try devide and conquer: PN(x) = pL(x) + xN/2 p’H(x)pL(x) = a0 + a1 x + ... aN/2 xN/2 pH(x) = aN/2+ ... aN/2 xN/2 P(x) Q(x) = pL(x) qL(x) + (pL(x) qH(x) + pH(x) qL(x)) xN/2 + pH(x) qH(x) xN

T(N) = 4 T(N/2) + c0 N T(N) = O(N2)

Better: define (pL(x) + pH(x))(qL(x) + qH(x)) – pL(x) qL(x) – pH(x) pH(x)

T(N) = 3 T(N/2) + c0 N T(N) = O(N)

5

Large Integer Multiply: Z = X * Y

Set: X & Y has at most N Digits: X= 137 = 1*102 + 3* 10 + 7

Y= 025 = 0*102 + 2* 10 + 5

Steps: Single Digit Multipliers and Adders with carry.

e.g. Standard method: Complexity: T(N) = const * N2 Is this best algorithm? Is there a best upper bound

1 3 7 A x 0 2 5 B

--------- 1 3 5 1 5 5 1 4 1 6 2---------------------- 3 4 2 5 C

6

Beating O(N ) by “Divide & Conquery

Split each N digit integer into two N/2 integers:

X = XH * 10N/2 + XL and Y = YH * 10N/2 + YL

X*Y = XH*YH 10N + (XH*YL + XL*YH) 10N/2 + XL*YL

Number of Multiplies: 4 * (N/2)2 = N2 NO CHANGE ?

BUT: XH*YL + XL*YH = (XH + XL)*(YH + YL) – XH*YH – XL*YL

“Really 3 times half problems or 3 (N/2)2 = 3N2/4

RECURSIVELY this gives T(N) = const Nlog(3)/log(2) = N1.59…

2

y See Weiss Sec. 10.2.4 page 419

7

Large Integer Multiplication: T(N) = 3 T(N/2) + c N

using:

adding:

8

Integer Multiplication: T(N) = ?Function Value Complexity

XL 6143 Input O(N)

XH 8521 Input O(N)

YL 9473 Input O(N)

YH 6407 Input O(N)

D1 = XH + XL 14664 N adds

D2 = YH + YL 15880 N adds

XL*YL 58192639 T(N/2)

XH*YH 54594047 T(N/2)

D1* D2 232864320 T(N/2)

D3 = D1*D2 - XH*YH - XL*YL 120077634 N adds

XH*YH + D3*104 + XH*YH 5820464730934047 Add/Output O(N)

Direct Method: T(N) = c1 N2 + c2 N O(c1 N2 + c2 N) = O(N2)

One Level: T(N) = 3 * (c1 (N/2)2 + c2 N/2) O(0.75 c1 N2) = O(N2)

Naive Recursion: T(N) = 4 T(N/2) + c N O(N2)

Smart Recursion: T(N) = 3 T(N/2) + c O( N llog(3)/log(2) )

9

Strassen’s Algorithm O(N2.81), p. 422

where:

T(N) = 7 T(N/2) + c N2

Naïve T(N) = c N^3

10

Zeros of PN(x): Bisection vs Newton’s MethodsP_N(x) has N complex roots.

No closed form for roots for Quintic (N=5) & above

Real roots by bisection:

accuracy = 1/N O(log(N))

Newton’s Method:

accuracy = 1/N O(log(N))

11

Multiplying Polynomials and FFT Multiply PN/2 (x), QN/2(x)

1. Evaluate at 2N points (k=0, N-1) O(N2)

2. W(xk) = Q(xk) P(xk) O(N)

3. Interpolate to find ck , n = 0,...,N-1 O(N2)

ai, bj c_k

Q(x_k) P(x_k)W(xk) = Q(xk) P(xk)

O(N2)

O(N log(N))

O(N)

O(N log(N))

FFTFFT

convolution

multiply

12

FFT: xk = kN = exp[i 2 k/N]

spit polynomial into low/high pieces:

even k

odd k

One N = Two N/2 Fourier transforms

13

Thus T(N) = 2 T(N/2) + co N: T(N) » N Log(N)

Insert butterfly and discuss bit reversal

14

End Ari’s Class