Introduction to Time Memory Tradeoffs

Introduction toTime Memory

Tradeoffs

Jin HongSNU

2006 SNU-KMS Winter Workshop on Cryptography 2

Today, we hope to learn ...

Birthday paradox Hellman tradeoff on blockciphers Babbage and Golic birthday paradox based tra

deoff on streamciphers Biryukov-Shamir tradeoff on streamciphers Recent developments

Birthday Paradox


Birthday paradox – layman’s version

If you have 23 people in one room, it’s a good idea to bet on finding two of them having the same birthday than not.

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

1 11 21 31 41 51 61


Birthday paradox - most cryptographers’ version

Consider a box containing N numbered balls. If you take out N½ balls, one at a time, with replacements, then there’s a large chance of seeing the same ball twice.


Birthday paradox - a more general version

Consider a set of size N, and two subsets of size A and B. If AB=N, there is a large chance that the two subsets intersect non-trivially.

(1+1/n) n̂

1.8

2.0

2.2

2.4

2.6

2.8

1 2 4 8 16 32 64 128

256

512

1024

2048

4096

8192

1638

4

3276

8

Hellman


Hellman tradeoff

Martin E. Hellman, A cryptanalytic time-memory trade-off. IEEE Trans. on Infor. Theory, 26 (1980).

A chosen-plaintext attack on blockcipher DES


Blockcipher Blockcipher is a parame

trized family of permutations.

Each k-bit key specifies a permutation on the set of n-bit strings.

Without knowledge of key, it is not possible to obtain plaintext from ciphertext.

blockcipherblockcipher

n-bit plaintextn-bit plaintext

k-bit ke

yk-b

it key

n-bit ciphertextn-bit ciphertext


Using a blockcipher The communicating parties share a c

ommon key through some other secure channel.

The long plaintext to be sent is broken into small blocks.

Each block is encrypted though the blockcipher using the common key.

Generated short ciphertext blocks are transmitted over insecure channel.

Receiving party decrypts each ciphertext block using the common key to recover each plaintext block.

The plaintext blocks are concatenated to bring back the whole plaintext.

share through

secure channel

transmit over

insecure channel

keykey

blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt

ciphertextciphertext

blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

keykey


Attacking a blockcipher The number of possible keys is much smaller than

the number of possible permutations on the space of plaintext blocks.

The keys size is usually comparable to plaintext size and the number of permutations being used in any blockcipher is comparable to the number of ciphertext blocks.

Hence, in principle, a small number of plaintext-ciphertext pair determines the key uniquely.

But, blockciphers are (or should be) designed so that it is computationally infeasible to find key from plaintext-ciphertext pairs.

If an adversary is successful in obtaining the key from a few plaintext-ciphertext pairs, it may be used to decrypt all other ciphertext blocks encrypted under the same key.


n-bit plaintextn-bit plaintext

k-b

it key

k-b

it key

n-bit ciphertextn-bit ciphertext


Chosen-plaintext attack on DES

DES: 56-bit key, 64-bit block Attacker is given the ciphertext cor

responding to a plaintext of his choice.

Objective of the attacker is to find key from the given ciphertext.

Note that the expected ratio of random mapping image points is (1-1/e)~0.632.

DESDES

fixed plaintextfixed plaintext

key

key



Two extreme attacks

Exhaustive search Try all keys until correct one is found. This takes quite a long time.

Table lookup Pre-compute all (key, ciphertext) pairs. Sort the list according to the ciphertexts. Read off answer from the dictionary, as soon as ci

phertext is given. This requires quite a large amount of storage.


Tradeoff

We could come somewhere in the middle of the two extreme solutions through a tradeoff between online time and storage space.

Offline phase Pre-compute all (key,ciphertext) pairs, and store a digest of the computation in a table smaller than th

e complete dictionary. Online phase

Given a target, using the incomplete table, find answer in time shorter than require for exhaustive search.


Notation

Denote DES encryption by C = EK(P) Define reduction function

R: (Z/2Z)64 (Z/2Z)56 to be any fixed “choosing” of 56 bits from 64 bits.

Fix plaintext P0 and definef: (Z/2Z)56 (Z/2Z)56 by f(K) = R◦EK(P0).

Attacker’s objective translates to that of finding K, given f(K)=R(C).


Hellman table

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp1 ep1

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp2 ep2

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp3 ep3. . . . . . .

. . . . . . .

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦spm epm

t


Hellman tradeoff

HT = {(spi,epi)}i, sorted according to the second component.

For j=0…t-1, successively check if the correct key belongs to the (t-j)th column by applying f to R(C) j-many times, and checking for existence of the result among the epi’s.

If key belongs to column t-j, it can be recovered from spi by applying f to it appropriately many times.

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .


Questions?

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp1 ep1

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp2 ep2

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp3 ep3. . . . . . .

. . . . . . .

◦ f◦ f ◦ f ◦ . . . . . . .


t


False alarm

Due to f being not injective, existence of f j(R(C)) among the epi’s do not guarantee that the correct key belongs to the (t-j)th column.

These false alarms cost t applications of f and its frequency is hard to analyze.

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .


Success probability

Let N=256 be the number of all keys. Birthday paradox gives the matrix

stopping rule: t2m = N. Success probability

= (# of distinct keys in HT)/N~ 0.8 tm/N (when t2m = N)

Success probability of t tables, that use different reduction functions= 1-(1-tm/N)t ~ 1-exp(-t2m/N) = 1-1/e

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .


Hellman tradeoff curve Pre-computation time: P=t2m=N Online time: T=t2 (applications of f) Storage: M = tm (sp-ep pairs) Tradeoff curve: TM2=N2

Conversely, given T and M satisfying TM2=N2, setting t = T½ and m = M/t results in a tradeoff algorithm requiring time T and storage M.

If cost is measured as T+M, the optimal tradeoff point is T=M=N2/3.

What we have discussed so far does not depend on structure of DES. It is applicable to any one-way function.


Inversion problem

Given a one-way function f: XY and a target point y∈Y, find any x∈X such that f(x)=y.Given a one-way function f: XY and a target point y∈Y, find any x∈X such that f(x)=y.

Inversion ProblemInversion Problem

Try out each x∈X until we see an x with f(x)=y.Try out each x∈X until we see an x with f(x)=y.Exhaustive SearchExhaustive Search

Pre-compute and store all (x,f(x)) pairs in a table (dictionary), sorted according to the second component. Read off answer when target point y∈Y is given.

Pre-compute and store all (x,f(x)) pairs in a table (dictionary), sorted according to the second component. Read off answer when target point y∈Y is given.

Table LookupTable Lookup


Time-memory tradeoff

Offline phase Pre-compute all (x,f(x)) pairs, and store a digest of the computation in a table s

maller than the complete dictionary. Online phase


Offline phase Pre-compute all (x,f(x)) pairs, and store a digest of the computation in a table s

maller than the complete dictionary. Online phase


TradeoffTradeoff


Hellman tradeoff summary If the keyspace is of size N (DES: 256), for any set of

values P, T, and M, satisfying

one may find the key in online time Tusing offline pre-computation time P and storage of size M for table.

Hellman’s algorithm may be used on arbitrary one-way functions.

TM2 = N2, P = NTM2 = N2, P = N

T = M = N2/3T = M = N2/3

Tweaksto Hellman’s

Methods


Distinguished points

Rivest, before 1982 (according to a book by Denning)

Distinguished point example: a binary string starting with 10 zeros.

To create each row of the Hellman table, function f is iterated until a pre-defined distinguished point is reached.

The length of rows is variable. This removes much of the table lookup time during t

he online phase.


Rainbow tables

Philippe Oechslin, Making a Faster Cryptanalytic Time-Memory Trade-Off. Crypto 2003.

◦ f3◦ f1

◦ f2

◦ . . . . . . .

◦ ft-1 ◦ ft ◦sp1 ep1

sp2 ep2

sp3 ep3. . . . . . .

. . . . . . .

spm epm

◦ f3◦ f1

◦ f2

◦ . . . . . . .

◦ ft-1 ◦ ft ◦

◦ f3◦ f1

◦ f2

◦ . . . . . . .

◦ ft-1 ◦ ft ◦

◦ f3◦ f1

◦ f2

◦ . . . . . . .

◦ ft-1 ◦ ft ◦


Rainbow tables

In a way, t Hellman tables corresponds to one rainbow table.

Compared to the original Hellman method, rainbow tables use half the online time for the same storage.

Using 1.4GB of data (two CD-ROMs) rainbow table method cracks 99.9% of all alphanumerical MS-Windows password hashes in 13.6 seconds.


Checkpoints

G. Avoine, P. Junod, P. Oechslin, Time-memory trade-offs: False alarms detection using checkpoints. Indocrypt 2005.

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp1 ep1

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp2 ep2

◦ f◦ f ◦ f ◦ . . . . . . .

◦ f ◦ f ◦sp3 ep3. . . . . . .

. . . . . . .

◦ f◦ f ◦ f ◦ . . . . . . .


t


Other neat tricks

Starting points need not be random. For the original Hellman method, they could be small counters concatenated with table numbers. This results in storage savings. (This is an argument against the usefulness of rainbow tables.)

After sorting, the endpoints that are close together have common significant bits. This also leads to storage savings.

Digression


Are tradeoffs meaningful? Tradeoff algorithms require exhaustive search. How can

such a thing be a meaningful attack? In constrained environments, systems of marginal

security are used. With tradeoff attacks, security level is meaningfully reduced.

Low (short-term) security may be all one wanted. With tradeoff attacks, the security of these systems may turn out to what was expected.

Your neighbor may be incapable of exhaustive search, but a network of hackers may have gotten together and published the needed table. Your adversary may have had such help from a third party.

As soon as exhaustive search is possible by someone, one cannot be sure of the security level provided by the affected system.


Affordable tradeoffs (www.rainbowcrack-online.com)

They have huge tables that implement Oechslin’s tradeoff algorithm and will recover passwords on a subscription basis.

Password hashing schemes based on MD5, LanManage, SHA1, Cisco PIX, NTLM, MySQL-323, MySQL-SHA1, and MD4 are served and they also sell these tables.

LanManager case details:

Babbage, Golic


Babbage Golic tradeoff

S. H. Babbage, Improved exhaustive search attacks on stream ciphers. European Convention on Security and Detection, 1995.

J. Dj. Goli , Cryptanalysis of alleged A5 streaćm cipher. Eurocrypt’97.

Attack on streamciphers.


Streamcipher Streamcipher is a pseudo-rand

om bit stream generator. The following two steps are rep

eated. Filter function is applied to inte

rnal state to produce a short bit sequence.

The internal state is updated. Each initial internal state, i.e.,

an element of (Z/2Z)s, specifies a long bit sequence (keystream).

internalstate

internalstate

internalstate

internalstate

stateupdatefunctio

n

keystreamfe

wbits

few

bits

few

bits

few

bits

filterfunctio

n

internalstate

internalstate

internalstate

internalstate

internal

stateinternal

state

few

bits

few

bits

few

bits

few

bits

few

bits

few

bits


Using a streamcipher1. The communicating parties sha

re a common initial internal state through some other secure channel.

2. A long keystream is generated from the common internal state.

3. Plaintext is added onto the carrier keystream.

4. Generated ciphertext is transmitted over insecure channel.

5. Receiving party generates the same keystream from shared initial state.

6. Plaintext is recovered from ciphertext by “subtracting” the keystream from ciphertext.

internalstate

internalstate

long keystream

pla

inte

xt

=

ciphertext

long keystream

ciphertext

=

pla

inte

xt

internalstate

internalstate

share through

secure channel

transmit over

insecure channel


keystreamkeystream

segment

keystream segm

entAttacking a streamcipher Anything that allows recovery of

whole keystream from a partial keystream segment is a successful attack.

An appropriate length of keystream segment determines the starting internal state uniquely.

But, streamciphers are designed so that it is computationally infeasible to recover the starting internal state from a finite keystream segment.

internalstate

internalstate


The crucial discovery Given a long keystream,

it suffices to find the internal state corresponding to any one of the keystream segments.

Once state is recovered, the cipher may be run forward to obatina future keystream.

internalstate

internalstate

keystreamsegm

entkeystreamsegm

ent

internalstate

internalstate

keystreamsegm

entkeystreamsegm

ent


Two extreme solutions

Exhaustive search Try all possible internal states until a known keystream seg

ment is produced. With N possible states and D keystream segments, N/D tri

es are expected until an answer is found. Table lookup

Pre-compute enough (state, keystream seg) pairs. Sort the list according to the keystream segments. When D keystream segments are given, look for them in the

table and read off answer. N/D pairs should be pre-computed and stored.


Babbage Golic tradeoff If the number of possible states is N, and the online target data set

will be of size D, for any set of values P, T, M, and D, satisfying

one may find the key in online time Tusing offline pre-computation time P, storage of size M for table, and online data of size D.

This birthday paradox based method does not depend on the structure of streamciphers, and hence may be used to invert arbitrary one-way functions.

TM = N, P = M ≥ N/DTM = N, P = M ≥ N/D

T = M = D = N1/2T = M = D = N1/2


Attack restatement in terms of one-way functions

Let there be N possible internal states. Define function one-way function by

f: internal state (ln N) bits of keystream. Attacker’s objective translates to that of finding any o

ne of the internal states, corresponding to any one of the keystream segments.

Given a one-way function f: XY and a target set S⊂Y, find at least one x∈X such that f(x)∈S.Given a one-way function f: XY and a target set S⊂Y, find at least one x∈X such that f(x)∈S.

Multi-target InversionMulti-target Inversion

Biryukov, Shamir


Hellman review

Go back to pages 24 and 16.


Birthday + Hellman

There’s no reason we can’t apply Hellman table method to the streamcipher situation.

This time, we have the advantage of not having to cover the whole search space.

During the offline phase, it suffices to deal with only N/D internal states.


Birthday + Hellman

(single target) Offline coverage

P = N t tables Online time

T = t t = t• 2

StorageM = m t = mt•

Tradeoff curveTM2 = N2

(multiple targets) Offline coverage

P = N/D t/D tables Online time

T = t (t/D) D = t• • 2

StorageM = m (t/D) = mt/D•

Tradeoff curveTM2D2 = N2


BS-tradeoff

A. Biryukov and A. Shamir, Cryptanalytic time/memory/data tradeoffs for stream ciphers. Asiacrypt 2000.

Combination of Hellman tradeoff and birthday paradox based tradeoff. keystream

internalstate

internalstate keystream

segment

keystreamsegm

ent

internalstate


segment

keystreamsegm

ent

stateupdate


BS-tradeoff

If the state size is N, and the online target data set will be of size D, for any set of values P, T, M, and D, satisfying

one may find the key in online time Tusing offline pre-computation time P, storage of size M for table, and online data of size D.

Biryukov-Shamir’s tradeoff algorithm does not depend on the structure of streamciphers, and hence may be used to invert arbitrary one-way functions.

TM2D2 = N2, P = N/D, D2 ≤ TTM2D2 = N2, P = N/D, D2 ≤ T

T = M = N1/2, D = N1/4T = M = N1/2, D = N1/4


TMD-tradeoff theory summary Even though not made explicit in the original

works, the tradeoff algorithms can be applied to arbitrary one-way functions.

Assume a one-way function to be inverted acting on a search space of size N.

For situations where single target inversion problem is applicable, there is a tradeoff algorithm of online complexity N2/3.

For situations where multiple target inversion problem is applicable, there is a tradeoff algorithm of online complexity N1/2.

Tradeoff on StreamciphersRevisited


Using a streamcipherinternal

stateinternal

state

long keystream

pla

inte

xt

=

ciphertext

long keystream

ciphertext

=

pla

inte

xt

internalstate

internalstate

share through

secure channel

transmit over

insecure channel


Another tradeoff on (old) streamciphers

keystream

internalstate


prefixkeystream

prefix

keysetupkey

setup

keykey Target one-way function is{key} {keystream prefix}.

Assume keystream prefix exposed due to protocol.

Once key is found, rest of keystream is exposed.

In some situations, multiple data tradeoff is possible. Example: Attacker wants to make bad reput

ation of one particular popular mobile telecom system. It suffices for him to decrypt any one message.

Even with single data tradeoff, online complexity of attack corresponds to 2/3 of key size.

This attack works irrespective of internal state size.

internalstate

internalstate


Another tradeoff on (recent) streamciphers Attacker wants to attack one particula

r user. Assume fixed user key with variable IV.

Target one-way function is{(key,IV)} {keystream prefix}.

It suffices to obtain any one (key,IV) pair. If found, all other sessions can be decrypted.

Assume keystream prefix exposed due to protocol.

Multiple data tradeoff possible. Online complexity of attack is half of k

ey size. This attack works irrespective of inter

nal state size.

internalstate

internalstate

keystreamkeystream

prefixkeystream

prefix

keysetupkey

setup

keykey IVIV


Example eSTREAM (ECRYPT Stream Cipher Project)

Profile 2 must accommodate 80-bit keys and at least one of 32-bit or 64-bit IVs.

BS-tradeoff on 80-bit key / 32-bit IV TM2D2 = N2, P = N/D, D2 ≤ T N = 2112, T = 264, M = 250, D = 230, P = 282

Doing 282 key setups as pre-computation, one prepares a table containing 250 data points.

Then, given 230 keystream prefixes, the key can be recovered using 264 key setups.

These numbers are large, but small enough to be considered a threat.

Tradeoff onBlockciphers

Revisited


Using a blockciphershare

through

secure channel

transmit over

insecure channel

keykey

blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her

pla

inte

xt

pla

inte

xt


blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

blo

ckcip

her

blo

ckcip

her


pla

inte

xt

pla

inte

xt

keykey


Blockcipher mode of operation

key

key

IVIV


Another tradeoff on blockciphers Assume chosen plaintext scenario

with multi-block chosen plaintext. Assume fixed key and variable IV. Target one-way function is

{(key,IV)} {ciphertext blocks}. It suffices to obtain any one (key,I

V) pair. If found, all other sessions can be decrypted.

Assume multiple ciphertexts corresponding to fixed chosen plaintext and different IV’s available due to protocol.

Multiple data tradeoff attack is possible.

fixed plaintext2fixed plaintext2

key

key

ciphertext2ciphertext2

IVIV






Another tradeoff on blockciphers

In CBC, any ciphertext block may be thought of as an IV for subsequent encryption.

Multiple data is possible even from a single session.

Online complexity of attack is half of key+IV size.


fixed plaintext2fixed plaintext2ke

yke

y

IVIV





If block size is smaller than key size, security is less than key size.







key

key


IVIVfixed plaintext1fixed plaintext1





IVIV








fixed plaintextfixed plaintextke

yke

y


IVIV










IVIV


Example

3GPP A5/3 128-bit key, 64-bit blockcipher KASUMI in modified OFB mo

de is used. But IV is a 22-bit counter and key is a double copy of a sing

le 64-bit key Only 228 bits of keystream used for each key IV.

BS-tradeoff Target one-way function is {(key,IV)} {keystream prefix}. TM2D2 = N2, P = N/D, D2 ≤ T N = 286

T = 243, M = 243, D = 221.5, P = 264.5

TMD-tradeoffis a Versatile Tool


Summary

Hellman family of TMD tradeoff techniques can be used to invert generic one-way functions.

It is possible to apply them to various situations other than that in which each algorithm was originally applied to, and also in many different ways.

Questions?

Documents

Introduction to Time Memory Tradeoffs