Upload
garrison-cunnington
View
214
Download
0
Embed Size (px)
Citation preview
Information theory
Multi-user information theory
A.J. Han VinckEssen, 2004
content
Some examples of channels Additive coding for the broadcasting Superposition coding for multi-access Coding for the two-way channel Coding for the switching channel Some more
Goal of the lectures:
Introduction of some classical models two-way; two access; broadcast;
Problems connected: calculation and formulation of capacity Development of coding strategies
Time Sharing (TDMA)
User 1
User 2
User 3
Time sharing: easy to organize
inefficient if not many users active
efficiency depends on channel
message
idleCommon channel
Two-way
X1 X2
Y1 Y2
X1 and X2 communicate by observing Y1 and Y2
R1= I(X1;Y2|X2)
R2= I(X2;Y1|X1)
Maximize (R1,R2) over any input
distribution P(X1,X2)
I(X1;Y2|X2) := H(X1|X2)-H(X1|X2,Y2)
Note:
I(X1;Y2|X2) := H(X1|X2)-H(X1|X2,Y2)
H(X1|X2) = minimum average # bits needed to specify X1 given X2
H(X1|X2,Y2) = minimum average # bits needed to specify X1 given X2
and the observation Y2
Difference = what we learned from the transmission over the channel
= the reduction in average specification length of X1|X2
Example: AND channel
X1 X2
Y Y
X1
0 1
X2 0 0 0
1 0 1
Y
When X1 = 0, he does not know X2
X1 = 1, he knows X2
Same for X2
A coding example
X1 01 10
01 01 00
X2
10 00 1
if y = 0
Transmit inverse (red)
If y = 0
Inputs are knownRate: 1/( 2*3/4 + 1*¼) = 4/7 = 0.57 > 1 !!
X1 X2
Y
Another coding example
X1
0 11 0 10 0 0
X2 0 10 1 01 1 00
0 0 1 00 1 1
Rate: log23/( 2*3/9 + 3 * 6/9 ) = .59 > 1 !!
X1 X2
Y
dependent inputs X1 and X2
P(X1=0, X2=0) = 0
P(X1=0, X2=1) = P(X1=1, X2=0) = p
P(X1=1,X2=1) = 1-2p
Then, P(X1=1) = P(X1=1, X2=0) + P(X1=1, X2=1) = 1-p.
R2 = R1 = I(X2;Y|X1) = I(X1;Y|X2)
= H(Y|X1) = (1-p)h(p/(1-p))
The maximum = 0.694
0 1
0 p
1 p 1-
2p
Note:
P(Y=0|X1=1) = P(Y=0, X1=1)/P(X1=1) = p/(1-p)
P(Y=1|X1=1) = P(Y=1, X1=1)/P(X1=1) = (1-2p)/(1-p)
A lower bound
Let X1 and X2 transmit independently
P(X1 = 1) = 1 – P(X1=0) = a
P(X2 = 1) = 1 – P(X2=0) = a
Then: R1 = I(X1;Y|X2) = H(Y|X2) – H(Y|X1,X2)
= ah(a) = R2
The maximum = 0.616 > 4/7
X1 X2
Y
The upper (outer) bound
The innerinner bound ( for independent transmission) is
< the Shannon outerouter bound ( X1 and X2 dependent).
For R1 = R2 inner rate = 0.616
outer rate 0.694
The exact capacity is unknown!
X1 X2
Y
bounds
R1
1
1 R2
Outer bound
inner bound
0
X1 X2
Y
Broadcast
X
Z
Y
Z transmits information to X
same information to Y
R1 I( X; Z )
R2 I( Y; Z )
R1 + R2 I ( Z; (X,Y))
= I( Z; X)
broadcast
X
Z
Y
Z transmits information to X
different information to Y
R1 I( X; Z )
R2 I( Y; Z )
R1 + R2 I( Z; (X,Y) )
= I( Z; X) + I(Z;Y|X)
example: Blackwell BC
Z X Y
0 0 0
1 0 1
2 1 1
R1 I( X; Z ) = H(X)-H(X|Z)
R2 I( Y; Z ) =H(Y) –H(Y|Z)
R1 + R2 I( Z; (X,Y)) log23
X
Z
Y
example
Y
00/11 01/10
00 00 10
X 01 12 02
10 21 20
Z
Z X Y
0 0 0
1 0 1
2 1 1
I(Y;Z) = 1
I(X;Z) = log23
Rsum = (1+ log23)/2
= 1.29 bit/tr.
X
Z
Y
2-access channel
X1
X2Y
X1 and X2 want to communicate with Y at the same time!
Obvious bound on the sum rate:
R1+R2 H(Y) – H(Y|X1,X2) H(Y)
Two-access models
Switching two-Adder
x1 x2 y x1 x2 y
00 00 0 01 01 1 10 0 10 1 11 1 11 2
Y Y
X1
X2
X1
X2y y
0 1
1 2
0
1
Two-adder (1)
Capacity region
0
1
1
2
X1 0 1
0
X2 1
X1
X2
Y
R1 1 from X1 to Y
R2 1 from X2 to Y
R1+R2 H(Y) 1.5
0 .5 1 R2
R1
1
0.5
0
timesharing
Two-adder (2)
Coding strategy: -error
User 1: transmit at rate R = 1 bit, i.e. P(X1 = 0) = ½
User 2: sees erasure channel.
0
User 2
1
0
1
2
.5
.5
.5
.5
Max H(X)-H(X|Y) = ½
Hence:
rate pair (R1,R2) = ( 1, ½ )
X1
X2
Y
Two-adder (3)
A simple a-symmetric strategy: 0-error
X1
X2 000 001 010 011 100 101 110
000 000 001 010 011 100 101 110
111 111 112 121 122 211 212 221
Efficiency = 1/3 log214 = 1.27
X1
X2
Y
Two-adder with feedback (1)
Question: can we enlarge the capacity region?
Yes (Wolf)! Is this a surprise?R1
1
0.5
0 0 .5 1 R2
Capacity region characterization:
modified by Cover-Leung
Willems
X1
X2
Y
Two-adder with feedback (2)
0
1
1
2
0 1
0
1
X1
X2
N independent transmissions
2
Nuncertainty Solve uncertainty
in steps 3log/2
N2
Total efficiency: = 1.52!3log2
2
2
NN
N
Joint output selection
X1
X2
Y
Switching channel (1)
X1 0 1
X2 0 0
1 1
Y
X2={0,1}
X1={0,1}
{,0,1}
tri-state logic
P( \ pass info ) = ( 1-a, a )
Rsum(max) = a + h(a) log23
simple coding example (2)
For n = 2:
code 1: 11 10 01
code 2 00 00 0 0
11 11 1 1
Sum rate: Rsum = (log23 + 1)/2 = 1.3
X2={0,1}
X1={0,1}
{,0,1}
Switching channel with general coding (3)
Strategy: code 1 (0 0 1 0 ... 0 1 0 ) < dmin zeros
Linear code 2 (0 1 1 0 ... 0 0 0 ) k n – dmin+ 1
receive: ( 1 ... 0 )
correct dmin - 1 erasures
PERFORMANCE: = C!!= C!! !Chn
n
n
1dnR n
mindminsum
n
Extensions (2)
T-user ADDER +{0,1}
{0,1}
{0,1}
{0,1, •••, T}•••
T-user ADDER + (1)
Input output
User 1 { 00 11} { 10 01 11 20 21 31 12 22 }
User 2 { 10 01}
User 3 { 00 10}
Efficiency: 3 * ½ = 1.5
T-user ADDER + (2)
Input
User 1 { 000 111}
User 2 { 110 001}
User 3 { 000 001 010 100 101 011}
Outputs: { 110 111 120 210 211 121 etc. }
Efficiency = ( 2 + log26 )/3 = 1.53 bits/tr.
record: 1.551 by van Tilborg (1991)