Bottom-Up Evaluation of 802.11 Performance Testing Dr. Michael D. Foegelle ETS-Lindgren September 2004 Foegelle, ETS-Lindgren doc.: IEEE 802.11-04-1156-00-000t

Bottom-Up Evaluation of 802.11 Performance Testing

Dr. Michael D. Foegelle

ETS-Lindgren

September 2004

Foegelle, ETS-Lindgren

doc.: IEEE 802.11-04-1156-00-000t

Submission

September 2004

Foegelle, ETS-LindgrenSlide 2

doc.: IEEE 802.11-04-1156-00-000t

Submission

Overview

• Introduction

• Layer Analysis

• Basic Operational Testing

• Environmental Effects

• Summary & Conclusion

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Introduction

• Top-down approach of defining usage cases and working down to resulting metrics and needed tests is a useful starting point.

• However, tendency to focus on usage case scenarios is hindering determination of wireless performance parameters needed in order to progress towards development of equivalent laboratory techniques .

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Introduction

• This presentation will use a bottom-up approach to identify and evaluate critical contributions of the wireless portion of the DUT, concentrating primarily on the physical layer/radio effects.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Layer Analysis

• In the ISO 7-layer network model, only the bottom two layers (Data Link and Physical) have anything to do with the device being wireless as opposed to any other network device.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Layer Analysis

• The tests necessary to evaluate the upper layers should not differ from testing performed on non-wireless network devices.– Only indirect contribution of lower levels make

this any different from non-wireless devices.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Layer Analysis

• Effects of environment (location, interference, etc.) directly affect performance of physical layer, which in turn affects the MAC layer and so on, all the way up the chain.

• Analysis of environmental effects on the physical layer, starting at the radio, will show that it is possible to determine well defined laboratory techniques to duplicate these effects.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Basic Operational Testing• Before we talk about performance testing,

there’s a level of testing required to verify that the device is actually operational.

• Radio functionality– Ability to generate required signals and

modulations at expected levels.– Ability to detect/decode input signals in a

nominal range.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Basic Operational Testing• Additional PHY layer functionality

– Ability to encode packets and handle shared media.

• MAC layer functionality– Ability to establish connection, perform

handshaking, and transfer data.

• Some level of standardized functionality testing is assumed prior to attempting performance testing.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Distance• The simplest environmental effect to evaluate

is that of distance on direct line-of-sight communication.

• Signal level drops as distance increases or as signal passes through intervening obstructions.– Relative phase relationship of signal remains

constant over time. – No actual degradation of signal content.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Distance• Implies evaluation of basic radio

performance (transmit power and sensitivity)– Ability to detect a coherent signal– Ability to demodulate/decode a given complexity

of modulation (data rate control) at a given signal level

– “Clean” signal – No interference

• Secondary effect of propagation delay unlikely to be significant for range of 802.11– ~0.33 S at 100 meters

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Distance• This testing can easily be performed as a

conducted test using a calibrated radio and a reference 802.11 interface.– Reference interface provides known good

communication behavior– Calibrated radio provides traceable:

• Power and signal quality measurements of DUT transmit signal at various data rates.

• Variable forward power received by the DUT to test sensitivity and data rate control/feedback.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Multipath• Multipath effects can be sub-divided and

investigated separately.

• Multipath fading in its traditional sense refers to the reduction in received signal due to the location of the receive antenna in a field null.– Null caused by the summation of one or more

out-of-phase signals with similar amplitudes.– Dynamic fading refers to the effect seen when

motion causes the receiver to move through nulls.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Multipath• The signal level effects of fading (i.e. similar path-

length, out-of-phase signals) would have a similar effect on performance as that for distance.– Since fading (null location) is frequency dependent,

spread spectrum signals offer some rejection to fading.

– Accurate test for fading would entail use of variable notch filters (fading simulators) to create a frequency dependent fade behavior.

• Testing for dynamic fading would entail determining the device’s response to dynamically changing faded signals.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Multipath• The term fading refers to audio behavior of

traditional amplitude modulation. – As signal amplitude fades, so does audio

amplitude. Sound fades out.– At audio frequencies, modulation effects of

multipath are negligible.

• At high data rates, modulation rates approach path lengths (11 MHz chip rate ~ 27 meters), so modulation corruption due to varying multipath lengths becomes an issue.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Multipath• Worst cases of modulation corruption likely for two

similar strength, in-phase signals with considerably different path lengths.– E.g. LOS through wall (lossy) and NLOS from distand

reflector

– Results in strong signal for portions of some bits, nulls for others (dependent on phase modulation at given bits).

• SS demodulation techniques are designed to decode through multipath effects, but suitable test cases are desirable.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Multipath• Again, behavior can be simulated through

cabled test.– Brute force: split signal and use delay lines and

attenuators to create a true multi-path input.– Custom phase locked transmitters capable of

generating identical signals with desired delay and amplitude variations.

• Note that multipath tests need only be receiver side tests.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Antenna• In general, the antenna is a modifier to the RF

performance previously discussed.– An antenna simply integrates (or adds up) the various

signals directed towards it, with a weighting factor as a function of both propagation and polarization directions.• Directionality can change magnitude of multipath effect seen by

radio by minimizing contribution of other paths, but does not change radio’s innate ability to handle a given multipath effect.

– Does interact with circuitry and nearfield environment, changing performance (see 11-04-0675)

• Radiated over-the-air tests need only verify radio TX/RX performance with antenna loading and interaction. -- No need to perform all tests OTA.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Network• Once an 802.11 device becomes part of a wireless

network with multiple endpoints, there are a variety of interactions relating to both the PHY and MAC layers that should be evaluated.– These include physical effects:

• Out of band signals and tolerances• Collisions, cross-talk, etc.• Hidden endpoints & network overlap.

– And protocol related effects:• Timing, holdoff, etc.• Multi-point access control• Handshaking

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Network• There are a number of “network stress

tester” type products on the market designed to simulate multiple devices on a network.– These products typically simulate a multitude of

MAC layers operating through one radio.• Single radio approach can’t simulate real world

collisions and other RF interaction behaviors.• Some level of synchronization already exists above the

PHY layer to allow unit to operate.

• A multi-radio device is needed to properly simulate real-world network behavior.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Environmental Effects: Interference• Interference testing may be more of a

functional test than a performance test.– Any properly operating radio/PHY layer should

expect to have similar susceptibility to a given level and type of interference.

– Outside interference is not a predictable event. • Can simulate a wide variety of interference, but the

resulting metric is not likely to represent any particular real-world behavior.

– As much EMC as operational or performance related.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Summary & Conclusion• This is an incomplete list intended to get the task

group thinking in terms of the input parameters to overall performance metrics.

• It’s not necessary to set up real-world networks to simulate real-world effects. Most tests can be done with a cabled connection to appropriately designed test and measurement equipment or systems.

• Any number of the metrics discussed by the TG may be evaluated, as needed, but they should be evaluated in terms of the factors that contribute to variation in each level of the 802.11 layer structure.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Summary & Conclusion• Many metrics may have the same interaction with

the available input parameters. It may not be necessary to test all conceivable metrics as a function of these parameters.

• We need to avoid getting caught up in upper level metrics that may not change due to the use of a wireless network as opposed to any other network.

• We should pick a starting point and develop some experimental test plans around currently available equipment for simulating the wireless effects.

September 2004


doc.: IEEE 802.11-04-1156-00-000t

Submission

Summary & Conclusion• Point-to-point behavior is easy to understand and

should be an initial starting point for radio/PHY evaluation.

• Network stress testers are a good starting point for MAC level testing.

• Additional effects and interactions may be defined and investigated as we proceed, building on what we learn with the simpler tasks.

• New test equipment or combined rack systems will need to be developed to carry this effort forward.

Documents

Bottom-Up Evaluation of 802.11 Performance Testing Dr. Michael D. Foegelle ETS-Lindgren September 2004 Foegelle, ETS-Lindgren doc.: IEEE 802.11-04-1156-00-000t