Paper No. 89-3260, AIAA Flight S~rnulation Technolog~es Conference 14-16 August 1989, Boston, MA

A COMPREHENSIVE COLLECTION OF PROCEDURES FOR SIMULATION VERIFICATION

Matthew 5 M~ddendorf rT h-

Willlam V Johnson ' f P sd Michael J Gilkey @

Terrence D McClurg 4 4-

, * Systems Research Laboratories, Inc

A Dlvls~on of Arv~n Calspan 8 ' Dayton, OH 45440

Abstract

Because sirnulators a re becoming more widely used for t r a i n l n g and research, i t i s becoming increas ingly impor t an t t h a t t he s imula to r s be verified to ensure tha t they behave a s designed and intended. The incorporation of new technologies has made simulators more complex which has forced the verification process to become more diverse and thorough.

The dyniimic n~odels used in the simulators have typically been the focus of the verification process. Discussing the verification of highly sophisticated models can be a whole paper in itself. This paper will touch upon the verification of simple models, but will concentrate on a wide range of o ther s imula tor components t ha t should be verified. This list of components and procedures results from nearly a decade of experien~.e in building research simulators.

Althuugh this list of components was compilcd while develup~rlg llight simulators used in research laboratories, rts applicability is not limited to those simulators. Components such as timing integrity, cornmunicatiun validity, and input control scaling a r e i m p o r t a n t t u a l l r e a l - t i m e s i m u l a t o r s . Procedures for vt~rlfylng these components, and several others, a r e covered in this paper.

Additional c,,rnponents include motion loop delay measuremenL, visual loop delay measu remen t , graphics e q u a t ~ o r ~ s , forcing function spectra and filtering, moti~jn t cling device scaling, time tags on c o l l e c t e d d a t i l , r a m p f u n c t i o n s , n u m e r i c a l integration, the Euler transformation matrix, and the Euler equations of motion. Each one of these c o m p o n e n t s m u s t be ve r i f i ed t h r u u g h some quantitative means.

Introduction

I t i s paramount t h a t research s i m u h t o r s be verified to ensure the validity of the collected data. If a long term research effort is conducted in a simulator tha t contains a mistake, then two serious consequences rnaq result. First, if the experimental d a t a a r e used to gu ide s i m u l a t o r des ign and development, then the simulator industry may be lead astray. Secundly, if the mistake is uncovered a t or near the end of a study, then considerable time and money is wasted. To reduce the likelihood of mistakes, a comprehensive set of rigorous procedures needs to be meticulously followed. The procedures described in this paper are mainly quantitative in n a t u r e , however , subjec t ive eva lua t ion by a n experienced individual also proves to be qui te valuable.

Copyrighr Q American Inst~tute of Aeronautics and r r ronaut tcs , Inc., 1989. All rtghts reserved.

Recently a test pilot flew one of the laboratory's fighter-type aircraft simulators and commcntcd tha t the dynamics seemed to be biased to the left. His specific comment about the dynamics was proven to be incorrect, but in flying the simulator the bias was i t pp t~ ren t . T h i s led to inves t iga t ion of o t h e r s imula to r components , speci f ica l ly t h e i n p u t controller (in this case a side-mounted force stick). I t was discovered tha t the input controller was rotated >even dcgrecs off-axis. T h i s f inding led to a n irnprovernent to the simulator and a n addition to the s imula to r ver i f ica t ion procedures . A l t h o u g h subjective comments may also be solicited, they are nut a subs t i t u t e for t h e objective p rocedures discussed in this paper.

Components

Several simulators used in research incorporate wind gus t models andlor sum-of-s ines forc ing functions tu gain spectral information about the human operator and the man-machine system.1 In such simulators it is very important to have stable timc frames to prevent smearing in the frequency domain. Multi-tasking operating systems can cause frame time instability. Several of these systems have 3 job scheduler t ha t runs a t 60 Ilertz. The interrupt f u r the job scheduler could coincide with an irlterrupt from a programmable clock tha t is used to drive the real-time simulation software. If t h i s occurs then there will be some jitter in the initiation uf the simulation time frames (see figure 1). This instability can reduce the signal-to-noise rat io of data that is already troubled with noise due to the nonlinear human in the loop. Some of the problems associated with multi-tasking operating systcrns can be avoided by turning off some system services, and in some cases, disabling the system clock.

IJsing the exact duration of the time frame in thc calculation of coefficients and integration constants can be as important as having stable t iming . If the time frame is off by just a little (59.8 Hertz compared tu 60.0), then the characteristics of the dynamics, the integrated values, and the overall duration of the trial (for experiments tha t use fixed length tr ials) will be slightly in error. Th i s problem i s most prevalent in multi-computer simulations where the computers need to be synchronized, par t icular ly when there is a separate graphics computer. One way to synchronize these computers is to detect the vertical retrace of the graphics monitor and use this signal to drive the other computers. If this method is used then the exact duration of the time frame can be measured us ing a p rogrammable clock. T h i s measurcrnent should be t a k e n eve ry t i m e t h e

simulator is pciwered up. 'I'wtr graphics computers made by t h e same vendor can have slightly different refresh rates . Also, switching between the two majur image preserltatiun formats (60 Hertz nun-interlaced a n d 30 H e r t z in te r laced) can have a s ign i f ican t i m p a c t o n t h e d u r a t i o n of' t h e t i m e f r a m e . Spec i f ica l ly , sonic 30 l i c r t z i n t e r l a c e d sys tcn ls actual ly have ~ e r t i c a l retraces occurring a t 66 Hertz. A digi tal ou tpu t h ~ t and a n oscilloscope can be used to verify both the s imulat ion t ime frame stabi l i ty a n d durat ion (see figure 1).

Each t ime fr ;~nie can he initiated by :in interrupt from a programmable clock or by a n in te r rup t from a n e x t e r n a l i n p u t , p re fe rab ly from t h e ver t i ca l retrace uf the graphics monitur. I n both cases the digi ta l ou tpu t hit can be se t when the interrupt first occurs a n d c1e;tred a f t e r a l l so f tware execu tes . Displaying thc va lue of t h e d ig i ta l o u t p u t on a n oscilloscope will yield both the exact durat ion of the t ime frame and t h e execution t ime of the software. 'I'o observe t rame t ime stability i t m a y be necessary to pu t two full f ' rxnes up on the scope because the scope m a y autoniatically trigger off the set t ing of the bi t t h u s eliminating t h e j i t ter on t h e f i rs t frarne. Ilowever, if ' there 15 instability then i t can be seen in t h e s t a r t of'thcl sccond frame (see figure 1 ).

tl E:xecut~iin time of software ta - I)ur:it~on i~f ' t irne frame t:: - Timing illstability

'I'he e x e c ~ ~ t i o r i rlme of' individual routines is alsci eas i ly m e a s u r e d us ing t h i s s e t u p . T h i s c a n be achieved by clear ing the bit before the par t i cu la r rout ine and set t ing the bi t a f te r t h e rout ine ( see figure 2).

+ t4 t~ - L x e c u t ~ o n t ime of a n

i n d i v ~ d u a l rout ine I I

Figure 2. l*:xtxution t ime measurement

Another aspect of t iming integri ty involves the verification of ramp t imes. Very often r a m p s a r e used to ensure snlooth ini t ia t ion of motion cuing. injection of independent variables, and introduction of gusts. If t h e subject/pilot is being scored based on h i s o r h e r pcr l 'ormance i n t h e s i m u l a t o r , i t i s i m p o r t a n t t h a t t h e r a m p s be completed before scoring begins (see figure 3). T h i s can be verified by

I i i ~ n d - ( , h e c k i n g t h e r a m p t i m e c o n s t a n t s a n d cwmptiring them to t h e t ime when scoring begins. Another way to verify the r a m p s is to review t ime his tory plots of t h e p e r t i n e n t v a r i a b l e s f r o m a cwllected da ta file. If t h e r a m p s a r e no t properly synchronized it could effect t h e scoring a lgor i thm, bias the experimental variables, a n d decrease s ignal- to-noise ratios if frequency domain analysis i s being used.

Scoring

af ter here

Ramp ,,.., +. +/

Time - - - - Gust r a m p . . . . . . . motion r a m p

- . - . - Independent variable r a m p

Figure : 3 . Simula t ion r a m p s

LVhcn t i m e h i s t o r y f i l e s a r e c o l l e c t e d i t is iluportant to verify the t ime tags (i.e. t h e effective sampling instance) for each channe l of da ta . I n most cases t h e t i m e t a g s s h o u l d m a t c h e a c h o t h e r . huwever, in some experiments t h e invest igator m a y request t h a t t h e t a g s m i s m a t c h f o r a n a l y t i c a l p u r p o s e s . I m p r o p e r l y m a t c h e d c h a n n e l s a r e p:irticularly problematic when performing frequency domain analysis. An incorrect t ime t a g will mani fes t itself a s a phase shif t when t h a t par t icular channe l i s used to form n t ransfer function. A simple example sing a difference equat ion of t h e form Y =(X,, - X, 1 J 1' f i r numerical differentiation (see figure 4) will i l lustrate how two collected c h a n n e l s of d a t a c a n h ; ~ v e m ~ s m a t c h e d t ime tags. T h i s equat ion f i ts t h e form of a f i n ~ t e impulse response filter (F111) a n d h a s a s igna l t ime de lay associated w i t h i t of T/2. 2 Another example where a mismatch i s a n ar t i fact of the specific numerical t echnique i s t h e use of a n Adams-Uashforth integrat ion algori thm. T h e use of th i s a lgori thm can introduce a t ime advance relat ive LO the d a t a collection t ime reference. 3 Additionally, feedback s igna ls implemented i n a sampled d a t a s y s t e m i n t r o d u c e d e l a y s d u e t o t h e o r d e r of operations t h a t m u s t be accounted for. D a t a channe l t i m e t a g s c a n b e c h e c k e d b y t i m e h i s t o r y examination of channe ls a f te r d r i v i n g t h e s y s t e m with a known forcing function. Frequency ana lys i s ~ i ' t h e same type of d a t a will reveal a n y mismatches by phase deviations from the expected values. These same considerations affect t h e validation of t ranspor t delay to be covered later .

I TI2 FIR Delay

I

Figure 4. Numerical delay

Another titnin,: integrity issue is the stationary ini t iat ion of da ta collection. 'I'his consideration applies specifically to research simulators whcrc the trial is initiated by a n experimenter. If the tr ial beg inn ing and subsequent d a t a collection a r c dependent on a button being pushed or a switch being thrown thcn a l l r e su l t an t act ions by the software m u s t be keyed off t h e same moment . Specifically, when the huttc~n is prcsscd or released, but not both. 'I'his is particularly important when time history data is being averaged. Verification of stationary data collection is simple when there is :r determinist ic da ta channel ( i .e . a gus t channel c o m p u t e d u s i n g a s u m - o f - s i n e s t e c h n i q u e ) . Examination of the first da ta point in the gus t channel across several files will accomplish this task.

The last ~ i tmnr : integrity issue is thc verification of de lay queues . I n some cases vehicle s t a t e s (attitude and p?sitim) arc delayed bcforc they arc sent to the motlon cuing device so tha t the motion i n f o r m a t i o n presented to t h e p i l o t c a n be synchronized with the visual information presented by the graphics system. In other cases, the oppositc is t rue. In some research simulators the delay in the visual loop i s rn i~nipula ted a s the independent variable therefirre requiring a delay queue. If delay queues are impletnented i t is important to verify t h a t they are producing the correct amount of delay. Care must be taken to see if the queue is updated before or after the delayed value i s pulled out to prevent a one f r ; ~ m e delay er ror . Th i s can he accomplished by Inspecting the delay queue logic and code. The inotion loop and visual loop delay measurement sections of this paper will present quantitative vcrificiition procedure.

Vehicle Dvnainics i'erification

As stated e a r l ~ e r , i t i s not the purpose of this paper to deal with the verification of sophisticated full nonlinear aerc~dynarnic models. Thc models used in the research laboratory where these procedures have evolved have been, for the most part, transfer function models. Verification of these models is u s traight forward process utilizing a linear ~u top i lo t and a sum-of-sines approximation to a Dryden 4 wind gust model (see figure 5). The simulation is run . with the autopilot engaged, and the collected data is analyzed using frequency domain techniques. Thc resultant frequency data for the inputs and outputs of each element in the aerodynamic model arc uscd to form t r ans fe r functions. For each t r ans fe r f u n c t i o n t h e e x p e r i n ~ e n t a l g a i n a n d p h a s c information i s computed. This experimental data is then compared to analytical data for verification of the aerodynamic model.

Visual Loop Delay Measurement

Research h a s shown t h a t excessive t ranspor t delay in aircraft simulators can have negative effects on pilot performance and t ransfer of training.5 The re fo re i t i s very i m p o r t a n t t o he a b l e to a c c u r a t e l y q u a n t i f y t h e t r a n s p o r t d e l a y i n simulators. More~~ver , the verification of the actual delay a a ins t the expected value i s a n excellent cheek ofother simulator components. The transport delay i s defined a s the delay due to the computer implementation of t he s imula t ion and does not include delay due to the dynamics of the simulated vehic1e.f; This delay is measured between pilot control input and pilot cuing. The measurement is accomplished using a frequency domain steady state

- Autopilot

1

Figure 5. Simplified Roll Dynamics

technique. A sine wave i s injected into the system in place of' the input controller and a photo-sensitive device is used to measure resultant changes on the graphics monitor. Us ing a frequency response analyzer, the t d a l phase shift is measured from the sine wave input to the photo-sensitive device output (see figure 6). Then the phase sh i f t due to the dynamics is subtracted from the total phase shift and the remaining phase shift is due to the transpurt delay. Because the input sine wave is a t a known rrequency, the phase shift can be converted to time, in seconds. Consider the following equation:

'I'ransport delay(seconds) = ctt~tel measured phaseineg) - Vehicle d namics phase(Deg)) * ( ~ e r i i l d input freqoencyi360) [ I )

T h e e m p i r i c a l l y d e r i v e d t r a n s p o r t d e l a y measurement is then compared to the analytically derived value . Tf they do not match then t h e simulation may contain an error, or, a component may not he accoun ted for i n t h e a n a l y t i c a l computation. Some sources of transport delay a re executiun Lirne, delay queues, numerical techniques, s ample a n d hold (zero o r d e r h o l d ) , g r a p h i c s pr~rcessirig, and delay due to orderofoperations.

ILl~ltiun I,uup Delay Measurement

For rn~~tic~n-based simulalurs, the motion loop is the path shown in the block d i ag ram, Figure 7, from the control input l o the molion display. The transport delay around the motion loop i s a s i m u l a t i o n parameter that must be characterized and controlled. This delay must be theoretically determined and experimentally verified.

, , I he sources of transport delay around the motion Iuop are three: 1 ) delay due to the order of I10 operations; 2) zero order hold delay or the digital-to- analog converters (DIA's); and 3 ) motion display device servc.~controllers. The delay due to order of c~perations occurs when the Am's are sampled before the I)/A's are output. This is commonly done to give the DiA's maximum settling time before sampling the A!l)'s and to preserve the periodicity of the 110 operations (e.g. due to variable execution time). Thus :in artificial delay equal to one simulation frame time is incurred. A zero order hold delay i s a n artifact I ) ( ' the recunstructivn of an analog signal from a digital signal resulting in an effective delay of 3 of

Inputs

Analyzer

Graphics Computer

Figure 6 . Visual Loop Delay Measurement Setup

Inputs

Motion Display Device 1

T Motion Sensors

Servo- controllers

t

~ l ~ ~ ~ i t h ~ Collection

Figure 7. Motion Loop Delay Measurement Setup

the frame time. L'ist of all, the servocontrollers can be expected to include some signal conditioning electronics, such a s low-pass filters, which affect the phase response of the motion device. The frequency characteristics of the servos are usually specified by the manufacturer or can be analytically determined and should be ~neasured empirically as well.

With the ahove information, the theoretical t ransport delay a round the motion loop can be determined. Once th is has been done, the actual sys t em m u s t be ver i f ied by compar ison wi th experimentally collected data . Using a frequency response analyzer, a sinusoid a t a known frequency can be injected into the loop a t point 1 in Fi ure 7

r f above. The frequency must be within the ban width of the system in order to achieve meaningful results. By feeding hack the system response a t point 2 to the analyzer, the gain and phase of the system between the two points is c.omputed. The total delay around

the loop a t that frequency is given by the following equation:

Time delays due to the vehicle dynamics and drive algorithms are not transport delays and must he s u b t r a c t e d f r o m t h e s e e x p e r i m e n t a l measurements. Care must be taken to account for 180' phase shifts due to sign changes in the dynamics or drive algorithm.

Motion Device Scaling

Motion Display Device

Motion Sensors

Figure 8 shows a simplif ied block d i a g r a m encompassing the motion display device a n d i t s associated control a n d recording functions. T h e motion drive a lgor i thm computes t h e type a n d amount of desired platform movement. This quanti ty is scaled appropriately and output to a digital to analog converter (DIA). The D/A outputs a n analog voltage proportional to the digital input. This signal serves as the input to the servocontroller. The servo drives the motion device according to i ts design.

Motion device scaling i s simply the process of verifying the calibration of the blocks between the drive algorithm and the display device. This ensures that the subject is experiencing the desired motion. Another related step which is often overlooked i s to verify the calibration of the blocks between the motion device and data collection point. This insures t h a t t he movemen t of t h e p l a t f o r m i s b e i n g accurately recorded.

There are two scaling steps required to achieve the desired platform motion from the drive algorithm block. First, the proper servo input voltage must be determined tha t will result in the desired movement. Next, the digital value for the DIA must be computed that will generate this voltage. The scaling of each motion actuator must be verified individually.

The following equations i l lustrate th is scaling using an example of l inear displacement a s t h e measure of motion.

The nnly assunlption tha t has been made is that the system is stationary. l'his simply means tha t the scale factors do not change with time.

The overall n1t)tinn s c ~ l i n g is driven by the ca l ib ra t ion of t h e servocontro l ler . T h i s i s a n electronic c ~ ~ n t r ~ i l system with feedback. The sc i~le factor t h a t couvt.rts i n p u t voltage to platform movement mus i he verified empirically us ing :I

precision vol tage source a n d t h e appropr i a t e measuring tt~ol. TI~c voltage source is used to inject a n arbitrary voltage into a servo a s shown a t point 1 in Figure 8. 'I'hc resulting travel of the individual actuator is then measured using a ruler, protractor, or similar device a t point 2. Data i s collected :it regular interval., within the active range of the servo. A least squ'tres fit is then performed on the da ta to determim the equation of the curve tha t yields the closest match. This often proves to be linear or near11 so. hut even nonlinear relationships can be easily nccok~nted for with software. A highly nonlinear fit may lndicate a hardware problem with t h e s e r v o o r n l r ~ t i o n d e v i c e a n d s h o u l d be investigated further. The best fit relationship tha t results is the function Fg of Equation (4) .

The next sci~lc factor to be verified is the one that converts the des i~ ,ed platform movement into the proper digital value to send to the D/A. The DIA's arc characterized by the number of bits of resolution and analog voltagc range. The desired coefficient i s simply the number of volts per bit. For example, n twelve bit DiA can represent 4096 distinct levels. If the output range is k10.0 volts, there are (20 volts + 4096=) 4.88 mill ivolts per level . Therefore, function Y1 of Lquation (3) is simply a multiplicative constant possihiy with an Offset a s shown in the example equation (5). This scale factor should be checked regularly to detect hardware problems with the digital-to-analog converter.

UIA lnput = 0.00488(volts-1) x F2(desircd displ (mrn))(volts) + Dig Offset (5)

The second !)art of motion device scaling is to verify the accurate scaling of motion sensor data to real world quantities for da ta collection purposes. T h i s i s s i m p l y a m a t t e r of c a l i b r a t i n g t h e transducers used in the device a s well as the ND's. The following equations illustrate this scaling.

AID lnpu t (volts) = P3 (Actual Displacement) ( 6 )

Recorded Displacement (mm) = F I (Output frotn N D )

For every t y p ~ ill motion there is a n appropriitto transducer t ha t allows the ~ l a t f o r m movement to be recorded. In the example of linear displacement used above, a linear variable differential transformer is used ( L V D T This device produces an analog voltage d i r e c t l y p r o p o r t i o n a l t o t h e LVU'I ' s h a f t displaccmcnt tind ~ t s characteristics are specified by the manufacturer. The calibration of these devices must be verified rcgularly. This is done by collecting d a t a on the t ransducer o u t p u t vol tage a s t h e platform m ~ v e s . l'his can be done simultaneously with data collection from the first par t of motion scaling. While > L voltage is being injected into the

system a t point 1 of Figure 8 and platform motion data is being collected a t point 2, t he transducer voltage should also be recorded a t point 3. Data analysis similar to tha t used in part 1 will yield the relationship Fa in equation (6) above.

A D scaling i s the inverse of t h e DIA s c a l i n g described above. Again, this scale factor should be checked regularly for hardware problems.

Cummunication Validation

The communication link between the simulation umpute r and the graphics computer must be verified tt) ensure the integrity of al l transmitted data. This is especially important if position deltas are being bent to and summed in the graphics computer. The usc of position del tas may be necessary d u e to hardware limitations (i.e. R8232 updated a t 6011~). Thc communication link can be easily verified by sending the f ina l posit ion v a l u e s back to t h e simulation computer to be compared with its final values. Another aspect of communication t h a t should be verified involves h igh speed para l le l transmission of data. When floating point values are passcd i t is impor t an t to verify t h a t t h e d a t a representation formats are the same. If not, then the conversion process needs to be verified using a wide range of values including boundary conditions. All data packets should include a checksum.

l'crification of Graphics Equations

The order of translat ions and rotat ions in the graphics computer needs to be verified. When the vehicle is ~naneuvered through large angles, errors ~n the graphics equations are obvious. llowever, for tasks tha t require small perturbations about zero there could be a n error in the equations t h a t may go unnoticed. Another important graphics parameter t ha t must be verified i s the field-of-view (FOV). Cmventionally the value used for the field-of-view closely m a t c h e s t h e a c t u a l h a r d w a r e s e t u p . Specifically th is value equa l s twice the inverse tangent of the distance from the eye to the screenlcrt dlvided by the half-width of the screenlcrt. Ca re should also be taken to see which axis the field-of- vlew is with respect to. Some modern graphics wmputers specify the field-of-view in the y-axis (vertical) while the measurement i s t a k e n wi th respect to the x-axis. graphics systems t h a t have >I four to three ( X to Y) aspect ratio, the field-of-view In the x-axis can be transformed to the y-axis using the following equ a t' ]on.

b'i)V,. = TAN-1 ( T A N (FOV,) *314 )

Verification of Gust Spectra and Gust Filters

T h i s ve r i f i ca t ion p rocedure i s speci f ic to simulations t h a t use gust , or turbulence, t h a t i s approximated by a sum-of-sine waves. The use of a sum-of-sines model to approximate a wind gus t i s wcll suited for research simulators because the gust inputs appear to be stochastic to the sub'ect and i t is useful for frequency domain analysis o ! the human operator and the man-machine system.7 If each component in the sum-of-sines profile is a n integer harmonic of the fundamenta l frequency (e.g. t h e reciprocal of the r u n l eng th ) a n d i s gene ra t ed without distortion. then the collected da ta can be analyzed using a fast-Fourier transform (FFT). The results of the FF'r can be used to verify the gus t spectra. The FFT of the gust spectra should contain

power clnlj ; i t L I I C nominal i npu t frequencies.; Ilowever. a11 of' tht: Fl4"1' data should be examined to make sure then. i h nu significant power a t non-input frequencies due tu harmonic distortion. Significant power is power i~bleve the n~)mina l remnant level due to sampling hor.mally when a sum-of-sines profile is selected t , ) riudc.l gust there is a desired RMS level for the signal. l;or example, if a moderate vertical gus t is needed to per turb the pitch s t a t e s of a s i m u l a t e d a i r c r d f t t h e n a l lMS l eve l of 7 . 7 feetisecond might t)c selected. The desired RMS level of the input signal can be verified using the FFl' data. The square root of the total power computed from the FYl' data in the input signal should equal the selected K M S ~ i i l u e .

In some rhc gust input is filtered to reflect the effect uf the vehicle's immersion in the gust rnediutn. S ~ c [ , ~ l d i y , the gust may be filtered t ~ , determine its eftet,t on the vehicle. The verificat~on of these filters v i i n hc. easily accomplished using the h'P?' data. 'I'he F171 data can be used to compute a t ransfer function for each of t he f i l te rs . The charac ter i s t ics of t he t ransfer function can be compared against the design spccifications of the filters for verification.

Controls Scaling aud Ueadband Verification

All flight simulations require user inputs of some type to allow control of the simulated aircraft. Thcse can be control sticks, throttles, or instrumentation switches. This paper will concentrate on the control stick and throt t le , since these a re the primary devices tha t close the man-machine loop.

Control stick> be either of a displacement or force type. Both are transducers which generate an output voltage proportional to stick movement or applied stick force. Since only side-mounted force sticks are used here in the laboratory where these procedures evolved, the following discussion will concentrate on these. The comments on force sticks can be directly extended to displacement sticks as well. The hardwnre and software issues of control scaling will be discussed here.

The stick 11as both roll and pitch rate control commanded with forces a long the l a t e r a l and longitudinal axes, respectively. The device must be checked to insure tha t these axes are orthogonal. 'I'his is done by applying a force along one stick axis and reading the voltage of the off-axis output with a voltmeter. 'l'he force should be rotated about the stick's vertical axis un t i l the output voltage is nulled. 'l'he direction of the force vector should be marked and thc: s tep repeated for t he opposite direction. 'l'he positive and negative directions of each axis s h ~ ~ u l d hc co-linear. This can then be d(~nc for t h e o t h e r a x i s . 'l'he two a x e s s h o u l d be orthogonal.

Now t h ; ~ ~ t i le control device axes have been characterized. measu remen t s m u s t be t aken to confirm their propcr alignment with respect to the fixed axes of the simulator. Often the control stick is required to be roiated several degrees off axis. A protractor can be used to verify the correct angle.

Cal ibra t~on , i f ' the control stick is similar to tha t of any trarlsduccr. A fcorce is applied to the stick and the resulting output voltage is recorded. The force vector m u s t 1 ~ . illong the t r u e ax i s of control determined previously. Care must be taken to apply the force : ~ t the s i m c point on the stick every time it

i:, cil1ibr;itcd to insure consistent results. This point should be a t the nominal hand grip centerpoint and t h r ~ ~ u g h the center of t he s t ick . D a t a m u s t be collected tit regular intervals throughout the active range c~f the device. This procedure mus t t hen be repeated for the opposite stick axis.

Once this has been completed, the da ta can be p I ~ ~ t t e d for visual inspection. A f i rs t order l ea s t squares fit is often helpful to determine the slope and intercept c)f t he l ine . The f i t should be l i nea r , qmmetr ica l , and have :+ small offset. Offsets reduce tile symmetrical dynamic range of the device. Offsets can be calculated by the simulation software before each run and corrected for in real-time if they are sm;ill. llardware problems should be suspected and invcstigatcd if the calibration procedure uncovers any sizable deviations from these characteristics.

'l'he raw stick command voltages a re sampled under software control and are usually conditioned by imposing a breakout force, scaling and limiting. The breakout force i s a n art if icial ly introduced ~.ontrol response deadband a t a n d n e a r zero a s depicted by k Min Input in figure 9. The deadband is useful for eliminating small, unintentional i npu t s rcL.g. from the hand r e s t ing on t h e s t i c k ) a n d sampling noise. Limiting sets the absolute maximum :and minimum allowable control input and is shown as +Max Output in figure 9. These critical points can be easily verified by comparison of time history data from the raw and conditioned stick signal from a s ~ ~ n u l a t i o n run with full-range control inputs.

I OUT

+ Min Input I /,IN

- -Max Ouput

A rnethod used to ensure maximum signal-to- riuise ratio on the control stick channel i s automatic gain switching. This entails simultaneously boosting the command voltage with three or more amplifiers of different gains. The software samples all of these signals a t the same time. The signal with the highest gain that is not saturated is scaled in accordance with its am lification and input to the simulation. 'I'he gain o f! each amplifier m u s t be de termined riccurately and checked f requent ly . T h i s can be accumplished easily using a frequency response analyzer. The gain circuits mus t exhibit a l inear phase so no time distortions are introduced.

E;ngine throttles in flight simulators are typically putentiometers adjusted by the throttle lever. Data must he collected on output voltage between the forward and rear lever limits to allow the correct s t a l e f a c t o r t o b e d e t e r m i n e d . T h e eng~nee r / t echn ic i an m u s t a l so ver i fy t h a t t h e throttle limits are reached before the analog-to- d lg l td converter saturates.

F l i g h t a ~ n ~ ~ i l ~ ~ t i o n s usua l ly inc lude v i s u a l displays which ,~rc, directly or indirectly affected by control input scalirig. Fur example, a fuel flow meter display is a direct function of throttle setting and must be calibrated to indicate the proper flow for any g iven t h r u t t l c lL,ver posit ion. Roll a n d p i tch commands from l he stick are typically scaled in degrees per second per pound-force. By applying w constant stick Gjrce and measuring the time ('S3fiu- ) to conlplete one. rx,volution, the actual control scalc factor can be c~ inpu ted using equation 9. This can then he verified h\, c.omparison with the theoretical scaling.

Control Scale E dc111r (&/sec/lb) = 360L - 'SJ(;~ (sec) - Force(lhs) (9)

This timing check provides an excellent end-to- end verificatiou as well because it also includes the aircraft dynamics, simulation to graphics computer communications and graphics equations.

Nurnericalihl;ither~~atical Considerations

Nun1eric:il integration of any vehicle state must be verified tu n u k e sure the integration technique is a p p r o p r i a t e , work ing p rope r ly , a n d i s b e i n g i n i t i a l i z e d a n d r e - i n i t i a l i z e d p r o p e r l y . I n s imula t ions u s i r ~ g a h igh sample r a t e , s imple trapezoidal i n t epa t ion is most likely appropriate. However, in other situations a more sophisticated technique may he required. After selecting the best technique fur a particular situatiun, the uperation of the integration software must be verified to make sure tha t the code is indeed integrating. Visual inspection of the time history da ta is not enough. Two other verification procedures are suggested. First, if a known function is input to the integrator, then the result can be compared to the analytical solution. Second, if the simulation is using a sum-of- sines a s a forcing function and there is the capability for frequency domain analys is , t hen a t ransfer function can he computed for the input and output of the integrator to verify tha t it is working properly. The transfer function should have a slope of -20 dB per decade for the gain and a conskdnt -90 degrees for phase . I t is iilso impor t an t to verify t h a t the in tegra tor is b e i l ~ g initialized and re-init ial ized properly. The ini t ial condition of the integrator must be set equal to the exact start ing value of the the vehicle. Also, if the simulation is being reset or re-initinlizeil, then i t is important to reset the initial conditions { t i ' integrators and to re-initialized the var iablds) that holds the past value(s) used in the integration equation(s).

The Eulcr ~~.arlsformatiun matrix and cquatiuns of motion should also be verified to make sure there are no terms mis s~ng . In research simulators where the task requires small perturbations about the initial vehicle state, i t is pc~ssible to have missing terms go unrlutic.eci. However, the missing terms can effect the collected data and thus the reporting of the results.

Verify the Scor111~ Algorithm

Simulators t h d t are used in research t y p ~ c a l l j incorporate .I sc,rl lng algori thm to measure pilot performance L trlfication of the scorlng algorithm 13

very Impor tnr~l Pre-calculnted patterns may be mjected Into t i J ( almula t lon to verlfy t h a t t he expected sct~rc I, c~lculatcd. It is also important t t ,

verify tha t the scoring starts and stops a t the roper t m e s The code should also be checked to veri!j t ha t a l l t empora ry v a r i a b l e s u s e d i n t h e s c o r i n g algorithm are being re-initialized properly between trials.

Conclusion

Although these procedures evolved in a research laboratory, their applicability i s not solely limited to rcsearch simulators. Some of these procedures appl tu al l real- t ime simulators, even beyond a i rcra r t simulators. The procedures discussed in this paper a r e mostly q u a n t i t a t i v e , however , s u b j e c t i v e cvaluaticm is also encouraged. Specifically, visual inspection of collected time history da ta , in the form of plots, can often reveal peculiaralities. Secondly, subjective comments from pilots can often lead to valuable investigation. Knowing tha t these set of procedures has been rigorously followed gives the researchers a h igh degree of confidence i n t h e collected data.

References

1 Levlson, W.11. ( M a y 1 9 8 2 ) . M e t h o d s fo r Identifying Pilot Dynamics. Procedings of t he Workshop on F l i g h t T e s t i n g t o Iden t i fy P i l o t Workload and Pilot Dynamics. AFFTC-TR-82-5.

2 Oppcnhcinl & Schafer (1975). Digital S ignal Prucessing, Prentice-IIall.

3. Gum, U.li. & Martin, E.A. (1987). 'l'he Flight Simulator Delay Problem. N A A paper 87-2369-CP, Montery, CA.

4 . Military Specification (1980). Flying Qualities of Piloted Airplanes. MIL-F-8785C, ASDIENESS, Wright-Patterson AFB, Dayton, Ohio.

5 Iticcio, G.E., Cress, J.U. & Johnson, W.V. (1987). The Effects of Simulator Delays on the Acquisition of Fl ight Control Skil ls: Control of H e a d i n q a n d Altitude. Procedings of the 31st Meeting of the l l uman Factors Society. New York, NY:Iiuman Facturs Society.

(3 . ,Johnson, W.V. & Middendorf, M.S. (1988) . Simulator Transpor t Delay Measuremen t us ing Steady State Techniques. AIAA paper 88-4619-CP, Atlanta, GA.

7. Levison, W.11. (1975). Techniques for U a t a Analysis a n d I n p u t WavefTrm G e n e r a t i o n for Manual Control Research. Technical Memorandum, CSD-75-2, Bolt, Beranek, and Newman. Cambridge, MA.