Dont Destroy the Archives

8/7/2019 Dont Destroy the Archives

1/10

Don't Destroy The Archives!by Michael GerzonAbstract

This report explains the value of not destroying original recordings once they have been transcribed to digital media. It is shown that there is information inthe original master recordings, whether in disc or analogue tape formats, that c

annot be recovered with present day technology which would allow a future technology to recover improved quality. By destroying original recordings, the possibility of such future improvements is permanently lost.1. INTRODUCTION1a. The problem

This report is prompted by alarming reports from a number of quarters that, after being archived in a modern digital format, original master recordings in the form of disc or analogue tape are being destroyed. What this article seeks to show is that there is in the original recording information which cannot be recovered with present day technology that will, at a future time with much more powerful signal recovery, storage and processing technology than is presently availabl

e, be usable to get much improved quality from the original recording than is currently possible. The very process of transcribing into a current digital formatthrows away over 99% of this information, which is thereby not available to future technologies. The conclusion is: never throw away a master recording, no matter how low quality it appears to be.1b. Reasons for destroying masters.

The reason why master recordings are destroyed is a combination of two factors:1) The storage of master recordings is often expensive, both in terms of cost ofspace and manpower, and of the cost of maintaining optimum storage conditions (temperature, humidity, etc.). 2) There is a widespread belief, which this article will show to be wholly unfounded, that modern digital transcription technologyis practically perfect, so that it is wrongly believed that the digital transcr

iption is virtually as good as the original master.

In fact, as we shall see, there is every reason to believe that future technologies will be able to recover a much more accurate sound from the original mastersthan can current transcription technologies, and we shall go into some detail about what this information in the master is and how it will be used in the future.1c. An awful warning

This belief that modern transcription technology is practically perfect is not anew one, and there are past awful warnings about how historical material can belost. By way of a typical example, during the 1960's and 1970's, whenever histo

rical 78 rpm material was remastered onto analogue tape for commercial release by some major record companies, the original master parts were then destroyed, unknown at the time to the remastering engineers involved - who of course were aware of how compromised the transcription was with available technology. The result is that it is not now possible to remaster to digital from the original masters, meaning that this material has the extra drop-out, modulation noise, flutterand other losses inherent in the analogue tape technology used but not inherentin the original 78 rpm medium.

Also, as recent revelations about the chemical self-destructing properties of certain master-tape media used around 10 years ago has made clear, newer storage media cannot be guaranteed to have good archival storage properties, and this isvery true of most known digital storage media, where the risk of an unrecoverabl

e loss of digits beyond the powers of error correction is very real, especiallyfor media such as DAT that are pushing storage technology to its limits.1d. Alternatives


2/10


3/10

This alone, plus the need for very high audio-quality grade recording with a wide bandwidth around 250 kHz, makes such transcription difficult to do currently,but should become possible in time.

In practice, algorithms to remove wow and flutter will need to be more complicated, since the phase shift on the recorded AC bias is also dependent on the recor

ded signal level, and means of compensating for this effect will also need to beincorporated. Additionally, stratagems to cope with temporary drop-outs or lossof the AC bias signal, to keep the playback speed steady when phase-lock is lost, will have to be incorporated, since it cannot be expected that there will beno drop-outs in the recovery of the AC bias signal.2.2 Print-through .

Another problem plaguing analogue tape is print-through, i.e. magnetisation of alayer of tape by adjacent layers during storage. Again surprisingly, there aremethods of greatly reducing such print-through, which work only if the originaltape is available, in this case making use of the magnetic properties of the tape. It is generally found that most print-through is due to the existence in the

tape coating of a low-coercivity population of magnetic particles among the generally much higher coercivity particles that make up most of the coating. This low-coercivity population is much more easily magnetised (hence its importance incausing print-through), but is also much more easily erased.

The basic principle of using a weak erase current through the erase head to erase the low-coercivity print-though particles without much affecting the high-coercivity particles has long been known, but this has generally not been used because if the erase current has enough effect to markedly reduce print-through, it also has enough effect to selectively erase wanted high frequencies on the tape,thereby damaging the wanted recording through loss of high frequencies. Moreover, this damage is irreversible, so that if done wrong, it can never be undone.

However, there is a greatly improved version of the erasure procedure that has very much lower risk of damaging the wanted recording. This improvement was originally discovered in the 1950's at the BBC Research Department, but was generallyforgotten until independently rediscovered by the writer.

This is based on an understanding of what parts of the magnetised tape layer aremost affected by print-through and by erasure. An erase head produces the strongest erasure effect at that "near" side of the magnetic layer nearest the head,with the erasure effect diminishing on the "far" side of the magnetic layer. High frequencies (i.e. short wavelengths) are, contrary to popular myth, recorded throughout the depth of the magnetic layer, but the positive and negative cyclesof the high frequency components at some distance from the playback surface tendto cancel out, meaning that high frequencies are played back mainly from the surface nearest the playback head. Therefore an erase head on the same side of thetape will tend selectively to erase signals at the near surface, and so will cause a loss of high frequencies on playback. However, placing an erase head on the opposite side of the tape to erase print-through will have least effect at theplayback surface, and so cause least loss of high frequencies.

A print-through erasure based on a weak erase current through an erase head on the "wrong" side of the tape will therefore be much more effective at erasing print-through while leaving the wanted signal unaffected. Nevertheless, experimentswith "unimportant" recordings on similar tape stock is advisable before riskingdamaging master recordings. There is also the danger that print-through erasuremay damage information contained in the ultrasonic recorded bias signal (see se

ctions 2.1 and 2.6).

Sections 2.4 and 2.5 mention other methods of reducing print-through from a mast


4/10

er tape by future playback technology that will not involve the risks of erasure.2.3 Track splitting.

Conventional analogue tape playback uses a single head to read the whole width of a tape track, but this averages information across the width of the tape track, thereby losing potentially useful information obtainable from looking at the w

ay the magnetisation varies across the width of the track.

For archiving purposes, ideally this information should be retained, and this ispossible by using a multitrack head to subdivide the original track into a large number of subtracks. Even splitting a track into two halves gives a worthwhileimprovement in the recovered information, although ideally splitting a track into 10 or 30 or 100 subtracks would be much better. One of the practical problemshere is designing a tape head that splits the track without losing significantamounts of information between the subtracks (i.e. making sure that the subtracks are truly contiguous) and also of ensuring that each subtrack individually isrecovered with good signal to noise ratio (S/N). Improvements in playback head technology are making such a goal closer. The availability of the individual subt

racks gives a profile of the magnetisation across the width of the tape track which may be used to analyse and to compensate for irregularities in this profile.

The main technical problems with the use of track splittings currently are (i) the inadequate performance for this application of multitrack heads, (ii) the amount of extra information that needs to be recorded - an increase of data rate bya factor of up to 50 or 100, (iii) the formidable amounts of digital signal processing required to process this large amount of data. We expect the future to solve all three problems.

We give some examples of improvements possible by such width profile analysis:2.3.1 Dynamic azimuth correction

Phase differences between the subtracks are evidence of azimuth errors, which can thereby be measured and compensated by digital time correction of the individual subtracks before adding them to get the original sound. Additionally, it is possible to compensate for departures from a straight line profile for the original record head (e.g. due to head wear or contamination). Such azimuth correctioncan be dynamic and fast in operation, so can compensate for example for the common problem of rapid periodic azimuth variations due for example to warping of the tape due to pressure on the tape spool.2.3.2 Dynamic drop-out correction

Tape dropouts in general will not be uniform across the width of a track, and bycomparing levels on the subtracks, and finding that subtrack with the highest wanted signal level and the relative gains of the other subtracks, it is possibleto determine (if desired even as a function of frequency) the degree of gain loss individually on each of the subtracks. In the absence of dropout, the optimumsound would simply be the sum of the subtracks, but a weighted sum (using a so-called "Weiner filtering" weighting), with overall gain determined by the levelfrom the highest subtrack level, will allow not only compensation for dropout, but the best possible S/N in the recovered signal on a moment by moment basis.2.3.3 Dynamic lateral head adjustment

If many subtracks are used, the process described in the last paragraph also hasthe automatic effect of dynamically compensating for misadjustments of track placement across the width of the tape, thereby ensuring optimum adjustment of theeffective positioning of the playback head on a moment by moment basis.

2.3.4 Crosstalk reduction

Moreover, by appropriate "deconvolution" of the magnetic profile across the widt


5/10

h of the tape, such effects as "bleed" between adjacent tape tracks can be reduced, thereby reducing crosstalk both between the tracks and between a track and the unwanted noise from guardbands between the tracks. In some cases, such guardbands may contain spurious signals due to imperfect tape erasure or to magnetisation of the tape before recording.2.3.5 Modulation noise reduction

A particular benefit of the track splitting approach is that the difference signal between subtracks contains information about the tape noise without the signal. This allows recovery not only of the steady background noise spectrum from the track differences, but also of modulation noise, i.e. variations of noise withthe wanted signal.

One of the problems with existing noise removal systems such as CEDAR or NO-NOISE is that they rely for effective operation on reasonable estimates of the noisespectrum, and it is currently not possible to estimate this noise spectrum on adynamic basis to optimally remove modulation noise effects. However, using differences between subtracks, the modulation noise spectrum can be determined dynamically on a moment-by-moment basis, thereby providing improved information for n

oise reduction processes. The subjective effect of modulation noise is one of the most serious problems with analogue magnetic tape, and this improvement will help reduce the problem.2.4 Double-sided playback

In section 2.2 above, we suggested the use of an erase head on the opposite sideof the tape from the playback head. It is also possible to recover additional information from a master tape by using a separate playback head on both sides ofthe tape. A playback head on the "wrong" side of the tape is of course spaced away from the wanted tape layer, causing a severe loss of high frequencies, but future head technology may reduce the resulting h.f. S/N penalties, at least up to middle audio frequencies.

The benefit of such double-sided playback, if the two head outputs are synchronised in time (this itself may require the use of DSP) is that the two heads are respectively more sensitive to magnetisation on their side of the thickness of the magnetic layer, thereby giving some indication of the profile of magnetisationacross the thickness of the layer. This helps distinguish the wanted signal (which has strongest magnetisation near the surface) and print-through from the back of the tape, and an appropriate linear combination of the two head outputs cancancel out this component of print-through.2.5 Reading non-transverse magnetisation

Conventional tape playback heads only "read" magnetisation in one direction relative to the motion of the tape, essentially a transversal component along tape surface. However, if in addition, magnetisation perpendicular to the tape surfaceis read, this provides additional means of separating the wanted signal from spurious signals, and also of separating out various modulation noise and distortion components. In particular, it is known that print-through from each side of the tape has a distinctive magnetisation angle relative to the tape surface, andagain this can be used to reduce print-through by taking appropriate linear combinations of the transversal and the perpendicular magnetisation component. The required head technologies may, for example, be based on the Hall effect..2.6 Analysing the bias signal

The AC bias signal recorded on the tape (see section 2.1) may contain extra information allowing playback with reduced distortion. The recorded AC bias frequency, and its harmonics, will be modulated, both in amplitude and phase by informat

ion related to the original signal. With further understanding of the distortionmechanisms in AC biasing, it may prove possible to use this additional information, along with nonlinear signal processing, to recover a less distorted version


6/10

of the wanted signal than can be recovered by direct playback on its own. As insection 2.1, this requires a much wider bandwidth than the normal audio band.

Additionally, aliasing distortion due to sidebands of the modulated bias frequency overlapping the audio baseband, are a significant cause of quality loss in recordings using bias frequencies below 100 kHz, and recovery of the bias signal may allow such aliasing to be computed and removed.

2.7 Combined methods

The above methods can be combined, although the practical problems of devising adouble-sided head divided into say 100 subtracks, each responsive to transversal and perpendicular magnetisation, responding up to say 500 kHz with audio-gradequality and good S/N are formidable by present-day standards!

Even if these technical problems are solved, the data rate involved will be thousands of times larger than for conventional digital audio, requiring the use ofrecorders with data rates comparable to or exceeding that of a digital HDTV recorder for data storage. The digital signal processing involved to use this information effectively is also formidable - many thousands of times more complex than

what is feasible economically at the moment.

At present, only very partial implementations are feasible, although we urge atleast the splitting of tracks into two subtracks, especially for mono material,so that the difference signal (or, more technically, the smaller of the principal eigenvalues of the 2-channel spectral matrix) can be used to determine the spectrum of the noise signal, and so that dynamic azimuth correction becomes possible.3. GROOVED RECORDS

The problem of recovering all information from recordings mastered, for example,in a 78 rpm grooved format is in its own way as formidable as the tape case above, although as we shall note, some past signal recovery technologies have alrea

dy used information thrown away by digital transcription!

We here consider a number of aspects that are difficult or impossible to recoverfrom a digital transcription.3.1 Impulse noise

78 rpm records typically have serious problems from what is often termed "scratch" noise, which is a noise consisting of a large number of added impulses. The first process to detect and suppress such impulses was implemented at EMI in thelate 1940's, and various analogue processes were subsequently developed by the present writer with Peter Craven in the early 1970's and commercially by Packburn, and currently, digital processes using more sophisticated predictor-type impulse detection and interpolation type replacement have been devised by CEDAR and by NO-NOISE.3.1a Ultrasonic components

However, such processes inevitably damage the integrity of the original signal,resulting in a less clean sound, and it is an important aim to minimise such damage. In separating the impulses from the music signal, it is found important topreserve a very wide bandwidth, preferably more than 40 kHz, since the durationof an impulse is inversely proportional to the bandwidth, so that short impulseswill only occur for wide bandwidths. Also, there is a relative lack of music-related signal energy above say 15 kHz, so that the location of impulses can muchmore easily be determined in the ultrasonic region above 15 kHz - indeed the EMIprocess in the 1940's remarkably used the ultrasonic components to localise the

impulses in time.

However, current digital recording technology, being bandlimited to 20 kHz, thro


7/10

ws away this ultrasonic information and smears out the duration of impulses, giving a signal in which separating out the music from the impulses is much harderto do well. Ideally, a bandwidth in the 50 to 100 kHz region would capture information for removing impulses much better.3.1b Stereo pick-up

A second piece of information that allows improved separation of impulses from t

he wanted signal, used both by the author's early work with Craven, and by Packburn, used the fact that record grooves have two walls, and the signals for eachwall can be separately recovered using a stereo pick-up. It is generally found that most impulses occur either on one groove wall or the other, so that on monorecords. the impulse alone can be recovered by taking the difference of the twowall signals. In practice, this "vertical" difference signal is contaminated byvarious distortions, but the use of two channels of recovered information nevertheless allows much more reliable detection of impulses.

Additionally, the fact that most impulses occur on only one groove wall means that it is possible to detect which groove wall and to switch recovery of the wanted signal to the other groove wall - a technique known as groove-wall switching.

There are various problems with this technique, ranging from residual cross-talk of impulse noise to the effects of stylus tracing distortion on the individualgroove wall signals.3.1c Combined information

Nevertheless, if from a mono 78 rpm record one can recover both groove walls separately with a bandwidth of 50 kHz or more, the resulting information can be processed much more reliably to remove impulse noise with minimal distortion of thewanted signal. This relies on using specialised digital recording technology, as the standard stereo digital formats have inadequate bandwidth. The DSP processing to make optimum use of the extra information has not yet been developed, butcould be near-future technology were there to be a demand for it (e.g. by a major record company commissioning the development of such a system from one of the

existing specialist suppliers such as CEDAR or NO-NOISE).3.2 Tracing Distortion

The ideal playback stylus would recover information from a point or line contactwith each groove wall, but actual styli have a finite radius of contact in thedirection of travel of the groove. This radius of contact causes what is termed"tracing distortion", whose theory was well developed by Shiga, Cooper and others in the 1960's. As first noted in 1975 by the writer, tracing distortion has the effect not only of adding nonlinear distortion to the wanted signal, but it also has the effect of prolonging the duration of unwanted noise impulses, therebyincreasing impulse noise. Therefore, optimum recovery of the signal to reduce impulse noise requires the use of a stylus with as small a contact minor radius as possible.

However, there is a practical limit to the size of minor radius that can be usedwith feasible playback styli, and electronic correction of tracing distortion,as in Cooper's elegant "skew sampling" technology of the 1960's, is the only feasible way of reducing tracing distortion further. However, such correction technology itself imposes three demanding requirements on information recovery: 1) that the two groove walls be recovered separately, since each has to be processedindependently to reduce tracing distortion, (2) that the recovered bandwidth bemuch wider than the audio bandwidth, as well as being phase-linear, since bandlimiting before tracing distortion correction itself introduces errors, and 3) That an accurate record be taken not only of the stylus radius, but also of the groove velocity at each point in the playback, since without a knowledge of both th

ese parameters, tracing distortion correction cannot be done.

Existing digital transcription throws away all this information, making both tra


8/10

cing distortion reduction and proper impulse length reduction impossible. Futuretechnology will allow both proper recovery and storage of this information andsignal processing to recover a clean tracing-distortion corrected signal of widebandwidth for each groove wall. Such signals form a better basis for the impulse-reduction processing methods described in section 3.1 above than the "raw" outputs of a pick-up cartridge.3.3 Groove wall profile

There is far more information that in principle can be recovered from an original disc or metal parts. Ideally, one would aim not simply to recover a kind of "average" of the signal on each groove wall over the contact area of a stylus, butto record separately the signal at each different height up the groove wall, soas to recover the complete profile of the cross section of the groove at each point along the groove.

Such a process divides the signal on each groove wall into a large number of parallel "subtracks", one for each different height, analogous to those suggested above for tape playback. Analysis of the differences between these subtracks canbe used not only to analyse noise as in section 2.3.5, but also to analyse at wh

ich heights the effects of record wear and noises from the "land" or the groovebottom are likely to be least serious. By this means, one would be able in effect to vary stylus height and contact profile adaptively by signal processing to optimise playback moment by moment.

As in the tape track splitting case, both the storage and subsequent signal processing demands of a groove wall profile approach are extremely high, and generally this technology is still in the future.

The actual playback technology required also does not yet exist. One might consider using an optical playback technology, but this has numerous problems both due to the size of the wavelength of light (too large!) to the fact that optical playback, unlike mechanical styli, does not push unwanted contamination out of th

e way. Probably the best way to recover profile information is to play back witha number of styli with different sizes, and then to use DSP to synchronise therecordings, to remove tracing distortion from each, and then to process the signals to recover a wall profile.

One of the uses of groove profile information is to correct for different effective angles of cutting stylus rake, including dynamic correction similar to the dynamic azimuth correction described above for tape in section 2.3.1.3.4 Other parameters

Besides the obvious geometric parameters for the groove surface, both metal parts and actual records have numerous other mechanical, chemical and physical parameters such as stress, elasticity, coefficient of friction and so forth. Each ofthese may provide additional information allowing deduction of distortions in the reproduced sound and permitting correction of the distortions. It is difficultto predict what parameters may be found useful in future, but they can clearlyonly be recovered from the original records or masters.

Aspects of elasticity may be recovered as signals by tracking the same recordingwith the same stylus at different stylus pressures, and synchronising the different recordings digitally. The difference signal between the recordings will (apart from noise signals due to contamination) contain information about the physical properties of the record.3.5 Other information

In addition to measuring information about the groove walls and the playback velocity, other information that may allow improved signal recovery includes: 1) playback of the "land" between the grooves, since this may correlate with noises i


9/10

n the groove itself 2) playback of the bottom of the groove, for similar reasons, and 3) measurement of the precise physical relationship between adjacent grooves (including distance and timing relationships), both to help reduce pre- and post-echo effects (the disc equivalent of print-though) and to detect periodic disturbances that may be filtered out by appropriate long-term comb-filter averaging.3.6 Information content

The information recoverable from original disc masters involves data rates of the order of 50 times greater than that of a conventional digital audio channel, due to the extra audio bandwidth, use of stereo channels, and the use of extra subchannels to record groove profile and (where relevant) elasticity information.As in the analogue tape case, this requires more powerful recording media (herea digital video recorder would have an adequate data rate), much more DSP powerthan is currently used to process the data to recover a signal, and finally theuse of multiple playback of the source master with careful measurement of all relevant physical parameters.

While not technologically as extreme as the requirements for the analogue tape c

ase, it will be not less than several more years before the appropriate technology is fully developed, and there is always the possibility of new unexpected data from section 3.4 above that may require new technology to be developed to improve wanted-signal recovery further.4. COPIES FROM MASTERS

While the above processing possibilities are most apt to masters of the recordings, similar techniques could be applied to first generation copies when these are all that are available. While the loss of information in the original copyingprocess cannot be undone, at least one will in future be able to reduce the effect of imperfections in the medium onto which the copy is made. The improvementsthat this will give will still be very worthwhile from a quality viewpoint.

This applies to copies from the master, whether on analogue tape or in the formof parts or discs cut from an original master tape. Often, the first release ofan LP, especially in the 'home' country of the recording, were cut from master tapes, so that the original parts or mint unplayed copies of such releases shouldbe considered an archive resource.

In many cases, either master tapes have already been lost or mislaid or they have significantly deteriorated - for example acquiring drop-outs. In this case, direct transcriptions made early on onto disc may be the best available source.

It should also be noted that poor or incorrect tape box labelling means that often one cannot be sure that tapes labelled as masters or copies are what they claim to be - it is not unknown for a so-called "copy" to be a master - and only attentive listening and investigations can decide the issue.

For these reasons, the greatest care should be taken to avoid disposing of "copytapes" until one is absolutely sure that they do not provide a useful access tothe original recording.5. DIGITAL CONVERTER IMPERFECTIONS

Besides the problems of recovering all relevant information from the original master, there is also the problem of imperfections in the transcription medium. The naivete of the early days of digital audio when it was thought to be essentially "perfect" has recently been replaced by an understanding of many of the mechanisms by which the ears hear faults that, according to traditional audio measure

ments were negligibly small. This work, based on the researches of Louis Fielderat Dolby Labs and Bob Stuart of Meridian Audio in modelling of masking in auditory perception, shows that there are still significant audible faults especially


10/10

in available analogue-to-digital converters (ADCs). This should not be a surprise, since such faults are heard not only by many audio professionals, but even by many lay listeners who hear a distinctive loss in digital transcriptions.

One of the known sources of audible faults is "jitter", i.e. tiny variations inthe timing of the digits. All digital recording media and signal interconnects introduce audibly significant amounts of jitter, and most DACs currently do not h

ave adequate de-jittering, resulting in each digital player having its own "sound". However, the problems of designing DACs to remove jitter have recently beensolved, and the choice of digital transcription medium itself should no longer be a serious quality problem provided that there is no loss of corrected digits.

However, jitter in the original ADC cannot subsequently be removed, and neithercan faults due to non-linearities, modulation noise and limit cycles in the ADC.Currently, the quality of most ADCs used to transcribe material in digital formstill leave a great deal to be desired, and these faults are clearly audible even on old archive disc and tape material. A great deal of listening is still required before the ADCs used are selected, and they should also be used in a way that minimises jitter effects in the conversion process itself.

6. CONCLUSIONS This report has shown that current digital transcription technology cannot yet recover most of the information in master disc or analogue tape recordings, and that future technologies will allow the recovery of extra information from the original master that cannot be recovered from a digital transcription. We have described improvements that we expect will become possible with future transcription technologies, although some of these may still be some time away due to limitations in current technologies.

It was also noted that even the sound quality of current transcription technology still leaves quite a lot to be desired.

It is therefore recommended in the strongest terms that original master recordings, or the closest available copies to these, should be preserved, since the dig

ital transcriptions are no substitute for the potential quality recoverable in the future from the masters. Some of these quality gains may be very substantial- e.g. virtual removal of print-through, modulation noise and wow and flutter from analogue master tapes.

Insofar as a digital transcription is required for archive purposes (e.g. for safety back-up or because of physical deterioration of the master), it is recommended that each mono track of the original be split into two subtracks as described above for disc (two groove walls) or tape (dividing the playback track into two), so that the extra information can be used by future signal processing shouldthe original master be lost. It is also recommended that such safety copies should use the best available ADCs in their preparation, since currently, they area quality bottleneck.

Documents

Dont Destroy the Archives