Introduction to Signal Processing

1 Introduction

In this article, we discuss about signals, systems and transforms. Signals canbe considered as functions. Audio signal is a one dimensional function of timewhereas an image is a two dimensional function that varies in space. Systemsmanipulate signals, i.e they can be considered as a map from functions to func-tions. Transforms make signal processing easier by changing the domain inwhich the underlying signal is represented.

2 Signals

Signals can be functions of time or space. For processing signals it is advanta-geous to represent them in in frequency domain. Several operations such as fil-tering out unnecessary frequencies can be performed efficiently in the frequencydomain. Generally low frequency components represents the overall shape ofthe signal and high frequency components are caused due to noise or presenceof edges in an image. To remove noise, high frequency components are reduced.To sharpen an image, high frequency components are emphasized. Thus it iseasier to carry many operations in the frequency domain.

2.1 From time domain to frequency domain

Fourier discovered that any periodic signal can be represented as combinationof sinusoids of different amplitudes, frequencies and phases. Any periodic sig-nal s(t) with period T0 (thus having fundamental frequency f0 = 1

T0can be

represented as follows

s(t) =∞∑

n=0

ansin(2πnf0t + φ) (1)

1

A phase shifted sine wave can be expressed in terms of sine and cosinefunctions due to the following identity.

sin(A + φ) = sin(A)cos(φ) + sin(φ)cos(A)

Thus we can express s(t) as sum of sines and cosines of different amplitudesand frequencies. Equation 1 becomes

s(t) =∞∑

n=0

ancos(2πnf0t) + bnsin(2πnf0t) (2)

which is also written as

s(t) = a0 +∞∑

n=1

ancos(2πnft) + bnsin(2πnft)

If the function has non zero value for f(0), it indicates the presence of cosinecomponents. a0 represents the average value of the function over a time period.This is because the average value of sines and cosines over a time period is zero.

Using the identities cos(t) = eit+e−it

2 and sin(t) = eit−e−it

2i and letting An

denote an−ibn

2 and A∗n denote an+ibn

2 , Equation 2 can be represented by meansof a single complex exponential.

s(t) =∞∑

n=0

Anej2πnf0t + A∗nej2πnf0t

which can be written as

s(t) =∞∑

n=−∞Anej2πnf0t

The above formula is known as fourier decomposition of s(t) into complexexponentials. This decomposition is a way of transforming the signal from thetime domain to frequency domain. In the next section we will see how a signaland its decomposition into fourier series can be viewed in the vector space model.

2.2 Vector Space Model

The set of signals forms a vector space. A vector space is a collection of vectorsthat is closed under linear combination, i.e. if two vectors ~v, ~w are in the space,then the vector a~v + b~w is also in the space for scalars a and b(The conditionsays the space is closed under vector addition and scalar multiplication). A

2

familiar example of vector space is the Euclidean space R3 . Any set of objectswith operations addition and multiplication which are closed under linear com-bination forms a vector space. Under this definition polynomials, matrices andfunctions also form vector space.

We can generalize familiar concepts in the Euclidean space to the moregeneral vector space. The length or norm of vector v is given by

‖v‖ =

√√√√ n∑i=0

v2i

The distance between two vectors v1 and v2 is the norm of the differencevector v1 − v2. The angle between two vectors is closely related to the innerproduct < v1, v2 > due to the following relation.

cosθ =< v1, v2 >

‖v1‖ ∗ ‖v2‖Two vectors are said to be orthogonal if their inner product is zero.

The concepts of length, distance and angle can be defined on any vectorspace. For instance, in the space of functions, the integral

∫f(x).g(x) gen-

eralizes the dot product in which summation is replaced by integration. Thefunctions f(x) and g(x) are said to be orthogonal on [a, b] if

∫ b

af(x).g(x) = 0.

For example the set of functions { ejn2πft } n=0,1,2... are orthogonal.

A basis is a minimal set of vectors that spans the entire space. If the elementsin basis are orthogonal and have unit length , then they form orthonormal basisas formed by (i, j, k) in the euclidean space. A point in the euclidean space givenby u = xi + yj + zk. The values x,y and z gives the magnitude of projection onthese basis. To compute the magnitude of projection along i, we compute theinner product of u and i, thus x = u.i.

A function s(t) can be viewed as a point in the space of functions. Thefourier decomposition of the function given by

s(t) =∞∑

n=−∞Anejn2πf0t

can be viewed as projection of the function onto orthonormal basis given by thecomplex exponential functions ejn2πf0t. Finding the magnitude of projection is

3

equivalent to finding the fourier coefficients. To compute the the magnitude ofprojection along ejn2πf0t ,we compute the inner product of s(t) and ejn2πf0t.Using the definition of inner product for functions, we get the fourier coefficientsas

An =1T0

∫ T0/2

−T0/2

s(t)e−jn2πf0tdt

3 Systems

Systems maps signals to signals. To characterize a system we need to find itsresponse for each and every input function, which is infeasible. Fortunately sys-tems that are linear and time invariant (LTI) can be characterized if its responsefor a small set of inputs is known. In a linear system, the principle of superpo-sition holds - if x1 and x2 produces responses y1 and y2 respectively, then inputax1 + bx2 produces the response ay1 + by2. A system is said to be time invari-ant if a delay in input by t seconds causes the output to be delayed by t seconds.

In the following sections, we will see that to characterize an LTI system it issufficient if we know its response for the impulse function and that the outputof an LTI system is just the convolution of the input with the impulse response.

3.1 Impulse Response

Consider a signal that lasts for w seconds and has amplitude 1/w. In the limit-ing case as w → 0, the signal is called impulse. LTI systems have a remarkableproperty that they can be completely characterized by its response to the im-pulse function.

@figureThe input signal is divided into smaller ones, each short enough to act as an

impulse to the system. In other words, the input signal is decomposed into aninfinite number of scaled and shifted impulse functions. Each of these impulsesproduces a scaled and shifted version of the impulse response in the outputsignal. The final output signal is then equal to the combined effect, i.e., thesum of all of the individual responses. Thus if the response to the impulsefunction is known, the response of the input signal can be calculated. We showthat the output is the convolution of the input with the impulse response of thesystem.

4

3.2 Convolution

Convolution is a mechanism by which an input signal is shaped by another sig-nal to produce an output signal. We introduce convolution by considering theproblem of smoothing a signal. Later we show that the output of a system isthe convolution of the input with the impulse response.

Consider a transformation that smoothes the signal by averaging adjacentsignals - bn = (an + an−1)/2. These kinds of transformations are used for re-ducing effects of noise. The transformation can also be described in terms ofpolynomial multiplication. If we represent the input sequence a0, a1, a2.... bythe polynomial A(x) = a0 + a1x + a2x

2 + a3x3.... , the coefficient of bn in the

product B(x) = A(x) 12 (1 + x) is given by bn = (an + an−1)/2 which is the

output sequence. The polynomial P (x) = 12 (1 + x) defines how the input signal

is manipulated(convolved) to produce the output. Convolution and polynomialmultiplication are closely related.

For the product B(x) = A(x) ∗ P (x), the coefficient of the convolved sig-nal bn is given by

∑ni=0 ai ∗ pn−i. In the continuous domain the summation

is replaced by integration and the convolution of two function f(x) and g(x)becomes C(u) =

∫ u

0f(x) ∗ g(u− x)

We now show that the output of a system can be viewed as a convolutionof the input signal with the impulse response of the system. Consider a systemwhose output is h(t) when given the input δ(t). Any input signal can be consid-ered as summation of scaled and shifted impulse functions i.e. For a continuoussignal, s(t) can be viewed as

∫∞−∞ f(τ)δ(t − τ)dτ . Since the response of δ(t) is

h(t), the response for s(t) is∫∞−∞ f(τ)h(t − τ)dτ . Thus the convolution of the

input signal with the impulse response gives the output signal.

4 Transforms and their applications

Transforms make it easier for systems to manipulate signals. Any linear trans-formation can be described as a projection onto a new set of basis functions.

Fourier Transform

5

We have seen the fourier series for a periodic signal having frequency f0 isgiven by s(t) =

∑∞n=−∞Anejn2πf0t and that An is given by

An =1T0

∫ T02

−T02

s(t)e−jn2πf0tdt

The following figure plots the frequency components and the magnitude ofAn. Note that here the amplitudes are function of frequencies defined at discretepoints.

@figureAs the frequency f0 decreases, these amplitudes become more closely spaced.

For a non periodic signal , the time period T0 →∞ and thus f0 → 0. The sepa-ration between successive harmonics becomes small and the amplitudes becomea continuous function of frequency. Thus nf0 becomes the continuous variablef and the above formula for An becomes

Af =1T0

∫ ∞

−∞s(t)e−j2πftdt (3)

The above transformation takes a function of time as input and producesa function of frequency i.e. s(t) → Af . This is called the Fourier Transform.The transform makes it easier to perform many operations that are difficult inthe time domain. The fourier transform of the convolution of two functionss1(t) and s2(t) is equal to product of their fourier transforms. Intuitively, mul-tiplication emphasizes those frequencies that are present in both the signals.Thus for producing the convolution of two functions, it is easier to transformthem into frequency domain, multiply them and transform back to time domain.

Discrete Fourier Transform

While processing signals on a computer, variables are discrete and the con-tinuous fourier transform is approximated by Discrete Fourier Transform. Wesample a continuous function s(t) at intervals of ∆, generating a sequence ofsampled values sk = s(k∆) collecting a total of N samples (1/∆ is the samplingfrequency, which according to Nyquist theorem must be at least twice the fre-quency of the highest component present in the wave).

In the continuous fourier transform, we have the values of the signal definedat each point of time and generate a function of frequency that is defined forall values. In DFT, we have the values of the signal at discrete points tk = k∆

6

represented as s(k) and generate a function of frequency that are valid at discretefrequencies An . In the discrete case Equation 3 becomes

An =N−1∑k=0

s(k)e−2πikn/N (4)

The equation xN = 1 is of degree N and thus has N roots given by ωnN ,

(n=0,1... N and ω = e2πiN . Thus Equation 4 can also be written as

An =N−1∑k=0

s(k)ωknN (5)

The DFT can be viewed as a transformation of N point signal into N fouriercoefficients or as a matrix that maps an N-dimensional vector to N-dimensionalvector.

DFT can be used to speed up polynomial multiplication. We have alreadyseen that polynomial multiplication is related to convolution and it is easy toconvolve signals in frequency domain. We use these facts to speedup multipli-cation.

Considering the coefficients of a polynomial (of degree N) as our signal, weget N fourier coefficients. (According to Equation 5, these correspond to thevalues of the polynomial evaluated at the complex roots of unity). These valuesgive the point-value representation of the polynomial. To multiply two polyno-mials f(x) and g(x), we evaluate the polynomials at roots of unity, multiply thecorresponding values to get the point-value representation of f(x) ∗ g(x) fromwhich we can construct the polynomial f(x) ∗ g(x).

Fast Fourier Transform The fast fourier transform is a computationallyfast method to compute the DFT. Computing DFT directly for N points requiresO(N2) complex multiplication. FFT exploits the structure of DFT to reducethe complexity. Rewriting Equation 5 we get

An =N−1∑k=0

s(k)ωknN

An =∑

k is even

s(k)ωknN +

∑k is odd

s(k)ωknN

An =(N/2)−1∑

j=0

s(2j)ω2jnN +

(N/2)−1∑j=0

s(2j + 1)ω(2j+1)nN

7

Using the relation ω2N = ωN/2, we get

An =(N/2)−1∑

j=0

s(2j)ωjnN/2 + ωn

N

(N/2)−1∑j=0

s(2j + 1)ωjnN/2

Thus we have reduced the problem of computing N point DFT for s(k) tofinding two N/2 point DFTs for s(j) and s(2j +1). This is a divide and conquerapproach where we have reduced the problem of size N to two subproblems ofsize N/2. It has complexity O(NlogN)

Discrete Cosine Transform If the function is an even function, then thefourier transform consists of only cosine components and the transform becomesDiscrete Cosine Transform. Thus DCT can be defined in terms of DFT by im-posing symmetry. DCT is used for compression in JPEG images. JPEG com-pression takes into account that the human eye, cannot detect high frequencycomponents. Using a process of quantization, we retain all low frequency com-ponents. (The high frequency components are retained only if they have largecoefficients). On the resulting coefficients run length encoding and huffmancompression is applied

5 Conclusion

We have seen how a signal can be viewed as a function in the vector spacemodel of functions. Systems are used to manipulate signals and a special classof systems called LTI systems can be easily characterized by its impulse function.Transforms make it easier to manipulate signals. Transforms can be viewed aschange of basis in the vector space model. Transformations have a number ofapplications including filtering, convolution, pattern recognition, solving partialdifferential equations, multiplication of polynomials and large numbers, analysisof time series, solving system of linear equations

8