Upload
ravi-prasad-kj
View
38
Download
2
Embed Size (px)
Citation preview
REGRESSION
Meaning
• In statistics, regression analysis is a statistical process for estimating the relationships among variables.
• Regression analysis:There are several types of regression:– Linear regression– Simple linear regression– Logistic regression– Nonlinear regression– Nonparametric regression– Robust regression– Stepwise regression
Meaning: regression
Noun1.the act of going back to a previous place or state; return or reversion.2.retrogradation; retrogression.3.Biology. reversion to an earlier or less advanced
state or form or to a common or general type.4.Psychoanalysis. the reversion to a chronologically earli
er or less adapted pattern of behaviour and feeling.5.a subsidence of a disease or its manifestations:
a regression of symptoms.
INTRODUCTION
• Many engineering and scientific problems – concerned with determining a relationship between a
set of variables.• In a chemical process, might be interested in– the relationship between
• the output of the process, the temperature at which it occurs, and
• the amount of catalyst employed.
• Knowledge of such a relationship would enable us– to predict the output for various values of temperature
and amount of catalyst.
INTRODUCTION
• Situation- – there is a single response variable Y , also called the dependent variable
– depends on the value of a set of input, also called independent, variables x1, . . . , xr
• The simplest type of relationship these varibles is a linear relationship
INTRODUCTION
• If this was the linear relationship between Y and the xi, i = 1, . . . , r, then it would be possible – once the βi were learned – to exactly predict the response for any set of input
values. • In practice, such precision is almost never
attainable• The most that one can expect is that Equation
would be valid subject to random error
INTRODUCTION
Introduction
• Suppose that the responses Yi corresponding to the input values xi, i = 1, . . . , n are to be observed and used to estimate α and β in a simple linear regression model.
• To determine estimators of α and β we reason as follows: – If A is the estimator of α and B of β, then the estimator of the response
corresponding to the input variable xi would be A + Bxi.
• Since the actual response is Yi , – the squared difference is (Yi − A − Bxi)2, and so if A and B are the
estimators of α and β, – then the sum of the squared differences between the estimated responses– the actual response values—call it SS — is given by
Joke on Regression
DISTRIBUTION OF THE ESTIMATORS
• To specify the distribution of the estimators A and B, it is necessary to make additional assumptions about the random errors aside from just assuming that their mean is 0.
• The usual approach is to assume that the random errors are independent normal random variables having mean 0 and variance σ2.
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
• Remarks
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
• Notation:
DISTRIBUTION OF THE ESTIMATORS
DISTRIBUTION OF THE ESTIMATORS
EXAMPLE
• The following data relate – X: the moisture of a wet mix of a certain product– Y: the density of the finished product
• Fit a linear curve to these data. Also determine SSR.
EXAMPLE
EXAMPLE
STATISTICAL INFERENCES ABOUT THEREGRESSION PARAMETERS
• Inferences Concerning β
Inferences Concerning β
Inferences Concerning β
Inferences Concerning β
EXAMPLE• An individual claims that the fuel consumption of his
automobile does not depend on how fast the car is driven.
• To test the plausibility of this hypothesis, the car was tested at various speeds between 45 and 70 miles per hour. The miles per gallon attained at each of these speeds was determined, with the following data resulting is given.
• Do these data refute the claim that the mileage per gallon of gas is unaffected by the speedat which the car is being driven?
EXAMPLE
EXAMPLE
Inferences Concerning α
Summary of Distributional Results
Summary of Distributional Results
THE COEFFICIENT OF DETERMINATION AND THE SAMPLE CORRELATION COEFFICIENT
THE COEFFICIENT OF DETERMINATION AND THE SAMPLE CORRELATION COEFFICIENT
THE COEFFICIENT OF DETERMINATION AND THE SAMPLE CORRELATION COEFFICIENT
ANALYSIS OF RESIDUALS: ASSESSING THE MODEL
The figure shows that, as indicated both by its scatter diagram and the random nature of its standardized residuals, appears to fit the straight-line model quite well.
The figure of the residual plot shows a discernible pattern, in that the residuals appear to be first decreasing and then increasing as the input level increases. This often means that higher-order (than just linear) terms are needed to describe the relationship between the input and response. Indeed, this is also indicated by the scatter diagram in this case.
The standardized residual plot shows a pattern, in that the absolute value of the residuals, and thus their squares, appear to be increasing, as the input level increases. This often indicates that the variance of the response is not constant but, rather, increases with the input level.
TRANSFORMING TO LINEARITY
• The mean response is not a linear function • In such cases, if the form of the relationship
can be determined it is sometimespossible, by a change of variables, to transform it into a linear form.
• For instance, in certain applications it is known that W(t), the amplitude of a signal a time t after its origination, is approximately related to t by the functional form
TRANSFORMING TO LINEARITY
EXAMPLE
• The following table gives the percentages of a chemical that were used up when an experiment was run at various temperatures (in degrees celsius).
• Use it to estimate the percentage of the chemical that would be used up if the experiment were to be run at 350 degrees.
EXAMPLE• Let P(x) be the percentage of the chemical that is used up when the experiment is
run at 10x degrees.
• Even though a plot of P(x) looks roughly linear, we can improve upon the fit by considering a nonlinear relationship between x and P(x).
• Specifically, let us consider a relationship of the form : 1 − P(x) ≈ c(1 − d)x
EXAMPLE
EXAMPLE
POLYNOMIAL REGRESSION
EXAMPLE
• Fit a polynomial to the following data.
EXAMPLE
EXAMPLE
MULTIPLE LINEAR REGRESSION
MULTIPLE LINEAR REGRESSION
MULTIPLE LINEAR REGRESSION
MULTIPLE LINEAR REGRESSION
Thank you