Überblick und Ausblick - Vorletzte Vorlesung Statistik · 2020-01-28 · Multivariate Statistik Spezielle Daten Theorie Sortiert nach dem Inhalt der Vorlesung Sortiert nach Daten

RuckblickMultivariate Statistik

Spezielle DatenTheorie

Uberblick und AusblickVorletzte Vorlesung Statistik

Prof. van den Boogaart

Vorlesung Datenanalyse und Statistik

Prof. van den Boogaart Uberblick und Ausblick



Gliederung1 Ruckblick

Sortiert nach dem Inhalt der VorlesungSortiert nach Daten

2 Multivariate StatistikKovarianzmatrizenKlusteranalyseHauptkomponentenanalyseFaktorenanalyseDiskriminazanalyse

3 Spezielle DatenZeitreihenanalyseGeostatistikKompositionsdatenZusammenfassung Spezielle Daten

4 TheorieWahrscheinlichkeitstheorieMaximum Likelihood SchatzungBayes Schatzung














Sümpfe des multiplen Testens

Rangviertel

Die unwegsamen Ausreißerberge

Die Datenminen

Todeswüste, der nicht erfüllten Voraussetzungen

Steig der

Nich

tparametrik

Gle

tsch

ersp

alt

e der

glei

chen

Mes

swer

te

Nacht der angenommen

Hypothesen

Klippe

der

unüber

prü

fbare

n

Vor

auss

etzu

nge

n Klippe der unüberprüfbaren

Voraussetzungen

robuster

Weg

Momentenmethoden u.

Lineare ModelleML-CityBayes-Land

t-DorfModell-Platz

Statistika

Vertrauensbereich

Normalviertel

Vorhersagebereich

Steppe der unwesentlich verletzten Voraussetzungen

Schätzervorstadt

Schlaraffia oder das Land des gelungen statistischen Nachweis Land des offenen Betrugs

Bonferroni

Passage

Benjamini

Passage

Sequenzielle Passage

Posthoc

Mathe

Schätzung

Daten

Test

Aussichtsturm

Grafingen

Riesige Halde mit

nichtrepräsentativen

Daten





Ruckblick

Daten:

RepresentativitatDatenmatrixDatentafelSkala

Statistische Graphik

Deskriptive Statistik

Statistische Tests

Lineare Modelle





Ruckblick

Daten:


Welche Graphik fur welche Daten?Welche Graphik zeigt was?Wie interpretiert man das Ergebnis?


Statistische Tests

Lineare Modelle





Ruckblick

Daten:



Lageparameter: Mittelwert, MedianStreuungsparameter: Varianz, IRQAbhangigkeiten: KorrelationSchiefe, Verteilung

Statistische Tests

Lineare Modelle





Ruckblick

Daten:



Statistische Tests

Hypothese, Alternative, Fehler 1. und 2. Art.Nachweis, α-Niveau, p-WerteBonferroni KorrekturAuswahl der Testsparametrisch, nichtparametrisch, robust

Lineare Modelle





Ruckblick

Daten:



Statistische Tests

Lineare Modelle

Modellgleichungen: +bT , +bG , +εiVarianzanalysetabellenVoraussetzungenDiagnostik














Univariate StatistikDatenUnivariate ParameterUnivariate Graphiken

Bivariate Statistik

Multivariate Statistik

Spezielle Daten





Univariate Statistik

Bivariate StatistikCorrelation, KontingenztafelnGepaarte, Zweistichproben, Mehrstichproben-TestsBivariate GraphikANOVA, Lineare Regression


Spezielle Daten






Bivariate Statistik

Multivariate StatistikGraphik: Streudiagrammmatrix, MosaikplotLineare Modell, multivariate lineare ModelleKovarianzmatrizenKlusteranalyseHauptkomponentenanalyse, Independent Component AnalysisDiskriminazanalyse

Spezielle Daten






Bivariate Statistik


Spezielle Daten

ZeitreihenZufallsfelderKompositionsdaten. . .




KovarianzmatrizenKlusteranalyseHauptkomponentenanalyseFaktorenanalyseDiskriminazanalyse










Kovarianzmatrizen

var(X1) cov(X1,X2) cov(X1,X3) cov(X1,X4)

cov(X2,X1) var(X2) cov(X2,X3) cov(X2,X4)cov(X3,X1) cov(X3,X2) var(X3) cov(X3,X4)cov(X4,X1) cov(X4,X2) cov(X4,X3) var(X4)














Klusteranalyse I

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

0.0 0.5 1.0 1.5

−0.

50.

00.

51.

0

X

Y





Klusteranalyse II

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

−0.5 0.0 0.5 1.0 1.5

0.0

0.5

1.0

X

Y














Hauptkomponentenanalyse I

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

−5 0 5

−4

−2

02

4

X

Y





Hauptkomponentenanalyse II

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

−5 0 5

−4

−2

02

46

X

Y





Hauptkomponentenanalyse III

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

−5 0 5

−4

−2

02

46

X

Y





Hauptkomponentenanalyse IV

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

−5 0 5

−4

−2

02

46

X

Y






●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

−5

0

5

−4−2

02

46

X

Y






●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

● ●●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

−5

0

5

−4

−2

0

2

4

6

X

Y






●

●

● ●

●

●

●

●

●●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●●●

●

−5

0

5

−4

−2

0

2

4

6

X

Y














Faktorenanalyse

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

−2 −1 0 1 2 3

−3

−1

12

3

Z

X

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●●●

●

●

●

●

●●●

●

● ●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

−2 −1 0 1 2 3

−4

02

46

ZY

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

−4 −2 0 2 4 6

−3

−1

12

3

Y

X














Diskriminazanalyse

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

−0.5 0.0 0.5 1.0 1.5

0.0

0.5

1.0

X

Y

●?





Diskriminazanalyse

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

0.0 0.5 1.0

0.0

0.5

1.0

1.5

X

Y

●





Diskriminazanalyse

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

0.0 0.5 1.0

0.0

0.5

1.0

1.5

X

Y

●●





Zusammenfassung MV-Statistik

Die Klusteranalyse versucht Gruppen in den Daten zu finden.

Die Hauptkomponentenanalyse versucht die Hauptrichtungder Streuung im Datensatz aufzufinden.

Die Faktorenanalyse versucht unbeobachtbare gemeinsameUrsachen in den Daten zu entdecken.

Die Diskriminazanalyse versucht Individuen einer Gruppezuzuordnen.




ZeitreihenanalyseGeostatistikKompositionsdatenZusammenfassung Spezielle Daten










Zeitreihen

●

●●

●●

●

●●

●

●●

●

●●

●

●●

●

●●

●

●●●

●

●

●●

●●

●

●

●●

●

●●

●●

●●

●●

●●

●●●

●●

●●●●

●●

●

●●●

●●

●

●●

●

●

●

●

●●●

●●●

●●●

●

●

●●

●

●●

●●

●

●

●●●●

●●

●●

●●

●

0 20 40 60 80 100

0.00

0.05

0.10

0.15

Tage

z





Eigenschaften von Zeitreihen

Daten werden in regelmaßigen Zeitabstanden erhoben

In kurzer Zeit andert sich nicht so viel

Daten sind also stochastisch abhangig

Daten verhalten sich oft zyklisch (z.B. immer große Werte imSommer)
































Methoden fur Zeitreihen

Test ob Abhangigkeit tatsachlich vorliegt (z.B.Ansari-Friedmann)

Abhangigkeit quantifizieren durch Autokovarianzfunktion:

c(h) = cov(Z (t + h),Z (t))

Vorhersage: Wie geht es weiter?

Welche Gesetze verbergen sich hinter der Zeitreihe?








c(h) = cov(Z (t + h),Z (t))










c(h) = cov(Z (t + h),Z (t))










c(h) = cov(Z (t + h),Z (t))
















Was ist eine geostatistische Vorhersage?

10 20 30 40 50

1020

3040

50

An unkown reality

x

y

Z () =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

Z (s) = grade at location sProf. van den Boogaart Uberblick und Ausblick





10 20 30 40 50

1020

3040

50

Observation Locations

x

y

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

Z () =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

si = Observation locationProf. van den Boogaart Uberblick und Ausblick





10 20 30 40 50

1020

3040

50

Observation Locations

x

y

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

Z (s) =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

Z (si ) = True value at location siProf. van den Boogaart Uberblick und Ausblick





10 20 30 40 50

1020

3040

50

What do we expect here?

x

y

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

●

?

Z (s) =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

Z (si ) = Infered value at location siProf. van den Boogaart Uberblick und Ausblick





10 20 30 40 50

1020

3040

50


x

y

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

●

?

Z (s) =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

wi (s) = Kriging weightsProf. van den Boogaart Uberblick und Ausblick





10 20 30 40 50

1020

3040

50


x

y

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

●

?

10 20 30 40 50

1020

3040

50

What in this block?

x

y A

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ●

Z (A) =

∫Af( n∑

i=1

wi (s)Z (si ), i = 1, . . . , n)ds

Z (A) = Average value in Block BProf. van den Boogaart Uberblick und Ausblick




What ist geostatististische Simulation?

10 20 30 40 50

1020

3040

50

An unkown reality

x

y

reality is unkown

prediction is tosmooth

predict conditionaldistribution

simulating possibleszenarios

... many possibleszenaries

are qualitativelysimilar

*but qualitativelydifferent from reality






10 20 30 40 50

1020

3040

50

Prediction

x

y

reality is unkown












10 20 30 40 50

1020

3040

50

Prediction

x

y

reality is unkown












10 20 30 40 50

1020

3040

50

Simulation 1

x

y

reality is unkown












10 20 30 40 50

1020

3040

50

Simulation 2

x

y

reality is unkown












10 20 30 40 50

1020

3040

50

Simulation 3

x

y

reality is unkown












10 20 30 40 50

1020

3040

50

Simulation 4

x

y

reality is unkown




















Kompositionsdaten

NA20.K20

45 50 55 60

● ●

●●

●

●

●●

●●

●

●●

● ●●

●●

●●●

●

●

2030

4050

●●

●●

●

●

● ●

●●●

●●

●●●

●●

●● ●

●

●

4550

5560

●

●

●●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●

●

●

Fe2O3

●

●

●●

●

●

●

●

●●●

●●

●●

●

●

●

●●

●

●

●

20 30 40 50

●●

●●

●●

●

●●

●●

●●●●●●

●

●●

●

●

●

●●

●●

●●

●

●●● ●

● ●●

● ● ●

●

● ●

●

●

●

5 10 15 20 25 30 35 40

510

2030

40

MgO





Kompositionsdaten

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

40 45 50 55

4550

5560

NA20.K20 + MgO

Fe2

O3





Kompositionsdaten

Komponenten addieren zu 100%

Komponenten sind also abhangig und automatisch negativkorreliert.

Komponenten sind einzeln Anteile, aber hangen zusammen.





Kompositionsdaten








Kompositionsdaten








Ternare Diagramme

NA20.K20 Fe2O3

MgO

●● ●●

● ●

●

● ●●●

●●●● ● ●

●

●●

●

●

●














Zusammenfassung SD

ZeitreihenanalyseDaten mit zeitlichen Abhangigkeiten

GeostatistikDaten mit raumlichen Abhangigkeiten

KompositionsdatenWerte addieren zu 1 bzw. Summe egal.




WahrscheinlichkeitstheorieMaximum Likelihood SchatzungBayes Schatzung




























Sümpfe des multiplen Testens

Rangviertel

Die unwegsamen Ausreißerberge

Die Datenminen

Todeswüste, der nicht erfüllten Voraussetzungen

Steig der

Nich

tparametrik

Gle

tsch

ersp

alt

e der

glei

chen

Mes

swer

te

Nacht der angenommen

Hypothesen

Klippe

der

unüber

prü

fbare

n

Vor

auss

etzu

nge

n Klippe der unüberprüfbaren

Voraussetzungen

robuster

Weg

Momentenmethoden u.

Lineare ModelleML-CityBayes-Land

t-DorfModell-Platz

Statistika

Vertrauensbereich

Normalviertel

Vorhersagebereich

Steppe der unwesentlich verletzten Voraussetzungen

Schätzervorstadt

Schlaraffia oder das Land des gelungen statistischen Nachweis Land des offenen Betrugs

Bonferroni

Passage

Benjamini

Passage

Sequenzielle Passage

Posthoc

Mathe

Schätzung

Daten

Test

Aussichtsturm

Grafingen

Riesige Halde mit

nichtrepräsentativen

Daten





Die letzte Frage





Ich wunsche Ihnen

eine bestandene Klausur

schone Ferien!

ein erfolgreiches Leben





Ich wunsche Ihnen


schone Ferien!






Ich wunsche Ihnen


schone Ferien!



Documents

Überblick und Ausblick - Vorletzte Vorlesung Statistik · 2020-01-28 · Multivariate Statistik Spezielle Daten Theorie Sortiert nach dem Inhalt der Vorlesung Sortiert nach Daten