Lexical and Co-occurrence Evidence for Subject Vocabulary Reconciliation

in ADS Databases

Jonghoon Lee, University of IllinoisMichael Kurtz, Harvard-Smithsonian Center for

AstrophysicsDavid Dubin, University of Illinois

PRESENTATOIN OUTLINE

• INTRODUCTION

• HETEROGENEOUS SUBJECT INDEXING

• LEXICAL MATCHING APPROACH

• SPREADING ACTIVATION MODEL

• EXPERIMENT

• PRELIMINARY RESULTS

• FUTURE STUDY

INTRODUCTION

NASA ADS DATABASES• Many Astronomers use ADS Abstract Service

• Search options: title, author, object name, text..

• No keyword-only searching capability

• Subject descriptors (keywords): central concepts

Problems• ADS uses several different indexing languages (STI, ApJ, etc.)

• Searching one form of a concept will miss documents

(M31 vs. Andromeda)

• Ways to merge or reconcile these inconsistent indexing languages

Heterogeneous Subject Indexing • Subject descriptors

- controlled vocabulary (keywords/index terms)

- standardized labels of concepts

- central & important concepts in a document

- improves information retrieval

• Indexing inconsistency

- different indexing schemes applied

- term-specificity, precoordination, ...

- different terms used to index the same concept

- journal-specific indexing vocabulary

Heterogeneous Subject Indexing

• Inconsistency of Indexing in ADS - STI (Scientific and Technical Information) index terms

from the NASA Thesaurus (1975 to 1995)

- Journal-specific subject descriptors

a variety of indexing vocabulary since 1995

i.e. author-assigned keywords in ApJ

• Examples [STI] [ApJ]

LOW MASS STARS STARS: LOW-MASS

DWARF NOVAE STARS: DWARF NOVAE

ANDROMEDA GALAXY GALAXIES:INDIVIDUAL MESSIER NUMBER: M31

ELECTRIC FIELDS ???

??? HIGH-LATITUDE OBJECTS

COSMOLOGY COSMOLOGY COSMOLOGY: EARLY UNIVERSE COSMOLOGY: THEORY COSMOLOGY: OBSERVATIONS

VOCABULARY MERGING

• How to reconcile different controlled vocabularies

• Vocabulary Switching System based upon Term Mapping

• Issues

- how to identify term relationship

- automatic vs. manual

• Sources of Evidence

- lexical resemblance: spelling variants

- syndetic structure: thesaurus

- co-occurrence data: consistent assignment of descriptors to the same document

LEXICAL MATCHING APPROACH

• Clustering Method based on Lexical Similarity• Term Signature: galaxies: individual: m3131galaxym

• Can merge terms like the followinggalaxies: individual (m 31) % 1galaxies: individual(m 31) % 1galaxies: m 31 % 5galaxies: individual: m 31 % 9

• Results were presented at LISAIII in April • Simple and computationally inexpensive • But, lexical similarity is not enough

• Need to combine with other evidence

SPREADING ACTIVATION MODEL

• Connectionist Approach • A Model of Human Associative Memory• A Spreading Activation Model based on Co-occurrence Data• Combine the networks of two databases through the

commonly indexed documents• Terms from different vocabularies (e.g., STI and ApJ)

consistently assigned to the same documents are identified.

• Network Representation (3-layer)- input term layer, document layer (hidden), output term layer

• Activation Process- feed-forward network- activation is spread from one term layer to the other.

Information Retrieval: 2-Layer Network

T1 T2 Tm

D1 D2 D3 Dn

Figure 1. Network Representation of Information Retrieval

Activation Rule

m

Dj = (Ti * wij)

i=1

TERM LAYER

DOCUMENT LAYER

Ti = activation of term i

Dj = activation of document j

wij = weight between term i and document j

T1 T2 Tm

D1 D2 D3Dn

Figure 2. Spreading Activation Process during Information Retrieval

TERM LAYER

DOCUMENT LAYER

w11 w12 wij

Vocabulary Merging: 3-Layer NetworkT1 ( I ) T2 ( I ) Tl ( I )

D1 D3

Dm

Figure 5. Network representation of vocabulary merging

T1 (O) T2 (O) Tn (O)

D2

Activation Rule l

Dj = (Si * wij) i=1

m

Tk = (Dj * wjk) j=1

Ti (I) = activation of an input term i

Dj = activation of a document j

Tk (O) = activation of an output term k

wij = weight between an input term i and a document j

wjk = weight between an output term k and a document j

INPUT TERMLAYER

OUTPUT TERMLAYER

DOCUMENTLAYER

Weighting Scheme

• Binary Indexing System- wij = 1 if document j is indexed by term i

= 0 else

• Conservation of Activation- Total activation is divided up in amounts

proportional to the link weights m

- w´ij = wij / wij i=1

- input activation = output activation

• Simple Weighting Rule - w´ij = 1 / Ni (where Ni = the number of links

from input term i to the document nodes)

- w´jk = 1 / Nj (where Nj = the number of links

from document j to the output term nodes)

Weighting Scheme (continued)

• Bi-Directionality- w´ij w´ji

w´ij = 0.33 w´ji = 1.0

• Probability Interpretation

w´ijw´ji

Weighting Scheme (continued)

Computation of Activation

Ti ( I ) = 1.0

Dj

Tk ( O )

w´ij = 1 / Ni

= 0.5

l

Dj = ( Ti ( I ) * wij) i=1

= 0.5

w´jk = 1 / Nj

= 0.33

m

Tk(O) = ( Dj * wjk) j=1

= 0.5 * 0.33 = 0.165

w´ij

w´jk

OUTPUT THRESHOLD FUNCTION

• Output of spreading activation

- terms in the target vocabulary ranked by activation level

- terms above a cutoff value are selected

• Cut-off criterion

- Mexican-Hat function (second-derivative of the Gaussian)

- to find the deepest drop in the slope of activation levels

Experiment

• Material I

- Two indexing languages from ADS database (1983 - 1998)- STI (Scientific and Technical Information) terms- ApJ (Astrophysical Journal) terms

• Two types of merging: ApJ STI, STIApJ

• Database statistics[STI] [ApJ]

# of documents 39,366 22,139 # of indexing terms 10,200 3,335 avg. # of postings 163 25 avg. # of terms 10.2 3.7

[STI-M] [ApJ-M] # of co-indexed doc. 14,956 14,956 # of indexing terms 4,120 2,305 avg. # of postings 34 23 avg. # of terms 9.6 3.5

Experiment

• Material II

- Two indexing languages from ADS database

- Consistent and small (1998)

- AJ (Astronomical Journal) terms

- ApJ terms

• Two types of merging: AJ ApJ, ApJ AJ

• Database statistics[AJ] [ApJ]

# of documents 221 221

# of indexing terms 259 276

avg. # of postings 2.8 2.5

avg. # of terms 3.3 3.2

RESULTS

1. TERM RELATIONSHIP

• Term mappings identified by spreading activation model

GALAXIES: THE GALAXY MILKY WAY GALAXY

GALAXIES: ISM INTERSTELLAR MATTER

ISM: ABUNDANCES ABUNDANCE INTERSTELLAR MATTER

ANDROMEDA GALAXY GALAXIES: INDIVIDUAL MESSIER NUMBER: M31

• Types of Term Mapping

Exact match ASTROMETRY ASTROMETRY

Different word order CLUSTERS: GLOBULAR GLOBULAR CLUSTERS

Spelling variants GALACTIC CLUSTERS GALAXIES: CLUSTERING

Pre-coordinated term to its components

COSMIC RAYS: ABUNDANCES ABUNDANCE COSMIC RAYS

Term omission COSMOLOGY: DARK MATTER DARK MATTER CATALOGS ASTRONOMICAL CATALOGS

Semantic factoring COSMOLOGY: THEORY COSMOLOGY

ASTRONOMICAL MODELS

INPUT TERM (STI) STATISTICSANDROMEDA GALAXY Number of co-indexed document(s): 92

Number of active terms ( 0.01):171 (23)Number of terms above the cutoff point: 1

OUTPUT TERM (ApJ)GALAXIES: INDIVIDUAL MESSIER NUMBER: M31 = 0.171GALAXIES: STELLAR CONTENT = 0.057GALAXIES: NUCLEI = 0.035GALAXIES: LOCAL GROUP = 0.031GALAXIES: DISTANCES = 0.028STARS: NOVAE = 0.026GALAXIES: STRUCTURE = 0.023CLUSTERS: GLOBULAR = 0.019GALAXIES: INTERNAL MOTIONS = 0.019GALAXIES: PHOTOMETRY = 0.019GALAXIES: INTERSTELLAR MATTER = 0.018GAMMA RAYS: BURSTS = 0.018GALAXIES: MAGELLANIC CLOUDS = 0.017INTERSTELLAR: MOLECULES = 0.016STARS: ABUNDANCES = 0.016COSMOLOGY = 0.014BLACK HOLES = 0.013NEBULAE: PLANETARY = 0.013STARS: NEUTRON = 0.012GALAXIES: REDSHIFTS = 0.012GALAXIES: ISM = 0.011

A Sample Output (STI ApJ)

INPUT TERM(STI) ANDROMEDA GALAXY

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

GALAXIES: INDIVIDUAL MESSIER NUMBER: M31GALAXIES: STELLAR CONTENT GALAXIES: NUCLEI GALAXIES: LOCAL GROUP GALAXIES: DISTANCES STARS: NOVAE

GALAXIES: STRUCTURE CLUSTERS: GLOBULAR GALAXIES: INTERNAL MOTIONS GALAXIES: PHOTOMETRY GALAXIES: INTERSTELLAR MATTER GAMMA RAYS: BURSTS GALAXIES: MAGELLANIC CLOUDS INTERSTELLAR: MOLECULES STARS: ABUNDANCES COSMOLOGY BLACK HOLES NEBULAE: PLANETARY STARS: NEUTRON GALAXIES: REDSHIFTS GALAXIES: ISM

ACTIVATION LEVEL

INPUT TERM (APJ) STATISTICS ISM: ABUNDANCES Number of co-indexed document(s): 177

Number of active terms ( 0.01): 500 (15)Number of terms above the cutoff point: 2

OUTPUT TERM (STI)ABUNDANCE = 0.059INTERSTELLAR MATTER = 0.057MOLECULAR CLOUDS = 0.028INTERSTELLAR GAS = 0.023ABSORPTION SPECTRA = 0.021LINE SPECTRA = 0.017ASTRONOMICAL SPECTROSCOPY = 0.016H II REGIONS = 0.015MILKY WAY GALAXY = 0.014ULTRAVIOLET ASTRONOMY = 0.013EMISSION SPECTRA = 0.012STAR FORMATION = 0.011ASTRONOMICAL MODELS = 0.011SPECTRUM ANALYSIS = 0.010ULTRAVIOLET SPECTRA = 0.010

A Sample Output (ApJ STI)

INPUT TERM(APJ) ISM: ABUNDANCES

CUTOFF DATA ANALYSIS

STI->APJ

TOTAL STI TERMS: 900

AVERAGE CUTOFF POINT: 1.90

AVERAGE ACTIVATION: 0.17

AVERAGE NUMBER OF ACTIVATED TERMS: 134.78

APJ->STI

TOTAL APJ TERMS: 338

AVERAGE CUTOFF POINT: 1.62

AVERAGE ACTIVATION: 0.10

AVERAGE NUMBER OF ACTIVATED TERMS: 353.93

ACTIVATION LEVEL BY NUMBER OF DOCUMENT

ApJ => STI NUMBER OF DOC ACT AVG ACT CUT AVG DOCUMENTS FREQ TERM ACT LEV OFF CUT

[ - 5]: 1737 13 0.073 0.45 4.32 0.10

[ 5- 10]: 154 47 0.021 0.16 2.20 0.07

[ 10- 20]: 76 82 0.012 0.12 1.83 0.07

[ 20- 50]: 104 155 0.006 0.10 1.84 0.06

[ 50-100]: 82 272 0.004 0.09 1.54 0.06

[100-200]: 72 402 0.002 0.09 1.50 0.06

[200- ]: 80 652 0.002 0.10 1.55 0.06

STI => ApJ NUMBER OF DOC ACT AVG ACT CUT AVG DOCUMENTS FREQ TERM ACT LEV OFF CUT

[ - 5]: 2216 5 0.185 0.79 3.03 0.26

[ 5- 10]: 539 17 0.059 0.31 2.08 0.15

[ 10- 20]: 465 31 0.032 0.25 2.14 0.11

[ 20- 50]: 385 57 0.017 0.20 2.03 0.10

[ 50-100]: 211 107 0.009 0.17 1.89 0.09

[100-200]: 138 168 0.006 0.15 1.80 0.09

[200- ]: 166 320 0.003 0.14 1.73 0.08

RESULTS

2. TERM RANKING PREDICTION

- Document: 1983ApJ...265..760W- Input terms (ApJ)

GALAXIES: THE GALAXY- Target term list (STI)

GALACTIC NUCLEIINFRARED ASTRONOMYINTERSTELLAR EXTINCTIONINTERSTELLAR GASMILKY WAY GALAXY

- Output terms (STI)MILKY WAY GALAXY = 0.095GALACTIC NUCLEI = 0.041GALACTIC STRUCTURE = 0.030INTERSTELLAR MATTER = 0.026MOLECULAR CLOUDS = 0.022

INPUT: STIOUTPUT: ApJ

INPUT: ApJ OUTPUT: STI

DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS

1 5 1 947 2601 0.02 ( 3 )

2 7 1 854 2188 0.12 ( 1 )

3 10 2 646 2578 0.09 ( 1 5 )

4 7 1 978 4454 0.03 ( 1 )

5 7 2 609 2386 0.07 ( 3 6 )

6 7 2 651 1710 0.35 ( 1 2 )

7 9 4 937 4449 0.22 ( 1 2 3 11 )

8 7 2 829 3648 0.08 ( 1 7 )

9 9 4 939 4702 0.20 ( 1 2 3 9 )

10 8 2 775 3460 0.13 ( 1 3 )

DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS

1 1 5 456 201 0.16 (1 2 6 37 146)

2 1 7 563 265 0.12 (1 6 25 28 36 120 275)

3 2 10 897 711 0.14 (1 6 9 14 17 96 102 143

4 1 7 181 46 0.17 (1 5 7 14 25 62 62)

5 2 7 586 205 0.12 (1 6 11 12 13 22 97)

6 2 7 779 430 0.14 (1 2 23 51 55 175 233)

7 4 9 1467 1875 0.18 (1 2 3 6 9 17 68 98 115)

8 2 7 987 688 0.16 (1 2 3 5 17 108 336)

9 4 9 1527 2088 0.19 (1 2 3 7 9 12 17 72 190)

10 2 8 1093 1165 0.15 (1 2 4 5 79 93 163 210)

TERM RANKING (AJ vs. ApJ)

AJ -> ApJ DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS 1 2 2 14 8 0.53 ( 1 2 ) 2 3 3 19 10 0.65 ( 1 2 3 ) 3 4 3 31 17 0.42 ( 1 2 3 ) 4 3 3 18 11 0.73 ( 1 2 3 ) 5 2 2 15 7 0.71 ( 1 2 ) 6 2 2 25 14 0.45 ( 1 2 ) 7 2 2 32 14 0.22 ( 1 3 ) 8 5 6 49 23 0.64 ( 1 2 3 4 4 4 ) 9 3 3 38 15 0.41 ( 1 2 3 ) 10 2 2 5 2 0.75 ( 1 1 )

ApJ -> AJ DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS 1 2 2 12 7 0.53 ( 1 2 ) 2 3 3 18 9 0.66 ( 1 2 3 ) 3 3 4 22 11 0.57 ( 1 2 3 4 ) 4 3 3 14 9 0.76 ( 1 2 3 ) 5 2 2 16 8 0.46 ( 1 2 ) 6 2 2 26 13 0.47 ( 1 2 ) 7 2 2 16 6 0.69 ( 1 2 ) 8 6 5 43 21 0.72 ( 1 2 3 4 4 ) 9 3 3 37 13 0.57 ( 1 2 3 ) 10 2 2 2 1 1.00 ( 1 1 )

EFFECT OF LEAVING OUT THE GIVEN DOCUMENT

1. including the given document in the network

DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS 1 1 5 456 201 0.16 ( 1 2 6 37 146 ) 2 1 7 563 265 0.12 ( 1 6 25 28 36 120 275 ) 3 2 10 897 711 0.14 ( 1 6 9 14 17 96 102 143 158 219) 4 1 7 181 46 0.17 ( 1 5 7 14 25 62 62 ) 5 2 7 586 205 0.12 ( 1 6 11 12 13 22 97 ) 6 2 7 779 430 0.14 ( 1 2 23 51 55 175 233 ) 7 4 9 1467 1875 0.18 ( 1 2 3 6 9 17 68 98 115 ) 8 2 7 987 688 0.16 ( 1 2 3 5 17 108 336 ) 9 4 9 1527 2088 0.19 ( 1 2 3 7 9 12 17 72 190 ) 10 2 8 1093 1165 0.15 ( 1 2 4 5 79 93 163 210 )

2. leaving the given document out of the network

DOCU SOURCE TARGET ACTIVE SHARE ACTIV RANKINGS FOR THE _ID TERM TERM TERM DOC LEVEL ACTIVE TARGET TERMS 1 1 5 456 200 0.16 ( 1 2 6 45 279 ) 2 1 7 562 264 0.12 ( 1 6 26 30 41 168 0 ) 3 2 10 897 710 0.13 ( 1 6 9 14 18 119 121 198 226 365 ) 4 1 7 179 45 0.15 ( 1 5 8 24 42 0 0 ) 5 2 7 585 204 0.10 ( 1 6 15 17 19 36 0 ) 6 2 7 779 429 0.13 ( 1 2 28 61 67 276 391 ) 7 4 9 1467 1874 0.18 ( 1 2 3 6 9 17 73 118 137 ) 8 2 7 986 687 0.15 ( 1 2 3 5 17 135 0 ) 9 4 9 1527 2087 0.19 ( 1 2 3 7 9 13 18 74 226 ) 10 2 8 1093 1164 0.15 ( 1 2 4 5 89 116 242 360 )

Ranking: 1 2 3 4 5 6 7 8 9

[DOCUMENT #:ID] 135: 1986ApJ...301..240D 325 co-indexed document(s)

[TERM LIST FROM THE SOURCE VOC. (APJ)] [TERM LIST FROM THE TARGET VOC. (STI)]1212 ACCRETION DISKSSTARS: ECLIPSING BINARIES ASTRONOMICAL PHOTOMETRYSTARS: INDIVIDUAL ALPHANUMERIC: KPD 1911 BALMER SERIESSTARS: INDIVIDUAL CONSTELLATION NAME: CATACLYSMIC VARIABLES V1315 AQUILAE ECLIPSING BINARY STARSSTARS: NOVAE EMISSION SPECTRA

NOVAESTELLAR PHYSICSSTELLAR SPECTRA

[ACTIVE TERMS: 636] * ECLIPSING BINARY STARS = 0.080 * CATACLYSMIC VARIABLES = 0.070 * NOVAE = 0.069 * STELLAR SPECTRA = 0.063 * ACCRETION DISKS = 0.062 * EMISSION SPECTRA = 0.062 * ASTRONOMICAL PHOTOMETRY = 0.062 * BALMER SERIES = 0.058 * STELLAR PHYSICS = 0.057

LIGHT CURVE = 0.015ASTRONOMICAL SPECTROSCOPY = 0.013WHITE DWARF STARS = 0.011SPECTRUM ANALYSIS = 0.011SKY SURVEYS (ASTRONOMY) = 0.011

[DOCUMENT #:ID] 652: 1986ApJ...306..532L 9 co-indexed document(s)

[TERM LIST FROM THE SOURCE VOC. (APJ)] [TERM LIST FROM THE TARGET VOC. (STI)]NEBULAE: INDIVIDUAL NGC NUMBER: NGC 7538 ABUNDANCE

ASTRONOMICAL SPECTROSCOPYELECTRON DENSITY (CONCENTRATION)INFRARED ASTRONOMYLIGHT SCATTERINGLINE SPECTRANEBULAEVISIBLE SPECTRUM

[ACTIVE TERMS:73]H II REGIONS = 0.088MOLECULAR CLOUDS = 0.084

* LINE SPECTRA = 0.048INTERSTELLAR MASERS = 0.048INFRARED SOURCES (ASTRONOMY) = 0.047EMISSION SPECTRA = 0.036AMMONIA = 0.029

* ASTRONOMICAL SPECTROSCOPY = 0.028STELLAR EVOLUTION = 0.022

* NEBULAE = 0.022FLOW VELOCITY = 0.018SPECTRAL SIGNATURES = 0.018FORMALDEHYDE = 0.016HYDROCYANIC ACID = 0.016MICROWAVE INTERFEROMETERS = 0.016HIGH RESOLUTION = 0.015EARLY STARS = 0.014STAR FORMATION = 0.014WATER MASERS = 0.014

* ELECTRON DENSITY (CONCENTRATION) = 0.014 * VISIBLE SPECTRUM = 0.014 * LIGHT SCATTERING = 0.014

PRE-MAIN SEQUENCE STARS = 0.014 * INFRARED ASTRONOMY = 0.014 * ABUNDANCE = 0.014

INTERSTELLAR MATTER = 0.012

Ranking: 3 8 9 19 19 19 19 19

Ranking: 63

[DOCUMENT #:ID] 652: 1986ApJ...306..532L 4363 co-indexed document(s)

[TERM LIST FROM THE SOURCE VOC. (STI)] [TERM LIST FROM THE TARGET VOC. (APJ)]ABUNDANCE NEBULAE: INDIVIDUAL NGC NUMBER: NGC 7538ASTRONOMICAL SPECTROSCOPYELECTRON DENSITY (CONCENTRATION)INFRARED ASTRONOMYLIGHT SCATTERINGLINE SPECTRANEBULAEVISIBLE SPECTRUM

[ACTIVE TERMS: 1430]INTERSTELLAR: MOLECULES = 0.028STARS: PRE--MAIN-SEQUENCE = 0.018INFRARED: SOURCES = 0.018NEBULAE: H II REGIONS = 0.016QUASARS = 0.015GALAXIES: NUCLEI = 0.015ATOMIC PROCESSES = 0.014GALAXIES: SEYFERT = 0.014ULTRAVIOLET: SPECTRA = 0.014POLARIZATION = 0.014INTERSTELLAR: MATTER = 0.013STARS: FORMATION = 0.013INFRARED: SPECTRA = 0.013STARS: ABUNDANCES = 0.012RADIATIVE TRANSFER = 0.011INTERSTELLAR: GRAINS = 0.011NEBULAE: SUPERNOVA REMNANTS = 0.011

* NEBULAE: INDIVIDUAL NGC NUMBER: NGC 7538 = 0.004

Rocchio’s Measure

• Normalized recall measure• Difference between Ideal ranking and Actual ranking (Actual rankings) - (Ideal rankings)• Normalization

n1 (N - n1) where n1 = number of target termsN = number of active terms

• Example Ideal ranking: (1 2 3 4 5) Actual ranking: (1 2 5 6 10)

Total active terms: 40 # target terms: 5 (Actual rankings) - (Ideal rankings) = 9

n1 (N - n1) = 5(40-5) = 175 Rocchio measure = 1 - (9/175) = .95

Rocchio Measure (ApJ STI)

0

500

1000

1500

2000

2500

10.96 0.92 0.88 0.84 0.8 0.76 0.72 0.68 0.64 0.6 0.55 0.44 0.32 0.13

Number of Cases

Rocchio Measure (STI ApJ)

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10.98 0.96 0.94 0.92 0.9 0.88 0.86 0.83 0.8

Number of Cases

Future study

• User evaluation

• Compatibility measures

• Comparison to other methods

• Visualization of term relationships

GALAXIES: KINEMATICS

AND DYNAMICS

ACCRETION STARS: FORMATION

INSTABILITIES

DYNAMIC MODELS

KINEMATICS ROTATING DISKS

GALAXIES: CLUSTERING

GALACTIC STURUCTURE

GALACTIC STURUCTURE

THANK YOU!!!