Extracting Relationships by Multi-Domain MatchingYitong Li, Michael Murias, Samantha Major, Geraldine Dawson, David E. Carlson

Duke University

Motivation & Contribution1) We study domain adaptation from multiple sources. All data share thesame labels (i.e. diagnosis), but the underlying reason for the decision(i.e. cause) may be different in each domain.2) The training corpus is constructed from only a multiple sources withina larger population.3) We hypothesize that each domain should be similar to a few otherdomains and share statistical strength, while many other domains areirrelevant (i.e different reasons for outcomes).4) We propose the Multiple Domain Matching Network (MDMN) toperform unsupervised domain adaptation as well as extract thesedomain relationships.5) Theoretical results (in paper) bound the target error using sourceerror and the discrepancy between domains.

Experiments on Digit Dataset• We use MNIST, MNIST-M, SVHN and USPS datasets.

• Treat MNIST-M as target and the other three as training.

Measure the Difference between Domains• All domains, including sources and target, are denoted as !", for s = 1,⋯(.

• We have feature encoder * ⋅; -. , and a domain discriminator /" ⋅ for each domain 0 = 1,⋯ , (.

• Distance between two domains !1 and !12:

3 !", !"2 = max78, 78 9:;

<=∼!8 /" *(@) − <=∼!82 /" *(@)

• Distance between domain !1 and all other domains !1:

3 !", !" = max78, 78 9:;

<=∼!8 /" *(@) − <=∼!8 /" *(@)

• !" comes from the weighted combination of other domains:

!1 = C"DE;

1

F""D !"D FGHℎ F" J= 1 KL3 F"" = 0

• The total domain loss function minimizes this pair-wise domain distance with N" as domain weights (target domain is upweighted)

ℒP = −C"E;

1

N"3 !", !"

Model Illustration

Calculating Domain Weights• Domain weights F" = F";, F"J,⋯ ,F"1 denotes the similarity

between domain !1 and all other domains.• First, calculate 3 !", !"2 for every two domains in the set,

including the target. Note that dRRD = 3 !", !" = 0.• Second, compute F" = softmax 3";,⋯ , 3"1 .• Can set a temperature variable in the softmax if desired• F" and parameters of the networks are updated iteratively.

Experiments on EEG Dataset• Classification Accuracy

MNISTMNISTM

SVHN

USPS

Source Domains

MNISTMNISTM

SVHN

USPS

Source Domains

Domain Adversarial Networks

MultipleDomain Matching

Network

MNIST

MNISTM

SVHN

USPS

Model Framework

The features should be learned to perform well on label prediction and perform poorly on a domain loss or prediction:

VGLWX,WYVK@WZℒ[ − ℒP

Learned Subject Relationship from Autism Spectral Disorder Dataset

Baseline MDANs

DANN MDMN

X Feature Extractor

Label Predictor

Domain Adapter

\]

\^

\_

Learned Graph from Domain Weights