Graph Matching for Adaptation in Remote Sensing

Embed Size (px)

Text of Graph Matching for Adaptation in Remote Sensing


    Graph Matching for Adaptation in Remote SensingDevis Tuia, Member, IEEE, Jordi Muoz-Mar, Member, IEEE, Luis Gmez-Chova, Member, IEEE, and

    Jesus Malo, Member, IEEE

    AbstractWe present an adaptation algorithm focused on thedescription of the data changes under different acquisition con-ditions. When considering a source and a destination domain,the adaptation is carried out by transforming one data set to theother using an appropriate nonlinear deformation. The eventuallynonlinear transform is based on vector quantization and graphmatching. The transfer learning mapping is defined in an unsuper-vised manner. Once this mapping has been defined, the samplesin one domain are projected onto the other, thus allowing theapplication of any classifier or regressor in the transformed do-main. Experiments on challenging remote sensing scenarios, suchas multitemporal very high resolution image classification andangular effects compensation, show the validity of the proposedmethod to match-related domains and enhance the application ofcross-domains image processing techniques.

    Index TermsDomain adaptation, model portability, multi-temporal classification, support vector machine (SVM), transferlearning.


    THE problem of adapting models to a new unseen butrelated data set is one of the major challenges for futureremote sensing image processing. For instance, the increasein temporal resolution of image acquisitions of modern sen-sors allows data users to acquire several scenes of the samearea, thus making multitemporal analysis [1][3], multiangularstudies [4][6], and accurate change detection [7][9] possible.However, since new generation satellites such as QuickBird orWorldView 2 acquire images at an almost daily frequency, it isnot realistic to provide supervised labeled information for eachimage to perform accurate image classification or biophysicalparameter retrieval. Users must thus often rely on single ac-quisition ground truth to process series of images. Althoughtempting, direct application of classifiers or regressors on newly

    Manuscript received December 22, 2011; revised March 30, 2012 andMay 10, 2012; accepted May 11, 2012. Date of publication June 20, 2012;date of current version December 19, 2012. This work was supported in partby the Swiss National Science Foundation under Grants PBLAP2-127713 andPZ00P2-136827), the Spanish Ministry for Science and Innovation throughCICYT projects TEC2009-13696 and CSD2007-00018, and the Universitat deValncia via project UV-INV-AE11-41223.

    D. Tuia was with the Image Processing Laboratory, Universitat de Valncia,46980 Valncia, Spain. He is currently with the Laboratoire des SystmesdInformation Gographique (LaSIG), Federal Institute of Technology ofLausanne (EPFL), 1015 Lausanne, Switzerland (e-mail:

    J. Muoz-Mar, L. Gmez-Chova, and J. Malo are with the Image Pro-cessing Laboratory, Universitat de Valncia, 46980 Valncia, Spain (;;

    Color versions of one or more of the figures in this paper are available onlineat

    Digital Object Identifier 10.1109/TGRS.2012.2200045

    acquired and similar images can lead to catastrophic results:even if the objects represented in the images are roughly thesame, differences in reflectance, illumination, and atmosphericconditions may induce local changes in the probability distri-bution function (PDF) instead of simple global linear changes.Therefore, finding an efficient PDF matching strategy becomesan urgent and complex task for new generation remote sensingprocessing chains.

    In machine learning literature, the problem of model adap-tation is usually referred to as domain adaptation or transferlearning[10]. Transfer learning aims at describing the changesoccurring among data distribution and to adapt models torelated tasks. This recent field of research attempts to re-solve the deformations occurring among similar domains butacquired under different conditions: in remote sensing, thiswould correspond to data sets coming from similar images. Thismeans that, in transfer learning, we are particularly interested intransferring the knowledge from one or more source domainsto a target domain rather than learning all source and targetdomains simultaneously [11].

    In remote sensing literature, generic attempts to describechanges in manifolds are hardly found. Currently, the adapta-tion problem has been mainly considered from the perspectiveof specific classifiers, i.e., improve/modify the classifier so thatit can cope with the shifted distribution. In the 1970s, suchchanges were studied under the name of signature extension[12], but the field received little interest until recently andremained specific to simple models and midresolution land-use applications [13][16]. Textural filters have also beenconsidered for increasing the robustness of classifiers amongacquisitions: in [17], the authors observe that the use of texturalfeatures reduces spectral variability and thus allows success-ful classification of unseen scenes sharing the same classes.However, this approach only describes indirect adaptation, asno direct attempt to compensate for data set shift is done andthe improvement in performance is related to the well-knowndecrease of intraclass variability related to the use of contextualfeatures.

    The advent of transfer learning theory renewed the interestof the community for these problems, but in most cases, theeffort is mainly focused on modifying the classifier while thetransform between the domains is not explicitly studied. In[18], the samples in the new domain are used to re-estimatethe classifier based on a Gaussian mixture model. This way, theGaussian clusters optimized for the first domain are expected tomatch the data observed in the second. In case of strong shifts,there is no guarantee of appropriate correspondence betweenthe identified clusters. In [19], randomly generated classifiersare applied in the destination domain. A measure of the

    0196-2892/$31.00 2012 IEEE


    diversity of the predictions is used to prune the classifiersensemble. In [20], a support vector machine (SVM) classifier ismodified by adding and removing support vectors from both do-mains in an iterative fashion: the model discards contradictoryold training samples and uses the distribution of the new imageto adapt the model to the new conditions. In [21], matchingof the first-order statistics of the data clusters in source anddestination domains is performed in a kernel space. Two kernelsaccounting for similarities among samples and clusters arecombined, and the SVM weights are computed using the labelsfrom samples in the first domain. Finally, in [22], the shiftsare compensated using a small set of examples sampled onthe new distribution: the algorithm learns the shifts amongdomains iteratively, and active learning queries [23], [24] areused to maximize the ratio between samples and increase inperformance. Although adaptable to any classifier, the abovemethods consider data set shift in an indirect way since they arefocused in the modifications to be done in the classifier.

    Recently, some remote sensing researches have explicitlyconsidered the distortions occurring between data manifolds: In[25], multitemporal sequences for each pixel are aligned basedon a measure of similarity between sequences barycenters, thusconsisting into a global measure of alignment. In [26], pixelsspectra are spatially detrended using Gaussian processes inorder to avoid shifts related to geometrical differences or tolocalized class variability. Even if very efficient when appliedto subregions of the same area, the approach can difficultlybe extended to spatially disconnected areas. Finally, in [27],a kernel machine is regularized iteratively to ensure matchingbetween the original and test distributions. Also, the use ofmanifold learning [28] techniques based on graph-Laplacianhave been shown very accurate in problems with a reducednumber of samples. Approximate methods for out-of-sampleprediction [29] are required to apply this last approach to large-scale problems.

    In this paper, we also study the possibility of using localmanifold techniques for adaptation. This is justified by thegenerally nonlinear and low-dimensional structure of remotesensing data [30], [31]. To propose a simple and easily scalablesolution, we focus on the description of the changes in themanifold by defining a nonlinear transform based on vectorquantization and graph matching. Vector quantization is usedto retrieve local properties of the data clouds. A clusteringalgorithm is used to define a proximity graph in each domain,where the nodes of the graph are the cluster centroids (orcodebook vectors). Graph matching has been strongly used inpattern recognition and computer vision to study similaritiesamong data structures representing objects [32][34]. The needof matching procedures relying to other criteria than Euclideandistances has generated a wide field of research about thedefinition of similarity measures between graphs [32], [35],optimization techniques [36][38], and structured estimations[39]. In the proposed algorithm, the graphs of the two domainsare matched using a procedure aiming at maximizing theirsimilarity, while at the same time preserving the structure of thetransformed graph. In our case, the number of nodes remainsunchanged during the procedure, and no supervised information(e.g., class labels) about the nodes is required.

    Once the nonlinear transform has been defined, the train-ing samples in the source are matched to the data distri-bution of the target domain, or inversely dep


View more >