A Roller - Fast Sampling-Based TextureSynthesis Algorithm

Michal HaindlInstitute of Information Theory and Automation

Academy of Sciences of the Czech RepublicCZ18208, Prague, Czech Republic

haindl@utia.cas.cz

Martin HatkaFaculty of Nuclear Sciencesand Physical Engineering

Czech Technical UniversityCZ11519 Prague, Czech Republic

hadis@email.cz

ABSTRACT

This paper describes a method for synthesizing natural textures that realistically matches given colour texture ap-pearance. The novel texture synthesis method, which we call the roller, is based on the overlapping tiling andsubsequent minimum error boundary cut. An optimal double toroidal patch is seamlessly repeated during the syn-thesis step. While the method allows only moderate texture compression it is extremely fast due to separation ofthe analytical step of the algorithm from the texture synthesis part, universal, and easily implementable in a graph-ical hardware for purpose of real-time colour texture rendering.

KeywordsVirtual Reality Models, Texture Synthesis.

1. INTRODUCTION

Virtual or augmented reality systems (VR) require ob-ject surfaces covered with realistic nature-like colourtextures to enhance realism in virtual scenes. To makevirtual worlds realistic detailed scene models must bebuilt. Satisfactory models require not only complex 3Dshapes accorded with the captured scene, but also life-like colour and texture. This will increase significantlythe realism of the synthetic scene generated. Texturesprovide useful cues to a subject navigating in such aVR environment, and they also aid in the accurate de-tailed reconstruction of the environment.

Permission to make digital or hard copies of all or part ofthis work for personal or classroom use is granted withoutfee provided that copies are not made or distributed forprofit or commercial advantage and that copies bear thisnotice and the full citation on the first page. To copyotherwise, or republish, to post on servers or to redistributeto lists, requires prior specific permission and/or a fee.

WSCG SHORT papers proceedings ISBN 80-903100-9-5WSCG’2005, January 31-February 4, 2005Plzen, Czech Republic.Copyright UNION Agency - Science Press

The purpose of a synthetic texture is to reproduce agiven digitized texture image so that both natural andsynthetic texture will be indiscernible. However mod-elling of a natural texture is a very challenging anddifficult task, due to unlimited variety of possible sur-faces, illumination and viewing conditions simultane-ously with the strong discriminative functionality ofthe human visual system. The related texture mod-elling approaches may be divided primarily into in-telligent sampling and model-based-analysis and syn-thesis, but no ideal method for texture synthesis ex-ists. Each of existing approaches has its advantagesand also limitations.

Model-based colour texture synthesis [Bes74],[Kas81],[Ben98], [Hai91a], [Hai00], [Zhu00],[Gri03],[Hai04] requires non-standard multi-dimensional(3D for static colour textures or even 7D for bidirec-tional texture function) models. If a 3D texture spacecan be factorized then these data can be modelled us-ing a set of less-dimensional 2D random field mod-els, otherwise it is necessary to use some 3D randomfield model. Among such possible models the Gaus-sian Markov random fields are advantageous notonly because they do not suffer with some prob-lems of alternative options (see [Hai91b], [Hai91a],

[Hai00], [Hai00] for details) but they are also rela-tively easy to synthesize and still flexible enough toimitate a large set of natural and artificial textures. Un-fortunately real data space can be decorrelated onlyapproximately, hence the independent spectral com-ponent modelling approach suffers with some lossof image information. Alternative full 3D models al-low unrestricted spatial-spectral correlation mod-elling, but its main drawback is large amount of pa-rameters to be estimated and in the case of Markovmodels also the necessity to estimate all these parame-ters simultaneously. Model-based methods are mostlytoo difficult to be implemented in modern graphi-cal card processors.Intelligent sampling approaches [DeB97], [Efr99],[Efr01], [Hee95], [Xu00] rely on sophisticated sam-pling from real texture measurements. Given arandomly selected starting block of texture in the im-age, they propagate out from it selecting new tex-ture blocks. For each new block in the image, allneighboring blocks that have already been gener-ated are checked and the example image (or im-ages) is searched for similar textures. The k best suchmatches are found and then randomly choosen the cor-responding new texture patch from among them. Themethods [Efr01],[Efr99], [Wei01] all vary in howthe blocks are represented, how similarity is deter-mined, and how the search is performed. Intelligentsampling approaches are based on some sort of orig-inal small texture sampling and the best of themproduce very realistic synthetic textures, usually bet-ter than the model-based methods. However thesemethods require to store original texture sample, of-ten produce visible seams, they are mostly compu-tationally demanding, they cannot generate texturesunseen by the algorithm, and they cannot even ap-proach the large compression ratio of the model-basedmethods.

The rest of the paper is organised as follows. The fol-lowing section describes a simple sampling approachbased on the repetition of a double toroidal tile carvedfrom the original texture measurement. Results andconclusions are reported in the last section.

2. DOUBLE TOROIDAL TILE

The double toroidal tile (see Fig.1) is limited by theselected minimal rectangle to be inscribed in from theoriginal texture measurement. The texture tile is as-sumed to be indexed on the regular two-dimensionaltoroidal lattice. The optimal lattice searched by the al-gorithm allows for seamless repetition in both horizon-tal and vertical directions, respectively.Let us define the overlap error for a pixelr as follows:

ψhr =

(Yr − Yr+[N−h,0]

)2 ∀r ∈ Ih ,

Figure 1. The roller principle - upper row in-put texture and toroidal tile, bottom row tex-ture generation and the result, respectively.

ψvr =

(Yr − Yr+[0,M−v]

)2 ∀r ∈ Iv ,

whereYr denotes a multispectral pixel indexed on theN ×M underlying lattice. The multiindexr has twocomponentsr = [r1, r2], the first component is rowand the second one column index, respectively. The in-dex setsIh, Iv are defined

Ih = (1, . . . , h)× (1, . . . ,M) ,Iv = (1, . . . , N)× (1, . . . , v) ,

h andv lie in preselected intervalsh ∈ 〈hmin;hmax〉and v ∈ 〈vmin; vmax〉. The optimal horizontal and ver-tical overlaps are found from the following two rela-tions forζ ∈ {h, v}:

ζ∗ = minζ

1ζ

∑∀r∈Iζ

ψζr

.

Optimal Cut

The optimal cuts for both the horizontal and verti-cal edge is searched using the dynamic programmingmethod. Alternatively we can use some other subop-timal search such as theA∗ algorithm if necessary tospeed up also the analytical part of the method. How-ever for most applications the fast synthesis is prereq-uisite while the computation time for separately andonly once solved analytical part is of no importance.Both optimal cuts have to minimize the overall patherror

Ψh∗

r = ψh∗

r + min{

Ψh∗

r−[1,1],Ψh∗

r−[0,1],Ψh∗

r+[1,−1]

},

Ψv∗

r = ψv∗

r + min{

Ψv∗

r−[1,1],Ψv∗

r−[1,0],Ψv∗

r+[−1,1]

}.

Figure 2. The optimal tile cuts in bothdirections.

The combination of both optimal vertical and horizon-tal cuts creates the toroidal tile as is demonstrated onthe Fig.2.

Synthesis

The synthesis of any required colour texture size issimple repetition of the created double toroidal tile inboth directions until the required texture is generated.There is no computation involved in this step hence itcan be easily implemented in real time or inside thegraphical card processing unit.The complete roller algorithm is as follows:

• Analysis

1. Find the optimal horizontal and verti-cal overlapsh∗, v∗.

2. Search for optimal horizontal and verticalcuts starting fromIh ∩ Iv.

3. Create the double toroidal texture tile.

• Synthesis

The range of horizontal and vertical overlapping inter-vals are the only parameters specified by the user. Theanalytical part is completely separated from the syn-thesis. The most time consuming part of the analysisis optimal cuts search whose time requirement is pro-portional to T ∝ v2hN − 2v2h2 +Mvh2 . The opti-mal overlap search time is negligible and the synthesisstep contains no computations.

3. RESULTS AND CONCLUSIONS

We have tested the algorithm on several hun-dred colour and grayscale textures from the VisTexdatabase [Pic95], Noctua database, Brodatz textures[Bro66] and mainly from our extensive Prague tex-ture database, which currently contains over 500colour textures. Tested textures were either naturalsuch as bark, wood, plants, water, etc., or man-madeknitwear, upholstery, brick wall, textiles, food prod-ucts and many others. Some results (bark, rattan,jeans cloth, sugar, text and wire mesh) are demon-strated in the following images.

Resulting textures are mostly surprisingly good forsuch a very simple algorithm. For example our re-sults on the text texture (the second from the bot-tom) are indistinguishable (see [Efr01]) from resultson the same texture using much more complicatedand slower image quilting algorithm [Efr01]. Obvi-ously there is no optimal texture modelling methodand also the presented method fails on some textures.However on most of our failure examples also somealternative intelligent sampling methods failed. Thetest results of our algorithm on our extensive natu-ral texture collection are encouraging. The presentedmethod is extremely fast, very simple and easily im-plementable even in the graphical processing unit. Themethod offers moderate compression ratio for trans-mission or storing texture information while it has neg-ligible computation complexity. The roller method canbe used for easy and fast seamless synthesis of any re-quired texture size for many natural or man made tex-tures. The method’s extension for bidirectional texturefunctions is straightforward.

ACKNOWLEDGMENTS

This research was supported by the EC projects no.IST-2001-34744 RealReflect, FP6-507752 MUSCLEand grants No.A2075302, 1ET400750407 of the GrantAgency of the Academy of Sciences CR.

References

[Ben98] BENNETT, J.,AND KHOTANZAD , A. 1998. Mul-tispectral random field models for synthesis and anal-ysis of color images.IEEE Trans. on Pattern Analysisand Machine Intelligence 20, 3 (March), 327–332.

[Bes74] BESAG, J. 1974. Spatial interaction and the statis-tical analysis of lattice systems.Journal of the RoyalStatistical Society, B-36, 2 (February), 192–236.

[Bro66] BRODATZ, P. 1966.Textures: A Photographic Al-bum for Artists and Designers. Dover Publications.

[DeB97] DE BONET, J. 1997. Multiresolution samplingprocedure for analysis and synthesis of textured im-ages. InProc. SIGGRAPH 97, ACM, 361–368.

[Efr01] EFROS, A. A., AND FREEMAN, W. T. 2001. Im-age quilting for texture synthesis and transfer. InSIG-

GRAPH 2001, Computer Graphics Proceedings, ACMPress / ACM SIGGRAPH, E. Fiume, Ed., 341–346.

[Efr99] EFROS, A. A., AND LEUNG, T. K. 1999. Tex-ture synthesis by non-parametric sampling. InProc.Int. Conf. on Computer Vision (2), 1033–1038.

[Gri03] GRIM , J., AND HAINDL , M. 2003. Texture mod-elling by discrete distribution mixtures.Computa-tional Statistics Data Analysis 41, 3-4 (Jan.), 603–615.

[Hai00] HAINDL , M., AND HAVL I CEK, V. 2000. A mul-tiresolution causal colour texture model. InAdvancesin Pattern Recognition, Lecture Notes in ComputerScience 1876, F. J. Ferri, J. M. Inesta, and P. Pudil,Eds. Springer-Verlag, Berlin, August, ch. 1, 114 –122.

[Hai02b] HAINDL , M., AND HAVL I CEK, V. 2002. A multi-scale colour texture model. InProceedings of the 16thIAPR International Conference on Pattern Recogni-tion, IEEE Press, Quebec City, R. Kasturi, D. Lauren-deau, and C. Suen, Eds., vol. I, 255 – 258.

[Hai04] HAINDL , M., GRIM , J., SOMOL, P., PUDIL , P.,AND KUDO, M. 2004. A gaussian mixture-basedcolour texture model. InProc. 17th IAPR Int. Conf. onPattern Recognition, IEEE Press, Los Alamitos, J. Kit-tler, M. Petrou, and M. Nixon, Eds., vol. 3, 177–180.

[Hai91a] HAINDL , M. 1991. Texture synthesis.CWI Quar-terly 4, 4 (December), 305–331.

[Hai91b] HAINDL , M. 1991. Texture synthesis. Tech. Rep.CS-R9139, Centrum voor Wiskunde en Informatica,Amsterdam, The Netherlands.

[Hai00] HAINDL , M. 2000. Texture modelling. InProceed-ings of the World Multiconference on Systemics, Cy-bernetics and Informatics, International Institute of In-formatics and Systemics, Orlando, USA, B. Sanchez,J. M. Pineda, J. Wolfmann, Z. Bellahse, and F. Ferri,Eds., vol. VII, 634–639.

[Hai03] HAINDL , M. FILIP, J. 2003. Fast btf texture mod-elling. In Texture 2003, IEEE Computer Society.

[Hee95] HEEGER, D., AND BERGEN, J. 1995. Pyramidbased texture analysis/synthesis. InProc. SIGGRAPH95, ACM, 229–238.

[Kas81] KASHYAP, R. 1981. Analysis and synthesisof image patterns by spatial interaction models. InProgress in Pattern Recognition 1, Elsevier, North-Holland, L. Kanal and A.Rosenfeld, Eds.

[Pic95] PICKARD , R., GRASZYK, C., MANN , S., WACH-MAN , J., PICKARD , L., AND CAMPBELL , L. 1995.Vistex database. Tech.rep., MIT Media Laboratory.

[Wei01] WEI, L., AND LEVOY, M. 2001. Texture synthe-sis over arbitrary manifold surfaces. InSIGGRAPH20001, Computer Graphics Proceedings, ACM Press /ACM SIGGRAPH / Addison Wesley Longman.

[Xu00] XU, Y., GUO, B., AND SHUM , H. 2000. Chaosmosaic: Fast and memory efficient texture synthesis.Tech. Rep. MSR-TR-2000-32, Redmont.

[Zhu00] ZHU, S., LIU , X., AND WU, Y. 2000. Explor-ing texture ensembles by efficient markov chain montecarlo - toward a “trichromacy” theory of texture.IEEETrans. on Pattern Analysis and Machine Intelligence22, 6 (June), 554–569.