Still Life with Blue Tablecloth, 1909, one of Matisse's many works that featured a piece of fabric found in a Paris shop. Photo: The State Hermitage Museum, St Petersburg Rainbow-coloured textiles were what rescued Henri Matisse from the flat, muddy sugar beet fields of northern France and made him one of the best loved artists of the 20th century.

A dazzle of the fabrics he collected all his life, from scraps bought from Parisian flea markets when he was a student to North African embroidered wall hangings and haute couture gowns collected in the 30s has gone on display in London - for the first time alongside the paintings they inspired.

He called them "my noble rags" and the curator, Ann Dumas, described the collection of fabrics he took everywhere, draping them over furniture or hanging them on walls every time he moved into a new flat or studio, as "a portable travelling tool kit".

The fabrics are instantly identifiable in many paintings: a luscious length of French silk woven with brilliant coloured posies of flowers which resurfaces in Tangier as the table cloth in a still life Picasso bought; elaborately embroidered African wall hangings that form the background of the "odalisque" paintings of languid, exotically costumed women; and a length of blue floral print which has been worn to rags in the service of art.

He spotted the cream French cotton/linen table cloth, printed in pale and indigo blue, as his bus passed a shop window in Paris. He leaped off to buy it, and used it in a string of paintings over the next 30 years, sometimes barely visible in the background, sometimes filling the canvas.

The paintings and drawings are loans from private collections and museums including the Hermitage, which owns a whole gallery of Matisse works incorporating brilliantly patterned fabrics, originally bought from the artist by a collector, Sergei Schukin, who also happened to be a textile merchant.

The blue and cream fabric, sadly frayed and with several careful darns, survived in the Matisse family archives and is in the exhibition.

"There was no great mystery in finding the fabrics," Hilary Spurling, who has just completed the second volume of her epic biography of the artist, said. "Nobody had thought to display them together with the paintings before, but as soon as the family was asked they came pouring out of chests and dressing-up boxes."

His grandson, Claude Duthuit, said Matisse had an excellent eye all his life for textiles of any kind: apart from his noble rags, he regularly bought lengths of suit or dress fabric as presents for his family. Although he is mortified to have lost most of the notes the artist sent him - "you know as a boy of 13 or 14 one has different ideas of what is important" - he does still have a jacket made from one such gift.

Ms Spurling believes textiles directly inspired his use of colour. Matisse was brought up in the textile town of Bohain-en-Vermandois, where there was no museum or gallery, and the curators believe he did no oil paintings before the age of 19 or 20.

But the town's weavers were famous for outrageous colour and pattern: sample albums sizzle with silks in hot pinks and purples, and iridescent blues and greens.

"This was a place where everything was pretty dull, except the fabrics, where the streams from the dye works would literally run pink or scarlet - that's what trained his eye in colour," Ms Spurling said.