A Folksonomy of styles, aka: other stylists also said and Subjective Influence Networks for...

AIM of the project: Effective human in the (ML) loop:

• Providing stylists with the best style insights for them to do the best recommendations to clients, on top of the recommendation algorithm

How? • Leveraging the feedback of other clients and the advice of other stylists to

help stylists write helpful and insightful notes by providing insights on every item in the inventory

Project 1: A sentence-to-garment-ID classifier algorithm and a Folksonomy of styles (as a module for a note writing assistant tool)

3

Results Project 1

•A sentence2garmentID classifier supervised algorithm •A crowdsourcing task to gather a ground truth dataset of sentences to validate the former (***)

•A folksonomy(*) of styles (totally unsupervised approach) with very high precision and coverage:

•Positively labelled garment ids: 94.869% •Positively labelled shipments: 88.417% (**) •Positively labelled sentences: 24.529%

(*) Definition: Folksonomy: a (less structured than an ontology) collection of entities (garments) and everything ever said about them (**) +88% of the garment ids have comments, i.e., something was said about them, e.g., how to wear/style such garment (***) 100% coverage

Project 2 (this presentation): Subjective Influence Networks to extend the Fashion Knowledge Graph

5

10

State of the art on fashion ontologies

What existing ontologies about fashion look like

11

What existing ontologies about fashion look like

12

13

Subjective Influence Networks

• A way to model influence in time and subjectivity for machine learning hypothesis testing

• A mathematical framework to better frame fuzzy and subjective, aesthetics hypothesis

15

Subjective Influence Networks

Example of addressable ML problems

16

Proprietary and confidential

Proprietary and confidential

Proprietary and confidential

Blending image clouds…

24

…with word clouds (as FoxType does)

25

Subjective Influence Networks:Evaluation and Modelling Obligations

26

27

Modelling obligations

For the model to be useful, an ontology must include: • Enumerating at least some of the members in G, N and M (Garment space, Styles and Mechanisms)

• Characterising a relevant set of subjective judge functions s() s.t. s(g) = x, is a region in our theoretical set of all clothing styles

•where x is a distinct style • Calculating, estimating or assuming values for the quads (t,i,m,a) for edges between the nodes x in N

• Consideration of cycles: e.g. 70s’ disco fashion has come back multiple times: explicit modelling despite being a heavily influenced new style is important

28

Modelling obligations

Types of influence mechanisms: • Explicit: The creator of a style explicitly declares its source of influence

• Calculated: algorithmic or other mechanical means may estimate mechanism and strength (based on garment features or causal cultural models)

• Extrinsic: caused by parallel cultural influences (music, sports, religion).

29

Task specific ontology expressiveness

How well the ontology maps the user’s mental model? Statistical measures of “expressiveness” in IR:

E.g.: Given a query: “Natural fabric buttons dress from Banana Republic”

30

Task specific ontology expressiveness

• We derive a mapping between concepts that appeared in the query and concepts encoded in the ontology.

• We ask expert judges to identify important classes ci in the query.

• For each class ci, we ask them to map it to the most similar label in the ontology. Then we compute the recall of concepts in the query as

Query concept recall:

31

Evaluation: Task-specific ontology expressiveness

Search result ranking: • We use rankings of search results for a given query as a proxy of

“golden standard” • Normalized Discounted Cumulative Gain (NDCG) metric can be

adapted to measure the ontology’s relevancy to search quality. • The gain is quantified by the ontology’s recall of concepts in

search results. We examine the top K returned search results. For each of them Dp at position p, we compute the ontology’s document concept recall, which is then discounted by its logarithmic rank log(p):

32

Evaluation quality metrics to assess the ontology’s quality

How well the ontology approximates empirical data in the fashion domain? • Categorical precision: how many styles encoded in the influence network are real-world recognisable styles?

• Temporal bias (time-invariance of features): the length of time span before the divergence is the representativeness timescale of the network, and the scope indicates its robustness

33

Evaluation quality metrics to assess the ontology’s quality

• Semantic similarity: This measurement provides a distance metric between a traditional ontology augmented with an influence network and the text data in fashion domain in terms of “meanings" they express. We project both labels (class names Oc and property names Op) in the ontology and the tokens in the document corpus D in the same vector space (e.g., using word2vec). Then we compute the overall similarity based on all labels' distances weighted by their importance within the network:

The importance score θ(ci) is a normalized score [0, 1], can be defined depending on the context and usage (e.g. θ(ci) refers to how knowledgeable a network is w.r.t. class ci, which can be approximated by the cumulative distribution function of its number of attributes ui:

Business Development Experience:Can this technology be monetized?

35

36

Consumption of Subjective Influence Networks by ML systems

General ML problems: • NLP as a service: e.g., NER, POS, KG. • Fashion data retrieval: integrations with schema.org,

Freebase or other publishers to build: • Crowd-sourcing systems • Search engine indexes

• Recommender system: • Taxonomy building as a service: to learn KG,

bottom-up approach • Deal with sparsity and cold start problem.

Business model proposals:• Real-time advice giving from stylists with a snapchat-

like UI Bot/Human assistant [The rise of the social bots, ACM, 16]

• Scaling a very domain specific writing assistant• Bot style checker for a look in a fix (box)

Business Modelling

37

Business Modelling

38

Business Modelling

39

40

Some survey answers…

Future life

42

• getAsteria.com• Learning Deep Learning

• through practical or fun projects (involving music, parasites and stars)

Current sabbatical time & futuristic projects

43

Thank you!diaz.rodriguez.natalia@gmail.com

MORE DETAILS:

• Paper • Poster• Stitch Fix Technology Blog: http://multithreaded.stitchfix.com/

44