PyDataLondon 2016 presentation Fashion is a visual medium so it makes sense for our models of fashion to include visual features. In this presentation, I'll describe how we've build a general purpose visual fashion representation using CNNs. The network is multi-task (multiple labels per image), multi-image (multiple images per label) and it runs on multiple GPUs. I'll visually explore what is going on inside the black box of a neural network and discover how a fashion specific model sees the world differently from generic visual models. Lastly, I'll demonstrate a multi-modal applications of the representation learned by the model.

1. Working with
Fashion Models
Eddie Bell - @ejlbell

2. Lyst

3. 3
Fashion

4. general purpose visual fashion representation
Aim
Build a general purpose visual fashion model

5. Deep Learning
Composition and representation

6. Composition

7. Representation

8. AI must fundamentally understand the
world around us and this can only be
achieved if it can learn to identify and
disentangle the underlying explanatory
factors hidden in the observed milieu of
low-level sensory data.
2014 - Representation Learning: A Review and New Perspectives. - Bengio et al.

9. Male
FemaleCat
Dog

10. Male
FemaleCat
Dog

11. Male
FemaleCat
Dog

12. The model

13. Male
Jacket
Beige / Brown
Iridescent
Zip

16. Training
20 epochs
5 hours per epoch
16 million parameters
4 million images

17. task accuracy
all 0.80
colour 0.73
gender 0.91
type 0.97
category 0.78
subcategory 0.61
task prediction conf
colour blue 0.28
gender men 0.93
type shoe 0.88
category sneaker 0.38
subcategory low-top 0.21

18. Internals

29. What can we use this for?

31. Multi-modal embeddings
This elegant long black coat is perfect for pydata
Positive Image Negative Image
Anchor word
Textual context
Visual context

32. w1 w2 lexical semantic
opening openings 1 0.57
flared flare 1 0.87
chic chino 2 0.18
loops belt loops 5 0.99

33. embroidered logo
button down
curved hem
front button fastening
button placket
point collar
breathable
classic
slim fit

34. Composition and Representation

35. Thanks
@ejlbell