8/12/2019 Generating art using robotics

1/23

Zrich

CSE Seminar in RoboticsPresentation by: Maria Veiga

8/12/2019 Generating art using robotics

2/23

Zrich

Introduction Motivation

What is out there

In depth projects Painting fool

eDavid

Conclusion

8/12/2019 Generating art using robotics

3/23

Zrich

Artists perspective & questions: Generative art introduces complexity to an art piece;

Often used as a tool, thus:

What techniques can be used?

How to introduce randomness to art?

Scientific perspective & questions: Interest in setting up an autonomous creative agent

How to formalize a creative process?

How to create an autonomous creative agent?

How to teach a robot to paint?

8/12/2019 Generating art using robotics

4/23

Zrich

TimelineMozart

18thcentury

Alfred Jensen:

The Apex is Nothing

(1960)

Ellsworth Kelly:Four Black and Whites

(1957)

John Cage:

Music of Changes

(1951)

The Painting Fool

(2006)

eDavid

(2009)

BNJMN

(2013)

AARON

(1973)

Zanelle

(2012)

Vangobot

(2012)

Frieder Nake

(1963)

8/12/2019 Generating art using robotics

5/23

Zrich

TimelineMozart

18thcentury

Alfred Jensen:

The Apex is Nothing

(1960)

Ellsworth Kelly:Four Black and Whites

(1957)

John Cage:

Music of Changes

(1951)

The Painting Fool

(2006)

eDavid

(2009)

BNJMN

(2013)

AARON

(1973)

Zanelle

(2012)

Vangobot

(2012)

Frieder Nake

(1963)

8/12/2019 Generating art using robotics

6/23

Zrich

Started in the Computational Creativity Research

group at Imperial College

In 2006, the software was given the name

Painting Fool

Objective is to build a software system that is

one day taken seriously as a creative artist

8/12/2019 Generating art using robotics

7/23Zrich

The question is: how to build software thatexhibits behaviours deemed creative in people?

8/12/2019 Generating art using robotics

8/23Zrich

s an Iproject

Simulation of paint strokes on a canvas

Aware of emotions of the subjectCultural context

Critical appraisal of its own work

8/12/2019 Generating art using robotics

9/23Zrich

s an Iproject

Simulation of paint strokes on a canvas

Aware of emotions of the subjectCultural context

Critical appraisal of its own work

Non-Photorealistic Rendering (NPR):By producing software that simulates

natural media

Allow artists to teach the software:

Allow artists, designers, and anyone else to

train the sofware in aspects of itsprocessing (from analysing digital

photographs to constructing and painting

scenes)

8/12/2019 Generating art using robotics

10/23Zrich

s an Iproject

Simulation of paint strokes on a canvas

Aware of emotions of the subjectCultural context

Critical appraisal of its own work

Mapping of painting style to emotions:Mapping composed of researchs group

choices and also randomly generated

Detect emotion in images and videos:

Emotion detection software that identifies:

1) The frame that shows more emotion2) Locations of the facial features

3) The emotion expressed by the subject

(Using methods by Valstar and Pantic)

8/12/2019 Generating art using robotics

11/23Zrich

s an Iproject

Simulation of paint strokes on a canvas

Aware of emotions of the subjectCultural context

Critical appraisal of its own work

Show intention in the paintings:Scan the internet for text and image

resources for production of artwork.

1) Software downloads news headlines

2) Use a text rank algorithm to find most

important nouns

3) Use the keywords to scan imagerepositories (Google, Flickr)

4) Create a piece by collage of selected

pictures and using non-photorealistic

rendering software

8/12/2019 Generating art using robotics

12/23Zrich

s an Iproject

Simulation of paint strokes on a canvas

Aware of emotions of the subjectCultural context

Critical appraisal of its own work

Still an active area of research:How to formalise the process of appraisal of

artwork?

Implemented abilities for the software to

calculate objective measures of abstract

art, e.g: location of symmetries

BUT hard to train a software to appreciateworks of art without being an extension of

its creators subjective views

8/12/2019 Generating art using robotics

13/23Zrich

Emotionally aware painter

Demonstrating intent Travelling salesman dancers

8/12/2019 Generating art using robotics

14/23Zrich

8/12/2019 Generating art using robotics

15/23Zrich

Objective: Build a robot able to paint with real

color on a real canvas.

How?

Commercial robotic arm + Visual System + Smart Algorithms

8/12/2019 Generating art using robotics

16/23Zrich

Objective: Build a robot able to paint with real

color on a real canvas.

How?

Commercial robotic arm + Visual System + Smart Algorithms

-one-arm industry robot used in car manufacturing

modified for painting

- several adapters for different mediums

8/12/2019 Generating art using robotics

17/23Zrich

Objective: Build a robot able to paint with real

color on a real canvas.

How?

Commercial robotic arm + Visual System + Smart Algorithms

8/12/2019 Generating art using robotics

18/23Zrich

Objective: Build a robot able to paint with real

color on a real canvas.

How?

Commercial robotic arm + Visual System + Smart Algorithms

Painting as an optimization processconsider spatial division, color and

content composition.

Iterative operation scheme to produce color distributions.

2 ways to compute the brush stroke:

- Pre-define a set of stroke candidates

- Dynamically generated strokesCompute image gradient using a Sobel

operator. Randomly generate initial positions. For each position, a path using

Line Integral Convolution of the gradient vector field is generated and then

Runge Kutta numerical integration is performed.

8/12/2019 Generating art using robotics

19/23Zrich

Objective: Build a robot able to paint with real

color on a real canvas.

How?

Commercial robotic arm + Visual System + Smart Algorithms

Semantic Hints:

- Use different techniques for different parts of a picture

- Automatically segment the picture

- Choose which methoddynamic or pre-defined brush strokesbest

minimizes the distance between canvas image C and target image T.

8/12/2019 Generating art using robotics

20/23Zrich

8/12/2019 Generating art using robotics

21/23Zrich

Pre-defined brush stroke vs dynamic brush strokes

8/12/2019 Generating art using robotics

22/23

8/12/2019 Generating art using robotics

23/23