Week 1 - An Introduction to Machine Learning & Soft Computing -Yosi Kristian-

Week 1 - An Introduction to Machine Learning & Soft Computing

-Yosi Kristian-

Soft Computing STTS – Yosi Kristian 2


Definition• Soft Computing is a term applied to a field within computer

science which is characterized by the use of inexact solutions to computationally hard tasks, for which there is no known algorithm that can compute an exact solution.

• Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation.


Still the Definition..• In effect, the role model for soft computing is the human

mind. • The guiding principle of soft computing is: Exploit the

tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost.


Components of soft computing

• Neural networks (NN)• Support Vector Machines (SVM)• Fuzzy logics (FL)• Evolutionary computation (EC), including:

o Evolutionary algorithms• Genetic algorithms• Differential evolution

o Meta heuristic and Swarm Intelligence• Ant colony optimization• Bees algorithms• Bat algorithm• Cuckoo search• Harmony search• Firefly algorithm• Artificial immune systems• Particle swarm optimization

What ???Are we going to learn them

allin this

subject?


Soft Computing in AI• Soft computing may be viewed as a foundation component for

the emerging field of conceptual intelligence.o Machine Learningo Fuzzy Systemso Evolutionary Computationo Probabilistic Reasoning

• Soft Computing is the CORE component of many Machine Learning System


Machine Learning• Arthur Samuel (1959). Machine Learning: Field of study that

gives computers the ability to learn without being explicitly programmed.

• Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.


Machine learning usage• Usage of Machine Learning is to develop applications

that can’t be programed by hand.• E.g., Autonomous helicopter, handwriting recognition,

most of Natural Language Processing (NLP), Computer Vision etc.

• Or a machine learning system could be trained on email messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new email messages into spam and non-spam folders.


Machine Learning Categorized By Data and Learning Process


Soft Computing In Machine Learning• Soft Computing is the soul of many machine

learning system.• Classification and Clustering is a very common

soft computing problems.


Intro to Supervised Learning


Example

0 500 1000 1500 2000 25000

100

200

300

400Housing price prediction.

Price ($) in 1000’s

Size in feet2

Regression: Predict continuous valued output (price)

Supervised Learning“right answers” given


Example

Breast cancer (malignant, benign)

ClassificationDiscrete valued output (0 or 1)

Malignant?

1(Y)

0(N)

Tumor Size

Tumor Size


Another Example

Tumor Size

Age

- Clump Thickness- Uniformity of Cell

Size- Uniformity of Cell

Shape…


Exercise• You’re running a company, and you want to develop learning

algorithms to address each of two problems.• Problem 1: You have a large inventory of identical items. You

want to predict how many of these items will sell over the next 3 months.

• Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised.

• Should you treat these as classification or as regression problems?


Answer• Treat both as classification problems. • Treat problem 1 as a classification problem, problem 2 as a

regression problem. • Treat problem 1 as a regression problem, problem 2 as a

classification problem. • Treat both as regression problems.


Classification Example• Another Example is for image Classification / Categorization

Training LabelsTraining

Images

Classifier

Training

Training

Image Features

Trained Classifier


Cont…

Image Features

Testing

Test Image

Trained Classifier

Outdoor

Prediction


Learning a classifier• Given some set of features with corresponding labels, learn a

function to predict the labels from the features• Training labels dictate that two examples are the same or

different, in some sense• Features and distance measures define similarity• Classifiers try to learn weights or parameters for features and

distance measures so that feature similarity predicts label similarity


Intro to Unsupervised Learning


Supervised Learning

x1

x2


Unsupervised Learning

x1

x2


Clustering Example


Contd…


ExerciseOf the following examples, which would you address using an

unsupervised learning algorithm? (Check all that apply.) o Given email labeled as spam/not spam, learn a spam filter.o Given a set of news articles found on the web, group them into set of

articles about the same story. o Given a database of customer data, automatically discover market

segments and group customers into different market segments. o Given a dataset of patients diagnosed as either having diabetes or not,

learn to classify new patients as having diabetes or not.


Warming Up….• Do 10 x Push Ups.


Linear Regression with one variable

500 1000 1500 2000 2500 30000

100000

200000

300000

400000

500000

500 1000 1500 2000 2500 30000

100000

200000

300000

400000

500000Housing Prices(Portland, OR)

Price(in

1000s of dollars)

Size (feet2)

Supervised Learning

Given the “right answer” for each example in the data.

Regression Problem

Predict real-valued output


Linear Regression with one variable

Notation:

m = Number of training examples n = Number of feature x’s = “input” variable / features y’s = “output” variable / “target” variable

Training set ofhousing prices(Portland, OR)


The Concept

Training Set

Learning Algorithm

hSize of

house

Estimated price

How do we represent h ?

Linear regression with one variable.Univariate linear regression.


Cost Function

How to choose ‘s ?

Training Set

Hypothesis:

‘s: Parameters


Contd..

0 1 2 30

1

2

3

0 1 2 30

1

2

3

0 1 2 30

1

2

3


Cost Function..

y

x

Idea: Choose so that is close to for our training examples


Simplification: For the sake of understanding

Hypothesis:

Parameters:

Cost Function:

Goal:

Simplified


Trial 1

0 1 2 30

1

2

3

y

x

(for fixed , this is a function of x) (function of the parameter )

-0.5 0 0.5 1 1.5 2 2.50

1

2

3


Trial 2

0 1 2 30

1

2

3

y

x


-0.5 0 0.5 1 1.5 2 2.50

1

2

3


Trial 3

-0.5 0 0.5 1 1.5 2 2.50

1

2

3

y

x


0 1 2 30

1

2

3


Done with simplification, back to real world.

Hypothesis:

Parameters:

Cost Function:

Goal:


The Contour Figures ….

How to find minimum of J in that?


Gradient Descent..• Next Week…

Documents

Week 1 - An Introduction to Machine Learning & Soft Computing -Yosi Kristian-