Download pptx - Introduction to Machine Learning

Shao-Chuan Wang, Academia Sinica 1

Introduction to Machine Learning

Shao-Chuan WangResearch Center for IT Innovation

Multimedia and Machine Learning LabAcademia Sinica

中央研究院資訊科技創新研究中心多媒體與機器學習實驗室

NTNU


Outline

• What is involved in intelligence?• Why is machine learning important?• What can machine learning do?• Overview of machine learning applications• Challenges of machine learning• Future of machine learning


What Is Intelligence?


What Is Involved in Intelligence?

• From Merriam-Webster: – “intelligence”: (1) the ability to learn or understand or

to deal with new or trying situations. (2) the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria

• Abstraction (finding the common patterns)

– V.S.• Adaptation– Learning is dynamic; e.g. a computer chess.


Why Is Machine Learning Important?(1/4)

The explosion of data



Some places areNOT for humans



Machine learning can help us understand human learning



Intelligent machines can help!


What Can Machine Learning Do?


Application One:Handwriting Recognition

Video


Application Two: Face Detection and Tracking

Video


Application Three: Autonomous Driving

Video


Overview of Machine Learning Applications

• Speech recognition

• Computer vision

• Bio-surveillance

• Robotics

• Data mining


What Is Learning?

15Shao-Chuan Wang, Academia Sinica

A Tree Recognition Example (1/2)

• Suppose that you have never seen trees before, and I give you some “EXAMPLES”

Trees examples ‘Not’ Trees examples


A Tree Recognition Example (2/2)

• I will ask you if these unseen photos are trees or not.

Is it a tree?YES

NOor

Query Images

(AND) How much confidence?


What Is Learning?

• (Mitchell 2002) Learning is to improve the performance measure P of the task T based on the past experience E.– T: To recognize a tree– P: Recognition accuracy– E: The examples that I gave to you

• Two key elements of learning:– Memorization of past experiences.– “Generalization” ability ( 舉一反三 ).

18Shao-Chuan Wang, Academia Sinica

A Simple Algorithm: Nearest Neighbor

• For a given query image– Find the nearest image to the query image in the

database– Assign the label of the nearest one to the query

image. Query

Difference = 1.5

Difference = 5.5 Difference = 10

Difference = 13

Difference = 11

Tree!

19

What Were We Modeling?

Human Concept(exist but unknown)

YESYESYESNONONO…

A Machine(A learning algorithm)

Infer

Query

Prediction: NO

TREE

Shao-Chuan Wang, Academia Sinica

Training…


What if we do not have label ground truth?? (or labels are very expensive)


Unsupervised Learning

• Clustering。 Each segment forms a “Cluster”.。 Pattern discover


Examples: Amazon.com

• Marketing– Recommendation on the similar goods.


Challenges of Machine Learning

• How do we model the “difference” between two images?– Data Representation– Difference Metric• What is the “score” or “difference” function?

– How did we calculate the distance value in the tree example?

• Learning– Does it model well? (can it accurately predict the

seen data?)– Does it generalize well? (can it be proved?)

OR ?


Example: sea bass or salmon?

• Suppose that we have only two kinds of fish, and we want a computer system that aids our distinction between sea bass and salmon.

• Process:

Take A

PictureComputer Decision



• How do we describe a fish? (Data representation)– What kinds of information can help us distinguish one

from the other?• Length, width, size of fins, tail shape, color, etc?

• How do we measure its distinctness under the chosen data representation? (Difference metric)– E.g. if we choose length, than their “distinctness” can

be measured using its absolute relative values.

|| basssalmon LLD



• Assume that a fisherman (prior domain knowledge) told us that salmon is generally longer than a sea bass.

• We may use length as a feature to discriminate between them.

• But how?

|| basssalmon LLD



• We use “past experiences” and we calculate a histogram of lengths for two types of fishes.

• Apply Nearest Neighbor to their average length.



• The difficulty comes from the ambiguity around the threshold value.

• Length itself is insufficient to “describe” the fishes.– Use more features like width and color, etc.– Other manipulation. E.g. use nearest neighbor to

“median” of the length; will it be better?• Let’s try one more feature: width



• We can use two features and wrote them down as a vector:

• Each fish image is represented as a 2-D feature vector:

2

1

x

xx

Length : x1

Width : x2



There are still misclassified training examples



• Why use Line?

• We can use complex boundary, but we radically change the boundary just because of some heretics. => may not generalize well.


Challenges of Machine Learning

• Conclusion on this example:– We have to incorporate prior knowledge to decide

which features we are going to use. At present, there is no universal learning machines.

– We want a feature that is invariant within certain specie but distinct between different species.

– There is a trade-off between complexity of decision model s and their “training errors”.


The Future of Machine Learning

• Theoretic foundations of learning• Scalability (Parallel)• Robustness to dynamic environment


Questions?


Thank you for your attention!


Learning schemes

• Supervised learning:– The tree example is a supervised learning

problem.– Supervised learning provides label ground truth.

• Unsupervised learning:– Unsupervised learning DOES NOT provide label

ground truth.• Reinforcement learning:– The way you train your pets.