Kunstig intelligens (MNFIT-272) - høst 2002.

Transkript

1 Kunstig intelligens (MNFIT-272) - høst Forelesning 10 Emner: Maskinlæring symbolorientert - intro til maskinlæring generelt - induktiv symbol-prosessering - læring i versjonsrom - læring av beslutningstrær

2 Hva er læring? Any process by which a system improves performance (H. Simon) Making useful changes in our minds (M. Minsky) The organisation of experience (M. Scott) Constructing or modifying representations of what is being experienced (R. Michalski)

3 Hva er maskinlæring? Metoder og teknikker som gjør datasystemer i stand til selv å oppdatere sin kunnskap og problemløsnings-evne

4 Hvorfor maskinlæring? Modellere menneskers læring Studere læring og intelligens som fenomen Automatisere utvikling av kunnskapsbaserte systemer det siste er mest vektlagt i dette kurset

5 Maskinlæringmetoder kan klassifiseres utifra flere perspektiver Grad av selvlæring - pugging (rote learning) - instruksjon - læring ved eksempler - læring ved analogi - læring ved oppdagelse Læringsmål - begrepslæring - problemløsning - situasjonsforståelse - produsere forklaringer Representasjonsspråk - attributt-verdi vektor - beslutningstrær - regler - predikatlogiske uttrykk - semantiske nett, rammer Grad av bakgrunnskunnskap - kunnskapsfattige metoder - kunnskapsrike metoder Grad av lærerstyring - veiledet læring - ikke-veiledet læring Grad av inkrementalitet - alle eksempler samtidig - inkrementell læring - læring gjennom problemløsing - lærlingteknikker

6 Maskinlæring - kort historikk Nevrale modeller, Rosenblatt s PERCEPTRON, Selvorganiserende systemer, Adaptive systemer, Læring som mønstergjenkjenning, Parameterjustering Samuel s Checkers Player Kunnskapsbaserte metoder, Winston s Analogy system, Symbolske teknikker for eksempelbasert læring, Michalski s AQ system, Mitchell s Version Space metode, Quinland s ID3 system, Discovery systemer, Langley s BACON, Lenat s AM. Kunnskapsrike metoder, Buchanan s Meta-Dendral, læring ved instruksjon, Davis Teresias, Ny giv for nevrale modeller, Forklaringsbasert læring (EBL), læring under problemløsning, case-basert læring, integrerte læremetoder Kunstig liv, Statistiske metoder, Data mining, Induktiv logikk-programmering, Genetiske algoritmer, Reinforcement læring, Bayesianske nett Integrerte læringsmetoder, læring under problemløsing

7 Eksempelbasert læring instansrommet: begrepsrommet: Generaliseringsbasert læring: eksempler hypoteser kandidater begrep Instansbasert læring: eksempler eksempler

8 Lærings-systemer i MNFIT 272: Generaliserings-baserte Læring i Versjonsrom (VS metoden) - Inkrementell læring - Lærer kun konjunktive begreper - Lærer klassifikasjonsregel Læring av beslutningstrær (ID3 metoden) - Ikke-inkrementell læring - Lærer konjunktive og disjunktive begreper - Lærer beslutningstre Forklaringsbasert læring (EBL) - bruker bakgrunnskunnskap Instans-baserte: CBR Analogi-læring

9 Version Space The version space is the set of all generalizations which are consistent with all of the examples It is described by the S and G sets: S is a generalization which is consistent with the examples and no other consistent generalization is more specific than it G is a generalization which is consistent with the examples and no other consistent generalization is more general than it

10 Version Space G more general consistent generalizations S more specific + examples move S up - examples move G down

11 Generality Lattice - Example (??) (? circle) (red?) (blue?) (? square) (red circle) (blue circle) (red square) (blue square)

12 Versjonsrom-algoritmen S summerer info fra de positive exempler G summerer info fra de negative exempler Når S=G er begrepet lært - dvs. flere eksempler tilfører ikke noe nytt Ikke nødvendig å ta vare på enkelteksemplene

13 Versjonsrom-algoritmen er en prinsipielt sterk læringsalgoritme - garanterer konsistens - er inkrementell men har store begrensninger i representasjonens uttrykkskraft - dvs. i hva som kan læres - kun konjunktive uttrykk - over-ivrig generalisering (mister muligheter)

14 ? (version-space example1) Example: (+ (BIG RED CIRCLE)) S= ((BIG RED CIRCLE)) G= ((???)) Example: (- (SMALL RED SQUARE)) S= ((BIG RED CIRCLE)) G= ((?? CIRCLE) (BIG??)) Example: (+ (SMALL RED CIRCLE)) S= ((? RED CIRCLE)) G= ((?? CIRCLE)) Example: (- (BIG BLUE CIRCLE)) S= ((? RED CIRCLE)) G= ((? RED CIRCLE)) Convergence. Concept must be: (? RED CIRCLE)

15 ? (version-space example2) Example: (+ (BIG RED CIRCLE)) S= ((BIG RED CIRCLE)) G= ((???)) Example: (- (SMALL BLUE TRIANGLE)) S= ((BIG RED CIRCLE)) G= ((?? CIRCLE) (? RED?) (BIG??)) Example: (+ (SMALL RED CIRCLE)) S= ((? RED CIRCLE)) G= ((?? CIRCLE) (? RED?)) Example: (- (MEDIUM GREEN SQUARE)) S= ((? RED CIRCLE)) G= ((?? CIRCLE) (? RED?)) Did not converge S= ((? RED CIRCLE)) G= ((?? CIRCLE) (? RED?))

16 ? (version-space example5) Example: (+ (BIG RED CIRCLE)) S= ((BIG RED CIRCLE)) G= ((???)) Example: (- (BIG BLUE CIRCLE)) S= ((BIG RED CIRCLE)) G= ((? RED?)) Example: (+ (SMALL BLUE SQUARE)) S= NIL G= NIL Langauage is insufficient to describe the concept

17 Decision Trees Data structure for clasifying objects Restricted to featural descriptions Allows for disjunction color red blue green size - shape big small circle square something red and small or green and square-shaped

18 Learning EXAMPLE Feature-names: (size color shape) (+ (big red circle)) (+ (small red square)) (- (medium red circle)) size small medium big The concept of a big or small, but not medium-sized, object

19 ID3 s information theoretic measure Picking the right feature for the root is crucial to building simple trees Choose feature which gives the most information regarding the class of an instance: I(C): Information content of tree for set of instances C P: Feature selected as splitting node E(P): Expected information needed to complete tree given splitting on P gain(p):information gain from feature P, gain(p) = I(C) - E(P)

20 ? *domains* ((BIG MEDIUM SMALL) (BLUE RED GREEN) (SQUARE TRIANGLE CIRCLE))? *feature-names* (SIZE COLOR SHAPE)? *categories* (+ -)? example1 ((+ (BIG RED CIRCLE)) (- (SMALL RED SQUARE)) (+ (SMALL RED CIRCLE)) (- (BIG BLUE CIRCLE)))? (id3 example1) Considering splitting on SIZE Gain = Chi square stat = (NOGOOD) Considering splitting on COLOR Gain = Chi square stat = (NOGOOD) Considering splitting on SHAPE Gain = Chi square stat = (NOGOOD) Gain Tie- Randomly choosing

21 Splitting on COLOR Looking at COLOR = BLUE All examples in class - Looking at COLOR = RED Considering splitting on SIZE Gain = Chi square stat = (NOGOOD) Considering splitting on SHAPE Gain = Chi square stat = (NOGOOD) Splitting on SHAPE Looking at SHAPE = SQUARE All examples in class - Looking at SHAPE = TRIANGLE No examples. Using most common class of parent node + Looking at SHAPE = CIRCLE All examples in class + Looking at COLOR = GREEN No examples. Using most common class of parent node + Feature: COLOR BLUE (0.250) Class is: - RED (0.750) Feature: SHAPE SQUARE (0.333) Class is: - TRIANGLE (0.000) Class is: + CIRCLE (0.667) Class is: + GREEN (0.000) Class is: + color red blue shape - square circle triangle green +

22 ? example5 ((+ (BIG RED CIRCLE)) (- (BIG BLUE CIRCLE)) (+ (SMALL BLUE SQUARE)))? (id3 example5) Considering splitting on SIZE Gain = Chi square stat = (NOGOOD) Considering splitting on COLOR Gain = Chi square stat = (NOGOOD) Considering splitting on SHAPE Gain = Chi square stat = (NOGOOD) Gain Tie- Randomly choosing Splitting on SHAPE Looking at SHAPE = SQUARE All examples in class + Looking at SHAPE = TRIANGLE No examples. Using most common class of parent node + Looking at SHAPE = CIRCLE Considering splitting on SIZE Gain = Chi square stat = (NOGOOD) Considering splitting on COLOR Gain = Chi square stat = (NOGOOD)

23 Splitting on COLOR Looking at COLOR = BLUE All examples in class - Looking at COLOR = RED All examples in class + Looking at COLOR = GREEN No examples. Using most common class of parent node + Feature: SHAPE SQUARE (0.333) Class is: + TRIANGLE (0.000) Class is: + CIRCLE (0.667) Feature: COLOR BLUE (0.500) Class is: - RED (0.500) Class is: + GREEN (0.000) Class is: + One way to treat missing data

24 Summary: Similarity-based Learning Empirical, data-intensive Requires many examples and counter-examples Knowledge poor Generates unjustified concept definitions

25 Alternative paradigm: - Explanation-based Learning Analytical Learns from 1 example Knowledge rich Generates justified concept definitions

26 Explanation-based Learning: Use exisiting domain knowledge to explain why an example is a member of a concept or why a plan works. Explanations determines relevant features Generalise the explanation made, to obtain a justified (explained) operational definition of the concept

27 Explanation-based Generalization Problem Given Goal concept Training example Domain theory Operationality criterion Determine A generalisation of the training example which is an operational sufficient concept definition

28 EBL method: 1. Explain Use the domain theory to explain why the example is an example of the goal concept 2. Generalise Determine the most general condictions under whch the explanation holds, and generalise the explanation

29 Meta-Dendral Lærer kløvningsregler for molekyler under massespektrometri Gitt: - et representasjonsspråk for å beskrive molekylstrukturer - et trenings-sett bestående av et sett molekyler fra en molekylfamilie, samt deres molekylstruktur og massespektrogram - en viss generell kunnskap om molekylkløving unde massespektrometri Finn: - et sett kløvningsregler for den aktuelle molekylfamilien