CSE 150 Lecture Notes - Lecture 12: Mixture Model, Bigram, 96.5 Wave Fm

25 views2 pages

turquoiseplatypus250

11 May 2018

School

UCSD

Department

Computer Science & Engineering

Course

CSE 150

Professor

Lawrence Saul

For unlimited access to Class Notes, a Class+ subscription is required.

Update rules

- When applied in parallel to ALL pi, will monotonically increase L = ∑t log P(Xt|X’t)

Markov Models of Language

Let wl = l’th word in sentence

How to model P(w1,...,wl) ? Shorthand: w’ = (w1,...,wL)

Model P(w’) MLE

Unigram ∏l P1(w’l) P1(w) = count(w)/L

Bigram P1(wl) ∏l P2(w’l|wl) P2(w’|w) = count(w → w’)

Trigram count(w → anything)

Train on corpus A: P1(w’) <= P2(w’) on corpus A

Test on corpus B: P1(w’) >= P2(w’) = 0 if there are unseen bigrams

Linear Interpolation (mixture model)

PM(wl|wl-1) = (1 - ƛ)P1(wl) + ƛP2(wl|wl-1) with ƛ ∈ [0,1]

How to estimate ƛ?

Hidden Variable Model

wl-1 → wlP(wl|wl-1, z) = { P1(wl) if z = 1

/ { P2(wl|wl-1) if z = 2

Z ∈ {1,2} P(z = 1) = 1 - ƛ

P(z = 2) = ƛ

How to estimate ƛ? From incomplete data {(wl-1,wl)}l =1

In this model:

P(wl|wl-1) = ∑z = 1

2 P(wl, z|wl-1) [marginalization]

= ∑z = 1

2 P(z|wl-1) P(wl|z, wl-1) [Product Rule]

= ∑z = 1

2 P(z) P(wl|z, wl-1) [Marginal Independence]

= (1 - ƛ)P1(wl) + ƛP2(wl|wl-1) [Reproduces mixture model]

E-step: compute posterior probs

P(z|wl-1,wl) = P(wl|z, wl-1) P(z|wl-1) [Bayes Rule]

P(wl|wl-1)

= P(wl|z, wl-1) P(z) [Marginal Independence]

P(wl|wl-1)

P(z = 2|wl-1,wl) = ƛP2(wl|wl-1)

(1-ƛ)P1(wl) + ƛP2(wl|wl-1)

M-step of EM alg:

For root nodes:

P(X = x) ← (1/T) ∑t P(X = x|V(t))

For mixture model:

P(Z = 2) ← (1/L) ∑l P(z = 2|wl,wl-1)

Unlock document

This preview shows half of the first page of the document.
Unlock all 2 pages and 3 million more documents.

Already have an account? Log in

Document Summary

When applied in parallel to all p i , will monotonically increase l = t log p(x t |x" t ) Test on corpus b: p 1 (w") >= p 2 (w") = 0. Hidden variable model w l-1 on corpus a if there are unseen bigrams. P(w l |w l-1 , z) = { p 1 (w l ) if z = 1. P 2 (w"|w) = count(w w") count(w anything) P(x = x) (1/t) t p(x = x|v (t) ) P(z = 2) (1/l) l p(z = 2|w l ,w l-1 ) Word-dependent value of (w) -- em used to estimate as many params of words in corpus. O t {1,2,,m} hidden state at time t observation at time t ( partial , noisy reflection of hidden state of world) S t { has to go, doesn"t have to go, went } O t { wagging tail, barking, standing by door, acting guiltily, }

Related Questions

9. Write a method with the following header public static void replace (String oldFile, string newfile, char oldChar, char newChar) throws IOException This method opens the file whose name is given by the String parameter oldFile (the input file) for reading, and opens the file whose name is given by the string parameter newlile (the output file) for writing. It then reads every lime in the input file. For each line, it creates a new line identical to the original line, creept for the fact that all occurrences of char oldchar in the original String have been replaced by occurrences of char newchar in the new line. When the new line has been properly created, it write it to the output file. Once all the lines in the input file have been read and a corresponding line has been written to the output file, the method clases both files. Additional notes: • You MAY assume that none of the parameters this method accepts are null; in other words, your method does not have to handle the case where one or more of the parameters are null. You MAY assume that all the 10-related classes have already heen imported properly . Finally, because the replace() method is specified to throw an exception object of type TORxception, you MAY assume that I/O related problems will occur while reading from the input file or writing to the output file. In other words, your method DOES NOT have to check that no such problems occur, and DOES NOT have to manage Texceptions using try-catch blocks. WRITE YOUR replace() METHOD IN THE SPACE BELOW: YOUR replace() METHOD CONTINUED: USE THIS PAGE IF YOU NEED ADDITIONAL SPACE. CLEARLY INDICATE WHICH QUESTION(S) YOU ARE ANSWERING HERE. USE THIS PAGE IF YOU NEED MORE ADDITIONAL SPACE. CLEARLY INDICATE WHICH QUES- TION(S) YOU ARE ANSWERING HERE. • String (package java.lang) Methods: - public String chal wale Al t ri g od med in the chance au - public int length(): R e ach of the string - public char chattin R echar w heefd index public chart to Chachay: Consthis String to a new c r y - public boolean equals Object anbject): Ce this string to and yect - public boolean equalsigmoce Case String anotherstring]: Cam p ing sider this String to anotherString - public int compareToString anotherString: Compare this String to anotherstring graphically returns a negative ef this string before anotherstring, apithe value of this Stringatering distries publie iw paneelgoro (String another Camp i ng Garden ces, string to anoth i ng li hically returns te value of this thing care anotherstring, apie as if this string e r anotherstring, Strings - public int indexin ch Rather within this string of the och ch-lil does not - public int index fint.ch, int from index): Returns the index within this string of the first Roe of character ch, starting the search at position from Index retums - 1 il ch does not in this String - public int indexor (String ste): Returns the index within this string of the first currence of ringt-d eur - public int indexOf(stringate, int from index): Resures the index within the string of the fare currence of shariaparting police froiden liste des not count i ng - public String subat ring it begintndax) Rema new string which is a webstring of this string compend of the chesting at position begintndex (inclusive) - puble string subat ring intaginindex intendindex: Returns a new sering that waring of this string, composed of the characters starting at position beginIndex (inclusive, and ending at - public String replace(char oldChar, char newchar): Returns a new string resulting from replacing a Tences of olchar in this string with new Char. puble string tolon Case(): Returns a new string consisting of all the characters in this sting - publle ting toppe : Roum ingiting of all the c on in this thing uverted to perce - publie ting tet): Returns a copy of this line with leading and wailing whitespace • Scanner (package java.util) Methods: - publie danner File source) thrown jatorio Foundation sa Scanner the fun the specified file - public scanner InputStream source): Cits Scanner that process and from the input stream - puble Scannerstag source): Corsa ww Scanner that produces wales w ed from the specified String -puble vold close(): Cwes this Scanner -puble boolean asext(): Return this can het in its input puble boolean asext.Double(): Return to the taken in this season al ante pasadala w ith Doble() ed publie boolean hat Into: Reutte le inse gule ed wat w eet Int() had - public boolean hati : Return true if this w ee - puble boolean has longt) Return true if the r in this cannerm ane das along the next long) with puble ting at the t e rm from the canner puble double Smerthea ter puble intention there the l ot. puble w ith the need them - puble Intolog m u long • Printstream ( kape Java.io) Methods: a - Pri le rlle) throw alo.Pilo t Pelat ihan the p rile - Printing the throw r alo. Pet Stichw e d the wind - le void lose them - le void phone boolean ve - void prichar public void print chart 6): the way of char public void print double d o uble bed, public void print (int ) Fins int value - public void printet a n net public void printing : Strings - public void printin (1 Tm the line by writing the line para ring - public void printinhoolna ) Prints hoolean when thinorminates the public void printin (cthar c h ar wheelthening the line public void print in char i ta way of cars and then the line public void print in double d in double valued and the terminates the line public void printin(int 1) Pints int value and then into the line public void printin ( jat ova t terminates the line pulle void print intring): Tits Strings then to the line Note that the printwriter class defines the same methods and constructors (except for the fact that the structors are called Print.writer instead of PrintStream) • Math (package java.lang) Methods: - please le podle double bear the value of a t the of - puble balle blanche a Returns they u se of dele with pe r than realto - pietate double random Ruma doble 0.0 wi th .. puble stat la doble sin double a: Rethe puble statie doble con double a Return ther l e, where a ininin we angle, wherea in in e -puble statie doble tan double a) R he i ngold , whereas o m public statie double toeges double agrad): C a ngrad v ideo puble salle de anable and we a r e - - estatie delle au t olewa idle de ahholooble • Character (packape java.lang) Methods: - public static boolean ibigit charch: D i chtechisadil public static int digit(charch, int radix: Return the meal of cho ch in the di radix-lich des representa digit - public statie char forbigit chat digit, ut radix) Rems the character of digit in the disa - public statie boolean isletter charch): Demines if character chalet publle statie boolean islowerCase(char ch): Determines if character ch is a character - public static boolean ipparate(char eh): Determines ir ductes chinuppercase cura nd - publicatie boliwhip (arch Termine if character his white a intola - public static char tolowrane(char chevert hech to low - public static char tolpperCase(char ch): Covers Chococh topp • Array list (package java.util) Methods: - public ArrayList(): Create a new em ArrayList which contains clements of type - public int (1 : R m the number of clans in this list - public boolean isimpty(): Rens true if this in contains no elements - public boolean contains (object o): Return true if this lintutins el compr a re performed using the equals() method on - public int IndexOf(obj ): Returns the index of the first currence of elemento in this list, or lir this listes contain this element companies are performed wing the equals() method - publiek galimt in ): Returns the elementat position index in this list - publieket fint inde element): Replace the element at the position index in this list with the - puble honlanad a ): Appends the specifiedalent to the end of this list. - public void (int in aw e t the specified element the price index in this - publiek veint index): Remove the de s ition index in the list - publie hoolean remove(object a s the fir e of the president of this if it is p r epared wing the equals() methodno - public void clear(): Remes all of the elements from this • Vile (package java.io) Methods: - puble File(String pathname: Creates a File representing the file at the given pathane

CSE 150 Lecture Notes - Lecture 12: Mixture Model, Bigram, 96.5 Wave Fm

Document Summary

Get access

Related Documents

CSE 150 Lecture Notes - Lecture 10: Posterior Probability, Bayes Estimator, Backtracking

MAD 3512 Study Guide - Midterm Guide: Dfa Records, Turtle Island Quartet, Concatenation

COMP 335 Lecture Notes - Lecture 3: Tape Head, Control Unit, Substring

Related Questions