Syntactic Analysis - Information Sciences Institutepedro/Teaching/CSCI565-Spring16/Lectures/... ·...

Spring 2016CSCI 565 - Compiler Design

Pedro Dinizpedro@isi.edu

Syntactic Analysis

Implementing a ParserLR parsing tables

Copyright 2016, Pedro C. Diniz, all rights reserved.Students enrolled in Compilers class at University of Southern California (USC)have explicit permission to make copies of these materials for their personal use.

Pedro Dinizpedro@isi.edu 2

Outline

• Implementing a Parser • Shift-Reduce Parser Example• Why is it hard to build a Parser Engine?• LR(k) Parser Tables

• Different Parsing Techniques– Each can handle some set of CFGs– Categorization of techniques

Implementing a Parser

– L - parse from left to right– R - parse from right to left

– L - leftmost derivation– R - rightmost derivation

– Number of lookahead characters

– Examples: LL(0), LR(1)

• We will be studying LR(k) parsers

( )L R k

• Implementing a Parser• Shift-Reduce Parser Example• Why is it hard to build a parser engine?• LR(k) parser tables• Constructing a LR(0) Parser Engine

Outline

Why use a LR(k) parser• Can be construct to recognize a large class of CFGs

– virtually all programming language constructs

• Most general non-backtracking parsing method

• Can build a very efficient very parser engine

• Can detect a syntactic error as soon as it is possible to do so

Let’s look at a Parser Implementation

• Workings of a LR(k) Parser• Parse from Left to Right • Rightmost Derivation

– Starts with the entire String– Ends with the Start Symbol

Actions of a Shift-Reduce Parser

ParseTree

ParseTreeParseTree

ParseTree

• Cannot create a full sub parse tree • Need the look ahead information• Thus, keep some state

ParseTree

teParseTree

Current Symbol

StackParser A

ParserEngine

• Shift– Shift the current element on top of the stack– Move the current input pointer

• Reduce– Apply a production– Top of the stack should match the RHS of a production– Remove those symbols from the stack– Add the LHS non-terminal to the stack

• Accept– End of stream reached &– Stack only has the start symbol

• Reject– End of stream reached but – stack has more than the start symbol

Shift-Reduce Parser Example

* ( + num )numnum

<expr> → num<op> → +<op> → -<op> → *

* ( + num )numnum

<expr> → num<op> → +<op> → -<op> → *

* ( + num )num

<expr> → num<op> → +<op> → -<op> → *

* ( + num )numnum

<expr> → num<op> → +<op> → -<op> → *

* ( + num )numnum

<expr> → num<op> → +<op> → -<op> → *

* ( + num )numnum

<expr> → num<op> → +<op> → -<op> → *

* ( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

<expr>

* ( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

<expr>

( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

*<expr>

( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

*<expr>

( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

*<expr>

( + num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

+ num )num

<expr>

<expr> → num<op> → +<op> → -<op> → *

+ num )num

<expr>

num * (

<expr> → num<op> → +<op> → -<op> → *

<expr>

+ num )num

<expr>

num * (

<expr> → num<op> → +<op> → -<op> → *

<expr>

* + num )

<expr>

num ( num

<expr> → num<op> → +<op> → -<op> → *

<expr>

* + num )

<expr>

num ( num

<expr> → num<op> → +<op> → -<op> → *

<expr>

* + num )

<expr>

num ( num

<expr> → num<op> → +<op> → -<op> → *

<expr>

* + num )

<expr>

num ( num

<expr> → num<op> → +<op> → -<op> → *

<expr>

* num )

<expr>

num ( num

<expr> → num<op> → +<op> → -<op> → *

<expr>

* num )

<expr>

num ( num +

<expr> → num<op> → +<op> → -<op> → *

<expr>+

<expr>

* num )

<expr>

num ( num +

<expr> → num<op> → +<op> → -<op> → *

<expr>+

<expr>

* num )

<expr>

num ( num +

<expr> → num<op> → +<op> → -<op> → *

<expr>+

<expr>

* num )

<expr>

num ( num +

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num +

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr><op>num

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr><op>num

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr><op>num

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>)

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>)

<expr>

num ( num + num

<expr> → <expr> <op> <expr><expr> → ( <expr> )

<expr> → - <expr><expr> → num

<op> → +<op> → -<op> → *

<expr>)

<expr>

num ( num + num

<expr> → - <expr><expr> → num

<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr> → num<op> → +<op> → -<op> → *

<expr>

num ( num + num

<expr>

num ( num + num

<expr>

num ( num + num

<expr>

What does the Parser Engine do?• If the top symbols of the stack match an RHS

of a production do the reduction – Pop the RHS from the top of the stack– Push the LHS symbol onto the stack

• If no production is found do the shift– Push the current input into the stack

• If the input is empty– Accept if only the start symbol is on the stack– Reject otherwise Parser

Engine

Outline• Implementing a Parser • Shift-Reduce Parser Example• Why is it Hard to build a Parser Engine?• LR(k) parser tables• Constructing a LR(0) Parser Engine

What does the Parser Engine do?• If the Top Symbols of the Stack Match an RHS

of a Production do the Reduction – Pop the RHS from the top of the stack– Push the LHS symbol onto the stack

• If No Production is Found do the Shift– Push the current input into the stack

• If the Input is Empty– Accept if only the start symbol is on the stack– Reject otherwise Parser

Engine

This is not that Simple!

• Many Choices of Reductions– Matches multiple RHS

• Choice between Shift and Reduce– Stack matches a RHS– But that may not be the right match– May need to shift and later find a different reduction

<expr> → - <expr><expr> → num<op> → +<op> → -<op> → *

• Change in the Grammar

<expr> → - <expr><expr> → num<op> → +<op> → -<op> → *

<expr> → <expr> -<expr> → num<op> → +<op> → -<op> → *

- numnum

<expr> → <expr> <op> <expr><expr> → ( <expr> )<expr> → <expr> -

<expr> → num<op> → +<op> → -<op> → *

- numnumnum

<expr> → num<op> → +<op> → -<op> → *

numnum

<expr> → num<op> → +<op> → -<op> → *

numnum

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum

<expr>

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

We have a choice!!!<op><expr>

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

But not the right thing to do!!<op><expr>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

But not the right thing to do!!<expr>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

num<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

num<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

num<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

But not the right thing to do!!No more actions!!!

<expr>

numnumnum -

• But this is perfectly valid input for the grammar• We chose a wrong production • Lets see what is the right production

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

We have a choice<op><expr>

The step before we went wrong

-<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

Use the other production

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<op> Use the other production

<expr> → num<op> → +<op> → -<op> → *

numnum -

num<op>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

num<op>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

num<op>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

E<expr>

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

<expr> → num<op> → +<op> → -<op> → *

numnum -

<expr>

This is not that Simple!

• Many Choices of Reductions– Matches Multiple RHS

• Choice between Shift and Reduce– Stack Matches a RHS– But that may not be the ‘right’ match– May need to Shift and later find a different Reduction

• Keep additional information

Outline

• Implementing a Parser • Shift-Reduce Parser Example• Why is it hard to build a Parser Engine?• LR(k) parser tables• Constructing a LR(0) Parser Engine

Constructing a LR(k) Parser

• We will construct few LR(k) parsers– LR(0), – SLR (or simple LR)– LR(1)

Constructing a LR(k) Parser

• We will construct few LR(k) parsers– LR(0), – SLR (or simple LR)– LR(1)

• We followed the Parsing Actions• What is in the Parse Engine?

– Decision between shift and reduce– Decide on the “right” reduction

Current Symbol

StackParser A

ParserEngine

Constructing a LR(k) Parser• Create a DFA

– Encodes all the possible states that the parser can be in– DFA state transition occurs on terminals and non-terminals

• Create an Parser table – Stores what action should be taken for the current

state and current input character

• Maintain a Stack of States – in parallel with the stack of symbols

LR(k) Parser Engine

Current Symbol

Parser Action

LR(k)ParserEngine

ACTION GotoState ( ) $ X

s0 shift to s2 error error goto s1s1 error error accept s2 shift to s2 shift to s5 error goto s3s3 error shift to s4 error s4 reduce (2) reduce (2) reduce (2) s5 reduce (3) reduce (3) reduce (3)

Parser Tables

• Look-up [top of state stack] [ input symbol] in the Parser Table

• Carry-out the Corresponding Action

ACTION GotoState ( ) $ Xs0 shift to s2 error error goto s1s1 error error accepts2 shift to s2 shift to s5 error goto s3s3 error shift to s4 errors4 reduce (2) reduce (2) reduce (2)s5 reduce (3) reduce (3) reduce (3)

• Shift to sn– Push input token into the symbol stack– Push sn into state stack and move to state sn– Advance to next input symbol

Parser Tables

• Reduce (n)– Pop both stacks as many times as the number of symbols on the RHS of rule n– Push LHS of production onto the symbol stack – Let t be top of state stack

– Lookup [top of the state stack t][top of symbol stack] in Goto section of table– Push that state (in Goto k) into state stack and move to that state

Parser Tables

• Accept– Stop parsing and report success

Parser Tables

• Error– Stop parsing and report failure

Parser Tables

LR Example

• The grammar<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

Question

• The grammar<S> → <X>$ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

• What is the language accepted by this CFG?

Parser Table in Action• The grammar

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

Parser Table in ActionAction Table

• The grammar<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

Action GotoState ( ) $

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

reduce (2)reduce (3)

shift to s5shift to s4

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

Action Table

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

s5 )(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s5 )(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

s5 )(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

s5 )(s2(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

s5 )(s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s4 )s3

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s4 )s3

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s4 )s3

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s4 )s3

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $

Action Table

s0 goto s1

s1s2s3s4s5

goto s3

errorerrorerror

accept

errorerror

shift to s2

<S> → <X> $ (1)<X> → ( <X> ) (2)<X> → ( ) (3)

s0 $)( ( ) $ Accept

Action Table

Summary

• Parser Implementation• Shift-Reduce Parsing• Examples

Syntactic Analysis - Information Sciences Institutepedro/Teaching/CSCI565-Spring16/Lectures/... ·...

Documents

CSCI 565 Compiler Design and Implementation Spring …pedro/Teaching/CSCI565-Spring16/Syllabus/CSCI... · 1 CSCI 565 Compiler Design and Implementation Spring 2016 Instructor: Dr

CSCI 565 -Compiler Design Spring 2017 Intermediate …pedro/Teaching/CSCI565-Spring17/... · · 2017-02-16CSCI 565 -Compiler Design Spring 2017 Pedro Diniz ... – Compiler inserts

Compiler Design - Information Sciences Institutepedro/Teaching/CSCI565-Spring...Prof. Pedro C. Diniz USC / Information Sciences Institute 4676 Admiralty Way, Suite 1001 Marina del

Compiler Design - Information Sciences Institutepedro/Teaching/CSCI565-Spring17/Practice/SDT-Sample.pdf · Compiler Design Spring 2016 Syntax-Directed Translation Sample Exercises

Overview of the Class - Information Sciences Institutepedro/Teaching/CSCI565-Spring15/Lectures/ClassIntroduction.pdf · Class-Taking Technique for Compilers • I will use projected

Compiler Design - Information Sciences Institutepedro/Teaching/CSCI565-Spring11/... · Compiler Design ... Given the Syntax-Directed Definition below construct the annotated parse

Problem Solution - courses.physics.ucsd.educourses.physics.ucsd.edu/2010/Winter/physics2b/hw/hw8.pdf · its rightmost edge enters a uniform ... (We substituted the given numerical

Yacc: Yet Another Compiler-Compilerpedro/Teaching/CSCI565-Fall15/Materials/Yacc.pdf · Yacc: Yet Another Compiler-Compiler PS1:15-3 The next several sections describe the basic process

Compilers: Principles, Techniques, and Tools, Second ...pedro/Teaching/CSCI565-Spring15/Materials/Dragon... · use informal descriptions and ... Compilers: Principles, Techniques,

1. 2 3 4 Adders Used to perform addition, subtraction, multiplication, and division (sometimes) Half-adder adds rightmost (least significant) bit Full-adder

Run-Time Environments - Information Sciences Institutepedro/Teaching/CSCI565-Fall16/Materials/... · · 2016-08-17Run-Time Environments ... management and organization of this logical

Software testing techniques Spring 2009 Csci565: Theory and Practice of Software Testing 1 Csci565: Graph Theory for Testers

Compiler Design - Information Sciences Institutepedro/Teaching/CSCI565-Fall15/Practice/SDT... · Grammar G = {{a,b,c}, {S}, S, P} for all the strings

On-Campus and Online Law School | USC Gould …...360V), and ‘danger: severe shock’ (375 – 420V). The two rightmost switches (435V and The two rightmost switches (435V and 450V)

Compilers: Principles, Techniques, and Tools, Second ...pedro/Teaching/CSCI565-Spring... · Compilers: Principles, Techniques, and Tools, Second Edition, by Alfred V. Aho, Monica

2–1 Manipulating Rightmost Bits - hackersdelight.orghackersdelight.org/basics2.pdf · 2–1 MANIPULATING RIGHTMOST BITS 13 As an example of the application of these formulas, =

Program 1 worksheet - ORTOP€¦ · Web viewClick on the rightmost icon under the word LabVIEW in the upper right-hand corner of the window

4-wave mixing for phase-matching free nonlinear optics in ... · respectively) moves the electron from leftmost quantum well to rightmost quantum well. Upon reaching the rightmost

Efﬁcient iterative algorithms for linear stability ... · Linear stability analysis of a dynamical system entails ﬁnding the rightmost eigenvalue for a series of eigenvalue problems

1 Parse Trees and Ambiguity (Dr. Torng) Parse/Derivation Trees –Leftmost derivations, rightmost derivations Ambiguous Grammars –Examples Arithmetic expressions