Skip to content


Pyparsing introduction: BNF to code

Finally we can wrap this together in our calculator script. Here’s a possible implementation:

from pyparsing import Word, nums, Literal, ParseException
from string import lowercase

class Calculator(object):
    nonzero = ''.join([str(i) for i in range(1, 10)])
    integer = Word(nonzero, nums)
    varname = Word(lowercase)
    equals = Literal('=').suppress()
    operator = Word('+-*/', exact=1)
    operand = integer ^ varname
    unaryoperation = operand
    binaryoperation = operand + operator + operand
    operation = unaryoperation ^ binaryoperation
    expression = varname + equals + operation

    def __init__(self):
        self._state = dict()

    def execute_command(self, cmd):
        try:
            parts = self._parse_command(cmd)
        except ParseException, err:
            print 'Exception while parsing command: %s' % err
            return

        if len(parts) == 2:
            self._do_basic_assignment(parts)
        else:
            self._do_calculated_assignment(parts)

    def dump_state(self):
        print self._state

    def _parse_command(self, cmd):
        return self.expression.parseString(cmd)

    def _do_basic_assignment(self, parts):
        value = self._get_value(parts[1])
        if value is None:
            print 'Unable to execute command'
            return
        self._state[parts[0]] = value

    def _get_value(self, s):
        value = None

        try:
            value = int(s)
        except ValueError:
            pass

        if value is None:
            try:
                value = self._state[s]
            except KeyError:
                print 'Unknown variable: %s' % s
                return None

        return value

    def _do_calculated_assignment(self, parts):
        op1 = parts[1]
        op2 = parts[3]
        operator = parts[2]

        op1 = self._get_value(op1)
        op2 = self._get_value(op2)

        if op1 is None or op2 is None:
            print 'Unable to execute command'
            return

        funcs = {
            '+': lambda a, b: a + b,
            '-': lambda a, b: a - b,
            '*': lambda a, b: a * b,
            '/': lambda a, b: a / b,
        }

        self._state[parts[0]] = funcs[operator](op1, op2)

def main():
    print 'Press ^C to quit'
    print

    calc = Calculator()

    while True:
        cmd = raw_input('> ')
        calc.execute_command(cmd)
        calc.dump_state()

if __name__ == '__main__':
    main()

Here’s a test-run:

Press ^C to quit

> a = 1
{'a': 1}
> b = 2
{'a': 1, 'b': 2}
> c = a + b
{'a': 1, 'c': 3, 'b': 2}
> d = c + 1
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
> e = f + g
Unknown variable: f
Unknown variable: g
Unable to execute command
{'a': 1, 'c': 3, 'b': 2, 'd': 4}

That’s all for now. I’ll write more about Pyparsing later on, as we only scratched the top of it’s capabilities for now. Stay tuned!

Posted in Development.

Tagged with , , , .


8 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. sb says

    Six pages?? Ouch. And no next button. Any chance you could put it all on one page next time? I feel like I’m reading some ad-infested hardware blog.

  2. jauco says

    very nice tutorial! thanks!

  3. Olivier Berger says

    What’s the difference to other parser systems like simpleparse ?

    Regards,

  4. Francis says

    I don’t see the difference in (except for the whitespaces)

    print sentence.parseString(‘hello world’) # notice >1 spaces
    # returns ['hello', 'world']
    print sentence.parseString(‘Hello world’)
    # raises a ParseException

    Why does the second one raise an exception ?

  5. Nicolas says

    Francis: I guess you’re referring to the snippet on page 2? It says:

    from pyparsing import OneOrMore
    sentence = OneOrMore(word)

    The definition of ‘word’ is given on the previous page:

    word = Word(lowercase)

    where ‘lowercase’ is imported from the ‘string’ module and equals

    abcdefghijklmnopqrstuvwxyz

    The definition of the BNF type ‘word’ is Word(lowercase), ie. a concatenation of any character in the string (or list, so you want) ‘lowercase’, which is a-z.

    A sentence is defined as OneOrMore words.

    The string ‘Hello world’ can not be parsed since it does not match OneOrMore(word): the first item in it (‘Hello’) contains characters not matching the definition of word: the ‘H’ (since we defined a word to be a concatenation of lowercase characters, it shouldn’t contain any uppercase characters).

    As you can see, on page 3 a better definition of sentence is constructed using a ‘startword’ definition which should be a concatenation of one uppercase character, followed by zero or more lowercase characters.The example shows ‘A valid sentence.’ can be parsed and validated. The string ‘Hello world!’ would be valid in this BNF construct too. ‘Hello world’ would not match since we’re missing a punctuation sign.

    Using the definitions from page 3

    almost_valid_sentence = startword + body

    or (even more limited)

    hello_caps = startword + word

    would validate and parse ‘Hello world’.

  6. GDR! says

    Good introduction – thank you!

    Although I share the feelings of “sb” about pagination.

  7. lerry says

    hi poh,, what if the expr is like this A=B+c?

  8. Wayne says

    Good introduction to pyparsing. Thanks Nicolas!



Some HTML is OK

or, reply to this post via trackback.