Parsing XML with Python using lxml.objectify

A couple years ago I started a series of articles on XML parsing. I covered lxml’s etree and Python’s included minidom XML parsing library. For whatever reason I didn’t notice lxml’s objectify sub-package, but I saw it recently and decided I should check it out. To my mind, the objectify module seems to be even more “Pythonic” than etree is. Let’s take a some time and go over my old XML examples using objectify and see how it’s different!

Let’s Get This Party Started!

If you haven’t already, go out and download lxml or you won’t be able to follow along very well. Once you have it, we can continue. We’ll be using the following piece of XML for our parsing pleasure:

    
        1181251680
        040000008200E000
        1181572063
        
        
        1800
        Bring pizza home
    
    
        1234360800
        1800
        Check MS Office website for updates
        
        604f4792-eb89-478b-a14f-dd34d3cc6c21-1234360800
        dismissed
  

Now we need to write some code that can parse and modify the XML. Let’s take a look at this little demo that shows a bunch of the neat abilities that objectify provides.

from lxml import etree, objectify

#----------------------------------------------------------------------
def parseXML(xmlFile):
    """"""
    with open(xmlFile) as f:
        xml = f.read()
        
    root = objectify.fromstring(xml)
    
    # returns attributes in element node as dict
    attrib = root.attrib
    
    # how to extract element data
    begin = root.appointment.begin
    uid = root.appointment.uid
    
    # loop over elements and print their tags and text
    for appt in root.getchildren():
        for e in appt.getchildren():
            print "%s => %s" % (e.tag, e.text)
        print
     
    # how to change an element's text
    root.appointment.begin = "something else"
    print root.appointment.begin
    
    # how to add a new element
    root.appointment.new_element = "new data"
    
    # print the xml
    obj_xml = etree.tostring(root, pretty_print=True)
    print obj_xml
    
    # remove the py:pytype stuff
    #objectify.deannotate(root)
    etree.cleanup_namespaces(root)
    obj_xml = etree.tostring(root, pretty_print=True)
    print obj_xml
    
    # save your xml
    with open("new.xml", "w") as f:
        f.write(obj_xml)
    
#----------------------------------------------------------------------
if __name__ == "__main__":
    f = r'path\to\sample.xml'
    parseXML(f)

The code is pretty well commented, but we’ll spend a little time going over it anyway. First we pass it our sample XML file and objectify it. If you want to get access to a tag’s attributes, use the attrib property. It will return a dictionary of the attribute’s of the tag. To get to sub-tag elements, you just use dot notation. As you can see, to get to the begin tag’s value, we can just do something like this:

begin = root.appointment.begin

If you need to iterate over the children elements, you can use iterchildren. You may have to use a nested for loop structure to get everything. Changing an element’s value is as simple as just assigning it a new value. And if you need to create a new element, just add a period and the name of the new element (see below):

root.appointment.new_element = "new data"

When we add or change items using objectify, it will add some annotations to the XML, such as xmlns:py=”http://codespeak.net/lxml/objectify/pytype” py:pytype=”str”. You may not want that included, so you’ll have to call the following method to remove that stuff:

etree.cleanup_namespaces(root)

You can also use “objectify.deannotate(root)” to do some deannotation chores, but I wasn’t able to get it to work for this example. To save the new XML, you actually seem to need lxml’s etree module to convert it to a string so you can save it.

At this point, you should be able to parse most XML documents and edit them effectively with lxml’s objectify. I thought it was very intuitive and easy to pick up. Hopefully you will find it useful in your endeavors as well.

Further Reading

• lxml Objectify documentation
• Example Parsing XML with lxml objectify
• Parsing XML with lxml etree