Parsing XML with Python using lxml.objectify

A couple years ago I started a series of articles on XML parsing. I covered lxml’s etree and Python’s included minidom XML parsing library. For whatever reason I didn’t notice lxml’s objectify sub-package, but I saw it recently and decided I should check it out. To my mind, the objectify module seems to be even more “Pythonic” than etree is. Let’s take a some time and go over my old XML examples using objectify and see how it’s different!

Let’s Get This Party Started!

If you haven’t already, go out and download lxml or you won’t be able to follow along very well. Once you have it, we can continue. We’ll be using the following piece of XML for our parsing pleasure:



    
        1181251680
        040000008200E000
        1181572063
        
        
        1800
        Bring pizza home
    
    
        1234360800
        1800
        Check MS Office website for updates
        
        604f4792-eb89-478b-a14f-dd34d3cc6c21-1234360800
        dismissed
  

Now we need to write some code that can parse and modify the XML. Let’s take a look at this little demo that shows a bunch of the neat abilities that objectify provides.

from lxml import etree, objectify

#----------------------------------------------------------------------
def parseXML(xmlFile):
    """"""
    with open(xmlFile) as f:
        xml = f.read()
        
    root = objectify.fromstring(xml)
    
    # returns attributes in element node as dict
    attrib = root.attrib
    
    # how to extract element data
    begin = root.appointment.begin
    uid = root.appointment.uid
    
    # loop over elements and print their tags and text
    for appt in root.getchildren():
        for e in appt.getchildren():
            print "%s => %s" % (e.tag, e.text)
        print
     
    # how to change an element's text
    root.appointment.begin = "something else"
    print root.appointment.begin
    
    # how to add a new element
    root.appointment.new_element = "new data"
    
    # print the xml
    obj_xml = etree.tostring(root, pretty_print=True)
    print obj_xml
    
    # remove the py:pytype stuff
    #objectify.deannotate(root)
    etree.cleanup_namespaces(root)
    obj_xml = etree.tostring(root, pretty_print=True)
    print obj_xml
    
    # save your xml
    with open("new.xml", "w") as f:
        f.write(obj_xml)
    
#----------------------------------------------------------------------
if __name__ == "__main__":
    f = r'path\to\sample.xml'
    parseXML(f)

The code is pretty well commented, but we’ll spend a little time going over it anyway. First we pass it our sample XML file and objectify it. If you want to get access to a tag’s attributes, use the attrib property. It will return a dictionary of the attribute’s of the tag. To get to sub-tag elements, you just use dot notation. As you can see, to get to the begin tag’s value, we can just do something like this:

begin = root.appointment.begin

If you need to iterate over the children elements, you can use iterchildren. You may have to use a nested for loop structure to get everything. Changing an element’s value is as simple as just assigning it a new value. And if you need to create a new element, just add a period and the name of the new element (see below):

root.appointment.new_element = "new data"

When we add or change items using objectify, it will add some annotations to the XML, such as xmlns:py=”http://codespeak.net/lxml/objectify/pytype” py:pytype=”str”. You may not want that included, so you’ll have to call the following method to remove that stuff:

etree.cleanup_namespaces(root)

You can also use “objectify.deannotate(root)” to do some deannotation chores, but I wasn’t able to get it to work for this example. To save the new XML, you actually seem to need lxml’s etree module to convert it to a string so you can save it.

At this point, you should be able to parse most XML documents and edit them effectively with lxml’s objectify. I thought it was very intuitive and easy to pick up. Hopefully you will find it useful in your endeavors as well.

Further Reading

10 thoughts on “Parsing XML with Python using lxml.objectify”

  1. Maybe rather avoid using the word “builtin” for a stdlib module, since there is also a concept of “built-in” functions and since it’s a bit inaccurate/misleading.

  2. It seems like PyQuery is already abandoned. BugTracking list looks horrible. I found 2 bugs on myself today. I would not recommend it to anybody…

  3. Pingback: Python: Creating XML with lxml.objectify | Hello Linux

  4. I tried this example and I am not entirely sure what I am missing here. The for loop rather than parsing the entire file and printing all the tags prints the first section only. Can you let me know why this is :

    My code :

    ————————————-

    from lxml import etree, objectify

    #http://python.dzone.com/articles/using-lxmlobjectify-parse-xml

    def parseXML(xmlFile):

    with open(xmlFile) as f:

    xml = f.read()

    root = objectify.fromstring(xml)

    attrib = root.attrib

    #How to get the first begin & uid in the XML file

    begin = root.appointment.begin

    uid = root.appointment.uid

    print (“BeginID : %s & UID: %s ” % (begin, uid))

    print

    #how to iterate over all the tags & their text

    for e in root.appointment.iterchildren():

    print (“%s => %s ” % (e.tag, e.text))

    print

    #how to change an elements text

    root.appointment.begin = “something else”

    print (“Changed tag is : %s” % (root.appointment.begin))

    print

    #how to add a new element(tag)

    root.appointment.new_element = “new data”

    objectify.deannotate(root)

    etree.cleanup_namespaces(root)

    obj_xml = etree.tostring(root, pretty_print=True)

    print (obj_xml)

    with open(“example4.xml”, “w”) as f:

    f.write(obj_xml)

    #——————————————–

    if __name__ == “__main__”:

    f = r”example3.xml”

    parseXML(f)

    —————————————————————————-

    Output :

    >>>

    BeginID : 1181251680 & UID: 40000008200.0

    begin => 1181251680

    uid => 040000008200E000

    alarmTime => 1181572063

    state => None

    location => None

    duration => 1800

    subject => Bring pizza home

    Changes tag is : something else

    something else

    040000008200E000

    1181572063

    1800

    Bring pizza home

    new data

    1234360800

    1800

    Check MS Office website for updates

    604f4792-eb89-478b-a14f-dd34d3cc6c21-1234360800

    dismissed

  5. I’m not sure why I did it that way, but basically what that code says is that it is going to loop over just the first appointment tag, as you’ve found out. I actually need a nested loop to make it right. I have updated the code in this article to show how to get the data from both appointments.

  6. Hi Mike,
    Thanks, I admire your to the point explanation. Could you let me know when you would use lxml & objectify module rather than others out there.
    Thanks.

  7. I use lxml’s objectify most of the time because it makes working with nested / complex XML very easy to use and create. There’s nothing wrong with ElementTree, which is included with Python. I just don’t think it’s ease of use is as good as lxml’s

  8. Pingback: Python: Creating XML with lxml.objectify - The Mouse Vs. The Python

  9. Pingback: Python 101 - Intro to XML Parsing with ElementTree - The Mouse Vs. The Python

Comments are closed.