Кратко: запросы к API и разбор XML-ответов. Python
Этот пост предназначен в первую очередь для новичков в разработке, впервые столкнувшихся с необходимостью отправить post/get запросы к какому-нибудь API и проанализировать полученный в XML ответ. Постаралась собрать необходимы минимум в одном месте.
1. Непосредственно API
Оно может быть от ваших коллег, партнеров, заказчиков, сторонних сервисов. Разной степени готовности и актуальности. Но у него как правило есть заголовки запросов, параметры запросов, параметры ответа и статус-коды ответов. Например:
Если проверить сам метод на работоспособность, например с помощью curl — можно сэкономить себе много нервов и сил, особенно актуально если API допиливается одновременно с вашей разработкой. После чего можно воспользоваться например вот таким сервисом https://reqbin.com/req/python/c-xgafmluu/convert-curl-to-python-requests для того что бы перевести curl-запрос в код под либу requests.
Воспользуемся модулем CaseInsensitiveDict из requests.structures для того что бы собирать заголовки запроса в привычные headers. Параметры запроса определим в параметре files. Тогда для метода с первого скрина post-запрос будет выглядеть следующим образом:
Вариант с ответом в json в этой статье разбирать не буду, по нему написано очень много материалов, поэтому остановлюсь подробнее на XML.
2. XML. Чтение и разбор
После получения XML-портянки на этапе отладки не лишним будет проверить визуально данные в ней. Например с помощью сервиса https://jsonformatter.org/xml-parser.
В приходящем респонсе байтовая кодировка(необходимая для раскладывания xml по дереву) находится только в атрибуте content. перепишем для дальнейшей работы его в отдельную переменную responce_xml_content.
В дебагере очень похоже выглядит атрибут text у полученного responce, но он там существует как utf-8.
Большая часть мануалов по парсингу xml написана под чтение из файла и под библиотеку etree. И метод для строки из переменной fromstring в каждом классе работает несколько по разному.
Поэтому оптимальным считаю использование etree из модуля lxml. С ним проверка существования пользователя get-запросом и добавление пользователя post-запросом выглядит лаконично.
3. Анализ XML. XPath или Bs4
XPath — язык на котором необходимо будет доставать цепочки элементов. И на нем необходимо будет писать выражения для поиска элемента по параметрам ответа.
Примечательно что родители знают о своих детях, но дети не знают о своих родителях. Напрямую, задав путь до проверки атрибута элемента на пятом уровне вложенности, в случае если такой найден, забрать атрибут с его третьего уровня вложенности не получится.
Один из немногих мануалов по тому как с ним обращаться http://www.k-press.ru/cs/2001/2/XPath/XPath4.asp
Например для того что бы достать из списка пользователей id пользователя по искомому email, находящегося в атрибуте value в элементе в котором атрибут name со значением email выражение будет выглядеть так:
Есть еще вариант использовать для этого Bs4. Тогда тот же поиск будет выглядеть так:
4. Размещаем эти запросы на flask.
Рекоммендуется использовать стандартную структуру страниц.
В api.py расположим основной обработчик страниц. Для обработки post-запроса с json-ом и примером формы. Для валидации самым лаконичным решением будет pydantic,из которого потребуется BaseModel, ValidationError, validator
Если потребуется styles.css то располагаться ему следует в /static/css.
Стандартное место расположения шаблонов страниц — /templates. Base.html для наследования общего стиля.
Форму регистрации сделаем например в таком виде
После чего мы можем например отправлять post-запросы к нашему сервису, таким же образом, curl-ом, через postman или руками заполняя формы на веб-странице.
The lxml.etree Tutorial
This is a tutorial on XML processing with lxml.etree. It briefly overviews the main concepts of the ElementTree API, and some simple enhancements that make your life as a programmer easier.
For a complete reference of the API, see the generated API documentation.
A common way to import lxml.etree is as follows:
If your code only uses the ElementTree API and does not rely on any functionality that is specific to lxml.etree, you can also use (any part of) the following import chain as a fall-back to the original ElementTree:
To aid in writing portable code, this tutorial makes it clear in the examples which part of the presented API is an extension of lxml.etree over the original ElementTree API, as defined by Fredrik Lundh’s ElementTree library.
The Element class
An Element is the main container object for the ElementTree API. Most of the XML tree functionality is accessed through this class. Elements are easily created through the Element factory:
The XML tag name of elements is accessed through the tag property:
Elements are organised in an XML tree structure. To create child elements and add them to a parent element, you can use the append() method:
However, this is so common that there is a shorter and much more efficient way to do this: the SubElement factory. It accepts the same arguments as the Element factory, but additionally requires the parent as first argument:
To see that this is really XML, you can serialise the tree you have created:
Elements are lists
To make the access to these subelements easy and straight forward, elements mimic the behaviour of normal Python lists as closely as possible:
Prior to ElementTree 1.3 and lxml 2.0, you could also check the truth value of an Element to see if it has children, i.e. if the list of children is empty:
This is no longer supported as people tend to expect that a «something» evaluates to True and expect Elements to be «something», may they have children or not. So, many users find it surprising that any Element would evaluate to False in an if-statement like the above. Instead, use len(element), which is both more explicit and less error prone.
There is another important case where the behaviour of Elements in lxml (in 2.0 and later) deviates from that of lists and from that of the original ElementTree (prior to version 1.3 or Python 2.7/3.2):
In this example, the last element is moved to a different position, instead of being copied, i.e. it is automatically removed from its previous position when it is put in a different place. In lists, objects can appear in multiple positions at the same time, and the above assignment would just copy the item reference into the first position, so that both contain the exact same item:
Note that in the original ElementTree, a single Element object can sit in any number of places in any number of trees, which allows for the same copy operation as with lists. The obvious drawback is that modifications to such an Element will apply to all places where it appears in a tree, which may or may not be intended.
The upside of this difference is that an Element in lxml.etree always has exactly one parent, which can be queried through the getparent() method. This is not supported in the original ElementTree.
If you want to copy an element to a different position in lxml.etree, consider creating an independent deep copy using the copy module from Python’s standard library:
The siblings (or neighbours) of an element are accessed as next and previous elements:
Elements carry attributes as a dict
XML elements support attributes. You can create them directly in the Element factory:
Attributes are just unordered name-value pairs, so a very convenient way of dealing with them is through the dictionary-like interface of Elements:
For the cases where you want to do item lookup or have other reasons for getting a ‘real’ dictionary-like object, e.g. for passing it around, you can use the attrib property:
Note that attrib is a dict-like object backed by the Element itself. This means that any changes to the Element are reflected in attrib and vice versa. It also means that the XML tree stays alive in memory as long as the attrib of one of its Elements is in use. To get an independent snapshot of the attributes that does not depend on the XML tree, copy it into a dict:
Elements contain text
Elements can contain text:
In many XML documents (data-centric documents), this is the only place where text can be found. It is encapsulated by a leaf tag at the very bottom of the tree hierarchy.
However, if XML is used for tagged text documents such as (X)HTML, text can also appear between different elements, right in the middle of the tree:
Here, the <br/> tag is surrounded by text. This is often referred to as document-style or mixed-content XML. Elements support this through their tail property. It contains the text that directly follows the element, up to the next element in the XML tree:
The two properties .text and .tail are enough to represent any text content in an XML document. This way, the ElementTree API does not require any special text nodes in addition to the Element class, that tend to get in the way fairly often (as you might know from classic DOM APIs).
However, there are cases where the tail text also gets in the way. For example, when you serialise an Element from within the tree, you do not always want its tail text in the result (although you would still want the tail text of its children). For this purpose, the tostring() function accepts the keyword argument with_tail:
If you want to read only the text, i.e. without any intermediate tags, you have to recursively concatenate all text and tail attributes in the correct order. Again, the tostring() function comes to the rescue, this time using the method keyword:
Using XPath to find text
Another way to extract the text content of a tree is XPath, which also allows you to extract the separate text chunks into a list:
If you want to use this more often, you can wrap it in a function:
Note that a string result returned by XPath is a special ‘smart’ object that knows about its origins. You can ask it where it came from through its getparent() method, just as you would with Elements:
You can also find out if it’s normal text content or tail text:
While this works for the results of the text() function, lxml will not tell you the origin of a string value that was constructed by the XPath functions string() or concat():
Tree iteration
For problems like the above, where you want to recursively traverse the tree and do something with its elements, tree iteration is a very convenient solution. Elements provide a tree iterator for this purpose. It yields elements in document order, i.e. in the order their tags would appear if you serialised the tree to XML:
If you know you are only interested in a single tag, you can pass its name to iter() to have it filter for you. Starting with lxml 3.0, you can also pass more than one tag to intercept on multiple tags during iteration.
By default, iteration yields all nodes in the tree, including ProcessingInstructions, Comments and Entity instances. If you want to make sure only Element objects are returned, you can pass the Element factory as tag parameter:
Note that passing a wildcard "*" tag name will also yield all Element nodes (and only elements).
In lxml.etree, elements provide further iterators for all directions in the tree: children, parents (or rather ancestors) and siblings.
Serialisation
Serialisation commonly uses the tostring() function that returns a string, or the ElementTree.write() method that writes to a file, a file-like object, or a URL (via FTP PUT or HTTP POST). Both calls accept the same keyword arguments like pretty_print for formatted output or encoding to select a specific output encoding other than plain ASCII:
Note that pretty printing appends a newline at the end.
For more fine-grained control over the pretty-printing, you can add whitespace indentation to the tree before serialising it, using the indent() function (added in lxml 4.5):
In lxml 2.0 and later (as well as ElementTree 1.3), the serialisation functions can do more than XML serialisation. You can serialise to HTML or extract the text content by passing the method keyword:
As for XML serialisation, the default encoding for plain text serialisation is ASCII:
Here, serialising to a Python unicode string instead of a byte string might become handy. Just pass the name 'unicode' as encoding:
The ElementTree class
An ElementTree is mainly a document wrapper around a tree with a root node. It provides a couple of methods for serialisation and general document handling.
An ElementTree is also what you get back when you call the parse() function to parse files or file-like objects (see the parsing section below).
One of the important differences is that the ElementTree class serialises as a complete document, as opposed to a single Element. This includes top-level processing instructions and comments, as well as a DOCTYPE and other DTD content in the document:
In the original xml.etree.ElementTree implementation and in lxml up to 1.3.3, the output looks the same as when serialising only the root Element:
This serialisation behaviour has changed in lxml 1.3.4. Before, the tree was serialised without DTD content, which made lxml lose DTD information in an input-output cycle.
Parsing from strings and files
lxml.etree supports parsing XML in a number of ways and from all important sources, namely strings, files, URLs (http/ftp) and file-like objects. The main parse functions are fromstring() and parse(), both called with the source as first argument. By default, they use the standard parser, but you can always pass a different parser as second argument.
The fromstring() function
The fromstring() function is the easiest way to parse a string:
The XML() function
The XML() function behaves like the fromstring() function, but is commonly used to write XML literals right into the source:
There is also a corresponding function HTML() for HTML literals.
The parse() function
The parse() function is used to parse from files and file-like objects.
As an example of such a file-like object, the following code uses the BytesIO class for reading from a string instead of an external file. That class comes from the io module in Python 2.6 and later. In older Python versions, you will have to use the StringIO class from the StringIO module. However, in real life, you would obviously avoid doing this all together and use the string parsing functions above.
Note that parse() returns an ElementTree object, not an Element object as the string parser functions:
The reasoning behind this difference is that parse() returns a complete document from a file, while the string parsing functions are commonly used to parse XML fragments.
The parse() function supports any of the following sources:
- an open file object (make sure to open it in binary mode)
- a file-like object that has a .read(byte_count) method returning a byte string on each call
- a filename string
- an HTTP or FTP URL string
Note that passing a filename or URL is usually faster than passing an open file or file-like object. However, the HTTP/FTP client in libxml2 is rather simple, so things like HTTP authentication require a dedicated URL request library, e.g. urllib2 or requests. These libraries usually provide a file-like object for the result that you can parse from while the response is streaming in.
Parser objects
By default, lxml.etree uses a standard parser with a default setup. If you want to configure the parser, you can create a new instance:
This creates a parser that removes empty text between tags while parsing, which can reduce the size of the tree and avoid dangling tail text if you know that whitespace-only content is not meaningful for your data. An example:
Note that the whitespace content inside the <b> tag was not removed, as content at leaf elements tends to be data content (even if blank). You can easily remove it in an additional step by traversing the tree:
See help(etree.XMLParser) to find out about the available parser options.
Incremental parsing
lxml.etree provides two ways for incremental step-by-step parsing. One is through file-like objects, where it calls the read() method repeatedly. This is best used where the data arrives from a source like urllib or any other file-like object that can provide data on request. Note that the parser will block and wait until data becomes available in this case:
The second way is through a feed parser interface, given by the feed(data) and close() methods:
Here, you can interrupt the parsing process at any time and continue it later on with another call to the feed() method. This comes in handy if you want to avoid blocking calls to the parser, e.g. in frameworks like Twisted, or whenever data comes in slowly or in chunks and you want to do other things while waiting for the next chunk.
After calling the close() method (or when an exception was raised by the parser), you can reuse the parser by calling its feed() method again:
Event-driven parsing
Sometimes, all you need from a document is a small fraction somewhere deep inside the tree, so parsing the whole tree into memory, traversing it and dropping it can be too much overhead. lxml.etree supports this use case with two event-driven parser interfaces, one that generates parser events while building the tree (iterparse), and one that does not build the tree at all, and instead calls feedback methods on a target object in a SAX-like fashion.
Here is a simple iterparse() example:
By default, iterparse() only generates events when it is done parsing an element, but you can control this through the events keyword argument:
Note that the text, tail, and children of an Element are not necessarily present yet when receiving the start event. Only the end event guarantees that the Element has been parsed completely.
It also allows you to .clear() or modify the content of an Element to save memory. So if you parse a large tree and you want to keep memory usage small, you should clean up parts of the tree that you no longer need. The keep_tail=True argument to .clear() makes sure that (tail) text content that follows the current element will not be touched. It is highly discouraged to modify any content that the parser may not have completely read through yet.
A very important use case for iterparse() is parsing large generated XML files, e.g. database dumps. Most often, these XML formats only have one main data item element that hangs directly below the root node and that is repeated thousands of times. In this case, it is best practice to let lxml.etree do the tree building and only to intercept on exactly this one Element, using the normal tree API for data extraction.
If, for some reason, building the tree is not desired at all, the target parser interface of lxml.etree can be used. It creates SAX-like events by calling the methods of a target object. By implementing some or all of these methods, you can control which events are generated:
You can reuse the parser and its target as often as you like, so you should take care that the .close() method really resets the target to a usable state (also in the case of an error!).
Namespaces
The ElementTree API avoids namespace prefixes wherever possible and deploys the real namespace (the URI) instead:
The notation that ElementTree uses was originally brought up by James Clark. It has the major advantage of providing a universally qualified name for a tag, regardless of any prefixes that may or may not have been used or defined in a document. By moving the indirection of prefixes out of the way, it makes namespace aware code much clearer and easier to get right.
As you can see from the example, prefixes only become important when you serialise the result. However, the above code looks somewhat verbose due to the lengthy namespace names. And retyping or copying a string over and over again is error prone. It is therefore common practice to store a namespace URI in a global variable. To adapt the namespace prefixes for serialisation, you can also pass a mapping to the Element factory function, e.g. to define the default namespace:
You can also use the QName helper class to build or split qualified tag names:
lxml.etree allows you to look up the current namespaces defined for a node through the .nsmap property:
Note, however, that this includes all prefixes known in the context of an Element, not only those that it defines itself.
Therefore, modifying the returned dict cannot have any meaningful impact on the Element. Any changes to it are ignored.
Namespaces on attributes work alike, but as of version 2.3, lxml.etree will ensure that the attribute uses a prefixed namespace declaration. This is because unprefixed attribute names are not considered being in a namespace by the XML namespace specification (section 6.2), so they may end up losing their namespace on a serialise-parse roundtrip, even if they appear in a namespaced element.
You can also use XPath with fully qualified names:
For convenience, you can use "*" wildcards in all iterators of lxml.etree, both for tag names and namespaces:
To look for elements that do not have a namespace, either use the plain tag name or provide the empty namespace explicitly:
The E-factory
The E-factory provides a simple and compact syntax for generating XML and HTML:
Element creation based on attribute access makes it easy to build up a simple vocabulary for an XML language:
One such example is the module lxml.html.builder, which provides a vocabulary for HTML.
When dealing with multiple namespaces, it is good practice to define one ElementMaker for each namespace URI. Again, note how the above example predefines the tag builders in named constants. That makes it easy to put all tag declarations of a namespace into one Python module and to import/use the tag name constants from there. This avoids pitfalls like typos or accidentally missing namespaces.
ElementPath
The ElementTree library comes with a simple XPath-like path language called ElementPath. The main difference is that you can use the
In addition to a full XPath implementation, lxml.etree supports the ElementPath language in the same way ElementTree does, even using (almost) the same implementation. The API provides four methods here that you can find on Elements and ElementTrees:
- iterfind() iterates over all Elements that match the path expression
- findall() returns a list of matching Elements
- find() efficiently returns only the first match
- findtext() returns the .text content of the first match
Here are some examples:
Find a child of an Element:
Find an Element anywhere in the tree:
Find Elements with a certain attribute:
In lxml 3.4, there is a new helper to generate a structural ElementPath expression for an Element:
As long as the tree is not modified, this path expression represents an identifier for a given element that can be used to find() it in the same tree later. Compared to XPath, ElementPath expressions have the advantage of being self-contained even for documents that use namespaces.
The .iter() method is a special case that only finds specific tags in the tree by their name, not based on a path. That means that the following commands are equivalent in the success case:
Note that the .find() method simply returns None if no match is found, whereas the other two examples would raise a StopIteration exception.
How to parse XML and get instances of a particular node attribute?
I have many rows in XML and I’m trying to get instances of a particular node attribute.
How do I access the values of the attribute foobar ? In this example, I want "1" and "2" .
19 Answers 19
I suggest ElementTree . There are other compatible implementations of the same API, such as lxml , and cElementTree in the Python standard library itself; but, in this context, what they chiefly add is even more speed — the ease of programming part depends on the API, which ElementTree defines.
First build an Element instance root from the XML, e.g. with the XML function, or by parsing a file with something like:
Or any of the many other ways shown at ElementTree . Then do something like:
minidom is the quickest and pretty straight forward.
There are many options out there. cElementTree looks excellent if speed and memory usage are an issue. It has very little overhead compared to simply reading in the file using readlines .
The relevant metrics can be found in the table below, copied from the cElementTree website:
As pointed out by @jfs, cElementTree comes bundled with Python:
- Python 2: from xml.etree import cElementTree as ElementTree .
- Python 3: from xml.etree import ElementTree (the accelerated C version is used automatically).
I suggest xmltodict for simplicity.
It parses your XML to an OrderedDict;
Taking your sample text:
Python has an interface to the expat XML parser.
It’s a non-validating parser, so bad XML will not be caught. But if you know your file is correct, then this is pretty good, and you’ll probably get the exact info you want and you can discard the rest on the fly.
Just to add another possibility, you can use untangle, as it is a simple xml-to-python-object library. Here you have an example:
Your XML file (a little bit changed):
Accessing the attributes with untangle :
The output will be:
More information about untangle can be found in «untangle».
Also, if you are curious, you can find a list of tools for working with XML and Python in «Python and XML». You will also see that the most common ones were mentioned by previous answers.
I might suggest declxml.
Full disclosure: I wrote this library because I was looking for a way to convert between XML and Python data structures without needing to write dozens of lines of imperative parsing/serialization code with ElementTree.
With declxml, you use processors to declaratively define the structure of your XML document and how to map between XML and Python data structures. Processors are used to for both serialization and parsing as well as for a basic level of validation.
Parsing into Python data structures is straightforward:
Which produces the output:
You can also use the same processor to serialize data to XML
Which produces the following output
If you want to work with objects instead of dictionaries, you can define processors to transform data to and from objects as well.
Name already in use
cpython / Doc / library / xml.etree.elementtree.rst
- Go to file T
- Go to line L
- Copy path
- Copy permalink
30 contributors
Users who have contributed to this file
- Open with Desktop
- View raw
- Copy raw contents Copy raw contents
Copy raw contents
Copy raw contents
The :mod:`xml.etree.ElementTree` module implements a simple and efficient API for parsing and creating XML data.
The :mod:`xml.etree.ElementTree` module is not secure against maliciously constructed data. If you need to parse untrusted or unauthenticated data see :ref:`xml-vulnerabilities` .
This is a short tutorial for using :mod:`xml.etree.ElementTree` ( ET in short). The goal is to demonstrate some of the building blocks and basic concepts of the module.
XML tree and elements
XML is an inherently hierarchical data format, and the most natural way to represent it is with a tree. ET has two classes for this purpose — :class:`ElementTree` represents the whole XML document as a tree, and :class:`Element` represents a single node in this tree. Interactions with the whole document (reading and writing to/from files) are usually done on the :class:`ElementTree` level. Interactions with a single XML element and its sub-elements are done on the :class:`Element` level.
We’ll be using the following XML document as the sample data for this section:
We can import this data by reading from a file:
Or directly from a string:
:func:`fromstring` parses XML from a string directly into an :class:`Element` , which is the root element of the parsed tree. Other parsing functions may create an :class:`ElementTree` . Check the documentation to be sure.
As an :class:`Element` , root has a tag and a dictionary of attributes:
It also has children nodes over which we can iterate:
Children are nested, and we can access specific child nodes by index:
Not all elements of the XML input will end up as elements of the parsed tree. Currently, this module skips over any XML comments, processing instructions, and document type declarations in the input. Nevertheless, trees built using this module’s API rather than parsing from XML text can have comments and processing instructions in them; they will be included when generating XML output. A document type declaration may be accessed by passing a custom :class:`TreeBuilder` instance to the :class:`XMLParser` constructor.
Pull API for non-blocking parsing
Most parsing functions provided by this module require the whole document to be read at once before returning any result. It is possible to use an :class:`XMLParser` and feed data into it incrementally, but it is a push API that calls methods on a callback target, which is too low-level and inconvenient for most needs. Sometimes what the user really wants is to be able to parse XML incrementally, without blocking operations, while enjoying the convenience of fully constructed :class:`Element` objects.
The most powerful tool for doing this is :class:`XMLPullParser` . It does not require a blocking read to obtain the XML data, and is instead fed with data incrementally with :meth:`XMLPullParser.feed` calls. To get the parsed XML elements, call :meth:`XMLPullParser.read_events` . Here is an example:
The obvious use case is applications that operate in a non-blocking fashion where the XML data is being received from a socket or read incrementally from some storage device. In such cases, blocking reads are unacceptable.
Because it’s so flexible, :class:`XMLPullParser` can be inconvenient to use for simpler use-cases. If you don’t mind your application blocking on reading XML data but would still like to have incremental parsing capabilities, take a look at :func:`iterparse` . It can be useful when you’re reading a large XML document and don’t want to hold it wholly in memory.
Finding interesting elements
:class:`Element` has some useful methods that help iterate recursively over all the sub-tree below it (its children, their children, and so on). For example, :meth:`Element.iter` :
:meth:`Element.findall` finds only elements with a tag which are direct children of the current element. :meth:`Element.find` finds the first child with a particular tag, and :attr:`Element.text` accesses the element’s text content. :meth:`Element.get` accesses the element’s attributes:
More sophisticated specification of which elements to look for is possible by using :ref:`XPath <elementtree-xpath>` .
Modifying an XML File
:class:`ElementTree` provides a simple way to build XML documents and write them to files. The :meth:`ElementTree.write` method serves this purpose.
Once created, an :class:`Element` object may be manipulated by directly changing its fields (such as :attr:`Element.text` ), adding and modifying attributes ( :meth:`Element.set` method), as well as adding new children (for example with :meth:`Element.append` ).
Let’s say we want to add one to each country’s rank, and add an updated attribute to the rank element:
Our XML now looks like this:
We can remove elements using :meth:`Element.remove` . Let’s say we want to remove all countries with a rank higher than 50:
Note that concurrent modification while iterating can lead to problems, just like when iterating and modifying Python lists or dicts. Therefore, the example first collects all matching elements with root.findall() , and only then iterates over the list of matches.
Our XML now looks like this:
Building XML documents
The :func:`SubElement` function also provides a convenient way to create new sub-elements for a given element:
Parsing XML with Namespaces
If the XML input has namespaces, tags and attributes with prefixes in the form prefix:sometag get expanded to
Here is an XML example that incorporates two namespaces, one with the prefix «fictional» and the other serving as the default namespace:
One way to search and explore this XML example is to manually add the URI to every tag or attribute in the xpath of a :meth:`
A better way to search the namespaced XML example is to create a dictionary with your own prefixes and use those in the search functions:
These two approaches both output:
This module provides limited support for XPath expressions for locating elements in a tree. The goal is to support a small subset of the abbreviated syntax; a full XPath engine is outside the scope of the module.
Here’s an example that demonstrates some of the XPath capabilities of the module. We’ll be using the countrydata XML document from the :ref:`Parsing XML <elementtree-parsing-xml>` section:
For XML with namespaces, use the usual qualified
Supported XPath syntax
Selects all child elements with the given tag. For example, spam selects all child elements named spam , and spam/egg selects all grandchildren named egg in all children named spam .
Selects all elements for which the given attribute does not have the given value. The value cannot contain quotes.
Selects all elements whose complete text content, including descendants, equals the given text .
Selects all elements whose complete text content, including descendants, does not equal the given text .
Selects all elements that have a child named tag whose complete text content, including descendants, does not equal the given text .
Predicates (expressions within square brackets) must be preceded by a tag name, an asterisk, or another predicate. position predicates must be preceded by a tag name.
This module provides limited support for XInclude directives, via the :mod:`xml.etree.ElementInclude` helper module. This module can be used to insert subtrees and text strings into element trees, based on information in the tree.
Here’s an example that demonstrates use of the XInclude module. To include an XML document in the current document, use the
By default, the href attribute is treated as a file name. You can use custom loaders to override this behaviour. Also note that the standard helper does not support XPointer syntax.
To process this file, load it as usual, and pass the root element to the :mod:`xml.etree.ElementTree` module:
The ElementInclude module replaces the
If the parse attribute is omitted, it defaults to «xml». The href attribute is required.
To include a text document, use the
The result might look something like:
Element class. This class defines the Element interface, and provides a reference implementation of this interface.
The element name, attribute names, and attribute values can be either bytestrings or Unicode strings. tag is the element name. attrib is an optional dictionary, containing element attributes. extra contains additional attributes, given as keyword arguments.
The following dictionary-like methods work on the element attributes.
The following methods work on the element’s children (subelements).
:class:`Element` objects also support the following sequence type methods for working with subelements: :meth:`
Caution: Elements with no subelements will test as False . Testing the truth value of an Element is deprecated and will raise an exception in Python 3.14. Use specific len(elem) or elem is None test instead.:
Prior to Python 3.8, the serialisation order of the XML attributes of elements was artificially made predictable by sorting the attributes by their name. Based on the now guaranteed ordering of dicts, this arbitrary reordering was removed in Python 3.8 to preserve the order in which attributes were originally parsed or created by user code.
In general, user code should try not to depend on a specific ordering of attributes, given that the XML Information Set explicitly excludes the attribute order from conveying information. Code should be prepared to deal with any ordering on input. In cases where deterministic XML output is required, e.g. for cryptographic signing or test data sets, canonical serialisation is available with the :func:`canonicalize` function.
In cases where canonical output is not applicable but a specific attribute order is still desirable on output, code should aim for creating the attributes directly in the desired order, to avoid perceptual mismatches for readers of the code. In cases where this is difficult to achieve, a recipe like the following can be applied prior to serialisation to enforce an order independently from the Element creation:
ElementTree wrapper class. This class represents an entire element hierarchy, and adds some extra support for serialization to and from standard XML.
element is the root element. The tree is initialized with the contents of the XML file if given.
This is the XML file that is going to be manipulated:
Example of changing the attribute «target» of every link in first paragraph:
QName wrapper. This can be used to wrap a QName attribute value, in order to get proper namespace handling on output. text_or_uri is a string containing the QName value, in the form
This class is the low-level building block of the module. It uses :mod:`xml.parsers.expat` for efficient, event-based parsing of XML. It can be fed XML data incrementally with the :meth:`feed` method, and parsing events are translated to a push API — by invoking callbacks on the target object. If target is omitted, the standard :class:`TreeBuilder` is used. If encoding [1] is given, the value overrides the encoding specified in the XML file.
:meth:`XMLParser.feed` calls target‘s start(tag, attrs_dict) method for each opening tag, its end(tag) method for each closing tag, and data is processed by method data(data) . For further supported callback methods, see the :class:`TreeBuilder` class. :meth:`XMLParser.close` calls target‘s method close() . :class:`XMLParser` can be used not only for building a tree structure. This is an example of counting the maximum depth of an XML file:
A pull parser suitable for non-blocking applications. Its input-side API is similar to that of :class:`XMLParser` , but instead of pushing calls to a callback target, :class:`XMLPullParser` collects an internal list of parsing events and lets the user read from it. events is a sequence of events to report back. The supported events are the strings «start» , «end» , «comment» , «pi» , «start-ns» and «end-ns» (the «ns» events are used to get detailed namespace information). If events is omitted, only «end» events are reported.
:class:`XMLPullParser` only guarantees that it has seen the «>» character of a starting tag when it emits a «start» event, so the attributes are defined, but the contents of the text and tail attributes are undefined at that point. The same applies to the element children; they may or may not be present.
If you need a fully populated element, look for «end» events instead.
XML parse error, raised by the various parsing methods in this module when parsing fails. The string representation of an instance of this exception will contain a user-friendly error message. In addition, it will have the following attributes available: