Efficient XML Parsing with Java
For a language that likes XML so much, it was rather difficult to find an efficient and elegant way to parse it.
There’s more than one way to parse XML?
There are 2 ways to work with XML documents, read the full document and build a full DOM object representing the document or read the document as stream extracting what you need from the stream. The DOM method is fine for small documents like configuration files but completely falls apart when working with documents of any real size.
When it comes to stream parsing in Java the SAX parser seems to be the most common choice. Most Stack Overflow Answers and Tutorials about parsing large XML files in Java point to the SAX parser. The only problem with the SAX parser is it’s event driven API is very awkward to work with. Defining an event listener (Handler) and waiting for it to be called removes your control of the stream. I like to have control of the execution of my program and have always preferred a pull based API when working with streams.
Fortunately, like most things in programming, I’m not the first person to experience these pain points. The StAX API was created to address this pain, by providing pull based API for a XML stream.
The advantages of the approach are best described by the JavaDocs:
Pull parsing provides several advantages over push parsing when working with XML streams:
With pull parsing, the client controls the application thread, and can call methods on the parser when needed. By contrast, with push processing, the parser controls the application thread, and the client can only accept invocations from the parser.
Pull parsing libraries can be much smaller and the client code to interact with those libraries much simpler than with push libraries, even for more complex documents.
Pull clients can read multiple documents at one time with a single thread.
A StAX pull parser can filter XML documents such that elements unnecessary to the client can be ignored, and it can support XML views of non-XML data.
Show me this magic!
So let’s use the wikipedia dataset as an example, below is a sample of the data:
And here’s the code to extract the name and id of each page:
Wait! What just happened?
You can see the full project here, the first little bit extracts the xml file from a zip archive stored in the resources folder.
The next part uses the XMLInputFactory to create a StAX reader from the stream and starts reading the file. The reader provides a pull API over the stream to pull XML elements from the stream. This first loop goes through the document looking for <page> start element’s and then passing over control to the parsePage function to handle the processing of the page element.
The parsePage function continues to loop through the events in the stream. First we check if we’ve reach the end of the page element and return, as we’ve read the page and completed the scope of this function.
Next the start element is searched for and action taken based off which one has been reached. This allows us to quickly iterate over all the elements contained within the page element, extracting what is needed and ignoring the rest. An additional case could easily be added to the switch statement to handle the revision , calling an additional function to handle the details of parsing that element.
Why was that so simple?
Writing a SAXParser feels like instructing a clown how to juggle. It’s awkward and makes what should be a relatively simple task of reading an XML file unnecessarily complicated. The StAX parser flips this on it’s head, instead been thrown elements with no control, control remains with the caller. Instead of surrendering control and providing functions to be called when events happen. Events are requested and handled as necessary, leaving control with the consumer.
Java is a great language, when used well, unfortunately that is not always the case. The SAXParser is one of those examples, while it does serve a purpose, that purpose is niche. It should not be the default answer for efficient XML parsing. Why it still is when a much better solution exists in the standard library boggles my mind.
How to read and write XML files?
I have to read and write to and from an XML file. What is the easiest way to read and write XML files using Java?
6 Answers 6
Here is a quick DOM example that shows how to read and write a simple xml file with its dtd:
First import these:
Here are a few variables you will need:
Here is a reader (String xml is the name of your xml file):
And here a writer:
getTextValue is here:
Add a few accessors and mutators and you are done!
Writing XML using JAXB (Java Architecture for XML Binding):
The above answer only deal with DOM parser (that normally reads the entire file in memory and parse it, what for a big file is a problem), you could use a SAX parser that uses less memory and is faster (anyway that depends on your code).
SAX parser callback some functions when it find a start of element, end of element, attribute, text between elements, etc, so it can parse the document and at the same time you get what you need.
Some example code:
The answers only cover DOM / SAX and a copy paste implementation of a JAXB example.
However, one big area of when you are using XML is missing. In many projects / programs there is a need to store / retrieve some basic data structures. Your program has already a classes for your nice and shiny business objects / data structures, you just want a comfortable way to convert this data to a XML structure so you can do more magic on it (store, load, send, manipulate with XSLT).
This is where XStream shines. You simply annotate the classes holding your data, or if you do not want to change those classes, you configure a XStream instance for marshalling (objects -> xml) or unmarshalling (xml -> objects).
Internally XStream uses reflection, the readObject and readResolve methods of standard Java object serialization.
You get a good and speedy tutorial here:
To give a short overview of how it works, I also provide some sample code which marshalls and unmarshalls a data structure. The marshalling / unmarshalling happens all in the main method, the rest is just code to generate some test objects and populate some data to them. It is super simple to configure the xStream instance and marshalling / unmarshalling is done with one line of code each.
Reading XML Data into a DOM
In this section, you will construct a Document Object Model by reading in an existing XML file.
Note — In Extensible Stylesheet Language Transformations, you will see how to write out a DOM as an XML file. (You will also see how to convert an existing data file into XML with relative ease.)
Creating the Program
The Document Object Model provides APIs that let you create, modify, delete, and rearrange nodes. Before you try to create a DOM, it is helpful to understand how a DOM is structured. This series of examples will make DOM internals visible via a sample program called DOMEcho, which you will find in the directory INSTALL_DIR/jaxp-version/samples/dom after you have installed the JAXP API.
Create the Skeleton
First, build a simple program to read an XML document into a DOM and then write it back out again.
Start with the normal basic logic for an application, and check to make sure that an argument has been supplied on the command line:
This code performs all the basic set up operations. All output for DOMEcho uses UTF-8 encoding. The usage() method that is called if no argument is specified simply tells you what arguments DOMEcho expects, so the code is not shown here. A filename string is also declared, which will be the name of the XML file to be parsed into a DOM by DOMEcho.
Import the Required Classes
In this section, all the classes are individually named so you that can see where each class comes from, in case you want to reference the API documentation. In the sample file, the import statements are made with the shorter form, such as javax.xml.parsers.*.
These are the JAXP APIs used by DOMEcho:
These classes are for the exceptions that can be thrown when the XML document is parsed:
These classes read the sample XML file and manage output:
Finally, import the W3C definitions for a DOM, DOM exceptions, entities and nodes:
Handle Errors
Next, add the error-handling logic. The most important point is that a JAXP-conformant document builder is required to report SAX exceptions when it has trouble parsing an XML document. The DOM parser does not have to actually use a SAX parser internally, but because the SAX standard is already there, it makes sense to use it for reporting errors. As a result, the error-handling code for DOM applications is very similar to that for SAX applications:
As you can see, the DomEcho class's error handler generates its output using PrintWriter instances.
Instantiate the Factory
Next, add the following code to the main() method, to obtain an instance of a factory that can give us a document builder.
Get a Parser and Parse the File
Now, add the following code to main() to get an instance of a builder, and use it to parse the specified file.
The file being parsed is provided by the filename variable that was declared at the beginning of the main() method, which is passed to DOMEcho as an argument when the program is run.
Configuring the Factory
By default, the factory returns a non-validating parser that knows nothing about name spaces. To get a validating parser, or one that understands name spaces (or both), you can configure the factory to set either or both of those options using the following code.
As you can see, command line arguments are set up so that you can inform DOMEcho to perform validation against either a DTD or an XML Schema, and the factory is configured to be name space aware and to perform whichever type of validation the user specifies.
Note — JAXP-conformant parsers are not required to support all combinations of those options, even though the reference parser does. If you specify an invalid combination of options, the factory generates a ParserConfigurationException when you attempt to obtain a parser instance.
More information about how to use name spaces and validation is provided in Validating with XML Schema, in which the code that is missing from the above extract will be described.
Handling Validation Errors
The default response to a validation error, as dictated by the SAX standard, is to do nothing. The JAXP standard requires throwing SAX exceptions, so you use exactly the same error-handling mechanisms as you use for a SAX application. In particular, you use the DocumentBuilder class's setErrorHandler method to supply it with an object that implements the SAX ErrorHandler interface.
Note — DocumentBuilder also has a setEntityResolver method you can use.
The following code configures the document builder to use the error handler defined in Handle Errors.
The code you have seen so far has set up the document builder, and configured it to perform validation upon request. Error handling is also in place. However, DOMEcho does not do anything yet. In the next section, you will see how to display the DOM structure and begin to explore it. For example, you will see what entity references and CDATA sections look like in the DOM. And perhaps most importantly, you will see how text nodes (which contain the actual data) reside under element nodes in a DOM.
Displaying the DOM Nodes
To create or manipulate a DOM, it helps to have a clear idea of how the nodes in a DOM are structured. This section of the tutorial exposes the internal structure of a DOM, so that you can see what it contains. The DOMEcho example does this by echoing the DOM nodes, and then printing them out onscreen, with the appropriate indentation to make the node hierarchy apparent. The specification of these node types can be found in the DOM Level 2 Core Specification, under the specification for Node. Table 3-1 below is adapted from that specification.
Table 3-1 Node Types
Name of attribute
Value of attribute
Content of the CDATA section
Content of the comment
Document Type name
Name of entity referenced
Entire content excluding the target
Content of the text node
The information in this table is extremely useful; you will need it when working with a DOM, because all these types are intermixed in a DOM tree.
Obtaining Node Type Information
The DOM node element type information is obtained by calling the various methods of the org.w3c.dom.Node class. The node attributes by exposed by DOMEcho are echoed by the following code.
Every DOM node has at least a type, a name, and a value, which might or might not be empty. In the example above, the Node interface's getNamespaceURI(), getPrefix(), getLocalName(), and getNodeValue() methods return and print the echoed node's namespace URI, namespace prefix, local qualified name and value. Note that the trim() method is called on the value returned by getNodeValue() to establish whether the node's value is empty white space and print a message accordingly.
For the full list of Node methods and the different information they return, see the API documentation for Node .
Next, a method is defined to set the indentation for the nodes when they are printed, so that the node hierarchy will be easily visible.
The basicIndent constant to define the basic unit of indentation used when DOMEcho displays the node tree hierarchy, is defined by adding the following highlighted lines to the DOMEcho constructor class.
As was the case with the error handler defined in Handle Errors, the DOMEcho program will create its output as PrintWriter instances.
Lexical Controls
Lexical information is the information you need to reconstruct the original syntax of an XML document. Preserving lexical information is important in editing applications, where you want to save a document that is an accurate reflection of the original-complete with comments, entity references, and any CDATA sections it may have included at the outset.
Most applications, however, are concerned only with the content of the XML structures. They can afford to ignore comments, and they do not care whether data was coded in a CDATA section or as plain text, or whether it included an entity reference. For such applications, a minimum of lexical information is desirable, because it simplifies the number and kind of DOM nodes that the application must be prepared to examine.
The following DocumentBuilderFactory methods give you control over the lexical information you see in the DOM.
To convert CDATA nodes to Text nodes and append to an adjacent Text node (if any).
To expand entity reference nodes.
To ignore comments.
To ignore whitespace that is not a significant part of element content.
The default values for all these properties is false, which preserves all the lexical information necessary to reconstruct the incoming document in its original form. Setting them to true lets you construct the simplest possible DOM so that the application can focus on the data's semantic content without having to worry about lexical syntax details. Table 3-2 summarizes the effects of the settings.
Table 3-2 Lexical Control Settings
Preserve Lexical Info
Focus on Content
The implementation of these methods in the main method of the DomEcho example is shown below.
The boolean variables ignoreComments, ignoreWhitespace, putCDATAIntoText, and createEntityRefs are declared at the beginning of the main method code, and they are set by command line arguments when DomEcho is run.
Printing DOM Tree Nodes
The DomEcho application allows you to see the structure of a DOM, and demonstrates what nodes make up the DOM and how they are arranged. Generally, the vast majority of nodes in a DOM tree will be Element and Text nodes.
Note — Text nodes exist under element nodes in a DOM, and data is always stored in text nodes. Perhaps the most common error in DOM processing is to navigate to an element node and expect it to contain the data that is stored in that element. Not so! Even the simplest element node has a text node under it that contains the data.
The code to print out the DOM tree nodes with the appropriate indentation is shown below.
This code first of all uses switch statements to print out the different node types and any possible child nodes, with the appropriate indentation.
Node attributes are not included as children in the DOM hierarchy. They are instead obtained via the Node interface's getAttributes method.
The DocType interface is an extension of w3c.org.dom.Node. It defines the getEntities method, which you use to obtain Entity nodes — the nodes that define entities. Like Attribute nodes, Entity nodes do not appear as children of DOM nodes.
Node Operations
This section takes a quick look at some of the operations you might want to apply to a DOM.
Searching for nodes
Obtaining node content
Removing and changing nodes
Creating Nodes
You can create different types nodes using the methods of the Document interface. For example, createElement, createComment, createCDATAsection, createTextNode, and so on. The full list of methods for creating different nodes is provided in the API documentation for org.w3c.dom.Document .
Traversing Nodes
The org.w3c.dom.Node interface defines a number of methods you can use to traverse nodes, including getFirstChild, getLastChild, getNextSibling, getPreviousSibling, and getParentNode. Those operations are sufficient to get from anywhere in the tree to any other location in the tree.
Searching for Nodes
When you are searching for a node with a particular name, there is a bit more to take into account. Although it is tempting to get the first child and inspect it to see whether it is the right one, the search must account for the fact that the first child in the sub-list could be a comment or a processing instruction. If the XML data has not been validated, it could even be a text node containing ignorable whitespace.
In essence, you need to look through the list of child nodes, ignoring the ones that are of no concern and examining the ones you care about. Here is an example of the kind of routine you need to write when searching for nodes in a DOM hierarchy. It is presented here in its entirety (complete with comments) so that you can use it as a template in your applications.
For a deeper explanation of this code, see Increasing the Complexity in When to Use DOM. Note, too, that you can use APIs described in Lexical Controls to modify the kind of DOM the parser constructs. The nice thing about this code, though, is that it will work for almost any DOM.
Obtaining Node Content
When you want to get the text that a node contains, you again need to look through the list of child nodes, ignoring entries that are of no concern and accumulating the text you find in TEXT nodes, CDATA nodes, and EntityRef nodes. Here is an example of the kind of routine you can use for that process.
For a deeper explanation of this code, see Increasing the Complexity in When to Use DOM. Again, you can simplify this code by using the APIs described in Lexical Controls to modify the kind of DOM the parser constructs. But the nice thing about this code is that it will work for almost any DOM.
Creating Attributes
The org.w3c.dom.Element interface, which extends Node, defines a setAttribute operation, which adds an attribute to that node. (A better name from the Java platform standpoint would have been addAttribute. The attribute is not a property of the class, and a new object is created.) You can also use the Document's createAttribute operation to create an instance of Attribute and then use the setAttributeNode method to add it.
Removing and Changing Nodes
To remove a node, you use its parent Node's removeChild method. To change it, you can use either the parent node's replaceChild operation or the node's setNodeValue operation.
Inserting Nodes
The important thing to remember when creating new nodes is that when you create an element node, the only data you specify is a name. In effect, that node gives you a hook to hang things on. You hang an item on the hook by adding to its list of child nodes. For example, you might add a text node, a CDATA node, or an attribute node. As you build, keep in mind the structure you have seen in this tutorial. Remember: Each node in the hierarchy is extremely simple, containing only one data element.
Running the DOMEcho Sample
To run the DOMEcho sample, follow the steps below.
- Navigate to the samples directory. % cd install-dir/jaxp-1_4_2-release-date/samples.
- Compile the example class. % javac dom/*
- Run the DOMEcho program on an XML file.
Choose one of the XML files in the data directory and run the DOMEcho program on it. Here, we have chosen to run the program on the file personal-schema.xml.
% java dom/DOMEcho data/personal-schema.xml
The XML file personal-schema.xml contains the personnel files for a small company. When you run the DOMEcho program on it, you should see the following output.
As you can see, DOMEcho prints out all the nodes for the different elements in the document, with the correct indentation to show the node hierarchy.
Основы Java XML – учебник для новичка
Это пособие для программистов знакомит с XML и его использованием с Java, статью подготовил Ларс Фогель, переведено нами на русский язык.
XML – это сокращение от Extensible Markup Language и установленный формат обмена данными. XML был определен в 1998 году Консорциумом World Wide Web (W3C).
Для удобства и экономии времени, вы также можете найти онлайн курс на сайте https://jms.university/. XML-документ состоит из элементов, каждый элемент имеет начальный тег, контент и конечный тег. Должен иметь ровно один корневой элемент (т. е. Один тег, который включает остальные теги). Различает заглавные и не заглавные буквы.
Файл XML должен быть правильно сформирован. Это означает, что он должен применяться к следующим условиям:
- XML-документ всегда начинается с пролога
- Каждый открывающий тег имеет закрывающий тег.
- Все теги полностью вложены.
Правильный XML-файл должен содержать ссылку на XML-схему и быть действительным в соответствии с этой схемой. Ниже приведен правильный, корректный XML-файл.
Сравнение XML с другими форматами
Обрабатывать XML-документ относительно легко по сравнению с двоичным или неструктурированным форматом. Это из-за следующих характеристик:
- простой текст
- представляет данные без определения способа отображения данных
- может быть преобразован в другие форматы через XSL
- может быть легко обработан с помощью стандартных анализаторов
- XML-файлы являются иерархическими
Если данные представлены в виде XML, размер этих данных является относительно большим по сравнению с другими форматами. JSON или двоичные форматы часто используются для замены, если важна пропускная способность данных.
Элементы XML
XML-документ всегда начинается с пролога, который описывает XML-файл. Этот пролог может быть минимальным, например <? xml version = “1.0”?>. Он также может содержать другую информацию, например, кодировку <? xml version = “1.0” encoding = “UTF-8” standalone = “yes”?>.
Тег, который не содержит никакого содержимого, называется «пустым тегом», например, .
Комментарии в XML определяются как: <! COMMENT>.
Обзор XML Java
Язык программирования Java содержит несколько методов для обработки и написания XML.
Более старые версии Java поддерживали только API DOM (объектная модель документа) и API SAX (простой API для XML).
В DOM вы получаете доступ к документу XML через дерево объектов. DOM может использоваться для чтения и записи файлов.
SAX (простой API для XML) – это Java API для последовательного чтения XML-файлов. SAX обеспечивает управляемую событиями обработку XML по модели Push-Parsing. В этой модели вы регистрируете слушателей в виде Handlers to the Parser. Они уведомляются через методы обратного вызова.
И DOM, и Sax – старые API, и я рекомендую больше их не использовать.
Stax (Streaming API for XML) – это API для чтения и записи XML-документов. Он был представлен в Java 6.0 и считается превосходящим SAX и DOM.
Архитектура Java для привязки XML (JAXB) – это стандарт Java, который позволяет преобразовывать объекты Java в XML и наоборот. JAXB определяет API для чтения и записи объектов Java в документы XML. Он также позволяет выбирать реализацию JAXB. JAXB применяет множество настроек по умолчанию, что делает чтение и запись очень простым.
Ниже объясняется интерфейс Stax.
Потоковый API для XML (StaX)
Потоковый API для XML, называемый StaX, представляет собой API для чтения и записи XML-документов.
StaX – это модель Pull-Parsing, может взять на себя управление анализом XML-документов, извлекая (принимая) события из анализатора.
Ядро StaX API делится на две категории, и они перечислены ниже.
- Курсор API
- Event Iterator API
Приложения могут использовать любой из этих двух API. Далее речь пойдет об API итератора событий, так как я считаю его более удобным в использовании.
API Event Iterator
API итератора событий имеет два основных интерфейса: XMLEventReader для синтаксического анализа XML и XMLEventWriter для генерации XML.
XMLEventReader – читаем пример XML
Этот пример хранится в проекте “de.vogella.xml.stax.reader”.
Приложения зацикливаются на всем документе. API-интерфейс Event Iterator реализован поверх API-интерфейса Cursor.
В этом примере мы прочитаем следующий XML-документ и создадим из него объекты.
Определим следующий класс для хранения отдельных записей.
Далее читается файл XML и создается список элементов объекта из записей.
Вы можете проверить парсер с помощью следующей тестовой программы. Обратите внимание, что файл config.xml должен существовать в папке проекта Java.
Пример записи файла XML
Этот пример хранится в проекте “de.vogella.xml.stax.writer”.
Предположим, вы хотели бы написать следующий простой XML-файл.
StaX не предоставляет функциональные возможности для автоматического форматирования, поэтому вам необходимо добавить в конец файла строки и информацию табуляции.
Средняя оценка 2.8 / 5. Количество голосов: 5
Спасибо, помогите другим — напишите комментарий, добавьте информации к статье.
Или поделись статьей
Видим, что вы не нашли ответ на свой вопрос.
Помогите улучшить статью.
Напишите комментарий, что можно добавить к статье, какой информации не хватает.