Fabio Vazquez (Sao Paulo, Brazil) is a Software Development Engineer for the Microsoft Dynamics Ax Globalization Team at Microsoft. He is a former Microsoft Most Valuable Professional (MVP) in C# and VFP and a frequent speaker at software development conferences in Brazil, including the Microsoft PDC (Professional Developers Conference) and Tech-Ed. Fabio has two books published and has a great passion for agile development practices.
Microsoft is working on a group of related technologies that will hopefully change the way we think when programming with .NET languages, especially C# and VB.NET. The new versions of these languages (C# 3.0 and VB 9) bring together some great compiler improvements that enable the so called Linq (Language Integrated Query) features. As the name suggests, Linq brings query constructions to these languages and will allow developers to think on the collections of their programs as sets of values that can be queried, as they do with database tables in SQL language. Any .Net type that implements the IEnumerable interface is eligible to be the source of a query using Linq constructs. Ordinary collections, Xml documents or even objects representing database entities can be queried using high level language constructs. Microsoft is focusing its efforts on enabling a great developer experience when it comes to using the Linq features against Xml documents and Database tables. The project focusing on Linq over Xml documents is called Linq to Xml. There is another project focusing on providing run-time infrastructure for managing relational data as objects, known as Linq to Sql. In this article we will focus on Linq to Xml, trying to show how working with Xml documents will become easier and more natural when Linq to Xml become available. Along the way, we will touch some of the important enabling features that are being incorporated into the next versions of C# and VB, which are the real power force behind all the Linq strategy.
After the setup, new versions of the C# and VB compilers will be installed on your machine. These versions, as the package name suggests, are a “preview” of what is to come on the next generation of these compilers from Microsoft. It is not recommended that you use these compiler versions to build binaries that will be used on a production environment. Use them primarily as study subject.
Linq Preview allows you to alter the Visual Studio 2005 configurations so that it gives support to the new version of the C# compiler and IntelliSense. In order to enable these features, go to the “bin” folder under the Linq Preview installation directory and run the “Install C# IDE Support.Vbs” file. This integration is completely reversible and can be undone by executing the file “Uninstall C# IDE Support.Vbs”, located in the same folder.
NOTE: There is a bug in this CTP that causes commands to disappear from Visual Studio, including some commands that control smart tag display. It would be extremely annoying if it was not possible to overcome this, but it is! You can find the steps to get rid of this bug on the following MSDN Forum link: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=419975&SiteID=1
From now on, every time you open a project or solution on Visual Studio 2005, a message box will be presented prompting that an unsupported version of the respective compiler will be used and that some features may not work as expected:
Figure 1: Message box warning that we are using an unsupported version of the compiler.
With this integration with Visual Studio being installed, we can have access to templates for additional Linq specific projects:
Figure 2: Linq project templates in Visual Studio after the installation of the CTP.
Linq to Xml and all language integrated query features are still being designed and will continue evolving until its final release at the “Orcas” timeframe. Orcas is the codename for the next version of Visual Studio, which is expected to ship in 2007, after Windows Vista and Office 2007 have been shipped. Therefore, the information presented in this article is subject to change as Linq features are developed and the community feedback is collected by Microsoft.
NOTE: For those that have already begun to study Linq and the underling new features that are going to be incorporated into C# 3.0 and VB 9 compilers, I would like to clarify that this article will not go into details about language features like Static classes, Extension Methods, Lambda Expressions, Anonymous Types and Object Initalizers. Even though these features are the basis for the excellent Linq functionalities, this article will try to adopt a more pragmatic approach, discussing the topics relevant to the manipulation of Xml documents and the major differences between the Linq model and the classic W3C DOM API implemented by the classes on the System.Xml namespace in .Net.
A brief introduction to Linq
Before going deep into the Linq to Xml API and queries, let’s see a brief introduction to the Integrated Query concepts. Consider the following code that uses the new Integrated Query features to query over an array of integers:
int[] numbers = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
IEnumerable<int> selected = from number in numbers
select number;
foreach (int number in selected)
{
Console.WriteLine(number);
}
Here we used a SQL like syntax to select all the numbers in the array. The construct in the form “from...in...select” is the real alien thing here. This is the new integrated query syntax that is being incorporated into C# and VB compilers and will hopefully take these .Net languages closer to some functional languages. Notice that the result of the query is returned as an IEnumerable (IEnumerable in the example), making it possible for us to iterate through this structure using a foreach loop. In this example we are not actually doing anything useful with the query, since we are selecting all the same elements present in the original array. Let’s go ahead and select only the even numbers:
IEnumerable<int> selected = from number in numbers
where number % 2 == 0
select number;
Now this is becoming interesting! As we can see, the “where” clause can receive any valid C# statement. This statement is evaluated for each member in the collection being iterated over (in our example, all number in the numbers collection).
If we wanted to order the result of the query in descending order, we might have used the orderby clause in the query, like this:
IEnumerable selected = from number in numbers
orderby number descending
select number;
Now, let’s see an example using an array of strings:
string[] names = { "Paul", "John", "George", "Ringo"};
var namesLength = from name in names
select name.Length;
foreach (int len in namesLength)
{
Console.WriteLine(len);
}
The query in this code selects the length in characters of each string in the array. It uses the Length property of the string class, since we are querying over an array of strings. If you play close attention to the code above, you will see that there is this strange “var” keyword. This is a new feature from C# 3.0 that allows us to omit the type of any local variable we use in our code. The type will be inferred by the compiler and cannot be changed along the execution of the program. So, this will not mean any revolutionary concept shift in the language and it also doesn’t have any parallel with the “variant” types existing in other languages. It is just a means of leaving to the compiler the task of specifying the type of the object, but it doesn’t take from the programmer the responsibility of knowing the exact type of each variable she is working on. The exception to this rule is when we are working with anonymous types, which are also a new feature of C# 3.0 and VB 9. Anonymous types will not be covered in this article, but there are excellent references about them on the Internet – please, see the References section at the end of this article for more information. Having said that, it is correct to say that these two lines of code are functionally equivalent:
var number = 5;
int number = 5;
NOTE: VB 9 will also have a feature equivalent to the C# “var”, but it will use the old VB programmers’ friend Dim keyword, as this:
Dim number = 5
Which is the same as:
Dim number As Integer = 5
In order to clarity the various possibilities that we have with Language Integrated Query, let’s see some miscellaneous use cases that will hopefully give a task of what can be accomplished with Linq:
To get all directories whose name start with the letter “T” that are children of the root folder:
DirectoryInfo rootDir = new DirectoryInfo("c:\\");
var dirsStartingWithT = from dir in rootDir.GetDirectories()
where dir.Name.ToUpper().StartsWith("T")
select dir;
foreach (DirectoryInfo dir in dirsStartingWithT)
{
Console.WriteLine(dir.FullName);
}
To get all processes that are eating more than 10 MB of memory:
Process [] processesRunning = Process.GetProcesses();
var heavyProcesses = from process in Process.GetProcesses()
where process.WorkingSet / 1024 / 1024 > 10
select process;
foreach (Process process in heavyProcesses)
{
Console.WriteLine
(
"Memory use of Process {0}: {1} MB",
process.ProcessName,
process.WorkingSet / 1024 / 1024
);
}
To get, via reflection, all public methods from a given type:
Type targetType = typeof(DateTime);
var constructors = from method in targetType.GetMethods()
where method.IsPublic
orderby method.Name
select method;
foreach (var method in constructors)
{
Console.WriteLine(method);
}
If you stop to think about how those pieces of code would be written without Linq resources, you may start do notice the huge difference in paradigm that Linq will bring for .Net developers along the time, as they start feeling comfortable with this new syntax and style.
The Linq to Xml object model
The Linq to Xml API is very simple. Few classes play the main role in this model, where we could highlight XNode, XElement, XDocument and XAttribute. If you take out the “X” letter from the beginning of the class name, you have a hint about what each class represents in the Xml DOM. All these classes are defined in the System.Xml.XLinq namespace in the assembly System.Xml.XLinq.Dll.
Figure 3: The Linq to Xml class model. All these classes are from the System.Xml.XLinq namespace.
Writing Xml Documents
When we use the classical DOM model, the task of creating a new document from scratch or composing Xml fragments is made in an imperative way. This means that we have to instruct the API into doing what we want. But the problem is that the API does not converge to the document structure, so we have to pay more attention to the API usage than to the real Xml structure.
Consider that we want to create this simple Xml document:
On the other hand, by using the new Linq to Xml classes available in the System.Xml.XLinq namespace, we can write something like the following:
XDocument doc =
new XDocument(
new XDeclaration("1.0", Encoding.UTF8.HeaderName, String.Empty),
new XComment("Sample Xml Document"),
new XElement("Customers",
new XElement("Customer",
new XAttribute("Id", "1"),
new XElement("Name", "Charles Darwin")
)
)
);
As we can notice, the Linq to Xml syntax is much more intuitive and directed towards the structure of the final document. This makes it easier to use and maintain this code, since it describes more clearly its intention. This style of composing Xml document in Linq to Xml is known as Functional Construction.
Another difference between the W3C model and the Linq to Xml one is the fact that with Linq, we are decoupled from the XDocument class when we create documents. We can use the XElement class if we don’t need to include nodes such as processing instructions and Xml declarations. The following code creates an Xml document without using the XDocument class:
XElement doc =
new XElement(
new XElement("Customers",
new XElement("Customer",
new XAttribute("Id", "1"),
new XElement("Name", "Charles Darwin")
)
)
);
If we had to write a functional equivalent code using the W3C API, we would have to use the factory methods in System.Xml.XmlDocument class in order to construct the nodes for the elements and attributes.
VB and Xml Literals
As in many other parts of Linq to Xml support, VB takes a more productive approach than C#. One of the differences VB 9 will have over C# 3.0 when it refers to the construction of Xml documents is the use of Xml Literals. To understand this concept better, take a look at the following VB 9 code that creates exactly the same Xml document as the C# code created before:
Figure 4: Xml Literals in VB 9.0 editor.
Notice that we can simply copy and paste the text of and Xml document and pass if to a variable of type XDocument. The string will be resolved by the VB compiler as a series of calls to the constructor of the appropriate class. The IDE has still some features that make the developer experience even better, such as expand/collapse and viewing of hidden text:
Figure 5: Enhanced editor in VB 9.0 supports expand/collapse and hidden text viewing.
Pretty Printing
One task that is unjustifiable hard to perform with the DOM API in .Net is pretty printing the Xml document loaded into an XmlDocument object. The following code shows how this can be done considering that we want to print to the console:
XmlDocument doc = new XmlDocument();
doc.LoadXml(_sampleXml);
XmlTextWriter tw = new XmlTextWriter(Console.Out);
tw.Formatting = Formatting.Indented;
tw.Indentation = 3;
doc.Save(tw);
With Linq to Xml, this task is available right in the ToString() method of XNode derived classes:
NOTE: The method XDocument.Parse() is functionally equivalent to the XmlDocument.LoadXml() method in System.Xml. It parses the string content passed in as parameter and loads it into the XDocument object.
Searching for Content
In the System.Xml API we have the great support to XPath at our disposal, which makes the task of searching content into Xml a very powerful and flexible activity. Despite the fact that XPath is an extraordinary tool for Xml developers, working with it is not the most obvious thing one can learn. Linq to Xml takes the notion of Xml Axes from XPath and provides some methods that can be used to navigate through the Xml document hierarchy. The examples below show how some of these methods can be used. Please consider that the following document was loaded into a string variable named “_xmlDoc”, as this:
string _xmlDoc =
@"<Customers>
<!-- List of our important customers -->
<Customer Id='1'>
<Name>Charles Darwin</Name>
</Customer>
<Customer Id='2'>
<Name>Isaac Newton</Name>
</Customer>
</Customers>";
To get all the elements with a specific name that are children of a given parent element, we can use the Element(XName) method, which is defined on the XContainer class and is so inherited by the XElement class:
XElement doc = XElement.Parse(_xmlDoc);
IEnumerable<XElement> elements = doc.Elements("Customer");
Console.WriteLine("Number of elements: {0}", elements.Count());
foreach (XElement element in elements)
{
Console.WriteLine(element);
}
This code is getting all “Customer” elements that are children of the document element (Customers). It generates the following output:
Number of elements: 2
<Customer Id="1">
<Name>Charles Darwin</Name>
</Customer>
<Customer Id="2">
<Name>Isaac Newton</Name>
</Customer>
The Elements() method returns a collection of XElement objects (IEnumerable). This means that we can iterate over each XElement in this collection with a foreach loop, for example.
Notice that the ToString() method of each XElement is being implicitly called by the Console.WriteLine() method, producing a pretty printed output of the element to the console.
To get all children nodes of an element, we can use something like this:
Number of nodes: 3
<!-- List of our important customers -->
<Customer Id="1">
<Name>Charles Darwin</Name>
</Customer>
<Customer Id="2">
<Name>Isaac Newton</Name>
</Customer>
This time 3 nodes were returned. That is because the comment node was also retrieved by the Nodes() method, as would had happened with any other node that might exist in the document.
There is a method named Element() on the XElement class that can be used to get a specific element by searching for its name:
Differently from the Elements() method, which returns an IEnumerable, the Element() method returns a single XElement. As there are two “Customer” child elements, the first one is returned, producing this output:
All these methods can be chained to produce more detailed and complex searches, for example, to get all the “Name” elements in the document, we could do this:
XElement doc = XElement.Parse(_xmlDoc);
IEnumerable<XElement> names = doc.Elements("Customer").Elements("Name");
Console.WriteLine("Number of Name elements: {0}", names.Count());
foreach (XElement name in names)
{
Console.WriteLine(name);
}
This would produce this output:
Number of Name elements: 2
<Name>Charles Darwin</Name>
<Name>Isaac Newton</Name>
Attributes can also be retrieved by means of the Attributes() method:
XElement doc = XElement.Parse(_xmlDoc);
IEnumerable<XAttribute> ids = doc.Elements("Customer").Attributes("Id");
Console.WriteLine("Number of Id attributes: {0}", ids.Count());
foreach (XAttribute id in ids)
{
Console.WriteLine
(
"Attribute {0} form {1}: {2}",
id.Name,
id.Parent.Element("Name").Value,
id.Value
);
}
This is the output produced by this code:
Number of Id attributes: 2
Attribute Id form Charles Darwin: 1
Attribute Id form Isaac Newton: 2
Notice that the Attributes() method returns an IEnumerable. The XAttribute class is much simplier than the XNode and XElement classes. A XAttribute is basically a name/value pair, as we can see from the use of the Name and Value properties shown on the code above. To get access to the parent element of the attribute we are using the Parent property. From the parent element, we can navigate to other parts of the document, as the code above does.
The VB Way of Transversing XML
As we have seen before, the VB Team is working on a set of language features that aim to make the developer more productive when working on Linq to Xml. In the document navigation arena, VB 9 will have an interesting resource known as Xml Axis Properties. Xml Axis Properties define a special syntax that allow accessing nodes directly specifying its names, rather than calling methods as we saw before.
This means that, instead of using the Elements() method to get all the elements children of a node, as this:
Dim elements As IEnumerable(Of XElement) = doc.Elements("Customer")
We could do the following with Linq to Xml in VB:
Dim elements As IEnumerable(Of XElement) = doc.<Customer>
To get the value of an attribute, we can use this syntax:
Dim ids As IEnumerable(Of XAttribute) = doc.<Customer>.@Id
We can even specify an element to navigate based on its position related to its parent:
Dim darwin As IEnumerable(Of XElement) = doc.<Customer>(0).<Name>
Dim isaac As IEnumerable(Of XElement) = doc.<Customer>(1).<Name>
In order to retrieve all descendant nodes from an element, we can do the following:
Dim names As IEnumerable(Of XElement) = doc...<Name>
This syntax with three dots means that we want to find all “Name” elements children from doc, no matter how deeply they occur in the hierarchy.
We can also compose the various elements of this syntax in order to make more specific searches:
Dim names As IEnumerable(Of XElement) = doc.<Customer>.<Name>
Query language over Xml Documents
In this section we are going to investigate some sample queries we may create with Linq to Xml. These queries might have being created using an ordinary DOM tree iteration process or XPath expressions with the traditional W3C API. Instead of that, we are going to use the Integrated Query features of the new C# and VB compilers to explore how we will be approaching this kind of scenario when Linq to Xml become available.
To show that, let’s consider a more elaborate Xml document, so that we can work with more relevant types of queries. The following Xml document represents a set of orders and their respective details.
For all the queries we are going to see in this section, please consider that the document above is stored as a string into a variable named “_sampleXml”.
Let’s start by writing a simple query that gets all the Order Id Attributes present in the document:
XElement doc = XElement.Parse(_sampleXml);
IEnumerable<XAttribute> orderIds = from anOrder in doc.Descendants("Order")
select anOrder.Attribute("Id");
foreach (XAttribute id in orderIds)
Console.WriteLine(id);
Output:
Id="10574"
Id="10577"
Id="10822"
This query is returning an IEnumerable with the Id attribute in each Order element in the document. We could retrieve only the value of the attribute instead of the whole XAttribute object:
XElement doc = XElement.Parse(_sampleXml);
IEnumerable<string> orderIds = from anOrder in doc.Descendants("Order")
select anOrder.Attribute("Id").Value;
foreach (string id in orderIds)
Console.WriteLine(id);
Output:
10574
10577
10822
The next query will get all Order elements that have items for the ProductID 62:
XElement doc = XElement.Parse(_sampleXml);
IEnumerable<XElement> orders =
from anItem in doc.Elements("Order").Elements("Item")
where (int) anItem.Element("ProductID") == 62
select anItem.Parent;
foreach (var order in orders)
{
Console.WriteLine(order.Attribute("Id"));
}
Notice that we used an explicit cast to transform the XElement returned by the Elements(“ProductID”) method to an int. The XElement class has cast operators for various types, which is very convinient in scenarios like these where we are performing comparitions on the “where” predicate. One of the cast operators we can use over a XElement type is a cast to DateTime. Comparing dates and times in Xml documents was always an arduous task and the source of many doubts among the developers. With Linq to Xml we wil be able to easily manipulate dates in Xml documents as shown in the next example, which returns all Order elements whose Date is greater than 01/01/1998:
XElement doc = XElement.Parse(_sampleXml);
var orders = from anOrder in doc.Elements("Order")
where (DateTime) anOrder.Attribute("Date") > new DateTime(1998, 01, 01)
select anOrder;
foreach (var order in orders)
{
Console.WriteLine
(
"Order {0} put at {1}",
order.Attribute("Id").Value,
((DateTime) order.Attribute("Date")).ToShortDateString()
);
}
Output:
Order 10822 put at 8/1/1998
Xml Transformations
When we refer to transformations in Xml, usually we think on XSLT. XSLT is a great technology and will certainly be the solution to lots of transformation problems on Xml projects in the future. Linq to Xml doesn’t indent to replace XSLT at all, but with the Language Integrated Query features, lots of transformation tasks will become simpler to implement. Imagine for example that we wanted to transform our Orders Xml document shown before to an Xml file having summarization information about the Orders, such as this:
XElement doc = XElement.Parse(_sampleXml);
XElement ordersSummary =
new XElement("OrdersSummary",
from anOrder in doc.Elements("Order")
select
new XElement("Order",
new XAttribute("Id", anOrder.Attribute("Id").Value),
new XElement("ItemsOrdered",
anOrder.Elements("Item").Count()
)
)
);
Console.WriteLine(ordersSummary);
Here we are using Functional Construction in conjunction with Language Integrated Queries, getting therefore the best of both worlds. One or the constructors of the XElement class takes a param [] objects as parameter, meaning that we can pass in an arbirary number of elements to it. Since any array in .Net implements the IEnumerable interface, the result of any query is elegible to be passed to the constructor of the XElement class. This impressively beautiful and at the same time very simple design enables this kind of construction pattern to happen in Linq to Xml.
In VB, we have an alternative the the pair functional construction and Language Integrated Query. Xml Literals can be combined with queries in the following way:
Figure 6: Combining Xml Literals with code in VB.
Notice that Xml literals can be breaked and code can be executed to compose the content dinamically. We define code into an Xml literal between < %= and % >. This resembles ASP.NET code – actually it was common place on classic ASP. This feature is being subject to much debate in the community, where some people think its great, but others find that it will promote bad programming habits. The better we can do is try for outselves and draw our own conclusions, having always in mind that anything is subject to change ultil the Linq ship date.
Transforming to HTML
One of the classic Xml transformation use cases is the transformation from an Xml document into HTML. Knowing how to work with Functional Construction and Linq queries, this task becomes trivial. Let’s see how our Orders Xml document can be transformed to this HTML document:
Figure 7: Html document generated through a Linq transformation form an Xml document.
Here is the code used to do that:
public void TransformOrdersXmlToHtml()
{
XElement doc = XElement.Parse(_sampleXml);
XElement html =
new XElement("html",
new XElement("head",
new XElement("title", "Transformation Sample"),
GetCssStyleElement()
),
GetHtmlBodyContent(doc)
);
html.Save("\\Orders.Htm");
}
private object GetCssStyleElement()
{
return
new XElement("style",
"h2 { background-color:Navy; color:white; margin-bottom:5pt; padding:3pt }",
"body { background-color:#cccccc; font-family:Arial; }",
"table { background-color:White; border:solid 1pt black; }",
"td { width:25%; border-bottom: solid 1pt whitesmoke; }"
);
}
private XElement GetHtmlBodyContent(XElement doc)
{
return
new XElement("body",
from anOrder in doc.Elements("Order")
select GetOrderDiv(anOrder)
);
}
private XElement GetOrderDiv(XElement orderElement)
{
return
new XElement("div",
new XElement("h2", "Order: " + orderElement.Attribute("Id").Value),
new XElement("table",
new XAttribute("width", "100%"),
new XElement("tr",
new XElement("th", new XAttribute("align", "center"), "Product Id"),
new XElement("th", new XAttribute("align", "right"), "Unit Price"),
new XElement("th", new XAttribute("align", "right"), "Quantity"),
new XElement("th", new XAttribute("align", "right"), "Total")
),
from item in orderElement.Elements("Item")
select
new XElement("tr",
new XElement("td", new XAttribute("align", "center"), item.Element("ProductID").Value),
new XElement("td", new XAttribute("align", "right"), item.Element("UnitPrice").Value),
new XElement("td", new XAttribute("align", "right"), item.Element("Quantity").Value),
new XElement("td", new XAttribute("align", "right"),
((double) item.Element("UnitPrice") * (double) item.Element("Quantity")).ToString("n2"))
)
)
);
}
Conclusion
Linq in general and Linq to Xml in particular are going to incorporate new important features and development patterns to the developers’ arsenal of techinques. Working with Xml in .Net will become much more confortable and natural, taking away from the developers much of the difficult concepts imposed by the classic W3C API. Today we still see many developers using Xml the wrong way, treating Xml documents as simple strings and bypassing the Xml parser. These are very negative practices that are, many times, influenced by the difficulties they face when working with Xml APIs. With its simplicity and easy of use, Linq to Xml will hopefully aproximate the developers to the right techniques they should use when dealing with Xml and related technologies. Besides that, Language Integrated Query in general will become available to all C# and VB developers, bringing funcional concepts to these languages and adding powefull resources that will allow them to query in memory collections of objects and even database tables by means of Linq to Sql. Consider studying Linq right now as an investiment for your carreer as a developer and for your future projects in .Net.
