Readin nested subtree with XmlReader

Question

I met a problem during parsing XML into database. My XML structure looks like that:

<atrists>
<artist>
<images>
<image/>
</images>
<id>
</id>
</artist>
</artists>

There is a lot of both artist tag. and in artist a lot of tags.

What i am doing at the moment:

while (reader.ReadToFollowing("artist"))
{
 XmlReader reader2 = reader.ReadSubtree();
while (reader2.Read())
{
 if (reader2.NodeType == XmlNodeType.Element)
  {
  if (reader2.Name == "images")
  {
 while (reader2.ReadToFollowing("image"))
  {
//here i am adding all images data to lists to query it later
  }
  }
if (reader2.Name == "id")
{
 id = reader2.ReadElementContentAsInt();
}
}

//here i want to do SQL query that will add many images and ID attateched to every one.

That id is not working well. It skips like 50% of ID's or is not changing at all. But i can see every possible image there. So i think there is problem with me going form node to node. Any ideas?

Consider learning and using System.Xml.Linq. Your task will be much easier. — Darek, Jul 14 '14 at 13:27
Reading 1GB XML files with LINQ is not a problem, as long as you have RAM to back it up. Another thought, perhaps you could deserialize to a concrete class? — Darek, Jul 14 '14 at 14:03
Link to deserialization sample: http://stackoverflow.com/questions/364253/how-to-deserialize-xml-document — Darek, Jul 14 '14 at 14:07
I've just saved a 1GB XML file with LINQ ... took about 9GB of my RAM ... :) — Darek, Jul 14 '14 at 16:01
at the test server i got only 3 GB RAM and that's where i am doing it = ) — Whencesoever, Jul 14 '14 at 17:21

Darek · Answer 1 · 2014-07-15T13:04:12.507

Try this then:

    async Task BigFileReader(System.IO.Stream stream)
    {
        XmlReaderSettings settings = new XmlReaderSettings();
        settings.Async = true;
        bool bIdEncountered = false;

        using (XmlReader reader = XmlReader.Create(stream, settings))
        {
            while (await reader.ReadAsync())
            {
                switch (reader.NodeType)
                {
                    case XmlNodeType.Element:
                        bIdEncountered = reader.LocalName.Equals("id");
                        break;
                    case XmlNodeType.Text:
                        var value = await reader.GetValueAsync();
                        if(bIdEncountered) Console.WriteLine("Run my SQL for {0}", value);
                        break;
                    case XmlNodeType.EndElement:
                        break;
                    default:
                        break;
                }
            }
        }
    }

Example 2

Perhaps this will give you better example:

    async Task BigFileReader(System.IO.Stream stream)
    {
        XmlReaderSettings settings = new XmlReaderSettings();
        settings.Async = true;
        bool bIdEncountered = false;
        bool bImageEncountered = false;
        List<string> images = new List<string>();
        int artistId = 0;

        using (XmlReader reader = XmlReader.Create(stream, settings))
        {
            while (await reader.ReadAsync())
            {
                switch (reader.NodeType)
                {
                    case XmlNodeType.Element:
                        bIdEncountered = reader.LocalName.Equals("id");
                        bImageEncountered = reader.LocalName.Equals("image");
                        if(reader.LocalName.Equals("images")) images.Clear();
                        break;
                    case XmlNodeType.Text:
                        //var value = await reader.GetValueAsync();
                        if (bIdEncountered) artistId = Convert.ToInt32(await reader.GetValueAsync());
                        if (bImageEncountered) images.Add(await reader.GetValueAsync());
                        break;
                    case XmlNodeType.EndElement:
                        if (reader.LocalName.Equals("artist")) Console.WriteLine("Saving artist {0} with images {1}", artistId, String.Join(",", images));
                        break;
                    default:
                        break;
                }
            }
        }
    }

Example with attributes

    async Task BigFileReader(System.IO.Stream stream)
    {
        XmlReaderSettings settings = new XmlReaderSettings();
        settings.Async = true;
        bool bIdEncountered = false;
        List<ImageNode> images = new List<ImageNode>();
        int artistId = 0;

        using (XmlReader reader = XmlReader.Create(stream, settings))
        {
            while (await reader.ReadAsync())
            {
                switch (reader.NodeType)
                {
                    case XmlNodeType.Element:
                        bIdEncountered = reader.LocalName.Equals("id");
                        //bImageEncountered = reader.LocalName.Equals("image");
                        if(reader.LocalName.Equals("images")) images.Clear();
                        if (reader.LocalName.Equals("image"))
                        {
                            images.Add( new ImageNode
                                    {
                                        Width = Convert.ToInt32(reader.GetAttribute("width")),
                                        Height = Convert.ToInt32(reader.GetAttribute("height")),
                                        Url = reader.GetAttribute("uri")
                                    });
                        }
                        break;
                    case XmlNodeType.Text:
                        if (bIdEncountered) artistId = Convert.ToInt32(await reader.GetValueAsync());
                        break;
                    case XmlNodeType.EndElement:
                        if (reader.LocalName.Equals("artist")) Console.WriteLine("Saving artist {0} with images {1}", artistId, String.Join(",", images));
                        break;
                    default:
                        break;
                }
            }
        }
    }


internal class ImageNode    
{
    public int Width { get; set; }

    public int Height { get; set; }

    public string Url { get; set; }

    public override string ToString() { return String.Format("{0}x{1}:{2}", Width, Height, Url); }
}

Synchronous version (VS2010)

    static void BigFileReader(System.IO.Stream stream)
    {
        XmlReaderSettings settings = new XmlReaderSettings();
        bool bIdEncountered = false;
        List<ImageNode> images = new List<ImageNode>();
        int artistId = 0;

        using (XmlReader reader = XmlReader.Create(stream, settings))
        {
            while (reader.Read())
            {
                switch (reader.NodeType)
                {
                    case XmlNodeType.Element:
                        bIdEncountered = reader.LocalName.Equals("id");
                        //bImageEncountered = reader.LocalName.Equals("image");
                        if (reader.LocalName.Equals("images")) images.Clear();
                        if (reader.LocalName.Equals("image"))
                        {
                            images.Add(new ImageNode
                            {
                                Width = Convert.ToInt32(reader.GetAttribute("width")),
                                Height = Convert.ToInt32(reader.GetAttribute("height")),
                                Url = reader.GetAttribute("uri")
                            });
                        }
                        break;
                    case XmlNodeType.Text:
                        if (bIdEncountered) artistId = Convert.ToInt32(reader.Value);
                        break;
                    case XmlNodeType.EndElement:
                        if (reader.LocalName.Equals("artist")) Console.WriteLine("Saving artist {0} with images {1}", artistId, String.Join(",", images));
                        break;
                    default:
                        break;
                }
            }
        }
    }

are u sure it's what i need? after reading and adding my images i got all info about them i needed in lists. I just cannot get ID, one simple int that is in tag on the same level as images : (. — Whencesoever, Jul 14 '14 at 14:09
Without much detail what your program is supposed to do, I can't give you an answer better then this. I am positive that this will get you all your IDs. The XmlReader in the example reads the file line-by-line, or rather element by element, and you have to react accordingly. For example, you could shift execution logic to the closing element for artist. In the sample above, I left those points open for modification by you. If you gave more description what exactly is it you are trying to accomplish. I might be able to do more. — Darek, Jul 14 '14 at 14:14
[link](http://pastebin.com/4dT5pa6B) it's a part of my file. For every image tag in it i need to make Query in SQL. `Query = "insert into myTable values (" + id + ", " + Height + ", " + Width + ", '" + Type + "', '" + Uri + "', '" + Uri150 + "')";` That ID is for many Image tags and i all the time have problem to provide it right way. — Whencesoever, Jul 14 '14 at 14:21
I got a problem. There is visual studio 2010 installed and it do not use async. — Whencesoever, Jul 14 '14 at 20:14

Readin nested subtree with XmlReader

1 Answers1