Silverlight 2 RC0 Doing Data Part VIII : Using the Threadpool

Silverlight 2 implements the ThreadPool, which can be useful for parallel operations on background threads. Here we aggregate a number of RSS Search feeds and display the combined results in a DataGrid.

The Threadpool performs operations on background threads via its QueueUserWorkItem method.  Instead of having to make blocking calls for each search operation, and wait for each to complete before we can start the next one, we can kick them all off at once on ThreadPool threads, and they will all be executed in parallel (subject to available ThreadPool Threads and the limitations of the browser HttpRequest API).

If we know in advance (as in this case) how many enqueued WorkItems we've kicked off, then it becomes a simple matter to detect that we're finished with all our background thread operations and then continue on with the process of aggregating all the results.

In order to illustrate how this can be useful, I picked four RSS - formatted search urls (Digg, Google Blog Search, MSDN and ittyurl.net), each of which exposes the required crossdomain.xml or clientaccesspolicy.xml file at the root of the site, enabling a Silverlight client to make cross-domain calls to get data.

My UI has a DataGrid, a TextBox for search term, and a "Search" button that kicks off the operation. First, let's look at the XAML for the UI:

<UserControl xmlns:my="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data"  x:Class="SLThreadPool.Page"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    Width="700" Height="500">
    <Grid x:Name="LayoutRoot" Background="White">
        <TextBlock Text="Silverlight Threadpool Sample" VerticalAlignment="Top" Width="500" HorizontalAlignment="Center" FontSize="20" />
        <my:DataGrid x:Name="dgFeeds" AutoGenerateColumns="False"   IsReadOnly="True" Height="400" Width="700" RowHeight="30"  HorizontalAlignment="Left" Margin="0 10 0 0" ColumnWidth="600" >
            <my:DataGrid.Columns>
                <my:DataGridTemplateColumn>
                    <my:DataGridTemplateColumn.CellTemplate>
                        <DataTemplate>
                            <StackPanel Margin="5 5 5 5" Width="600">                             
                                <HyperlinkButton Content="{Binding Title}" NavigateUri="{Binding Link}" TargetName="_blank"></HyperlinkButton>
                            </StackPanel>
                        </DataTemplate>
                    </my:DataGridTemplateColumn.CellTemplate>
                </my:DataGridTemplateColumn>
            </my:DataGrid.Columns>
            <my:DataGrid.RowDetailsTemplate>
                <DataTemplate>
                    <StackPanel Margin="5 5 5 5" Width="600" Height="75" >
                        <TextBlock Text="{Binding Description}" TextWrapping="Wrap" Width="600" FontSize="10"/>
                    </StackPanel>
                </DataTemplate>
            </my:DataGrid.RowDetailsTemplate>
        </my:DataGrid>

        <TextBox x:Name="Text1" HorizontalAlignment="Left" VerticalAlignment="Bottom" Padding="5"  Width="200" Text="Silverlight"></TextBox> 
        <Button Width="150" Height="25" HorizontalAlignment="Right" VerticalAlignment="Bottom" Content="Search!" Click="Button_Click"></Button>
    </Grid>
</UserControl>

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.ServiceModel.Syndication;
using System.Text;
using System.Threading;
using System.Windows;
using System.Windows.Controls;
using System.Xml;

namespace SLThreadPool
{
    public class FeedItem
    {
        public FeedItem(string description, string link, string title)
        {
            Description = description;
            Link = link;
            Title = title;
        }

        public String Description { get; set; }
        public string Link { get; set; }
        public string Title { get; set; }
    }

    public partial class Page : UserControl
    {
        private readonly string[] feedUrls = 
        {
              "http://digg.com/rss_search?area=all&type=both&age=7&search=",
             "http://66.102.1.103/blogsearch_feeds?hl=en&spell=1&oi=spell&ie=utf-8&num=100&output=rss&q=",
             "http://lab.msdn.microsoft.com/search/data.ashx?query=",
             "http://ittyurl.net/RSS2.aspx?t="
         };

        public SyndicationFeed bigFeed;
        public int ctr;
        public int feedsCount;
        public List<SyndicationItem> itemList;

        public Page()
        {
            InitializeComponent();
            bigFeed = new SyndicationFeed();
            itemList = new List<SyndicationItem>();
        }

        private void Button_Click(object sender, RoutedEventArgs e)
        {
            string srch = Text1.Text;
            feedsCount = feedUrls.Length;
            foreach (string url in feedUrls)
            {
                ThreadPool.QueueUserWorkItem(ProcessItem, url + srch);
            }
        }

        private void ProcessItem(object stateInfo)
        {
            var wc = new WebClient();
            wc.DownloadStringCompleted += wc_DownloadStringCompleted;
            wc.DownloadStringAsync(new Uri(stateInfo.ToString()));
        }

        private void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
        {
            try
            {
                string res = e.Result;
                if (res == "")
                {
                    feedsCount--;
                    return;
                }

                var ms = new MemoryStream(Encoding.UTF8.GetBytes(res));
                var s = new XmlReaderSettings();
                s.DtdProcessing = DtdProcessing.Parse;
                XmlReader reader = XmlReader.Create(ms, s);
                SyndicationFeed feed = SyndicationFeed.Load(reader);
                foreach (SyndicationItem itm in feed.Items)
                    itemList.Add(itm);
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex.ToString());
            }

            ctr++;
            //Display some progress
            Dispatcher.BeginInvoke(() => Text1.Text = "Feed: "+ctr.ToString() +" Items=" +itemList.Count.ToString());
            Debug.WriteLine(itemList.Count.ToString());

            if (ctr >= feedsCount)
            {
                bigFeed.Items = itemList;
                Dispatcher.BeginInvoke(() => BindGrid());
            }
        }

        private void BindGrid()
        {
            dgFeeds.RowDetailsVisibilityMode = DataGridRowDetailsVisibilityMode.VisibleWhenSelected;        
            Text1.Text = bigFeed.Items.Count() + " Items.";
            // Create custom simple Items collection to bind to grid
            var items = new List<FeedItem>();
            foreach (SyndicationItem itom in bigFeed.Items)
            {
                items.Add(new FeedItem(itom.Summary.Text, itom.Links[0].Uri.ToString(), itom.Title.Text));
            }
            dgFeeds.ItemsSource = items;
        }
    }
}

private void BindGrid()

dgFeeds.RowDetailsVisibilityMode = DataGridRowDetailsVisibilityMode.VisibleWhenSelected;

Text1.Text = bigFeed.Items.Count() + " Items.";

// Create custom simple Items collection to bind to grid

var items = new List<FeedItem>();

foreach (SyndicationItem itom in bigFeed.Items)

items.Add(new FeedItem(itom.Summary.Text, itom.Links[0].Uri.ToString(), itom.Title.Text));

Thanks for working this solution out, it was very helpful.  I wanted to download n images each on a seperate thread which I was able to do with you example.  With Silverlight 2 is there a way to do this using a backgroundworker and the event model?  I tried by calling bw.RunWorkerAsync inside my foreach loop but DoWork only gets called once.  i don't want to create n backgroundworkers because that seems ineffecient.  Maybe that thinking is not correct though?