How to speed adding items to a ListView?

Question

i'm adding a few thousand (e.g. 53,709) items to a WinForms ListView.

Attempt 1: 13,870 ms

foreach (Object o in list)
{
   ListViewItem item = new ListViewItem();
   RefreshListViewItem(item, o);
   listView.Items.Add(item);
}

This runs very badly. The obvious first fix is to call BeginUpdate/EndUpdate.

Attempt 2: 3,106 ms

listView.BeginUpdate();
foreach (Object o in list)
{
   ListViewItem item = new ListViewItem();
   RefreshListViewItem(item, o);
   listView.Items.Add(item);
}
listView.EndUpdate();

This is better, but still an order of magnitude too slow. Let's separate creation of ListViewItems from adding ListViewItems, so we find the actual culprit:

Attempt 3: 2,631 ms

var items = new List<ListViewItem>();
foreach (Object o in list)
{
   ListViewItem item = new ListViewItem();
   RefreshListViewItem(item, o);
   items.Add(item);
}

stopwatch.Start();

listView.BeginUpdate();
    foreach (ListViewItem item in items)
        listView.Items.Add(item));
listView.EndUpdate();

stopwatch.Stop()

The real bottleneck is adding the items. Let's try converting it to AddRange rather than a foreach

Attempt 4: 2,182 ms

listView.BeginUpdate();
listView.Items.AddRange(items.ToArray());
listView.EndUpdate();

A bit better. Let's be sure that the bottleneck isn't in the ToArray()

Attempt 5: 2,132 ms

ListViewItem[] arr = items.ToArray();

stopwatch.Start();

listView.BeginUpdate();
listView.Items.AddRange(arr);
listView.EndUpdate();

stopwatch.Stop();

The limitation seems to be adding items to the listview. Maybe the other overload of AddRange, where we add a ListView.ListViewItemCollection rather than an array

Attempt 6: 2,141 ms

listView.BeginUpdate();
ListView.ListViewItemCollection lvic = new ListView.ListViewItemCollection(listView);
lvic.AddRange(arr);
listView.EndUpdate();

Well that's no better.

Now it's time to stretch:

• Step 1 - make sure no column is set to "auto-width":

  

  Check

• Step 2 - make sure the ListView isn't trying to sort the items each time i add one:

  

  Check

• Step 3 - Ask stackoverflow:

  

  Check

Note: Obviously this ListView is not in virtual mode; since you don't/cannot "add" items to a virtual list view (you set the VirtualListSize). Fortunately my question is not about a list view in virtual mode.

Is there anything i am missing that might account for adding items to the listview being so slow?

---

Bonus Chatter

i know the Windows ListView class can do better, because i can write code that does it in 394 ms:

ListView1.Items.BeginUpdate;
for i := 1 to 53709 do
   ListView1.Items.Add();
ListView1.Items.EndUpdate;

which when compared to the equivalent C# code 1,349 ms:

listView.BeginUpdate();
for (int i = 1; i <= 53709; i++)
   listView.Items.Add(new ListViewItem());
listView.EndUpdate();

is an order of magnitude faster.

What property of the WinForms ListView wrapper am i missing?

Answer 1

I took a look at the source code for the list view and I noticed a few things that may make the performance slow down by the factor of 4 or so that you're seeing:

in ListView.cs, ListViewItemsCollection.AddRange calls ListViewNativeItemCollection.AddRange, which is where I began my audit

ListViewNativeItemCollection.AddRange (from line: 18120) has two passes through the entire collection of values, one to collect all the checked items another to 'restore' them after InsertItems is called (they're both guarded by a check against owner.IsHandleCreated, owner being the ListView) then calls BeginUpdate.

ListView.InsertItems (from line: 12952), first call, has another traverse of the entire list then ArrayList.AddRange is called (probably another pass there) then another pass after that. Leading to

ListView.InsertItems (from line: 12952), second call (via EndUpdate) another pass through where they are added to a HashTable, and a Debug.Assert(!listItemsTable.ContainsKey(ItemId)) will slow it further in debug mode. If the handle isn't created, it adds the items to an ArrayList, listItemsArray but if (IsHandleCreated), then it calls

ListView.InsertItemsNative (from line: 3848) final pass through the list where it is actually added to the native listview. a Debug.Assert(this.Items.Contains(li) will additionally slow down performance in debug mode.

So there are A LOT of extra passes through the entire list of items in the .net control before it ever gets to actually inserting the items into the native listview. Some of the passes are guarded by checks against the Handle being created, so if you can add items before the handle is created, it may save you some time. The OnHandleCreated method takes the listItemsArray and calls InsertItemsNative directly without all the extra fuss.

You can read the ListView code in the reference source yourself and take a look, maybe I missed something.

In the March 2006 issue of MSDN Magazine there was an article called Winning Forms: Practical Tips for Boosting The Performance of Windows Forms Apps.

This article contained tips for improving the performance of ListViews, among other things. It seems to indicate that its faster to add items before the handle is created, but that you will pay a price when the control is rendered. Perhaps applying the rendering optimizations mentioned in the comments and adding the items before the handle is created will get the best of both worlds.

Edit: Tested this hypothesis in a variety of ways, and while adding the items before creating the handle is suuuper fast, it is exponentially slower when it goes to create the handle. I played with trying to trick it to create the handle, then somehow get it to call InsertItemsNative without going through all the extra passes, but alas I've been thwarted. The only thing I could think might be possible, is to create your Win32 ListView in a c++ project, stuff it with items, and use hooking to capture the CreateWindow message sent by the ListView when creating its handle and pass back a reference to the win32 ListView instead of a new window.. but who knows what the side affects there would be... a Win32 guru would need to speak up about that crazy idea :)