Very slow UpdatePanel refresh when containing big ListBoxes of DropDownLists

This is just a copy of this really nice entry here and hence not an original post: siderite

Update: this fix is now on CodePlex: CodePlex. Get the latest version from there.

The scenario is pretty straightforward: a ListBox or DropDownList or any control that renders as a Select html element with a few thousand entries or more causes an asynchronous UpdatePanel update to become incredibly slow on Internet Explorer and reasonably slow on FireFox, keeping the CPU to 100% during this time. Why is that?

Delving into the UpdatePanel inner workings one can see that the actual update is done through an _updatePanel Javascript function. It contains three major parts: it runs all dispose scripts for the update panel, then it executes _destroyTree(element) and then sets element.innerHTML to whatever content it contains. Amazingly enough, the slow part comes from the _destroyTree function. It recursively takes all html elements in an UpdatePanel div and tries to dispose them, their associated controls and their associated behaviours. I don't know why it takes so long with select elements, all I can tell you is that childNodes contains all the options of a select and thus the script tries to dispose every one of them, but it is mostly an IE DOM issue.

What is the solution? Enter the ScriptManager.RegisterDispose method. It registers dispose Javascript scripts for any control during UpdatePanel refresh or delete. Remember the first part of _updatePanel? So if you add a script that clears all the useless options of the select on dispose, you get instantaneous update!

First attempt: I used select.options.length=0;. I realized that on Internet Explorer it took just as much to clear the options as it took to dispose them in the _destroyTree function. The only way I could make it work instantly is with select.parentNode.removeChild(select). Of course, that means that the actual selection would be lost, so something more complicated was needed if I wanted to preserve the selection in the ListBox.

Second attempt: I would dynamically create another select, with the same id and name as the target select element, but then I would populate it only with the selected options from the target, then use replaceChild to make the switch. This worked fine, but I wanted something a little better, because I would have the same issue trying to dynamically create a select with a few thousand items.

Third attempt: I would dynamically create a hidden input with the same id and name as the target select, then I would set its value to the comma separated list of the values of the selected options in the target select element. That should have solved all problems, but somehow it didn't. When selecting 10000 items and updating the UpdatePanel, it took about 5 seconds to replace the select with the hidden field, but then it took minutes again to recreate the updatePanel!

Here is the piece of code that fixes most of the issues so far:

 /// 

    /// Use it in Page_Load.
    /// lbTest is a ListBox with 10000 items
    /// updMain is the UpdatePanel in which it resides
    /// 
    private void RegisterScript()
    {
        string script =
            string.Format(@"
var select=document.getElementById('{0}'); 
if (select) {{
    // first attempt
    //select.parentNode.removeChild(select);


    // second attempt
//    var stub=document.createElement('select');
//    stub.id=select.id;
//    for (var i=0; i
//        if (select.options[i].selected) {{
//            var op=new Option(select.options[i].text,select.options[i].value);
//            op.selected=true;
//            stub.options[stub.options.length]=op;
//        }}
//    select.parentNode.replaceChild(stub,select);


    // third attempt
    var stub=document.createElement('input');
    stub.type='hidden';
    stub.id=select.id;
    stub.name=select.name;
    stub._behaviors=select._behaviors;
    var val=new Array();
    for (var i=0; i
        if (select.options[i].selected) {{
            val[val.length]=select.options[i].value;
        }}
    stub.value=val.join(',');
    select.parentNode.replaceChild(stub,select);
    
}};",
                          lbTest.ClientID);
        ScriptManager sm = ScriptManager.GetCurrent(this);
        if (sm != null) sm.RegisterDispose(lbTest, script);
    }

What made the whole thing be still slow was the initialization of the page after the UpdatePanel updated. It goes all the way to the WebForms.js file embedded in the System.Web.dll (NOT System.Web.Extensions.dll), so part of the .NET framework. What it does it take all the elements of the html form (for selects it takes all selected options) and adds them to the list of postbacked controls within the WebForm_InitCallback javascript function.

The code looks like this:

if (tagName == "select") {
            var selectCount = element.options.length;
            for (var j = 0; j < selectCount; j++) {
                var selectChild = element.options[j];
                if (selectChild.selected == true) {
                    WebForm_InitCallbackAddField(element.name, element.value);
                }
            }
        }

function WebForm_InitCallbackAddField(name, value) {
    var nameValue = new Object();
    nameValue.name = name;
    nameValue.value = value;
    __theFormPostCollection[__theFormPostCollection.length] = nameValue;
    __theFormPostData += name + "=" + WebForm_EncodeCallback(value) + "&";
}

That is funny enough, because __theFormPostCollection is only used to simulate a postback by adding a hidden input for each of the collection's items to a xmlRequestFrame (just like my code above) in the function WebForm_DoCallback which in turn is called only in the GetCallbackEventReference(string target, string argument, string clientCallback, string context, string clientErrorCallback, bool useAsync) method of the ClientScriptManager which in turn is only used in rarely used scenarios with the own mechanism of javascript callbacks of GridViews, DetailViews and TreeViews.

And that is it!! The incredible delay in this javascript code comes from a useless piece of code! The whole WebForm_InitCallback function is useless most of the time!

So I added this little bit of code to the RegisterScript method and it all went marvelously fast: 10 seconds for 10000 selected items.

    string script = @"WebForm_InitCallback=function() {};";
        ScriptManager.RegisterStartupScript(this, GetType(), "removeWebForm_InitCallback", script, true);

ASP.NET 3.5 Page Life Cycle

original source:  msdn

Page request

The page request occurs before the page life cycle begins. When the page is requested by a user, ASP.NET determines whether the page needs to be parsed and compiled (therefore beginning the life of a page), or whether a cached version of the page can be sent in response without running the page.

Start

In the start step, page properties such as Request and Response are set. At this stage, the page also determines whether the request is a postback or a new request and sets the IsPostBack property. Additionally, during the start step, the page's UICulture property is set.

Page initialization

During page initialization, controls on the page are available and each control's UniqueID property is set. Any themes are also applied to the page. If the current request is a postback, the postback data has not yet been loaded and control property values have not been restored to the values from view state.

Load

During load, if the current request is a postback, control properties are loaded with information recovered from view state and control state.

Validation

During validation, the Validate method of all validator controls is called, which sets the IsValid property of individual validator controls and of the page.

Postback event handling

If the request is a postback, any event handlers are called.

Rendering

Before rendering, view state is saved for the page and all controls. During the rendering phase, the page calls the Render method for each control, providing a text writer that writes its output to the OutputStream of the page's Response property.

Unload

Unload is called after the page has been fully rendered, sent to the client, and is ready to be discarded. At this point, page properties such as Response and Request are unloaded and any cleanup is performed.

Reading XSD & XSL files from embedded resources of an assembly

*The following post is on VS 2008 & .NET Framework 3.5

Say we have an assembly called "MyAssembly" which could be seen on the project settings.

Let's first start with reading anything from embedded resources:

If we have a resource, and .xml file in this case, named as "MyXmlFile.xml", the resource name for this file would be "MyAssembly.MyXmlFile.xml"

Here's the code that streams the contents of this file:

Stream xmlStream = a.GetManifestResourceStream("CRFSCommandLine.WorkOrderTransform.xsl");

Now we have a stream that we can play with, let's now just display what it has in it:

StreamReader sr = new StreamReader(xmlStream);
Console.WriteLine(sr.ReadToEnd());

Now let's move on to doing validation and transform:

Here's what a validation would look like

Stream xsdStream= a.GetManifestResourceStream("MyAssembly.MySchema.xsd");
XmlSchemaSet schemas = new XmlSchemaSet();
XmlSchema x = XmlSchema.Read(schemaStream,
new ValidationEventHandler(HandleValidationEvent));
schemas.Add(x);


readerDoc.WhitespaceHandling = WhitespaceHandling.None;

//XmlValidatingReader is obsolete with .NET 3.5, we need to use XmlReader and //appropriate XmlReaderSettings settings
XmlReaderSettings validationReaderSettings = new XmlReaderSettings();
validationReaderSettings.ValidationType = ValidationType.Schema;
validationReaderSettings.Schemas.Add(schemas);
validationReaderSettings.ValidationEventHandler += new ValidationEventHandler(HandleValidationEvent);

//docReader is the XmlReader we have for the XML document we want to validate
XmlReader newReader = XmlReader.Create(readerDoc, validationReaderSettings);

while ( newReader.Read());
newReader.Close();

This should do the trick, below is what the transformation looks like:

XmlDocument originalDoc = new XmlDocument();
originalDoc.Load(fileName);

XslCompiledTransform trans = new XslCompiledTransform(true);
trans.Load(new XmlTextReader(transformStream));


XmlTextWriter xtw = new XmlTextWriter("TransformOutput.xml", Encoding.Unicode);
xtw.Formatting = Formatting.Indented;
trans.Transform(originalDoc.CreateNavigator(), xtw);

StringWriter sw = new StringWriter();
trans.Transform(originalDoc.CreateNavigator(), null, sw); //sw.ToString() would be //the string output here


That's all