Deadlocks causing 'Server failed to resume the transaction' with NHibernate and distributed transactions

Question

We are having an issue when using NHibernate with distributed transactions.

Consider the following snippet:

//
// There is already an ambient distributed transaction
//
using(var scope = new TransactionScope()) {
    using(var session = _sessionFactory.OpenSession())
    using(session.BeginTransaction()) {
        using(var cmd = new SqlCommand(_simpleUpdateQuery, (SqlConnection)session.Connection)) {
            cmd.ExecuteNonQuery();
        }

        session.Save(new SomeEntity());
        session.Transaction.Commit();
    }
    scope.Complete();
}

Sometimes, when the server is under extreme load, we'll see the following:

1. The query executed with cmd.ExecuteNonQuery is chosen as a deadlock victim (we can see it in SQL Profiler), but no exception is raised.
2. session.Save fails with the error message, "The operation is not valid for the state of the transaction."
3. Every time this code is executed after that, session.BeginTransaction fails. The first few times, the inner exception varies (sometimes it is the deadlock exception that should have been raised in step 1). Eventually it stabilizes to "The server failed to resume the transaction. Desc:3800000177." or "New request is not allowed to start because it should come with valid transaction descriptor."

If left alone, the application will eventually (after seconds or minutes) recover from this condition.

Why is the deadlock exception not being reported in step 1? And if we can't resolve that, then how can we prevent our application from temporarily becoming unusable?

The issue has been reproduced in the following environments

• Windows 7 x64 and Windows Server 2003 x86
• SQL Server 2005 and 2008
• .NET 4.0 and 3.5
• NHibernate 3.2, 3.1 and 2.1.2

I've created a test fixture which will sometimes reproduce the issue for us. It is available here: http://wikiupload.com/EWJIGAECG9SQDMZ

Answer 1

We've finally narrowed this down to a cause.

When opening a session, if there is an ambient distributed transaction, NHibernate attaches an event handler to the Transaction.TransactionCompleted, which closes the session when the distributed transaction is completed. This appears to be subject to a race condition wherein the connection may be closed and returned to the pool before the deadlock error propagates across, leaving the connection in an unusable state.

The following code will reproduce the error for us occasionally, even without any load on the server. If there is extreme load on the server, it becomes more consistent.

using(var scope = new TransactionScope()) {
    //
    // Force promotion to distributed transaction
    //
    TransactionInterop.GetTransmitterPropagationToken(Transaction.Current);

    var connection = new SqlConnection(_connectionString);
    connection.Open();

    //
    // Close the connection once the distributed transaction is
    // completed.
    //
    Transaction.Current.TransactionCompleted += 
        (sender, e) => connection.Close();

    using(connection.BeginTransaction())
        //
        // Deadlocks but sometimes does not raise exception
        //
        ForceDeadlockOnConnection(connection);

    scope.Complete();
}

//
// Subsequent attempts to open a connection with the same
// connection string will fail
//

We have not settled on a solution, but the following things will eliminate the problem (while possibly having other consequences):

• Turning off connection pooling
• Using NHibernate's AdoNetTransactionFactory instead of AdoNetWithDistributedTransactionFactory
• Adding error handling that calls SqlConnection.ClearPool() when the "server failed to resume the transaction" error occurs

According to Microsoft (https://connect.microsoft.com/VisualStudio/feedback/details/722659/), the SqlConnection class is not thread-safe, and that includes closing the connection on a separate thread. Based on this response we have filed a bug report for NHibernate (http://nhibernate.jira.com/browse/NH-3023).