How can I send MailMessages using multiple threads?

Question

I hope you guys will bear with me for my total lack of direction in case of Threading. I have to implement a Mail Queue Processing System where I have to send emails queued up in a database through a Windows Service.

It is not a producer-consumer pattern. I have fetched say 10 rows at a time into a datable. The datatable contains the serialized MailMessage object and SMTP sending details. If I have to use say a fixed number of threads (say 6 threads) Now how would I go about fetching a row from the datatable in a thread and then send the mail and again return to see if any more rows are remaining?

Any simple logic to implement this will do, preferably with a simple example in C#.

i am using .NET 3.5

Answer 1

Since sending emails is an I/O bound process, so spawning threads to send emails won't achieve much (if any) speedup.

If you're using the SMTP server that's part of Windows then when you "send" an email it doesn't actually get sent at that instant. It sits in a queue on the server and the server sends them as fast as it can. Sending emails is actually a slow process.

I guess what I'm saying is there are two options:

1. Just send them sequentially and see if that meets your performance requirements.
2. You could use a parallel programming concept called "Data Parallel" I've exampled it with examples in an blog post Data Parallel – Parallel Programming in C#/.NET

Basically, what you're doing is, you'll get all of your data (in one go). The reason is getting data in batches will also slow your process down so if you're interested in performance (which is why I'm guessing you're attempting to use threads), then don't do multiple rounds trips to the database server (which is also I/O bound on two levels, network I/O as well as disk I/O).

So get your data, and split it into chunks or partitions. This is all explained in the article I pointed to. The naive implementation would be the number of chunks equals the number of cores on the machine.

Each chunck is processed by one thread. When all threads are done, you're done. Withthe new features of the ThreadPool in .NET 4.0 (if you use Parallel.For or PLINQ or Tasks) you'll get some other benefits such as "work stealing" to further speed up the work.

Parallel.For/Parallel.ForEach will work well for you I'd think.

EDIT

Just noticed the .NET 3.5 requirement. Well the concepts still apply, but you don't have Parallel.For/ForEach. So here is an implementation (modified from my blog post) that uses the ThreadPool and using a Data Parallel technique.

    private static void SendEmailsUsingThreadPool(List<Recipient> recipients)
    {
      var coreCount = Environment.ProcessorCount;
      var itemCount = recipients.Count;
      var batchSize = itemCount / coreCount;

      var pending = coreCount;
      using (var mre = new ManualResetEvent(false))
      {
        for (int batchCount = 0; batchCount < coreCount; batchCount++)
        {
          var lower = batchCount * batchSize;
          var upper = (batchCount == coreCount - 1) ? itemCount : lower + batchSize;
          ThreadPool.QueueUserWorkItem(st =>
          {
            for (int i = lower; i < upper; i++)
              SendEmail(recipients[i]);
            if (Interlocked.Decrement(ref pending) == 0)
              mre.Set();
          });
        }
        mre.WaitOne();
      }      
    }

    private static void SendEmail(Recipient recipient)
    {
      //Send your Emails here
    }
  }

  class Recipient
  {
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string EmailAddress { get; set; }
  }

So, get your data and call SendEmailUsingThreadPool() passing it your data. Of course don't call your method that :). If you have a DataSet/DataTable then simply modify the implementation to accept a DataSet/DataTable. This methods takes care of partioning your data into chunks so you don't have to worry about any of that. Simply call it.