How to improve the Performance of FtpWebRequest?

Question

I have an application written in .NET 3.5 that uses FTP to upload/download files from a server. The app works fine but there are performance issues:

1. It takes a lot of time to make connection to the FTP server. The FTP server is on a different network and has Windows 2003 Server (IIS FTP). When multiple files are queued for upload, the change from one file to another creates a new connection using FTPWebRequest and it takes a lot of time (around 8-10 seconds).

2. Is is possible to re-use the connection? I am not very sure about the KeepAlive property. Which connections are kept alive and reused.

3. The IIS-FTP on Windows Server 2003 does not support SSL so anyone can easily see the username/password through a packet sniffer such as WireShark. I found that windows Server 2008 supports SSL over FTP in its new version if IIS 7.0.

I basically want to improve the upload/download performance of my application. Any ideas will be appreciated.

** Please note that 3 is not an issue but I would like people to have comments on it

Answer 1

I have done some experimentation (uploading about 20 files on various sizes) on FtpWebRequest with the following factors

KeepAlive = true/false

ftpRequest.KeepAlive = isKeepAlive;

Connnection Group Name = UserDefined or null

ftpRequest.ConnectionGroupName = "MyGroupName";

Connection Limit = 2 (default) or 4 or 8

ftpRequest.ServicePoint.ConnectionLimit = ConnectionLimit;

Mode = Synchronous or Async

see this example

My results:

1. Use ConnectionGroupName,KeepAlive=true took (21188.62 msec)

2. Use ConnectionGroupName,KeepAlive=false took (53449.00 msec)

3. No ConnectionGroupName,KeepAlive=false took (40335.17 msec)

4. Use ConnectionGroupName,KeepAlive=true;async=true,connections=2 took (11576.84 msec)

5. Use ConnectionGroupName,KeepAlive=true;async=true,connections=4 took (10572.56 msec)

6. Use ConnectionGroupName,KeepAlive=true;async=true,connections=8 took (10598.76 msec)

Conclusions

1. FtpWebRequest has been designed to support an internal connection pool. To ensure, the connection pool is used, we must make sure the ConnectionGroupName is being set.

2. Setting up a connection is expensive. If we are connecting to the same ftp server using the same credentials, having the keep alive flag set to true will minimize the number of connections setup.

3. Asynchronous is the recommended way if you have a lot of files to ftp.

4. The default number of connections is 2. In my environment, a connection limit of 4 will give to me the most overall performance gain. Increasing the number of connections may or may not improve performance. I would recommend that you have the connection limit as a configuration parameter so that you can tune this parameter in your environment.

Hope you would find this useful.