How to download directories from FTP using VB.NET

Question

I am trying to download multiple directories from FTP server to my local machine,

I have tried this,

Const localFile As String = "C:\Documents and Settings\cr\Desktop\T\New Folder\"
Const remoteFile As String = "textbox.Text"
Const host As String = "ftp://ftp.example.com"
Const username As String = "username"
Const password As String = "password"

For i1 = 0 To ListBox1.SelectedItems.Count - 1
    Dim li As New ListViewItem
    li = ListView1.Items.Add(ListBox1.SelectedItems(i1))
    Dim URI1 As String = host + remoteFile & "/" & ListBox1.SelectedItems(i1)
    Dim ftp1 As System.Net.FtpWebRequest = CType(FtpWebRequest.Create(URI1), FtpWebRequest)
    ftp1.Credentials = New System.Net.NetworkCredential(username, password)
    ftp1.KeepAlive = False
    ftp1.UseBinary = True
    ftp1.Method = System.Net.WebRequestMethods.Ftp.DownloadFile
    Using response As System.Net.FtpWebResponse = CType(ftp1.GetResponse, System.Net.FtpWebResponse)
        Using responseStream As IO.Stream = response.GetResponseStream

            Dim length As Integer = response.ContentLength
            Dim bytes(length) As Byte

            'loop to read & write to file
            Using fs As New IO.FileStream(localFile & ListBox1.SelectedItems(i1), IO.FileMode.Create)
                Dim buffer(2047) As Byte
                Dim read As Integer = 1

                Do
                    read = responseStream.Read(buffer, 0, buffer.Length)
                    fs.Write(buffer, 0, read)

                Loop Until read = 0 'see Note(1)
                responseStream.Close()
                fs.Flush()
                fs.Close()
            End Using
            responseStream.Close()
        End Using

        response.Close()
    End Using
    li.BackColor = Color.Aquamarine
Next

But here the problem is that I am able to download multiple files from folders, but unable to download the sub directories and their contents from the main directory.

Basically the main directory consist of files and sub directories both. So is there any possible way to download sub directory and its contents from FTP?

Thanks in advance.

Answer 1

Translating my answer to C# Download all files and subdirectories through FTP to VB.NET:

The FtpWebRequest does not have any explicit support for recursive file download (or any other recursive operation). You have to implement the recursion yourself:

• List the remote directory
• Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)

A tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.

Your options are:

• Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
• You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
• You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)

Sub DownloadFtpDirectory(
        url As String, credentials As NetworkCredential, localPath As String)
    Dim listRequest As FtpWebRequest = WebRequest.Create(url)
    listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails
    listRequest.Credentials = credentials

    Dim lines As List(Of String) = New List(Of String)

    Using listResponse As FtpWebResponse = listRequest.GetResponse(),
          listStream As Stream = listResponse.GetResponseStream(),
          listReader As StreamReader = New StreamReader(listStream)
        While Not listReader.EndOfStream
            lines.Add(listReader.ReadLine())
        End While
    End Using

    For Each line As String In lines
        Dim tokens As String() =
            line.Split(New Char() {" "}, 9, StringSplitOptions.RemoveEmptyEntries)
        Dim name As String = tokens(8)
        Dim permissions As String = tokens(0)

        Dim localFilePath As String = Path.Combine(localPath, name)
        Dim fileUrl As String = url + name

        If permissions(0) = "d" Then
            If Not Directory.Exists(localFilePath) Then
                Directory.CreateDirectory(localFilePath)
            End If
            DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath)
        Else
            Dim downloadRequest As FtpWebRequest = WebRequest.Create(fileUrl)
            downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile
            downloadRequest.Credentials = credentials

            Using downloadResponse As FtpWebResponse = downloadRequest.GetResponse(),
                  sourceStream As Stream = downloadResponse.GetResponseStream(),
                  targetStream As Stream = File.Create(localFilePath)
                Dim buffer As Byte() = New Byte(10240 - 1) {}
                Dim read As Integer
                Do
                    read = sourceStream.Read(buffer, 0, buffer.Length)
                    If read > 0 Then
                        targetStream.Write(buffer, 0, read)
                    End If
                Loop While read > 0
            End Using
        End If
    Next
End Sub

Use the function like:

Dim credentials As NetworkCredential = New NetworkCredential("user", "mypassword")
Dim url As String = "ftp://ftp.example.com/directory/to/download/"
DownloadFtpDirectory(url, credentials, "C:\target\directory")

---

If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.

For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:

' Setup session options
Dim SessionOptions As SessionOptions = New SessionOptions
With SessionOptions
    .Protocol = Protocol.Ftp
    .HostName = "ftp.example.com"
    .UserName = "user"
    .Password = "mypassword"
End With

Using session As Session = New Session()
    ' Connect
    session.Open(SessionOptions)

    ' Download files
    session.GetFiles("/directory/to/download/*", "C:\target\directory\*").Check()
End Using

Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.

The Session.GetFiles method is recursive by default.

(I'm the author of WinSCP)