FTP directory partial listing with wildcards
First I asked that: ftp directory listing timeout. Huge number of subdirs. I got the answer.
Still because I can have hundred thousands of FTP objects in the directory it could take really long time to scan that. However I thought it might be possible to retrieve all the objects that begin with 'A' and then 'B' and so on... As it retrieves directories it could start processing them on the other thread without waiting till it gets the entire list.
Is it possible to do FTP directory listing with wildcards using standard FtpWebRequest?
Answer
The most recent update to the FTP specification (RFC 3659) explicitly forbids it. From section 2.2.2 of that specification, titled "Wildcarding" (emphasis mine):
For the commands defined in this specification, all pathnames are to be treated literally. That is, for a pathname given as a parameter to a command, the file whose name is identical to the pathname given is implied. No characters from the pathname may be treated as special or "magic", thus no pattern matching (other than for exact equality) between the pathname given and the files present in the NVFS of the server-FTP is permitted.
Clients that desire some form of pattern matching functionality must obtain a listing of the relevant directory, or directories, and implement their own file name selection procedures.
That said, if your server supports it, you could still use the FtpWebRequest class, but you'd have to process the response yourself to handle the list of items, as the .NET classes won't understand your server-specific extensions.