How many threads for reading and writing to the hard disk?
i am developing an application that gathers a list with all the files of the hard drive and also afterwards it does write files to the hard drive.
I want to ask : what is the optimum number of concurrent threads that will do this task ?
I mean how many threads should i have that read the hard drive without making the hard drive to get slow because so many threads are reading it concurrently.
Thank you !
Answer
At first, I say one!
It actually depends whether the data to read need complex computations for being elaborated. In this case, it could be convenient to instantiate more than one thread to elaborate different disk data; but this is convenient only if you have a multiple CPU on the same system.
Otherwise, more than one thread make the HDD more stressed than necessary: concurrent reads from different threads will issue seek operations for reading the file blocks(*), introducing an overhead which could slow down the system, depending on the number of files read and the size of the files.
Read the files sequentially.
(*) The OS really tries to store the same file blocks sequentially in order to speed up the read operations. Disk fragmentation happens, so non-sequential fragments requires a seek operation which required really more time respect the read operation in the same place. Try to read multiple files in parallel, will cause a bunch of seeks because single file blocks are contiguous, while multiple files blocks could be not contiguous.