What is the recommended batch size for SqlBulkCopy?

.net sql-server performance ado.net
Jonathan Allen picture Jonathan Allen · Apr 23, 2009 · Viewed 55.1k times · Source

What is the recommended batch size for SqlBulkCopy? I'm looking for a general formula I can use as a starting point for performance tuning.

Answer

Alric picture Alric · Jul 24, 2009

I have an import utility sitting on the same physical server as my SQL Server instance. Using a custom IDataReader, it parses flat files and inserts them into a database using SQLBulkCopy. A typical file has about 6M qualified rows, averaging 5 columns of decimal and short text, about 30 bytes per row.

Given this scenario, I found a batch size of 5,000 to be the best compromise of speed and memory consumption. I started with 500 and experimented with larger. I found 5000 to be 2.5x faster, on average, than 500. Inserting the 6 million rows takes about 30 seconds with a batch size of 5,000 and about 80 seconds with batch size of 500.

10,000 was not measurably faster. Moving up to 50,000 improved the speed by a few percentage points but it's not worth the increased load on the server. Above 50,000 showed no improvements in speed.

This isn't a formula, but it's another data point for you to use.

Related questions

Is it better to execute many sql commands with one connection, or reconnect every time?

Here's my test code, which seems to suggest that it's better to connect multiple times instead of connecting just once. Am I doing something wrong? int numIts = 100; Stopwatch sw = new Stopwatch(); sw.Start(); using (SqlConnection connection = new SqlConnection(connectionParameters)) { connection.…

Read More
how to insert datetime into the SQL Database table?

How can I insert datetime into the SQL Database table ? Is there a way to insert this query through the insert command in C# / .NET?

Read More
C# Equivalent of SQL Server DataTypes

For the following SQL Server datatypes, what would be the corresponding datatype in C#? Exact Numerics bigint numeric bit smallint decimal smallmoney int tinyint money Approximate Numerics float real Date and Time date datetimeoffset datetime2 smalldatetime datetime time Character Strings …

Read More