Hi!
Unfortunately, we encounter a deadlock situation when using the SqlBulkCopy class to insert rows into a single table using multiple machines / processes concurrently.
In order to achieve best database write-out performance we use the SqlBulkCopy class with SqlBulkCopyOptions.TableLock. According to the msdn documentation SqlBulkCopy will "obtain a bulk update lock for the duration of the bulk copy operation" . Reading about bulk update locks it is stated that "Bulk update (BU) locks allow multiple threads to bulk load data concurrently into the same table while preventing other processes that are not bulk loading data from accessing the table."
Investigating our deadlock graph I can see that 2 processes are holding a BU lock on the destination table while both processes now request an exclusive X lock on that table. Hence, the deadlock situation. The destination table is a heap so there should not be any index update problems.
After some more investigation using the official Microsoft SqlBulkCopy sample from Adventureworks I realized that performing a bulkcopy operation to an empty table leads to an acquisition and release of a BU lock (on that table). Hence, no concurrency issue here.
However, performing a bulkcopy operation on a non-empty table leads to a BU lock (on that table) followed by an X (exclusive) lock. Hence the pattern I see in my deadlock graph.
Does anyone know why SqlBulkCopy obtains an X lock during bulk-insert in case the destination table is not empty?
How can I concurrently bulk-insert data in a single non empty table using multiple threads in a safe way?
I think originally we used the default row level locking but also ran into conversion deadlock issues when threads were escalating their row locks to page and table level concurrently.
Any insights and help is greatly appreciated.
Thanks!
Stefan
Stefan
Unfortunately, we encounter a deadlock situation when using the SqlBulkCopy class to insert rows into a single table using multiple machines / processes concurrently.
In order to achieve best database write-out performance we use the SqlBulkCopy class with SqlBulkCopyOptions.TableLock. According to the msdn documentation SqlBulkCopy will "obtain a bulk update lock for the duration of the bulk copy operation" . Reading about bulk update locks it is stated that "Bulk update (BU) locks allow multiple threads to bulk load data concurrently into the same table while preventing other processes that are not bulk loading data from accessing the table."
Investigating our deadlock graph I can see that 2 processes are holding a BU lock on the destination table while both processes now request an exclusive X lock on that table. Hence, the deadlock situation. The destination table is a heap so there should not be any index update problems.
After some more investigation using the official Microsoft SqlBulkCopy sample from Adventureworks I realized that performing a bulkcopy operation to an empty table leads to an acquisition and release of a BU lock (on that table). Hence, no concurrency issue here.
However, performing a bulkcopy operation on a non-empty table leads to a BU lock (on that table) followed by an X (exclusive) lock. Hence the pattern I see in my deadlock graph.
Does anyone know why SqlBulkCopy obtains an X lock during bulk-insert in case the destination table is not empty?
How can I concurrently bulk-insert data in a single non empty table using multiple threads in a safe way?
I think originally we used the default row level locking but also ran into conversion deadlock issues when threads were escalating their row locks to page and table level concurrently.
Any insights and help is greatly appreciated.
Thanks!
Stefan
Stefan