Quantcast
Channel: SQL Server Database Engine forum
Viewing all articles
Browse latest Browse all 12963

Best practices for batch processing without SSIS

$
0
0

Hi,

The gist of this question is in general how should a C# client insert/update batches records using stored procedures. The ideas I can think of are:

1) create 1 SP with a parameter of type XML, and pass say 100 records at a time, on 1 thread.  The SP reads the XML as a table and does a single INSERT.

2) create 1 SP with many parameters, that inserts 1 records.  I can either build a big block of EXEC statements for say 100 records at a time, or call the SP 1 and a time, on 1 thread.  Obviously this seems the slowest.

3) Parallel processing version of either of the above: Pass 100 records at a time via XML parameter, big block of EXEC statements, or 1 at a time, and use PLINQ to make multiple connections to the database.

The records will be fairly wide, substantial records.

Which scenario is likely to be fastest and avoid lock contention?

(We are doing batch processing and there is not a current SSIS infrastructure, so it's manual: fetch data, call web services, update batches.  I need a batch strategy that doesn't involve SSIS - yet).

Thanks.


Viewing all articles
Browse latest Browse all 12963

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>