I am having a problem and I want to ask some opinions on the best approach to accomplish this task.
The process that I am implementing is a purge solution for big databases. The tables can have a variable amount of rows that can go from a few thousands to big millions. What the process will do is to delete just the data for a specific year. Partitioning is not an option here.
I have a some initial key tables that were chosen to start the process, and from these tables I will recursively delete data, using some stored procedures built for this purpose (in order to maintain data integrity and delete only the necessary data).
I am using SSIS to be able to start the data deletion in multiple tables at the same time but I am having problems with locks.
The deletion process as the following steps:
- Drop all indexes on table
- Delete data in batches
- Rebuild the primary key indexes (this needs to be done for performance, because there are table with some primary keys non clustered)
I have tried:
- Using transactions and named transactions (still have locks)
- Using sp_getapplock (still have locks)
In SSIS I have serializable as the default isolation for the transactions.
My main question is, because I have to do some DML and DDL in the same transaction, and this will take some time to execute, I want all tasks that need to obtain a certain lock to be waiting until the lock is released and not be targeted as deadlock victim (very the same as a mutex).
The database will have no activity during this operation, only this process will be executed.
This is the code I use to delete the data, and here is where the deadlocks are coming from.
select @query = N' BEGIN TRANSACTION; DECLARE @result int; EXEC @result = sp_getapplock @Resource = ''[dbo].'+@dep_tbl_nm+''', @LockMode = ''Exclusive'', @LockOwner = ''Session'', @LockTimeout = -1; EXEC CreateIndexes ''dbo.'+@dep_tbl_nm+''' EXEC CreateCoveringIndexes '''+@tbl_nm+''','''+@dep_tbl_nm+''','''+@dep_tmp_tbl_nm+''' WHILE(1=1) BEGIN DELETE TOP(50000) FROM a OUTPUT deleted.* into '+@dep_tmp_tbl_nm+' FROM dbo.'+ @dep_tbl_nm + ' AS A INNER JOIN '+@tmp_tbl_nm+' AS B ON ' + @on_list +' if(@@ROWCOUNT = 0) break; END exec (''ALTER INDEX ALL ON '+@dep_tbl_nm+' REBUILD WITH (FILLFACTOR = 80)'') COMMIT TRANSACTION;' print(@query) exec(@query)
Comments are welcome
Regards