When a Case is deleted, the association between the Case and the Collection is deleted.  The Collection will no longer be associated with any case.  No Task or Set is affected by the Case deletion.

When a Collection is deleted, no Task is removed from the Destination Server. 
Collection Sets are never affected by Collection deletions.
 

In order to allow the Collection Tasks to be deleted from the Destination Server, the following ESA Property must be in place prior to the deletion of the Collection or Collection Task.
esa.icp.task.deleteOnDiskTaskData   = true

Once the above ESA property is in place, deleting a Collection or Task will remove the Task from the Destination Server.

Steps to Change / Add ESA Properties:

1. Navigate and Select Property Browser from Support Features

System | Support Features | Property Browser

2. Place the following values:

'Name of property to change': esa.icp.task.deleteOnDiskTaskData

'New Value (leave blank to remove)': true

3. Check the box for 'Confirm change. Are you sure?'

4. Press the Submit button

Note: Service restart is not required.