You need to close the file handles owned by your current process. To do this:
- Use the NtQuerySystemInformationAPI with undocumentedSystemHandleInformationparameter.
- This gives you an array of all handles open in the system
- Iterate over array and select only ones which match your process PID, and are file handles
- You can then further narrow it down using GetFinalPathNameByHandleto get paths of opened files e.g. you could select specific file names or all files withtmpin their name.
- For any files you want to delete, call CloseHandle()to force close the handle, then of courseDeleteFile()on the path.
Some code (without any error checking):
SYSTEM_HANDLE_INFORMATION* pInfo=NULL;
DWORD dwSize=0;
NTSTATUS status=0;
do
{
  // keep reallocing until buffer is big enough
  status = NtQuerySystemInformation(SystemHandleInformation, pInfo, dwSize, &dwSize);
  if (status==STATUS_INFO_LENGTH_MISMATCH)
     pInfo = (SYSTEM_HANDLE_INFORMATION*)realloc(pInfo, dwSize);
} while(status!=0);
// iterate over every handle
for (DWORD i=0; i<pInfo->dwCount; i++)
{
  if (pInfo->handles[i].ProcessID==GetCurrentProcessId() && pInfo->handles[i].HandleType==28)
  {
     TCHAR szPath[MAX_PATH];
     GetFinalPathNameByHandle((HANDLE)pInfo->handles[i].HandleNumber, szPath, MAX_PATH, 0);
     if (_tcsstr(szFilePath, L"filename_I_want_to_delete"))
     {
       CloseHandle((HANDLE)pInfo->handles[i].HandleNumber);
       DeleteFile(szPath);
     }
  }
}
This is assuming all the files you need to delete are owned by the process doing the deletion. If any of the files belong to another process you will need an extra step using DuplicateHandle() with the DUPLICATE_CLOSE_SOURCE option. Assuming you have suitable permissions this gives you the handle, which you can then close and delete the file as before.
There is some good sample code here.