Cherry pick from 93b72f20b1: Check if process still ACTUALLY running and if not go ahead and start the test