Fix intermittent time based test failures