Fix random failures in ManualClientEndTest