Revert "Parallel sampling eviction"

[sunggg]

Reverts #157

Multi-gpu has been broken after this PR has merged.
@masahi, would you follow-up? Let me revert this for now.

How to reproduce:

/opt/bin/cuda-reserve.py --num-gpu 2 python3 serve/benchmarks/benchmark_throughput.py --local-id mixtral-8x7b-instruct-v0.1-q0f16-presharded-2gpu --max-num-batched-tokens 16000 --dataset /opt/models/dataset/ShareGPT_V3_unfiltered_cleaned_split.json --greedy-sampling-ratio 0.0

Error:

terminate called after throwing an instance of 'tvm::runtime::InternalError'
  what():  [07:00:52] /home/spark/tvm/src/runtime/disco/nccl/nccl.cc:68: InternalError: Check failed: (e == cudaSuccess || e == cudaErrorCudartUnloading) is false: CUDA: an illegal memory access was encountered
Stack trace:
  0: tvm::runtime::nccl::SyncWorker()
  1: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::TypedPackedFunc<void ()>::AssignTypedLambda<void (*)()>(void (*)(), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)
  2: tvm::runtime::SyncWorker()
  3: tvm::runtime::DiscoWorker::Impl::MainLoop(tvm::runtime::DiscoWorker*)
  4: 0x00007f0b4a338792
  5: start_thread
        at /build/glibc-wuryBv/glibc-2.31/nptl/pthread_create.c:477
  6: __clone
        at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
  7: 0xffffffffffffffff


terminate called after throwing an instance of 'tvm::runtime::InternalError'
  what():  [07:00:53] /home/spark/tvm/src/runtime/cuda/cuda_device_api.cc:142: InternalError: Check failed: (e == cudaSuccess || e == cudaErrorCudartUnloading) is false: CUDA: an illegal memory access was encountered
Stack trace:
  0: tvm::runtime::CUDADeviceAPI::FreeDataSpace(DLDevice, void*)
  1: tvm::runtime::NDArray::Internal::DefaultDeleter(tvm::runtime::Object*)
  2: _ZN3tvm7runtime18SimpleObjAllocator12ArrayHandlerINS0_9ArrayNodeENS0_9ObjectRefEE8Del
  3: tvm::runtime::DiscoWorker::~DiscoWorker()
  4: tvm::runtime::WorkerProcess(int, int, long, long) [clone .cold]
  5: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::TypedPackedFunc<void (int, int, long, long)>::AssignTypedLambda<void (*)(int, int, long, long)>(void (*)(int, int, long, long), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)