Enable split rotary fusion for batched model

[masahi]

This is based on the recent work of @Lunderberg that enabled split-rotary fusion for single-sequence prefill. The same technique can be used to enable fusion for both prefill / decode in our batched model. This optimization can improve perf by reducing kernel launches and runtime memory allocations (thus more cache blocks can be allocated).

The latest https://github.com/masahi/tvm/tree/contrib-vllm (in particular, the commit masahi/tvm@a0cce50) is needed to use this feature.

Perf improvement is small but noticeable.

Vicuna 7B fp16

Before

Throughput: 7.04 requests/s, 3366.86 tokens/s

After

Throughput: 7.17 requests/s, 3427.27 tokens/s

llama 2 13B fp16

Before

Throughput: 3.79 requests/s, 1814.50 tokens/s

After

Throughput: 3.84 requests/s, 1834.09 tokens/s