Code

Add chunked_prefill#38

Code

Open

gc-fu wants to merge 12 commits into0.5.4analytics-zoo/vllm:0.5.4from add-chunked-prefill-kernelanalytics-zoo/vllm:add-chunked-prefill-kernelCopy head branch name to clipboard

+1,307-81

Commits

Commits on Sep 25, 2024

• Refactor prepare_model_input

  gc-fu

  committed
  000f061

  Copy full SHA for 000f061

• Enable ipex-attn.py backend with new model_input

  gc-fu

  committed
  b6b1418

  Copy full SHA for b6b1418

• Kernel added

  gc-fu

  committed
  5963dd4

  Copy full SHA for 5963dd4

• Enable chunked-prefill branch

  gc-fu

  committed
  cd9ec72

  Copy full SHA for cd9ec72

• fix minor error in decoding path

  gc-fu

  committed
  9ec2a46

  Copy full SHA for 9ec2a46

• Fix generation error

  gc-fu

  committed
  ee4500b

  Copy full SHA for ee4500b

Commits on Sep 26, 2024

• Fix long input error for v1

  gc-fu

  committed
  aa65457

  Copy full SHA for aa65457

Commits on Oct 7, 2024

• Fix context_attention_v2

  gc-fu

  committed
  4ffaa66

  Copy full SHA for 4ffaa66

• add environment variable to control

  gc-fu

  committed
  6dbaf87

  Copy full SHA for 6dbaf87

Commits on Oct 8, 2024

• remove unused comments

  gc-fu

  committed
  59ec323

  Copy full SHA for 59ec323

• add head_dim 64

  gc-fu

  committed
  b641e44

  Copy full SHA for b641e44

• add more head_dim for v2

  gc-fu

  committed
  0831fd1

  Copy full SHA for 0831fd1