Code

PagedAttention operation#867

Code

Closed

ilya-lavrenov wants to merge 22 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino_contrib:masterfrom ilya-lavrenov:paged-attentionilya-lavrenov/openvino_contrib:paged-attentionCopy head branch name to clipboard

Commits

Commits on Feb 2, 2024

• Introduced first version of PagedAttention operation

  ilya-lavrenov

  committed
  dc4553a

  Copy full SHA for dc4553a

• Added CPU implementation for reshape_and_cache

  ilya-lavrenov

  committed
  1542b7a

  Copy full SHA for 1542b7a

• Attention can be compiled

  ilya-lavrenov

  committed
  cbeff36

  Copy full SHA for cbeff36

• Temp commit

  ilya-lavrenov

  committed
  cb708a0

  Copy full SHA for cb708a0

Commits on Feb 5, 2024

• Creare SPDA operation for prefill case

  ilya-lavrenov

  committed
  b340cb3

  Copy full SHA for b340cb3

• Finalize PagedAttention implementation

  ilya-lavrenov

  committed
  b6c35d6

  Copy full SHA for b6c35d6

• Added framework map to map to PyTorch module

  ilya-lavrenov

  committed
  e9d5e44

  Copy full SHA for e9d5e44

• Use Tensor::set_shape instead of view

  ilya-lavrenov

  committed
  04e7014

  Copy full SHA for 04e7014

Commits on Feb 7, 2024

• Intermediate state

  ilya-lavrenov

  committed
  675de7b

  Copy full SHA for 675de7b

Commits on Feb 11, 2024

• Passes accuracy

  ilya-lavrenov

  committed
  980fabc

  Copy full SHA for 980fabc

Commits on Feb 12, 2024

• Merge remote-tracking branch 'upstream/master' into paged-attention

  ilya-lavrenov

  committed
  b36e57a

  Copy full SHA for b36e57a

Commits on Feb 13, 2024

• No explicit registration of frontend::OpExtension, removed unnecessary extension opset.

  slyalin

  committed
  7f9ae41

  Copy full SHA for 7f9ae41

Commits on Feb 14, 2024

• Merge pull request #1 from slyalin/simplified_extension_registration

  Show description for e1eda1a
  ilya-lavrenov

  authored
  e1eda1a

  Copy full SHA for e1eda1a

Commits on Feb 15, 2024

• Merge remote-tracking branch 'upstream/master' into paged-attention

  ilya-lavrenov

  committed
  3a368f8

  Copy full SHA for 3a368f8

• Remove print of performance

  ilya-lavrenov

  committed
  5b17873

  Copy full SHA for 5b17873

• Supported GQA, MQA

  ilya-lavrenov

  committed
  f4bc7c8

  Copy full SHA for f4bc7c8

Commits on Feb 17, 2024

• microsoft/phi-1_5 works

  ilya-lavrenov

  committed
  264e86d

  Copy full SHA for 264e86d

Commits on Feb 18, 2024

• Suppport mistral with sliding_windows

  ilya-lavrenov

  committed
  793350d

  Copy full SHA for 793350d

• Fixed sliding_window

  ilya-lavrenov

  committed
  c4705ac

  Copy full SHA for c4705ac

Commits on Feb 19, 2024

• Passing alibi_slopes and sliding_window to PagedAttension kernel

  slyalin

  committed
  bf920e3

  Copy full SHA for bf920e3

• Fixed overflow with maximal sliding_window

  slyalin

  committed
  4e1cfd3

  Copy full SHA for 4e1cfd3

• Merge pull request #2 from slyalin/paged_attention_remaining_parameters

  Show description for d6388a9
  ilya-lavrenov

  authored
  d6388a9

  Copy full SHA for d6388a9