Code

Generate pipeline#334

Code

Merged

Wovchena merged 158 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino.genai:masterfrom pavel-esir:generate_pipelinepavel-esir/openvino.genai:generate_pipelineCopy head branch name to clipboardJun 7, 2024

Commits

Commits on Apr 17, 2024

• initial generate

  pavel-esir

  committed
  ba91fde

  Copy full SHA for ba91fde

• LLM pipeline

  pavel-esir

  committed
  9d85a0e

  Copy full SHA for 9d85a0e

• Added calculating for several batches

  pavel-esir

  committed
  b21c6c1

  Copy full SHA for b21c6c1

• Greedy search works

  pavel-esir

  committed
  e52e90d

  Copy full SHA for e52e90d

• rename to GenerationConfig

  pavel-esir

  committed
  745a804

  Copy full SHA for 745a804

• Add fluent interface

  pavel-esir

  committed
  8895ed0

  Copy full SHA for 8895ed0

• Update text_generation/causal_lm/cpp/generate_pipeline/generate_pipeline.hpp

  pavel-esir

  committed
  b24977d

  Copy full SHA for b24977d

• cosmetic changes in main

  pavel-esir

  committed
  c933ca0

  Copy full SHA for c933ca0

• greedy search with batches and left padding works

  pavel-esir

  committed
  c43e901

  Copy full SHA for c43e901

• combine LLModel with LLMPipeline

  pavel-esir

  committed
  5a914f6

  Copy full SHA for 5a914f6

• wip: enable calling tokenize/detokenize for LLMPipeline

  pavel-esir

  committed
  c1e0c9d

  Copy full SHA for c1e0c9d

• add callback to generate

  pavel-esir

  committed
  8d66353

  Copy full SHA for 8d66353

• cleanup generate_sample.cpp

  pavel-esir

  committed
  fa12da7

  Copy full SHA for fa12da7

• add speculative decoding

  pavel-esir

  committed
  5ceb9d5

  Copy full SHA for 5ceb9d5

Commits on Apr 19, 2024

• separate Tokenizer

  pavel-esir

  committed
  a5083c7

  Copy full SHA for a5083c7

Commits on Apr 23, 2024

• wip

  pavel-esir

  committed
  7692160

  Copy full SHA for 7692160

• add apply_chat_template

  pavel-esir

  committed
  964a5e8

  Copy full SHA for 964a5e8

Commits on Apr 24, 2024

• add start/stop conversation

  pavel-esir

  committed
  d3f6339

  Copy full SHA for d3f6339

Commits on Apr 26, 2024

• use text in streamer instead of raw tokens

  pavel-esir

  committed
  3776433

  Copy full SHA for 3776433

• fix difference between accumulating conversation as text and keeping history in KV cache

  pavel-esir

  committed
  e57aa4c

  Copy full SHA for e57aa4c

• cleanup

  pavel-esir

  committed
  d0c1341

  Copy full SHA for d0c1341

• add Jinja2cpp submodule

  pavel-esir

  committed
  8dcea1f

  Copy full SHA for 8dcea1f

Commits on May 2, 2024

• add ov namespace

  pavel-esir

  committed
  754a462

  Copy full SHA for 754a462

• return scores for batched outputs

  pavel-esir

  committed
  9b19c6f

  Copy full SHA for 9b19c6f

Commits on May 3, 2024

• add AnyMap

  pavel-esir

  committed
  9bf6caa

  Copy full SHA for 9bf6caa

• Merge remote-tracking branch 'upstream/master' into generate_pipeline

  pavel-esir

  committed
  39fd73c

  Copy full SHA for 39fd73c

• cleanup

  pavel-esir

  committed
  63d8f6d

  Copy full SHA for 63d8f6d

Commits on May 6, 2024

• before moving to pimpl

  pavel-esir

  committed
  a833760

  Copy full SHA for a833760

• move to separate include & src

  pavel-esir

  committed
  1681654

  Copy full SHA for 1681654

• pimpl implementation

  pavel-esir

  committed
  9fe73c6

  Copy full SHA for 9fe73c6

• temporary disable jinja2cpp

  pavel-esir

  committed
  053708f

  Copy full SHA for 053708f

Commits on May 7, 2024

• add python api draft, hide implementations from user & refactor implementation

  pavel-esir

  committed
  bd6849a

  Copy full SHA for bd6849a

• extract decoding methods to separate files

  pavel-esir

  committed
  62c471e

  Copy full SHA for 62c471e

• extended python api, added python api test

  pavel-esir

  committed
  f1d54f4

  Copy full SHA for f1d54f4

Commits on May 8, 2024

• remove call method

  pavel-esir

  committed
  3c82e11

  Copy full SHA for 3c82e11

• init

  Wovchena

  committed
  5543cee

  Copy full SHA for 5543cee

• add_subdirectory

  Wovchena

  committed
  abb8835

  Copy full SHA for abb8835

• add files

  Wovchena

  committed
  0998abc

  Copy full SHA for 0998abc

• add __init__.py

  Wovchena

  committed
  15492c4

  Copy full SHA for 15492c4

• removed set_streamer

  pavel-esir

  committed
  005d3fb

  Copy full SHA for 005d3fb

• use std::optional

  pavel-esir

  committed
  cc44bc8

  Copy full SHA for cc44bc8

• started to add Readme docs

  pavel-esir

  committed
  d8cab05

  Copy full SHA for d8cab05

• reoder Readme

  pavel-esir

  committed
  2535394

  Copy full SHA for 2535394

Commits on May 9, 2024

• rm generate_pipeline/python

  Wovchena

  committed
  95c1bfb

  Copy full SHA for 95c1bfb

• update Readme; cleanup LLMPipeline and add docstring

  pavel-esir

  committed
  4510f71

  Copy full SHA for 4510f71

• refactor folder structure

  pavel-esir

  committed
  507bc49

  Copy full SHA for 507bc49

Commits on May 10, 2024

• cleanup generation_config and ov::Tokenizer

  pavel-esir

  committed
  af747d4

  Copy full SHA for af747d4

• move includes to a separate openvino/genai folder

  pavel-esir

  committed
  c6620d9

  Copy full SHA for c6620d9

• Merge branch 'generate_pipeline' into package

  Wovchena

  committed
  59c3e0b

  Copy full SHA for 59c3e0b

• align names

  Wovchena

  committed
  be84345

  Copy full SHA for be84345

• Dont modify text_generation/causal_lm/cpp/CMakeLists.txt

  Wovchena

  committed
  bced64a

  Copy full SHA for bced64a

• rm -r text_generation/causal_lm/cpp/generate_pipeline/python-bindings/

  Wovchena

  committed
  f4e82b6

  Copy full SHA for f4e82b6

• fix build

  Wovchena

  committed
  5b2b0ca

  Copy full SHA for 5b2b0ca

• add tokenizers only once

  Wovchena

  committed
  0dd8f59

  Copy full SHA for 0dd8f59

• change cmake.source-dir

  Wovchena

  committed
  23638ff

  Copy full SHA for 23638ff

• restore openvino/genai inits

  Wovchena

  committed
  d8c5349

  Copy full SHA for d8c5349

• Integrate JinjaCpp

  ilya-lavrenov

  committed
  24faefe

  Copy full SHA for 24faefe

• install genai lib

  Wovchena

  committed
  598dda3

  Copy full SHA for 598dda3

• Merge pull request #2 from ilya-lavrenov/jinja-integration-pavel

  Show description for f274b93
  pavel-esir

  authored
  f274b93

  Copy full SHA for f274b93

• import openvino for win and lin

  Wovchena

  committed
  02d0eae

  Copy full SHA for 02d0eae

• Merge branch 'generate_pipeline' into package

  Wovchena

  committed
  e6695f3

  Copy full SHA for e6695f3

• put the line back

  Wovchena

  committed
  a27c5a7

  Copy full SHA for a27c5a7

• Added cmake build type before project clause

  ilya-lavrenov

  committed
  0849c41

  Copy full SHA for 0849c41

• one line properties

  Wovchena

  committed
  34cddff

  Copy full SHA for 34cddff

• Merge pull request #3 from ilya-lavrenov/cmake-build-type

  Show description for 023cf1e
  pavel-esir

  authored
  023cf1e

  Copy full SHA for 023cf1e

• Export API symbols

  ilya-lavrenov

  committed
  6a5d750

  Copy full SHA for 6a5d750

• Merge pull request #4 from ilya-lavrenov/generate_pipeline

  Show description for 27f385e
  pavel-esir

  authored
  27f385e

  Copy full SHA for 27f385e

• Merge branch 'generate_pipeline' into package

  Wovchena

  committed
  a9332f0

  Copy full SHA for a9332f0

• rename

  Wovchena

  committed
  9ef488c

  Copy full SHA for 9ef488c

• add .github/workflows/genai_lib.yml

  Wovchena

  committed
  4fad7d5

  Copy full SHA for 4fad7d5

• on: pull_request

  Wovchena

  committed
  51e03a2

  Copy full SHA for 51e03a2

• spelling

  Wovchena

  committed
  e23a7bb

  Copy full SHA for e23a7bb

• install openvino

  Wovchena

  committed
  fc5b753

  Copy full SHA for fc5b753

• add syntacis sugar for geenrate, optimize value passing by reference

  Show description for 09f8806
  pavel-esir

  committed
  09f8806

  Copy full SHA for 09f8806

Commits on May 11, 2024

• remove speculative decoding

  pavel-esir

  committed
  af22a8a

  Copy full SHA for af22a8a

Commits on May 13, 2024

• update

  Wovchena

  committed
  e7db7e8

  Copy full SHA for e7db7e8

• add rpath

  Wovchena

  committed
  f279363

  Copy full SHA for f279363

• add rpath to libopenvino.so

  Wovchena

  committed
  83d77c8

  Copy full SHA for 83d77c8

• py_generate_pipeline

  Wovchena

  committed
  167f924

  Copy full SHA for 167f924

• reorder tokenizer.cpp, add comments to BaseStreamer

  pavel-esir

  committed
  a111a3f

  Copy full SHA for a111a3f

• install centos7

  Wovchena

  committed
  813d80a

  Copy full SHA for 813d80a

• install nightly

  Wovchena

  committed
  6227b65

  Copy full SHA for 6227b65

• Merge branch 'generate_pipeline' into package

  Wovchena

  committed
  74fc107

  Copy full SHA for 74fc107

• propagate _GLIBCXX_USE_CXX11_ABI

  Wovchena

  committed
  9b83a7e

  Copy full SHA for 9b83a7e

• Populate python with the libraries to allow skipping wheel installation

  Wovchena

  committed
  2d15752

  Copy full SHA for 2d15752

• run setupvars

  Wovchena

  committed
  8025554

  Copy full SHA for 8025554

• update .gitignore, install numpy

  Wovchena

  committed
  2b14286

  Copy full SHA for 2b14286

• quotes

  Wovchena

  committed
  1c11bc7

  Copy full SHA for 1c11bc7

• fix PYTHONPATH

  Wovchena

  committed
  e7fce82

  Copy full SHA for e7fce82

• fix PYTHONPATH

  Wovchena

  committed
  64608d1

  Copy full SHA for 64608d1

• quotes

  Wovchena

  committed
  43b87c7

  Copy full SHA for 43b87c7

Commits on May 14, 2024

• reorder vars

  Wovchena

  committed
  fef9674

  Copy full SHA for fef9674

• openvino.genai-

  Wovchena

  committed
  b21286c

  Copy full SHA for b21286c

• Merge pull request #1 from Wovchena/package

  Show description for d393f89
  pavel-esir

  authored
  d393f89

  Copy full SHA for d393f89

• Merge branch 'master' into generate_pipeline

  pavel-esir

  authored
  2b8954d

  Copy full SHA for 2b8954d

• Update CMakeLists.txt

  pavel-esir

  authored
  11e872b

  Copy full SHA for 11e872b

• move group beam searcher to src

  pavel-esir

  committed
  442dcbf

  Copy full SHA for 442dcbf

Commits on May 15, 2024

• Update .gitignore (#5)

  Show description for 53d534e
  Wovchena

  authored
  53d534e

  Copy full SHA for 53d534e

• Merge remote-tracking branch 'origin/generate_pipeline' into generate_pipeline_latest

  pavel-esir

  committed
  dcb4b86

  Copy full SHA for dcb4b86

• fixed difference between old greddy sample and generate

  pavel-esir

  committed
  72c045e

  Copy full SHA for 72c045e

• tokenizer minor fixes

  pavel-esir

  committed
  11fbaa2

  Copy full SHA for 11fbaa2

• apply comments

  pavel-esir

  committed
  264e99f

  Copy full SHA for 264e99f

• remove accidentally added test_cpp_samples.py

  pavel-esir

  committed
  11032b4

  Copy full SHA for 11032b4

• fix build

  pavel-esir

  committed
  7d0c80b

  Copy full SHA for 7d0c80b

• fix causal_lm comparison error

  pavel-esir

  committed
  2e3cd73

  Copy full SHA for 2e3cd73

• fix different outputs

  pavel-esir

  committed
  e7fa974

  Copy full SHA for e7fa974

Commits on May 20, 2024

• Archive (#7)

  Show description for 78d0b88
  Wovchena

  authored
  78d0b88

  Copy full SHA for 78d0b88

Commits on May 22, 2024

• add tests

  pavel-esir

  committed
  5eb59ea

  Copy full SHA for 5eb59ea

• Apply suggestions from code review

  Show description for ce4eb00
  

  3 peopleauthored
  ce4eb00

  Copy full SHA for ce4eb00

• names correction

  pavel-esir

  committed
  aa90e9d

  Copy full SHA for aa90e9d

• update URL_HASH

  Wovchena

  committed
  54cbb52

  Copy full SHA for 54cbb52

• remove submodules from .gitmodules

  Wovchena

  committed
  82a9449

  Copy full SHA for 82a9449

• install openvino_tokenizers for genai_python_lib

  pavel-esir

  committed
  5a0079b

  Copy full SHA for 5a0079b

• Update Jinja2Cpp fork commit

  Wovchena

  committed
  73e4312

  Copy full SHA for 73e4312

• remove group_beam_searcher.hpp; copy fast_tokenizer

  pavel-esir

  committed
  75b7c37

  Copy full SHA for 75b7c37

Commits on May 23, 2024

• Fix archive (#8)

  Show description for 70f1177
  Wovchena

  authored
  70f1177

  Copy full SHA for 70f1177

Commits on May 24, 2024

• Apply suggestions from code review

  Show description for da729ba
  

  4 peopleauthored
  da729ba

  Copy full SHA for da729ba

• add groups to GenerationConfig docstring

  Show description for 28c313b
  pavel-esir

  committed
  28c313b

  Copy full SHA for 28c313b

• refactor namespace ov::* -> ov::genai::*

  pavel-esir

  committed
  c395a8d

  Copy full SHA for c395a8d

• removed ov_tokenizers_path when ov::gena::Tokenizer is passed to LLMPipeline

  pavel-esir

  committed
  bbc8c25

  Copy full SHA for bbc8c25

Commits on May 27, 2024

• Add sampling decoding (#6)

  Show description for 9e37273
  

  as-suvorov

  and

  wenyi5608

  authored
  9e37273

  Copy full SHA for 9e37273

Commits on May 28, 2024

• Fix library loading by updating dependencies (#10)

  Show description for 81ec069
  Wovchena

  authored
  81ec069

  Copy full SHA for 81ec069

Commits on May 29, 2024

• Add extension near to genai library, tokenizers from fork (#11)

  Show description for 88c44fe
  Wovchena

  authored
  88c44fe

  Copy full SHA for 88c44fe

• set openvino_tokenizers path via environment; cleared LLMPipeline constructors

  pavel-esir

  committed
  220035d

  Copy full SHA for 220035d

• update environment util for win

  pavel-esir

  committed
  5c6c14f

  Copy full SHA for 5c6c14f

• Add callback binding (#12)

  Show description for 174f67a
  

  Wovchena

  and

  pavel-esir

  authored
  174f67a

  Copy full SHA for 174f67a

• Add streamer binding (#13)

  Show description for 6709a67
  Wovchena

  authored
  6709a67

  Copy full SHA for 6709a67

• remove reset_state, multibatch tests added

  pavel-esir

  committed
  1a4bd68

  Copy full SHA for 1a4bd68

• fix win build

  pavel-esir

  committed
  9389930

  Copy full SHA for 9389930

Commits on May 30, 2024

• map stop_criteria in pybind;

  Show description for 7d1d616
  pavel-esir

  committed
  7d1d616

  Copy full SHA for 7d1d616

• fix chat_sample build on Win

  pavel-esir

  committed
  9208110

  Copy full SHA for 9208110

• fix tests failing

  pavel-esir

  committed
  7021c87

  Copy full SHA for 7021c87

Commits on May 31, 2024

• add return bool to streamer to stop generation

  pavel-esir

  committed
  680e362

  Copy full SHA for 680e362

• Add tests for macOS (#9)

  Show description for 9ba0a71
  yatarkan

  authored
  9ba0a71

  Copy full SHA for 9ba0a71

Commits on Jun 3, 2024

• add genai into llm_bench (#15)

  eaidova

  authored
  ac26bf8

  Copy full SHA for ac26bf8

• allow model without position_ids

  Show description for dd619e6
  pavel-esir

  committed
  dd619e6

  Copy full SHA for dd619e6

• Merge remote-tracking branch 'origin/generate_pipeline' into generate_pipeline

  pavel-esir

  committed
  04003d4

  Copy full SHA for 04003d4

• Return eos_token from decoding algos (#16)

  Show description for 1718bfb
  as-suvorov

  authored
  1718bfb

  Copy full SHA for 1718bfb

• Merge remote-tracking branch 'origin/generate_pipeline' into generate_pipeline

  pavel-esir

  committed
  1b35935

  Copy full SHA for 1b35935

Commits on Jun 4, 2024

• Cache a model, rename genai target, fix Windows (#14)

  Show description for 2c2a34a
  

  6 peopleauthored
  2c2a34a

  Copy full SHA for 2c2a34a

• Merge remote-tracking branch 'origin/generate_pipeline' into generate_pipeline

  pavel-esir

  committed
  b180faf

  Copy full SHA for b180faf

• Merge branch 'master' into generate_pipeline

  eaidova

  authored
  59c1096

  Copy full SHA for 59c1096

Commits on Jun 5, 2024

• read special tokens only from tokenizer_config.json and config.json

  Show description for 28ebc87
  pavel-esir

  committed
  28ebc87

  Copy full SHA for 28ebc87

• Leftovers (#18)

  Show description for da96019
  Wovchena

  authored
  da96019

  Copy full SHA for da96019

• minor typos fix

  pavel-esir

  committed
  a7f73a6

  Copy full SHA for a7f73a6

• Split text samples to separate folders (#19)

  Show description for a74baa2
  Wovchena

  authored
  a74baa2

  Copy full SHA for a74baa2

• update llm_bench (#17)

  eaidova

  authored
  13ebf9f

  Copy full SHA for 13ebf9f

• Assume GenAI is installed (#20)

  Show description for 67b1cfa
  Wovchena

  authored
  67b1cfa

  Copy full SHA for 67b1cfa

• fix segfault in tests

  pavel-esir

  committed
  0bd9cb3

  Copy full SHA for 0bd9cb3

Commits on Jun 6, 2024

• fix converting unfinished utf strings

  pavel-esir

  committed
  b618673

  Copy full SHA for b618673

• load special tokens leftovers

  pavel-esir

  committed
  80a17be

  Copy full SHA for 80a17be

• add config loading tests

  pavel-esir

  committed
  743f348

  Copy full SHA for 743f348

Commits on Jun 7, 2024

• commit forgotten py_generate_pipeline.cpp

  pavel-esir

  committed
  51a9a73

  Copy full SHA for 51a9a73

• fix ScopedVar in Tokenizer for ov_tokenizers_path

  pavel-esir

  committed
  2494df1

  Copy full SHA for 2494df1

• skip config modification in tmp dir on Win

  pavel-esir

  committed
  8f1399f

  Copy full SHA for 8f1399f

• return back win tests after disabling cleanup

  pavel-esir

  committed
  57830ba

  Copy full SHA for 57830ba

• Disable unfinished utf string test in Win

  pavel-esir

  authored
  2175796

  Copy full SHA for 2175796

• disable failing win workflows

  pavel-esir

  committed
  7c07136

  Copy full SHA for 7c07136