Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple images miniCPM-V-2_6 #919

Merged
Merged
Changes from 1 commit
Commits
Show all changes
226 commits
Select commit Hold shift + click to select a range
8d793c5
Create /chatglm_cpp
wenyi5608 Nov 29, 2023
212de8b
Delete llm/chatglm_cpp
wenyi5608 Nov 29, 2023
669dd53
Create CMakeLists.txt
wenyi5608 Nov 29, 2023
c04d1c7
Update CMakeLists.txt
wenyi5608 Nov 29, 2023
6847898
Add files via upload
wenyi5608 Nov 29, 2023
9a368bb
add chatglm_cpp demo
wenyi5608 Nov 29, 2023
b9fe37f
Update README.md
wenyi5608 Nov 29, 2023
a2bbd4a
Update chatglm.cpp
wenyi5608 Nov 29, 2023
0b480c9
Update README.md
wenyi5608 Dec 1, 2023
5eb8c4a
Update model compile from xml
wenyi5608 Dec 7, 2023
995c82d
Update README.md
wenyi5608 Dec 7, 2023
0823c6c
Update README.md
wenyi5608 Dec 7, 2023
8d70a45
Update README.md
wenyi5608 Dec 7, 2023
d74fddd
Update README.md
wenyi5608 Dec 13, 2023
cafdec4
Update README.md
wenyi5608 Dec 13, 2023
3319533
Update chatglm.cpp
wenyi5608 Dec 13, 2023
417555f
Update chatglm.cpp
wenyi5608 Dec 13, 2023
a244f9b
Update chatglm.cpp
wenyi5608 Dec 13, 2023
20931c6
Update chatglm.cpp
wenyi5608 Dec 14, 2023
5c09800
Update README.md
wenyi5608 Dec 14, 2023
85a3945
Update README.md
wenyi5608 Dec 14, 2023
6af1acd
Update chatglm.cpp
wenyi5608 Dec 15, 2023
fefc606
Add files via upload
wenyi5608 Dec 15, 2023
36ca098
Update chatglm.cpp
wenyi5608 Dec 15, 2023
c45b85c
Update README.md
wenyi5608 Dec 15, 2023
b2909b8
Update README.md
wenyi5608 Dec 15, 2023
67b68f5
Update README.md
wenyi5608 Dec 15, 2023
dd60a8f
Update README.md
wenyi5608 Dec 15, 2023
035a7d6
Update chatglm.cpp
wenyi5608 Dec 15, 2023
7b6f607
Update chatglm.cpp
wenyi5608 Dec 20, 2023
ba888bb
Update chatglm.cpp
wenyi5608 Dec 27, 2023
673d86e
Update README.md
wenyi5608 Dec 27, 2023
c30bdf8
Update chatglm.cpp
wenyi5608 Dec 27, 2023
fcadd45
Update README.md
wenyi5608 Dec 27, 2023
35a9d66
Update chatglm.cpp
wenyi5608 Dec 29, 2023
57b9f68
Update chatglm.cpp
wenyi5608 Jan 23, 2024
1703821
Create clip.h
wenyi5608 Jul 26, 2024
8fd2fe7
minicmpv_2 demo
wenyi5608 Jul 26, 2024
2dda201
Update submodul
yangsu2022 Jul 26, 2024
59b394f
Add minicpmv2
wenyi5608 Jul 26, 2024
392dd62
multi-round chat
wenyi5608 Aug 11, 2024
c4f9a19
multi-round chat
wenyi5608 Aug 11, 2024
ae735a8
multi-round chat
wenyi5608 Aug 11, 2024
4879efc
Update minicpmv.h
wenyi5608 Aug 11, 2024
6507280
multi-round chat
wenyi5608 Aug 11, 2024
9b67765
Add files via upload
wenyi5608 Aug 11, 2024
f10989f
Update README.md
wenyi5608 Aug 11, 2024
a20db15
Update minicpmv_ov.cpp
wenyi5608 Aug 11, 2024
5c4f254
using namespace ov
Wovchena Aug 12, 2024
bbefc17
Merge branch 'wenyi5608-stateful' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 12, 2024
73db9a6
Add VLMPipeline
Wovchena Aug 12, 2024
8d0ad82
Add python
Wovchena Aug 13, 2024
4bc8a6d
Add constructor
Wovchena Aug 13, 2024
bea9a46
Move ModelConfig
Wovchena Aug 14, 2024
08ad5a3
generate(imgage)
Wovchena Aug 14, 2024
9dbf0e0
rename
Wovchena Aug 15, 2024
448a35e
Tensor
Wovchena Aug 15, 2024
8f02b9f
Remove output_fixed_len
Wovchena Aug 15, 2024
ceba7c9
merge generate
Wovchena Aug 15, 2024
6420a39
Add Clip
Wovchena Aug 15, 2024
6f5c256
remove n_threads
Wovchena Aug 16, 2024
fb2dcd0
VisionEncoder
Wovchena Aug 16, 2024
6b2aaa3
tensor
Wovchena Aug 16, 2024
f54e458
tensor
Wovchena Aug 16, 2024
4611834
tensor
Wovchena Aug 19, 2024
45d2f85
tensor
Wovchena Aug 19, 2024
431be5f
drop n_img_pos
Wovchena Aug 19, 2024
93d3eba
Split encoder and resampler
Wovchena Aug 19, 2024
1b2bc1e
Add encode for VisionEncoder
Wovchena Aug 19, 2024
0ea4c09
clean up
Wovchena Aug 19, 2024
18fa984
Merge branch 'master' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 20, 2024
3e4af2c
Remove int4
Wovchena Aug 21, 2024
02c79fc
workaround
Wovchena Aug 21, 2024
1124ddf
fix
Wovchena Aug 21, 2024
79e3e04
public
Wovchena Aug 21, 2024
3edfd91
workaround
Wovchena Aug 21, 2024
b5f04fc
x and y ratios
Wovchena Aug 21, 2024
2514ae2
Merge branch 'master' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 21, 2024
066ff0c
Merge branch 'master' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 21, 2024
5785cb2
Delete resize from ov_minicpm-v2-test.py
Wovchena Aug 21, 2024
0d28a64
resample
Wovchena Aug 22, 2024
c18219b
Update llm/mincpmv2_cpp/export_MiniCPM-V-2.py
Wovchena Aug 22, 2024
db8330a
ref
Wovchena Aug 23, 2024
b176c26
fix dims
Wovchena Aug 23, 2024
7c5f631
Batch inference
Wovchena Aug 26, 2024
7972bc2
Hide headers
Wovchena Aug 26, 2024
800bcc6
Return string
Wovchena Aug 26, 2024
15dbbaf
Remove
Wovchena Aug 26, 2024
7b4713c
Factor vision_encoder.hpp out
Wovchena Aug 28, 2024
e3dd4f6
Merge branch 'master' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 28, 2024
2d1d6b7
Update README.md
Wovchena Aug 28, 2024
8a9f4b9
fix warnings
Wovchena Aug 28, 2024
b51b981
Encoder AnyMap
Wovchena Aug 28, 2024
3fce7c7
Reproduce
Wovchena Aug 29, 2024
f5eb2b3
Remove members
Wovchena Aug 29, 2024
9c18ff9
relative link
Wovchena Aug 29, 2024
16d9185
relative link
Wovchena Aug 29, 2024
f6cc888
vlm_config.hpp
Wovchena Aug 29, 2024
19381a0
Add generate methods
Wovchena Aug 29, 2024
914293f
Add vlm_config
Wovchena Aug 29, 2024
dd30ca0
m_
Wovchena Aug 29, 2024
8a155b5
Merge branch 'master' into add-openbmb/MiniCPM-V-2-sample
Wovchena Aug 29, 2024
8e1819f
[GHA] Update OV refs to 2024.4
akladiev Aug 30, 2024
f169ec8
Remove unused
Wovchena Aug 30, 2024
21110e2
Add overloads
Wovchena Aug 30, 2024
4cfba8a
[GHA] Update OV refs to 2024.4 (#810)
Wovchena Aug 30, 2024
0052922
Tensor->EncodedImage
Wovchena Sep 2, 2024
86187dd
Add VisionEncoder docstrings
Wovchena Sep 2, 2024
4d79fda
public
Wovchena Sep 2, 2024
52a7c12
Update src/cpp/src/vision_encoder.cpp
Wovchena Sep 3, 2024
278ff39
Document shapes
Wovchena Sep 4, 2024
94a2f3f
Remove generate(AnyMap)
Wovchena Sep 4, 2024
675c8f0
Beautify sample
Wovchena Sep 4, 2024
b79874b
fix PerfMetrics
pavel-esir Sep 4, 2024
b4955f5
TTTFT -> TTFT
pavel-esir Sep 4, 2024
c3f895c
Merge branch 'releases/2024/4' into add-openbmb/MiniCPM-V-2-sample
Wovchena Sep 4, 2024
94573f3
Replace stable diffusion 1.5 model
yatarkan Sep 4, 2024
88c100c
std::move
Wovchena Sep 4, 2024
9a737f2
add tests
pavel-esir Sep 4, 2024
8ccfdf7
Change Stable Diffusion v1.5 model (#824)
Wovchena Sep 4, 2024
5773ae3
some corrections
pavel-esir Sep 4, 2024
41db203
temporary disable causal_lm win tests
pavel-esir Sep 4, 2024
6321b4d
fix PerfMetrics (#823)
Wovchena Sep 5, 2024
d1e5c13
Merge branch 'releases/2024/4' into add-openbmb/MiniCPM-V-2-sample
Wovchena Sep 5, 2024
6c0e656
[2024.4] update optimum intel commit to include mxfp4 conversion (#828)
eaidova Sep 5, 2024
ba0491a
[2024.4] use perf metrics for genai in llm bench (#830)
eaidova Sep 5, 2024
0c2f4c9
Tokenizerrs update
ilya-lavrenov Sep 5, 2024
7734510
Use tarballs
ilya-lavrenov Sep 5, 2024
18fc740
Use OpenVINO 2024.4 RC2
ilya-lavrenov Sep 6, 2024
1914a70
updated pybind version
mryzhov Sep 6, 2024
c8b3e3f
Merge branch 'releases/2024/4' into add-openbmb/MiniCPM-V-2-sample
Wovchena Sep 6, 2024
2d99414
improve utf8 handling for DecodedResults in pybind
pavel-esir Sep 6, 2024
52ea56d
alighn printing DecodedResults scores in py_generate_pipeline and llm…
pavel-esir Sep 6, 2024
fb8c7a4
Improve utf8 handling for DecodedResults in pybind (#837)
pavel-esir Sep 6, 2024
a75df54
replace invalid utf8 bytes when file is opened
pavel-esir Sep 6, 2024
355240e
print with 'cat pred.txt' for debug
pavel-esir Sep 6, 2024
1aa61f8
replace � -> ""
pavel-esir Sep 6, 2024
6d3d6b6
Apply suggestions from code review
pavel-esir Sep 6, 2024
95ceb92
Merge branch 'releases/2024/4' into add-openbmb/MiniCPM-V-2-sample
Wovchena Sep 6, 2024
8f87768
accept vector of tensors
Wovchena Sep 6, 2024
1a896d2
Fails at runtime with out of bounds
Wovchena Sep 10, 2024
adf0db3
temp
Wovchena Sep 10, 2024
3928162
Temo
Wovchena Sep 10, 2024
7920053
Temo
Wovchena Sep 10, 2024
8f51232
nan or seg fault
Wovchena Sep 19, 2024
daa8f6c
Still need to fix images
Wovchena Sep 19, 2024
348ec37
update README.md
Wovchena Sep 19, 2024
f9358d4
uncomment conversion
Wovchena Sep 19, 2024
e92b778
Comment out dumped tokens
Wovchena Sep 19, 2024
7de6d00
Merge branch 'master' into miniCPM-V-2_6
Wovchena Sep 20, 2024
f88b45e
Resolve conflicts
Wovchena Sep 20, 2024
3a0ad0b
Resolve conflicts
Wovchena Sep 20, 2024
dce2d79
Remove dump
Wovchena Sep 20, 2024
2450d96
Merge branch 'master' into miniCPM-V-2_6
Wovchena Sep 30, 2024
1bcecdf
chat template
Wovchena Oct 1, 2024
bdc4a65
temp
Wovchena Oct 2, 2024
5f30821
Tokenizers bug
Wovchena Oct 2, 2024
d43d06b
Merge branch 'master' into miniCPM-V-2_6
Wovchena Oct 2, 2024
bde9fb7
Read use_image_id
Wovchena Oct 2, 2024
6a44ba7
Merge branch 'miniCPM-V-2_6' into image-miniCPM-V-2_6
Wovchena Oct 2, 2024
c4d6fec
increase timeout
Wovchena Oct 2, 2024
a7e18a7
Delete comment
Wovchena Oct 2, 2024
5f9e5f7
prints
Wovchena Oct 3, 2024
63cdd24
Fix
Wovchena Oct 4, 2024
1e68d7a
Clean up
Wovchena Oct 4, 2024
17f69ec
workflow
Wovchena Oct 4, 2024
ce665b8
unpipe
Wovchena Oct 4, 2024
ea946d1
unzip
Wovchena Oct 4, 2024
64c93b8
dot
Wovchena Oct 4, 2024
450b6f7
cmd
Wovchena Oct 4, 2024
3356f2a
l
Wovchena Oct 4, 2024
23d087f
ls
Wovchena Oct 4, 2024
c796323
ls
Wovchena Oct 4, 2024
1e15496
env
Wovchena Oct 4, 2024
ca6b9ac
ubuntu
Wovchena Oct 4, 2024
416323d
torchvision
Wovchena Oct 4, 2024
de7d768
rename
Wovchena Oct 4, 2024
08d133c
docstrings
Wovchena Oct 4, 2024
447e745
add set_chat_template
Wovchena Oct 4, 2024
ebaadb7
Finilize chat_template
Wovchena Oct 4, 2024
bfad182
Simplify test
Wovchena Oct 4, 2024
be9e203
Fix multiple images
Wovchena Oct 4, 2024
3d0e5ba
Temp
Wovchena Oct 4, 2024
57bb044
Add macos
Wovchena Oct 6, 2024
5efc24b
and
Wovchena Oct 6, 2024
c473d2a
cmp
Wovchena Oct 6, 2024
c780a2e
cmp
Wovchena Oct 6, 2024
68d0134
matrix
Wovchena Oct 6, 2024
7f4cafb
runner
Wovchena Oct 6, 2024
b316c92
env
Wovchena Oct 6, 2024
797d8ed
shorten
Wovchena Oct 6, 2024
3d9886b
matrix
Wovchena Oct 6, 2024
5b00440
4 cores
Wovchena Oct 6, 2024
798f42d
./
Wovchena Oct 6, 2024
ee057ba
env
Wovchena Oct 6, 2024
e88ae49
ld
Wovchena Oct 6, 2024
382bcbc
ld
Wovchena Oct 6, 2024
30dc931
tbb
Wovchena Oct 6, 2024
4c61b12
and
Wovchena Oct 6, 2024
1ff0889
8
Wovchena Oct 6, 2024
3a800a1
LD_LIBRARY_PATH
Wovchena Oct 6, 2024
8a3a21b
TBB_DIR
Wovchena Oct 6, 2024
99911bc
move
Wovchena Oct 6, 2024
6ac98c5
echo
Wovchena Oct 6, 2024
53f3154
16
Wovchena Oct 6, 2024
55af01e
LD
Wovchena Oct 6, 2024
f064bdc
Remove ls
Wovchena Oct 6, 2024
21da18b
timeout-minutes: 1
Wovchena Oct 6, 2024
44f21c0
timeout-minutes: 2
Wovchena Oct 6, 2024
02c928e
setuvars
Wovchena Oct 6, 2024
f327e8c
fix jump
Wovchena Oct 6, 2024
0eb693e
tbb
Wovchena Oct 6, 2024
ddb1c29
DYLD_LIBRARY_PATH
Wovchena Oct 6, 2024
0ea2656
source ./ov/setupvars.sh
Wovchena Oct 6, 2024
c1d1f3b
4
Wovchena Oct 6, 2024
7a00aa5
timeout-minutes: 8
Wovchena Oct 7, 2024
33e1cef
timeout-minutes: 16
Wovchena Oct 7, 2024
12a7134
timeout-minutes: 32
Wovchena Oct 7, 2024
c166e5f
Update the model
Wovchena Oct 7, 2024
df9420d
Fix macos compilation
Wovchena Oct 7, 2024
871f334
spelling
Wovchena Oct 7, 2024
68e9870
Drop mac test
Wovchena Oct 8, 2024
6c93bbd
Merge branch 'master' into multiple-images-miniCPM-V-2_6
Wovchena Oct 9, 2024
bf11700
rename
Wovchena Oct 9, 2024
cb17470
reanme
Wovchena Oct 9, 2024
f6a4143
Remove Python test
Wovchena Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update README.md
wenyi5608 authored Dec 15, 2023
commit b2909b87a802a1e56655ef8e7fd3f3a4a90218b6
12 changes: 12 additions & 0 deletions llm/chatglm_cpp/README.md
Original file line number Diff line number Diff line change
@@ -62,6 +62,18 @@ python ./llm/cpp/convert_tokenizers.py ./build/thirdparty/openvino_contrib/modul
```

4. Run

Paramters:
* `-m, --model` PATH Chatglm OpenVINO model path (default: openvino_model.xml)\n"
* `-token` PATH Tokenizer model path (tokenizer.xml)\n"
* `-detoken` PATH DeTokenizer model path (detokenizer.xml)\n"
* `-d, --device` Device (default: GPU)\n"
* `--convert_kv_fp16` Convert kvcache fp16 (default: False)\n"
* `--do_sample` Search (default: False)\n"
* `--top_k` N top-k sampling (default: 0)\n"
* `--top_p` N top-p sampling (default: 0.7)\n"
* `--temp` N temperature (default: 0.95)\n"
* `--repeat_penalty` N penalize repeat sequence of tokens (default: 1.0, 1.0 = disabled)\n";

For Linux