Code

[Tokenizers] Tokenizer fix decode#767

Code

Merged

ilya-lavrenov merged 162 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino_contrib:masterfrom ilya-lavrenov:tokenizer-fix-decodeilya-lavrenov/openvino_contrib:tokenizer-fix-decodeCopy head branch name to clipboardNov 29, 2023

Commits

Commits on Apr 25, 2023

• Added string tensor implementation with explicit pointer unpack

  slyalin

  committed
  70f867a

  Copy full SHA for 70f867a

Commits on Apr 28, 2023

• Merged from master

  slyalin

  committed
  1fac3de

  Copy full SHA for 1fac3de

Commits on May 2, 2023

• Started to migrate to extension-only support of string operations with and without string support in OV core. Moved StringTensorUnpack and reworked it to be aligned with the new approach. Reworked …

  Show description for 821dee5
  slyalin

  committed
  821dee5

  Copy full SHA for 821dee5

Commits on May 10, 2023

• Started to merge string/tokenizer related stuff from a dedicated OV branch to contrib in form compatible with both master and the branch with string tensors support. Added CaseFoldUTF8 from that br…

  Show description for b9b0693
  slyalin

  committed
  b9b0693

  Copy full SHA for b9b0693

• Rename CaseFoldUTF8 to name from opset proposal: CaseFold, added NormalizeUnicode

  slyalin

  committed
  c785ec1

  Copy full SHA for c785ec1

Commits on May 11, 2023

• Added a stub for RegexNormalization operation, WA for CPU bug with empty constants, register StringTensorPack and StringTensorUnpack as OV operations to be able to read IRs with those operations

  slyalin

  committed
  1d129ac

  Copy full SHA for 1d129ac

• Implemented Reshape for decomposed string tensors

  slyalin

  committed
  71bc5bf

  Copy full SHA for 71bc5bf

Commits on May 12, 2023

• Added RaggedTensorPack, sophisticated stup for RegexSplit and overridden Const translator for TF to intercept string constants

  slyalin

  committed
  6c5eec0

  Copy full SHA for 6c5eec0

Commits on May 15, 2023

• Fixes for both master and element::string branches of OpenVINO; better conditional compilation based on available features in OpenVINO

  slyalin

  committed
  29dfe38

  Copy full SHA for 29dfe38

Commits on May 16, 2023

• Debug output of indices in RaggedTensorPack

  slyalin

  committed
  40063c1

  Copy full SHA for 40063c1

Commits on May 17, 2023

• Implemented a stub for WordpieceTokenizer. Supported conversion of a combination of WordpieceTokenizeWithOffsets and LookupTableFindV2 from TensorFlow

  slyalin

  committed
  cc47b12

  Copy full SHA for cc47b12

• Disabled debug output

  slyalin

  committed
  7644231

  Copy full SHA for 7644231

Commits on May 18, 2023

• Define default values for custom operations attributes to make attribute initialization optional (needed for core.make_node)

  slyalin

  committed
  80b8023

  Copy full SHA for 80b8023

Commits on May 20, 2023

• Added fast_tokenizer lib to the build. Implemented CaseFold based on fast_tokenizer.

  slyalin

  committed
  46c82b8

  Copy full SHA for 46c82b8

• Removed debug output

  slyalin

  committed
  d7ca2ab

  Copy full SHA for d7ca2ab

• Implemented RaggedToDense always in pad_right=true mode and with boolean mask extra output

  slyalin

  committed
  2baac3d

  Copy full SHA for 2baac3d

Commits on May 23, 2023

• Provided real implementations for NormalizeUnicode, RegexNormalization and RegexSplit based on paddle fast_tokenizer lib. Limited implementation, not all of the features of ops and TF translated op…

  Show description for d270dd6
  slyalin

  committed
  d270dd6

  Copy full SHA for d270dd6

• Implemented WordpieceTokenizer with fast_tokenizer library

  slyalin

  committed
  119d6e9

  Copy full SHA for 119d6e9

Commits on May 25, 2023

• Renamed behaviours to be verbs instead of adjectives

  slyalin

  committed
  4d4ad89

  Copy full SHA for 4d4ad89

• Added modified version of HF tokenizer parser from Artur; implemented necessary steps to complete HF bert preprocessing conversion (not validated)

  slyalin

  committed
  f4eee84

  Copy full SHA for f4eee84

• Renamed apply_tokenizer to connect_tokeniser and removed obsolete handling of model name

  slyalin

  committed
  1e50352

  Copy full SHA for 1e50352

Commits on May 31, 2023

• CombineSegments is implemented, used in HF converter. Stitching of tokenizer and main model is fixed partially (still produces topologically incorrect model)

  slyalin

  committed
  0966b8a

  Copy full SHA for 0966b8a

• Fixed stitching of two models by connecting with names of inputs/outputs, now Bert and its tokenizer are connected together correctly

  slyalin

  committed
  61d7983

  Copy full SHA for 61d7983

Commits on Jun 1, 2023

• WA for CPU bug with scalar inputs, correct truncation and dynamic padding, fix bugs for batches processing

  slyalin

  committed
  5609ee6

  Copy full SHA for 5609ee6

• Fixed conversion of HF tokenizer if part of outputs are omitted. Disabled debug output

  slyalin

  committed
  062acf3

  Copy full SHA for 062acf3

Commits on Jun 19, 2023

• Add BPE Tokenizer

  apaniukov

  committed
  0f772dc

  Copy full SHA for 0f772dc

Commits on Jun 20, 2023

• Add BytesToChars Node for BBPE

  apaniukov

  committed
  10e3d18

  Copy full SHA for 10e3d18

• Delete print

  apaniukov

  committed
  c413cb6

  Copy full SHA for c413cb6

• Clip max value for max_length to int32

  apaniukov

  committed
  8c8994c

  Copy full SHA for 8c8994c

Commits on Jun 22, 2023

• Fix RegexNormalization and Splitter, Add Digits Splitter

  apaniukov

  committed
  8750ae6

  Copy full SHA for 8750ae6

Commits on Jun 23, 2023

• Bug fixes

  apaniukov

  committed
  be6dc3f

  Copy full SHA for be6dc3f

Commits on Jun 29, 2023

• Add decoding step, BytesToChars refactoring

  Show description for e4dcdda
  apaniukov

  committed
  e4dcdda

  Copy full SHA for e4dcdda

Commits on Jun 30, 2023

• Fix some regex bugs for byte-level splitter

  apaniukov

  committed
  b45e5ec

  Copy full SHA for b45e5ec

Commits on Jul 7, 2023

• Fix bug with VocabDecoder shape

  apaniukov

  committed
  5f03ed0

  Copy full SHA for 5f03ed0

Commits on Jul 10, 2023

• Minor changes for natively supported strings

  slyalin

  committed
  2a65502

  Copy full SHA for 2a65502

• Merge remote-tracking branch 'artur/string_tensors_add_bpe' into string_tensors

  slyalin

  committed
  2e34b92

  Copy full SHA for 2e34b92

• Suppressed minor^Carnings about int32 -> unsigned implicit

  slyalin

  committed
  a6f9110

  Copy full SHA for a6f9110

• Restructured sentence_piece directory to tokenizer directory: split all ops, translators and helper into individual files. To build use tokenizer custom op name in cmake instead of sentence_piece.

  slyalin

  committed
  5c29254

  Copy full SHA for 5c29254

Commits on Jul 17, 2023

• Add regex to detokenizer pipeline, all splitters have 5 inputs

  apaniukov

  committed
  f8d0e0d

  Copy full SHA for f8d0e0d

Commits on Jul 27, 2023

• Add Caching for RegexNormalization

  apaniukov

  committed
  10c10c5

  Copy full SHA for 10c10c5

• Add Caching for RegexSplit

  apaniukov

  committed
  4eb12f8

  Copy full SHA for 4eb12f8

Commits on Jul 28, 2023

• Add Wordpiece Cache

  apaniukov

  committed
  c5efaf0

  Copy full SHA for c5efaf0

Commits on Jul 31, 2023

• Add NodeFactory

  apaniukov

  committed
  239acc4

  Copy full SHA for 239acc4

Commits on Aug 4, 2023

• Fix regex nodes init

  apaniukov

  committed
  38552b0

  Copy full SHA for 38552b0

Commits on Aug 10, 2023

• Fix Wordpiece Cache

  apaniukov

  committed
  597ccd4

  Copy full SHA for 597ccd4

• Add BPE Cache

  apaniukov

  committed
  e6933b7

  Copy full SHA for e6933b7

Commits on Aug 11, 2023

• Fix RegexNormalization

  apaniukov

  committed
  bd7f9d9

  Copy full SHA for bd7f9d9

Commits on Sep 6, 2023

• Refactor CombineSegments and Padding

  apaniukov

  committed
  99c603f

  Copy full SHA for 99c603f

Commits on Sep 7, 2023

• Refactoring

  apaniukov

  committed
  6cc9b36

  Copy full SHA for 6cc9b36

Commits on Sep 8, 2023

• Clean-up commented code

  apaniukov

  committed
  973c52d

  Copy full SHA for 973c52d

Commits on Sep 27, 2023

• Sentencepiece Model Encoder from Transformers Tokenizer

  apaniukov

  committed
  1fa02b2

  Copy full SHA for 1fa02b2

• Add tests for tokenizers

  apaniukov

  committed
  e37f89d

  Copy full SHA for e37f89d

Commits on Oct 2, 2023

• Add detokenizer for Sentencepiece models

  apaniukov

  committed
  88bf7c6

  Copy full SHA for 88bf7c6

Commits on Oct 4, 2023

• Update README.md

  apaniukov

  authored
  bb1b57a

  Copy full SHA for bb1b57a

• Update README.md

  apaniukov

  authored
  6b4be05

  Copy full SHA for 6b4be05

• Update README.md

  apaniukov

  authored
  539797f

  Copy full SHA for 539797f

• OVTokenizer as python package

  apaniukov

  committed
  79c3e09

  Copy full SHA for 79c3e09

• Merge branch 'openvinotoolkit:master' into tokenizer-fix-decode

  apaniukov

  authored
  203ffbb

  Copy full SHA for 203ffbb

Commits on Oct 5, 2023

• Update README.md

  apaniukov

  committed
  45c0068

  Copy full SHA for 45c0068

Commits on Oct 6, 2023

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  372465b

  Copy full SHA for 372465b

• Add sentencepiece detokenizer test

  apaniukov

  committed
  64567ea

  Copy full SHA for 64567ea

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  apaniukov

  committed
  f54076e

  Copy full SHA for f54076e

Commits on Oct 9, 2023

• Unified interface for fast and sentencepiece tokenizers

  apaniukov

  committed
  c42d1bd

  Copy full SHA for c42d1bd

Commits on Oct 11, 2023

• Add Full Pipeline example for Sentencepiece

  Show description for 8b29443
  apaniukov

  committed
  8b29443

  Copy full SHA for 8b29443

• Update third-party-programs.txt

  apaniukov

  committed
  2ee3707

  Copy full SHA for 2ee3707

Commits on Oct 12, 2023

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  4b57fcc

  Copy full SHA for 4b57fcc

Commits on Oct 13, 2023

• Add Constants

  apaniukov

  committed
  803d831

  Copy full SHA for 803d831

Commits on Oct 16, 2023

• Add CPP pack/unpack_strings functions

  Show description for 72f6d9f
  apaniukov

  committed
  72f6d9f

  Copy full SHA for 72f6d9f

Commits on Oct 17, 2023

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  386cb02

  Copy full SHA for 386cb02

• Move tests to tokenizer dir

  apaniukov

  committed
  79bd05f

  Copy full SHA for 79bd05f

Commits on Oct 18, 2023

• Fix import

  apaniukov

  committed
  24a60b3

  Copy full SHA for 24a60b3

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  apaniukov

  committed
  f01afee

  Copy full SHA for f01afee

• Fix imports

  apaniukov

  committed
  b22569f

  Copy full SHA for b22569f

• Sort Imports

  apaniukov

  committed
  96673f5

  Copy full SHA for 96673f5

Commits on Oct 19, 2023

• Add Streaming Sentencepiece Decoder

  apaniukov

  committed
  0e7ae87

  Copy full SHA for 0e7ae87

• Change Authors

  apaniukov

  committed
  5ebdb1f

  Copy full SHA for 5ebdb1f

Commits on Oct 23, 2023

• Update modules/custom_operations/user_ie_extensions/tokenizer/utils.cpp

  Show description for 6a55877
  

  apaniukov

  and

  Wovchena

  authored
  6a55877

  Copy full SHA for 6a55877

• Configure tests

  apaniukov

  committed
  06d5159

  Copy full SHA for 06d5159

Commits on Oct 24, 2023

• Skip Java Tests

  apaniukov

  committed
  fa5360d

  Copy full SHA for fa5360d

• Add Regression Test

  apaniukov

  committed
  e855193

  Copy full SHA for e855193

• Skip traceback

  apaniukov

  committed
  d495d3b

  Copy full SHA for d495d3b

• Add Win64 Fast Tokenizer lib

  apaniukov

  committed
  d7bebd0

  Copy full SHA for d7bebd0

• Fix WorkingDir

  apaniukov

  committed
  b2e35ed

  Copy full SHA for b2e35ed

• Return TB

  apaniukov

  committed
  f81bd18

  Copy full SHA for f81bd18

• Fix dependencies install

  apaniukov

  committed
  0bd23b5

  Copy full SHA for 0bd23b5

• Add byte tokens handling for sentencepiece

  apaniukov

  committed
  12ac9f8

  Copy full SHA for 12ac9f8

Commits on Oct 25, 2023

• Drop black, use ruff format instead

  apaniukov

  committed
  9e6ae6f

  Copy full SHA for 9e6ae6f

Commits on Oct 26, 2023

• Temp remove tokenizers from windows CI

  apaniukov

  committed
  f5d2d4c

  Copy full SHA for f5d2d4c

• CI check

  apaniukov

  committed
  cf039b9

  Copy full SHA for cf039b9

Commits on Oct 28, 2023

• Compile fast_tokenizers from source code

  ilya-lavrenov

  committed
  795306d

  Copy full SHA for 795306d

Commits on Oct 30, 2023

• Export pack_strings() and unpack_strings()

  Wovchena

  committed
  9c200c2

  Copy full SHA for 9c200c2

Commits on Oct 31, 2023

• Merge pull request #1 from ilya-lavrenov/tokenizer-fix-decode

  Show description for 0e9b960
  apaniukov

  authored
  0e9b960

  Copy full SHA for 0e9b960

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  95aa47c

  Copy full SHA for 95aa47c

• Merge branch 'tokenizer-fix-decode' into export-pack_strings-and-unpack_strings

  Wovchena

  authored
  e1de338

  Copy full SHA for e1de338

• Build tokenizer target on windows

  apaniukov

  committed
  f23e59b

  Copy full SHA for f23e59b

Commits on Nov 2, 2023

• Merge branch 'tokenizer-fix-decode' into export-pack_strings-and-unpack_strings

  Wovchena

  committed
  dbec117

  Copy full SHA for dbec117

Commits on Nov 3, 2023

• Add icu4c patch

  apaniukov

  committed
  ce25397

  Copy full SHA for ce25397

Commits on Nov 8, 2023

• Added include dir to nlohmann headers

  ilya-lavrenov

  committed
  d46f594

  Copy full SHA for d46f594

• Fixed compilation on ubuntu 18.04 arm64

  ilya-lavrenov

  committed
  6f213ab

  Copy full SHA for 6f213ab

• Fixed Windows

  ilya-lavrenov

  committed
  6ed52e4

  Copy full SHA for 6ed52e4

• Merge pull request openvinotoolkit#3 from ilya-lavrenov/nlohmann

  Show description for ca62321
  apaniukov

  authored
  ca62321

  Copy full SHA for ca62321

• Supported prebuild Fast Tokenizers on all platforms

  ilya-lavrenov

  committed
  52bfe5a

  Copy full SHA for 52bfe5a

Commits on Nov 9, 2023

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  b504013

  Copy full SHA for b504013

• Add tiktoken support WIP

  apaniukov

  committed
  cc663dc

  Copy full SHA for cc663dc

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  apaniukov

  committed
  4c9ceed

  Copy full SHA for 4c9ceed

• Unskip java tests

  apaniukov

  committed
  745e969

  Copy full SHA for 745e969

• Merge pull request openvinotoolkit#4 from ilya-lavrenov/prebuilt-fast-tokenizers

  Show description for 48564b7
  apaniukov

  authored
  48564b7

  Copy full SHA for 48564b7

Commits on Nov 10, 2023

• Fixed compilation with re2 on Windows

  ilya-lavrenov

  committed
  056eb9f

  Copy full SHA for 056eb9f

• Merge pull request openvinotoolkit#5 from ilya-lavrenov/windows-re2

  Show description for 309b8e9
  apaniukov

  authored
  309b8e9

  Copy full SHA for 309b8e9

• Merge branch 'tokenizer-fix-decode' into export-pack_strings-and-unpack_strings

  Wovchena

  committed
  b193cb2

  Copy full SHA for b193cb2

• Move unpack_strings(), create sepparate include dir

  Wovchena

  committed
  debcb5d

  Copy full SHA for debcb5d

• openvino_extensions

  Wovchena

  committed
  b739ffd

  Copy full SHA for b739ffd

Commits on Nov 13, 2023

• Fixed link stage on Windows

  ilya-lavrenov

  committed
  e70a3f2

  Copy full SHA for e70a3f2

• Merge pull request openvinotoolkit#6 from ilya-lavrenov/windows-linkage

  Show description for 2ce27cd
  apaniukov

  authored
  2ce27cd

  Copy full SHA for 2ce27cd

Commits on Nov 14, 2023

• i64 is default tokenizer output type

  apaniukov

  committed
  3022a5a

  Copy full SHA for 3022a5a

• Add support for more tiktoken tokenizers

  apaniukov

  committed
  c467a8c

  Copy full SHA for c467a8c

Commits on Nov 15, 2023

• Merge branch 'tokenizer-fix-decode' into export-pack_strings-and-unpack_strings

  Wovchena

  committed
  1ec4c5f

  Copy full SHA for 1ec4c5f

• Check Azure CI

  apaniukov

  committed
  8505b51

  Copy full SHA for 8505b51

• Fix Azure Win CI

  apaniukov

  committed
  82639e6

  Copy full SHA for 82639e6

• Merge pull request #2 from Wovchena/export-pack_strings-and-unpack_strings

  Show description for fb37580
  apaniukov

  authored
  fb37580

  Copy full SHA for fb37580

• Define python version for setupvars.bat

  apaniukov

  committed
  a45b826

  Copy full SHA for a45b826

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  apaniukov

  committed
  35cc136

  Copy full SHA for 35cc136

• Add support for tiktoken detokenizers

  apaniukov

  committed
  244a593

  Copy full SHA for 244a593

Commits on Nov 16, 2023

• Merge branch 'master' into tokenizer-fix-decode

  andrei-kochin

  authored
  84686f4

  Copy full SHA for 84686f4

• Add ChatGLM tokenization support.

  apaniukov

  committed
  ad1c589

  Copy full SHA for ad1c589

• Add ChatGLM detokenization and tests

  apaniukov

  committed
  0f63c3d

  Copy full SHA for 0f63c3d

Commits on Nov 17, 2023

• Add ChatGLM detokenization and tests

  apaniukov

  committed
  0f1c1cc

  Copy full SHA for 0f1c1cc

• Fix mac sha256

  apaniukov

  committed
  3edb73b

  Copy full SHA for 3edb73b

• Skip Lin Java Tests

  apaniukov

  committed
  48bba34

  Copy full SHA for 48bba34

• Add Mac Tokenziers Tests and Skip Mac Java Step

  apaniukov

  committed
  fe507ff

  Copy full SHA for fe507ff

• Fix Mac SHA

  apaniukov

  committed
  4b0c4ec

  Copy full SHA for 4b0c4ec

• Del WA for CPU Bug

  apaniukov

  committed
  4656238

  Copy full SHA for 4656238

Commits on Nov 18, 2023

• Fix Mac CI Pipeline

  apaniukov

  committed
  2f5cc1c

  Copy full SHA for 2f5cc1c

• Change Mac CI

  apaniukov

  committed
  1568727

  Copy full SHA for 1568727

Commits on Nov 20, 2023

• Fixed compilation

  ilya-lavrenov

  committed
  fa822c2

  Copy full SHA for fa822c2

• Merge pull request openvinotoolkit#7 from ilya-lavrenov/compilation-fix

  Show description for 6ddb2a6
  apaniukov

  authored
  6ddb2a6

  Copy full SHA for 6ddb2a6

• Add setupvars to mac CI

  apaniukov

  committed
  14f993b

  Copy full SHA for 14f993b

• Merge branch 'master' into tokenizer-fix-decode

  apaniukov

  authored
  cae3098

  Copy full SHA for cae3098

• Change detokenizer output type

  apaniukov

  committed
  b59204d

  Copy full SHA for b59204d

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  apaniukov

  committed
  e54b42e

  Copy full SHA for e54b42e

• Fix SegFault on AddedTokens For BPE tokenizer

  apaniukov

  committed
  6c3bae3

  Copy full SHA for 6c3bae3

• Add SP Space handling for decoder

  apaniukov

  committed
  d34d401

  Copy full SHA for d34d401

Commits on Nov 22, 2023

• Removed SHA for macOS x86_64

  ilya-lavrenov

  committed
  c4cac64

  Copy full SHA for c4cac64

• More fixes

  ilya-lavrenov

  committed
  2b4aa4f

  Copy full SHA for 2b4aa4f

• Merge remote-tracking branch 'upstream/master' into tokenizer-fix-decode

  ilya-lavrenov

  committed
  14bde86

  Copy full SHA for 14bde86

• Fixed macos

  ilya-lavrenov

  committed
  3eca50b

  Copy full SHA for 3eca50b

Commits on Nov 24, 2023

• Merge remote-tracking branch 'upstream/master' into tokenizer-fix-decode

  ilya-lavrenov

  committed
  3e441ff

  Copy full SHA for 3e441ff

• Enabled tests

  ilya-lavrenov

  committed
  bac9791

  Copy full SHA for bac9791

Commits on Nov 25, 2023

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  ilya-lavrenov

  committed
  38790c8

  Copy full SHA for 38790c8

• Fixed warnings

  ilya-lavrenov

  committed
  f68123f

  Copy full SHA for f68123f

• Use developer package

  ilya-lavrenov

  committed
  2da5353

  Copy full SHA for 2da5353

Commits on Nov 28, 2023

• Split build

  ilya-lavrenov

  committed
  5209f50

  Copy full SHA for 5209f50

• Update windows.yml

  ilya-lavrenov

  authored
  f7bea4c

  Copy full SHA for f7bea4c

• Added missed IMPLEMENT_OPENVINO_EXTENSION_API

  ilya-lavrenov

  committed
  3153964

  Copy full SHA for 3153964

• Merge remote-tracking branch 'origin/tokenizer-fix-decode' into tokenizer-fix-decode

  ilya-lavrenov

  committed
  87c1cee

  Copy full SHA for 87c1cee

• Update windows.yml

  ilya-lavrenov

  authored
  b10148f

  Copy full SHA for b10148f

• Update windows.yml

  ilya-lavrenov

  authored
  1ef316b

  Copy full SHA for 1ef316b

Commits on Nov 29, 2023

• Update windows.yml

  ilya-lavrenov

  authored
  8de402c

  Copy full SHA for 8de402c

• Update windows.yml

  ilya-lavrenov

  authored
  c9dd82f

  Copy full SHA for c9dd82f

• Update .ci/azure/windows.yml

  Show description for d74d4af
  ilya-lavrenov

  authored
  d74d4af

  Copy full SHA for d74d4af

• Update windows.yml

  ilya-lavrenov

  authored
  96867ca

  Copy full SHA for 96867ca

• Update windows.yml

  ilya-lavrenov

  authored
  02d0488

  Copy full SHA for 02d0488