Skip to content

v0.6.0
Compare
Choose a tag to compare
@asafgardin asafgardin released this 28 Dec 14:28
· 33 commits to main since this release
v0.6.0
7b8348d

v0.6.0 (2023-12-28)

Chore

  • chore: add test case for encode with is_start=False (#74)

  • chore: add test case for encode with is_start=False

  • fix: split is_start=False to a different testcase (77c0a39)

Feature

  • feat: Add decode with offsets (#73)

  • feat: Add decode_with_offsets() to JurassicTokenizer

  • refactor: remove kwargs from decode_with_offsets since it's not used

  • chore: Add unittest for decode and for offsets

  • fix: test only decode_with_offsets

  • fix: dummy for returned offsets in decode_with_offsets (a5a7bb4)

  • feat: Add the is_start parameter to JurassicTokenizer.encode() (#72)

  • feat: Add the is_start parameter to JurassicTokenizer.encode()

  • refactor: take 'is_start' from kwargs (296bda5)

Assets 2
Loading