Skip to content

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Nov 4, 2024

Bumps sentencepiece from 0.1.99 to 0.2.0.

Release notes

Sourced from sentencepiece's releases.

v0.2.0

Major changes

N/A

New features

  • [ALL] Added SentencePieceNormalizer class in C++/Python. It supports almost the equivalent feature of spm_normalize. Python Sample C++ Sample
  • [ALL] Added SentencePieceProcessor::Normalize method in C++/Python Python Sample C++ Sample
  • [ALL] Added functionality to override the normalization spec before the processing. Python Sample

Bug fixes & minor changes

  • Introduce better support of using external abseil and protobuf google/sentencepiece#869
  • Build universal binary in OSX release package google/sentencepiece#892
  • Add the set_min_log_level function to python to change the loglevel from the python wrapper. google/sentencepiece#893
  • Uses the logsumexp techniques in marginal probabilities of n-best tokenization to avoid underflow.
  • Support Python 3.12 google/sentencepiece#932
  • Improves the thread utilization in batch encoding/decoding.
  • Fix nasty bug in BPE position encoding.
  • Fix bugs in the handling of duplicated bigrams

v0.2.0pre1

Major changes

N/A

New features

  • [ALL] Added SentencePieceNormalizer class in C++/Python. It supports almost the equivalent feature of spm_normalize. Python Sample C++ Sample
  • [ALL] Added SentencePieceProcessor::Normalize method in C++/Python Python Sample C++ Sample
  • [ALL] Added functionality to override the normalization spec before the processing. Python Sample

Bug fixes & minor changes

  • Introduce better support of using external abseil and protobuf google/sentencepiece#869
  • Build universal binary in OSX release package google/sentencepiece#892
  • Add the set_min_log_level function to python to change the loglevel from the python wrapper. google/sentencepiece#893
  • Uses the logsumexp techniques in marginal probabilities of n-best tokenization to avoid underflow.
  • Support Python 3.12 google/sentencepiece#932
  • Improves the thread utilization in batch encoding/decoding.
  • Fix nasty bug in BPE position encoding.
  • Fix bugs in the handling of duplicated bigrams
Commits
  • 17d7580 suppress warnings in testharnress
  • 4a3cd1c Merge pull request #975 from google/dependabot/pip/dot-github/workflows/requi...
  • 670d2e7 Bump cryptography in /.github/workflows/requirements
  • 2b8772a Merge pull request #974 from xunkai55/patch-1
  • ffd8e9e Fix a typo in api.md
  • 03243af Merge pull request #970 from google/dependabot/pip/dot-github/workflows/requi...
  • d0fe405 Merge pull request #972 from google/dependabot/pip/dot-github/workflows/requi...
  • a8a618f Bump cryptography in /.github/workflows/requirements
  • bbbe548 Bump the build-time-deps group
  • 53de765 allows to load precomputed seed sentencepieces for unigram from a file.
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [sentencepiece](https://github.com/google/sentencepiece) from 0.1.99 to 0.2.0.
- [Release notes](https://github.com/google/sentencepiece/releases)
- [Commits](google/sentencepiece@v0.1.99...v0.2.0)

---
updated-dependencies:
- dependency-name: sentencepiece
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Nov 4, 2024
Copy link

sonarqubecloud bot commented Nov 4, 2024

@bact bact merged commit 2987f32 into dev Nov 5, 2024
39 checks passed
@dependabot dependabot bot deleted the dependabot/pip/sentencepiece-0.2.0 branch November 5, 2024 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant