DJL Monthly Update - July+ 2022 #1882

zachgk · 2022-08-09T22:45:44Z

zachgk
Aug 9, 2022
Maintainer

Also including content from several previous months

Released DJL 0.18.0 (Release Notes)

Key Features:

AArch64 support for PyTorch
linux AArch64 support for PyTorch
macOS M1 chip (#1696)
JDK 17 support (#1672
Full support of PyTorch Get Indexing for NDArrays (#1719
Full support of PyTorch Set Indexing for NDArrays (#1755
Updates Dataset documentation (#1686
Moves djl-bench to DJL Serving (#1743
Engines and Extensions
- TensorFlow 2.7.0 (#1674
- New djl-audio extension (#1681
- Adds GPU support for XGBoost (#1680
- tokenizers 0.12.0 (#1739
- sentencepiece 0.1.96 (#1745
- TensorRT 8.4.1 (#1758
- XGBoost version 1.6.0 (#1624)
- PaddlePaddle version 2.2.2 (#1601)
- ONNXRuntime version 1.11.0 (#1602)
- PyTorch version 1.11.0 (#1583)
- Apache MXNet version 1.9.0 (#1429)
Newly Added Datasets
- Goemotions dataset (#1598
- Daily Delhi Climate Dataset (#1667
- Tablesaw Dataset (#1679
- Universal Dependencies Corpus for English (#1595
- Movielens 100k dataset (#1718
- PennTreebank dataset (#1580)
- WikiText-2 dataset (#1545)
Support freezing parameters for transfer learning (#1544)

Community

Come attend DJL's biweekly community call for discussions about DJL development, events, and projects. All are welcome. You can find the agenda and meeting links on the meeting discussion page to learn more.

Blog post Scaling-up PyTorch inference: Serving billions of daily NLP inferences with ONNX Runtime by Faith Xu

Blog post Deploy HuggingFace NLP Models in Java With Deep Java Library by Kexin Feng and Jake Lee

Blog post Using Deep Java Library on Spring Boot by Arika Saputro

Paid course Get Started with Deep Java Library teaches how to train a model on Cifar10

Blog post Detecting objects with Groovy, the Deep Java Library (DJL), and Apache MXNet by Paul King

Feature Enhancements

DJL

Separate AbstractSymbolBlock from AbstractBlock (#1555)
Update checkstyle to require inheritDoc on override (#1564)
Add better error message for libstdc++ tf errors (#1570)
basicdataset Add Stanford Question Answering Dataset (#1554)
xgb Set default missing value to NaN (#1571)
allow empty batch data (#1569)
pytorch Upgrade to pytorch 1.11.0 (#1583)
pytorch Allows load libtroch from pip installation package (#1577)
HF Tokenizer: get charspans (#1584)
benchmark Allows benchmark run on aarch64 for PyTorch (#1591)
add pytorch cuDNN acceleration (#1592)
Paddle Update Paddle version to 2.2.2 (#1601)
bump up onnxruntime version (#1602)
add paddle CU110 (#1604)
ONNXRuntime add tensorRT option (#1611)
api Refactor metrics API (#1613)
bom Update bom dependency (#1615)
pytorch Use precxx11 build for aarch64 native library (#1619)
pytorch use precxx11 for aarch64 (#1620)
basicdataset Add PennTreebank dataset (#1580)
Update xgboost to 1.6.0 (#1624)
Support gather of pytorch (#1622)
bump up versioning (#1633)
update android package versions (#1635)
integration Enable gather unit test for windows (#1638)
Increase Final Version for main branch (#1644)
Increase build version to 0.18.0-SNAPSHOT (#1645)
Support of take from pytorch (#1627)
Upgrade JNA to 5.11.0 (#1655)
api Improves ServingTranslator output handling (#1654)
basicdataset Add a Goemotions dataset (#1598)
add width/height conversion to ObjectDetection (#1651)
pytorch Add macOS M1 chip support. (#1656)
add openCV find rectangle method (#1662)
basicdataset Add Daily Delhi Climate Dataset (#1667)
update TF versions (#1674)
remove unnecessary logics (#1676)
training Add Cyclical Tracker (#1671)
Extension Add djl-audio extension (#1681)
xgb Add GPU support for XGBoost (#1680)
Add support of take on MXNet engine (#1649)
Create Tablesaw Dataset (#1679)
Follow symlinks when loading models. (#1692)
basicdataset Add a Universal Dependencies Corpus for English (#1595)
TabNet/WIP Implement GhostBatchNorm (#1666)
allow indexer to attach specific manager (#1688)
Upgrade android module to use DJL 0.18.0 (#1693)
pytorch Uses pytorch to test API and aws-ai module (#1695)
xgb Avoid download cudf dependency at build time (#1694)
bump up versions (#1691)
osx m1 Allow DJL build on macOS M1 chip (#1696)
Create tabular dataset (#1699)
Creates PreparedFeaturizer (#1700)
Normalized Numeric Featurizer (#1701)
api Refactor ServingTranslatorFactory (#1702)
Added "capped" state to NDManager (#1683)
added LinearCollection block (#1658)
add android test code (#1714)
Change DJL repo names from aws-samples (#1716)
Adding Movielens 100k tabular dataset to basicdatasets (#1718)
add serving deb file publish (#1721)
Ensure PreparedOneHotStringFeaturizer encodes categorical mappings consistently across dataset usages (#1723)
Add ai.djl.audio and ai.djl.tablesaw to BOM (#1728)
format Upgrade java format to 1.15.0 (#1727)
api Add name to LambdaBlock (#1726)
add disable static option in DJL to allow some model running (#1735)
Bump tokenizers version to 0.12.0 (#1739)
Criteria.toBuilder() (#1741)
Bump sentencepiece version to 0.1.96 #1745)
Enable better textual description of neural net (#1720)
Advanced indexing that supports all indexing features on PyTorch getter (#1719)
api Ignore hidden files for nested model directory (#1754)
build jni for aarch64 (#1756)
Trt version upgrade (#1758)
Remove unnecessary packages from tensorrt dockerfile (#1760)
api Add log for custom Translator loading (#1761)
store indices with batch (#1750)
Add put feature with linear indexing on PyTorch engine (#1749)
pytorch Add NDList to IValue unit test (#1762)
make tensorflow always dense (#1763)
pytorch Adds IValue Dict(str, IValue) support (#1765)
Add tensor setter feature with advanced indexing on PyTorch engine (#1755)
ort Add support for registerCustomOpLibrary. (#1771)
Move djl-bench to DJL Serving (#1743)
api Supports Image output for ImageServingTranslator (#1772)
Implementing inverse operation (#1768)
api Allows user specify model name in serving.properties file (#1780)
Adding model zoo implementation (#1781)
change the sagemaker model to s3 (#1769)
Improvements to image coloring (#1784)
paddle paddle model zoo should not have compile time dependency on opencv (#1785)
Increase DJL version for main branch (#1790)
Update README for 0.18.0 Release (#1791)
Increase build version to 0.19.0 (#1792)
Backward compatible with MXNet indexing. (#1802)
Compatibility changes for Semantic Segmentation app (#1808)
android Bumpup android version to 0.19.0-SNAPSHOT (#1812)
model-zoo Add deeplabvs metadata.json (#1818)
api Allows convert float32 NDArray to Image. (#1814)
Add MobileNetV1 into modelZoo and add corresponding test class MobileNetV1Test (#1817)
tokenizer Add encodeDual support for huggingface tokenizer (#1826)
pt-model-zoo Make PtBertQATranslator compatible with huggingface model (#1827)
Added Huggingface QuestionAnsweringTranslator (#1828)
pytorch Allows block only model for PyTorch (#1831)
Updated image factories for HWC and CHW (#1833)
upgrade onnxruntime (#1838)
Add script to import huggingface model into djl model zoo (#1835)
api Support queryString for JarRepository (#1842)
Filtered out non-english models in huggingface_converter (#1845)
Expose tokenizers decode method (#1843)
Added fill-mask support for converting huggingface model to model zoo (#1849)
Neural machine translation upload (#1851)
api Set arguments in serving.properties as model properties (#1853)
api Adds IdentityBlockFactory for demo/test purpose (#1854)
Add ARM support for ONNXRuntime update unit test (#1856)
Allow overriding special token flags in encode and decode methods (#1855)

DJL Serving

Multi-device model support (#77)
serving Removes redundant logging (#79)
python configure model loading/prediction timeout (#80)
serving Upgrade dependency version (#82)
python Support custom metric for python engine (#81)
serving Avoid load MXNet unnecessarily (#83)
Bump async from 2.6.3 to 2.6.4 in /central (#85)
serving Add server metrics (#86)
serving Install engine plugin if needed (#87)
python Improves pip installation logging (#91)
serving Add install dependency in commandline (#92)
bump up djlserving version (#93)
support more python use cases (#94)
add python test (#95)
add integration tests (#96)
Disable python alternative engine by default (#97)
add pytest for binary mode (#98)
Adding model property check for max workers (#99)
python Reformat python code (#100)
add Java engine for Java client purpose (#101)
create Java client testing (#102)
serving Fix selfsign certificate issue for JDK 15+ (#105)
add Test Translator plugin (#104)
add DeepSpeed docker image (#106)
add cicd publish (#107)
add two new instances (#111)
add aarch64 and inferentia build (#112)
format Upgrade java formatter to 1.15.0 (#116)
Docker Create CPU docker image (#113)
add matrix build and skip aarch64 (#118)
use python 3.7 paths (#120)
add aarch64 build instruction (#121)
Docker GPU docker image (#119)
update headers (#123)
Docker Support full package installation (#124)
yes to remove spaces (#127)
change the order (#128)
Move djl-bench to serving repo (#125)
clean up pip cache (#129)
djl-bench Add .npz output (#131)
docker rename tag inf1 to pytorch-inf1 (#133)
Download models (#135)
Docker Integration add test client script for curl (#132)
launch docker container and wait (#136)
Add mac and windows (#140)
add integration tests (#139)
ModelInfo from Criteria (#130)
add inferentia,gpu,aarch tests (#143)
serving Support nested folder in model archieve file (#144)
add timeout to prevent instance hanging (#147)
docker install pytorch-jni for release docker (#148)
docker install docker in start_instance.sh (#150)
make retry to 2min to avoid retry timeout (#151)
Parameterize WLM (#142)
Add model registration to WLM (#153)
Add gpu and inf1 test (#157)
Adding more engine tests (#152)
install deepspeed specialized engine (#156)
Create test of standalone WLM (#154)
python Support serve huggingface transformers model (#158)
workaround for sagemaker configs (#160)
serving Avoid unnecessary download MXNet engine (#161)
docker numpy is required by djl python engine by default (#162)
upgrade netty to latest version (#165)
Upgrade to DJL 0.18.0 (#168)
Create WlmBenchmark (#166)
tests Use local java engine to test java client (#171)
bump to latest djl snapshot version (#170)
docker Add -XX:+UseContainerSupport option (#172)
update deepspeed version (#173)
Bump terser from 5.12.1 to 5.14.2 in /central (#175)
update snapcraft file to use correct repo and remove dangling symlink (#174)
Update macOS instructions to download from zip (#176)
add docker image migration process (#179)
install awscli (#180)
serving Allows user override OMP_NUM_THREADS (#183)
serving Change scale worker per device (#185)
Catch security exception in get environment call (#186)
serving Respect per model minWorkers setting (#188)
wlm Removes AutoClosable for ModelServer, Endpoint and Workflow (#187)

DJL Spring Boot Starter

Release djl-spring-boot-starter 0.15 (#20)
api Uses new DJL api to load model (#21)
update djl-springboot-version 0.16 → 0.17 (#22)
Update to DJL 0.18.0 (#24)

DJL Clojure

update to djl 0.16.0, clj-djl 0.1.9
add build.clj, a new method
Update clojure.yml
Change tools.deps version
update document, use build.clj and add loss test
simplify test cases
Merge branch 'master' of github.com:kimim/clj-djl

Documentation & Examples

DJL

Add release note to README (#1565)
add troubleshoot issue (#1600)
Update badge (#1610)
Update inference_performance_optimization.md (#1621)
update pytorch build instruction on android (#1630)
docs Update pytorch and paddle version (#1634)
Update README for 0.17.0 Release (#1646)
Fix broken and redirected links (#1647)
docs Clarified typo in example documentation (#1685)
Update the Dataset documentation (#1686)
Update description for latest javadoc location (#1708)
docs Create README for DJL Android PyTorch 1.11 builds (#1704)
Add serving to docs site (#1715)
Fixing broken javadoc links in jupyter notebooks (#1722)
docs Readme updates for PyTorch 1.11 (#1709)
doc Update CVSDataset example README file (#1729)
mxnet Update document to use MXNet 1.9.0 (#1737)
Update docs\development\setup.md, using JDK11 instead of JDK8 (#1767)
Add documentation on loading TF extension libraries for running certain TF models in DJL (#1776)
Adding semantic segmentation example (#1764)
Update bert classification notebook to reflect changes in CSVDataset (#1786)
fix typo in bert notebook (#1787)
docs Updates troubleshooting.md to remove -native-auto package (#1793)
docs Add audio and tablesaw to dependency document (#1794)
pytorch Document PYTORCH_PRECXX11 usage in README (#1807)
doc Immutable array output from InferenceMode PyTorch (#1822)
doc Remove pytorch cu102 package for Windows in README file (#1836)
doc Fixes NDIndex javadoc issue (#1841)