Skip to content

Commit 48e0622

Browse files
authored
Encodec notebook (openvinotoolkit#864)
1 parent 73ed1f7 commit 48e0622

File tree

5 files changed

+869
-0
lines changed

5 files changed

+869
-0
lines changed

README.md

+1
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@ More amazing notebooks here!
153153
|[231-instruct-pix2pix-image-editing](notebooks/231-instruct-pix2pix-image-editing/)<br>| Image editing with InstructPix2Pix | <img src=https://user-images.githubusercontent.com/29454499/219943222-d46a2e2d-d348-4259-8431-37cf14727eda.png width=225> |
154154
|[232-clip-language-saliency-map](notebooks/232-clip-language-saliency-map/)<br>| Language-Visual Saliency with CLIP and OpenVINO™ | <img src=https://user-images.githubusercontent.com/29454499/218967961-9858efd5-fff2-4eb0-bde9-60852f4b31cb.JPG width=225> |
155155
|[233-blip-visual-language-processing](notebook/233-blip-visual-language-processingp/)<br>| Visual Question Answering and Image Captioning using BLIP and OpenVINO™ | <img src=https://user-images.githubusercontent.com/29454499/221933762-4ff32ecb-5e5d-4484-80e1-e9396cb3c511.png width=225> |
156+
|[234-encodec-audio-compression](notebooks/234-encodec-audio-compression/)<br>| # Audio compression with EnCodec and OpenVINO™ | <img src=https://github.com/facebookresearch/encodec/raw/main/thumbnail.png width=225> |
156157
</details>
157158
</p>
158159

notebooks/234-encodec-audio-compression/234-encodec-audio-compression.ipynb

+839
Large diffs are not rendered by default.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Audio compression with EnCodec and OpenVINO
2+
3+
CCompression is an important part of the Internet today because it enables people to easily share high-quality photos, listen to audio messages, stream their favorite shows, and so much more. Even when using today’s state-of-the-art techniques, enjoying these rich multimedia experiences requires a high speed Internet connection and plenty of storage space. AI helps to overcome these limitations: "Imagine listening to a friend’s audio message in an area with low connectivity and not having it stall or glitch."
4+
5+
In this tutorial, we consider how to use OpenVINO and EnCodec algorithm for hyper compression of audio.
6+
EnCodec is a real-time, high-fidelity audio codec that uses AI to compress audio files without losing quality. It was introduced in [High Fidelity Neural Audio Compression](https://arxiv.org/pdf/2210.13438.pdf) paper by Meta AI. More details about this approach can be found in [Meta AI blog](https://ai.facebook.com/blog/ai-powered-audio-compression-technique/) and original [repo](https://github.com/facebookresearch/encodec).
7+
8+
9+
## Notebook Contents
10+
11+
This notebook demonstrates how to convert and run EnCodec model using OpenVINO.
12+
13+
Notebook contains the following steps:
14+
1. Instantiate and run EnCodec audio compression pipeline.
15+
2. Convert models to OpenVINO IR format using Model Optimizer Python API
16+
3. Integrate OpenVINO to EnCodec pipeline
17+
18+
As the result, we get pipeline that accepts input audio file and converts it to compressed representation, ready for being saved on disk or sent to a recipient. After that, it can be successfully decompressed back to audio.
19+
20+
## Installation Instructions
21+
22+
If you have not done so already, please follow the [Installation Guide](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/README.md) to install all required dependencies.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
--find-links https://download.pytorch.org/whl/torch_stable.html
2+
torchaudio==0.13.1; sys_platform == 'darwin'
3+
torchaudio==0.13.1+cpu; sys_platform == 'linux' or platform_system == 'Windows'
4+
encodec;python_version>"3.7"
5+
git+https://git@github.com/eaidova/encodec#egg=encodec;python_version=="3.7"

notebooks/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,8 @@ More amazing notebooks here!
153153
|[231-instruct-pix2pix-image-editing](231-instruct-pix2pix-image-editing/)<br>| Image editing with InstructPix2Pix | <img src=https://user-images.githubusercontent.com/29454499/219943222-d46a2e2d-d348-4259-8431-37cf14727eda.png width=225> |
154154
|[232-clip-language-saliency-map](232-clip-language-saliency-map/)<br>| Language-Visual Saliency with CLIP and OpenVINO™ | <img src=https://user-images.githubusercontent.com/29454499/218967961-9858efd5-fff2-4eb0-bde9-60852f4b31cb.JPG width=225> |
155155
|[233-blip-visual-language-processing](233-blip-visual-language-processingp/)<br>| Visual Question Answering and Image Captioning using BLIP and OpenVINO™ | <img src=https://user-images.githubusercontent.com/29454499/221933762-4ff32ecb-5e5d-4484-80e1-e9396cb3c511.png width=225> |
156+
|[234-encodec-audio-compression](234-encodec-audio-compression/)<br>| # Audio compression with EnCodec and OpenVINO™ | <img src=https://github.com/facebookresearch/encodec/raw/main/thumbnail.png width=225> |
157+
156158

157159

158160
</details>

0 commit comments

Comments
 (0)