failed to build pypesq on runpod with both pytorch 2.1 and 2.4

[lyramakesmusic]

i'm installing stable-audio-tools latest commit () on runpod and running into the following error. I tried on both runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04 and runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04, fresh environment with nothing installed besides SAT, and it broke in both. pre-0.0.17 commits install just fine (pypesq was added as a dependency in 0.0.17).

Building wheels for collected packages: stable-audio-tools, encodec, pypesq, argbind, pathtools, progressbar, wget, julius, randomname, fire
  Building editable for stable-audio-tools (pyproject.toml) ... done
  Created wheel for stable-audio-tools: filename=stable_audio_tools-0.0.17-0.editable-py3-none-any.whl size=4125 sha256=fe17a2f666fb0db512de0a4b71a4b6f87d2b2865bce249849f3bc3e358410384
  Stored in directory: /tmp/pip-ephem-wheel-cache-phjn77md/wheels/28/00/44/fb003b9bfe70a787958db92f0ab2b30e077e20d4ce0cedb3dc
  Building wheel for encodec (setup.py) ... done
  Created wheel for encodec: filename=encodec-0.1.1-py3-none-any.whl size=45759 sha256=6d3d33c6ca9d1f01b2348655fc3da50460efea0c664e77e276073e23996c9f9a
  Stored in directory: /root/.cache/pip/wheels/fc/36/cb/81af8b985a5f5e0815312d5e52b41263237af07b977e6bcbf3
  Building wheel for pypesq (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [106 lines of output]
      /usr/local/lib/python3.10/dist-packages/setuptools/installer.py:33: UserWarning: Module numpy was already imported from /usr/local/lib/python3.10/dist-packages/numpy/__init__.py, but /tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066 is being added to sys.path
        import pkg_resources  # Delay import to avoid unnecessary side-effects
      /usr/local/lib/python3.10/dist-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!
      
              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************
      
      !!
        dist.fetch_build_eggs(dist.setup_requires)
      running bdist_wheel
      running build
      running build_py
      file numpy.py (for module numpy) not found
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/pypesq
      copying pypesq/__init__.py -> build/lib.linux-x86_64-cpython-310/pypesq
      file numpy.py (for module numpy) not found
      running build_ext
      building 'pesq_core' extension
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/pypesq
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/dsp.c -o build/temp.linux-x86_64-cpython-310/pypesq/dsp.o
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/pesq.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesq.o
      In file included from /usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy/ndarraytypes.h:1940,
                       from /usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                       from /usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy/arrayobject.h:5,
                       from pypesq/pesq.c:2:
      /usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
         17 | #warning "Using deprecated NumPy API, disable it with " \
            |  ^~~~~~~
      pypesq/pesq.c: In function ‘_pesq’:
      pypesq/pesq.c:61:34: warning: passing argument 1 of ‘compute_pesq’ from incompatible pointer type [-Wincompatible-pointer-types]
         61 |     float pesq = compute_pesq(ref->data, deg->data, ref->dimensions[0], deg->dimensions[0], fs);
            |                               ~~~^~~~~~
            |                                  |
            |                                  char *
      In file included from pypesq/pesq.c:5:
      pypesq/pesq.h:287:28: note: expected ‘short int *’ but argument is of type ‘char *’
        287 | float compute_pesq(short * ref, short * deg, long ref_n_samples, long deg_n_samples, long fs);
            |                    ~~~~~~~~^~~
      pypesq/pesq.c:61:45: warning: passing argument 2 of ‘compute_pesq’ from incompatible pointer type [-Wincompatible-pointer-types]
         61 |     float pesq = compute_pesq(ref->data, deg->data, ref->dimensions[0], deg->dimensions[0], fs);
            |                                          ~~~^~~~~~
            |                                             |
            |                                             char *
      In file included from pypesq/pesq.c:5:
      pypesq/pesq.h:287:41: note: expected ‘short int *’ but argument is of type ‘char *’
        287 | float compute_pesq(short * ref, short * deg, long ref_n_samples, long deg_n_samples, long fs);
            |                                 ~~~~~~~~^~~
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/pesqdsp.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesqdsp.o
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/pesqio.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesqio.o
      pypesq/pesqio.c: In function ‘load_src’:
      pypesq/pesqio.c:200:10: warning: unused variable ‘s’ [-Wunused-variable]
        200 |     char s;
            |          ^
      pypesq/pesqio.c:198:10: warning: unused variable ‘count’ [-Wunused-variable]
        198 |     long count;
            |          ^~~~~
      pypesq/pesqio.c:196:10: warning: unused variable ‘to_read’ [-Wunused-variable]
        196 |     long to_read;
            |          ^~~~~~~
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/pesqmain.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesqmain.o
      pypesq/pesqmain.c: In function ‘compute_pesq’:
      pypesq/pesqmain.c:118:10: warning: unused variable ‘names’ [-Wunused-variable]
        118 |     int  names = 0;
            |          ^~~~~
      pypesq/pesqmain.c: In function ‘pesq_measure’:
      pypesq/pesqmain.c:405:25: warning: variable ‘end’ set but not used [-Wunused-but-set-variable]
        405 |             long start, end;
            |                         ^~~
      pypesq/pesqmain.c:405:18: warning: variable ‘start’ set but not used [-Wunused-but-set-variable]
        405 |             long start, end;
            |                  ^~~~~
      pypesq/pesqmain.c:242:9: warning: unused variable ‘i’ [-Wunused-variable]
        242 |     int i;
            |         ^
      pypesq/pesqmain.c:404:12: warning: ‘resultsFile’ may be used uninitialized [-Wmaybe-uninitialized]
        404 |         if (resultsFile != NULL) {
            |            ^
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipypesq -I/usr/local/lib/python3.10/dist-packages/numpy/core/include/numpy -I/usr/include/python3.10 -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -c pypesq/pesqmod.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesqmod.o
      pypesq/pesqmod.c: In function ‘utterance_split’:
      pypesq/pesqmod.c:298:10: warning: variable ‘Utt_Delay’ set but not used [-Wunused-but-set-variable]
        298 |     long Utt_Delay;
            |          ^~~~~~~~~
      pypesq/pesqmod.c: In function ‘pesq_psychoacoustic_model’:
      pypesq/pesqmod.c:806:13: warning: variable ‘peak’ set but not used [-Wunused-but-set-variable]
        806 |     float   peak;
            |             ^~~~
      pypesq/pesqmod.c:782:24: warning: variable ‘power_deg’ set but not used [-Wunused-but-set-variable]
        782 |     float   power_ref, power_deg;
            |                        ^~~~~~~~~
      pypesq/pesqmod.c:782:13: warning: variable ‘power_ref’ set but not used [-Wunused-but-set-variable]
        782 |     float   power_ref, power_deg;
            |             ^~~~~~~~~
      x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-x86_64-cpython-310/pypesq/dsp.o build/temp.linux-x86_64-cpython-310/pypesq/pesq.o build/temp.linux-x86_64-cpython-310/pypesq/pesqdsp.o build/temp.linux-x86_64-cpython-310/pypesq/pesqio.o build/temp.linux-x86_64-cpython-310/pypesq/pesqmain.o build/temp.linux-x86_64-cpython-310/pypesq/pesqmod.o -L/usr/lib/x86_64-linux-gnu -o build/lib.linux-x86_64-cpython-310/pesq_core.cpython-310-x86_64-linux-gnu.so
      /usr/bin/ld: build/temp.linux-x86_64-cpython-310/pypesq/pesqdsp.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: multiple definition of `Nb'; build/temp.linux-x86_64-cpython-310/pypesq/pesq.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: first defined here
      /usr/bin/ld: build/temp.linux-x86_64-cpython-310/pypesq/pesqio.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: multiple definition of `Nb'; build/temp.linux-x86_64-cpython-310/pypesq/pesq.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: first defined here
      /usr/bin/ld: build/temp.linux-x86_64-cpython-310/pypesq/pesqmain.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: multiple definition of `Nb'; build/temp.linux-x86_64-cpython-310/pypesq/pesq.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: first defined here
      /usr/bin/ld: build/temp.linux-x86_64-cpython-310/pypesq/pesqmod.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: multiple definition of `Nb'; build/temp.linux-x86_64-cpython-310/pypesq/pesq.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesq.h:143: first defined here
      /usr/bin/ld: build/temp.linux-x86_64-cpython-310/pypesq/pesqmod.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesqpar.h:123: multiple definition of `InIIR_Nsos'; build/temp.linux-x86_64-cpython-310/pypesq/pesqdsp.o:/tmp/pip-install-qf4w0fv8/pypesq_cb14de9496334f8eb2ec7d11731ce066/pypesq/pesqdsp.c:136: first defined here
      collect2: error: ld returned 1 exit status
      error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pypesq
  Running setup.py clean for pypesq
  Building wheel for argbind (setup.py) ... done
  Created wheel for argbind: filename=argbind-0.3.9-py2.py3-none-any.whl size=11733 sha256=3831d666528c9b679763c590616f96122ec26c79975a56119d0f55cb6c8688bf
  Stored in directory: /root/.cache/pip/wheels/ed/ab/ff/64eb14a776ae6525e1a7d6ad38b73ba020ecc4262d83a7889d
  Building wheel for pathtools (setup.py) ... done
  Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8790 sha256=39a06fa2bcc9f606dcdb6406cdeb8ad381060f2dc99ebab5d2176eac8a69d60c
  Stored in directory: /root/.cache/pip/wheels/e7/f3/22/152153d6eb222ee7a56ff8617d80ee5207207a8c00a7aab794
  Building wheel for progressbar (setup.py) ... done
  Created wheel for progressbar: filename=progressbar-2.5-py3-none-any.whl size=12067 sha256=56a6209eff0e91fc6f7cdc61a85fb22c4a1e989bee622df7a7ee28048c627154
  Stored in directory: /root/.cache/pip/wheels/cd/17/e5/765d1a3112ff3978f70223502f6047e06c43a24d7c5f8ff95b
  Building wheel for wget (setup.py) ... done
  Created wheel for wget: filename=wget-3.2-py3-none-any.whl size=9657 sha256=ebd868faddc73a9727c545effbe9d0dc235633a61662509e91a0fcc6dd00e978
  Stored in directory: /root/.cache/pip/wheels/8b/f1/7f/5c94f0a7a505ca1c81cd1d9208ae2064675d97582078e6c769
  Building wheel for julius (setup.py) ... done
  Created wheel for julius: filename=julius-0.2.7-py3-none-any.whl size=21869 sha256=df16880143ebb7e2323033c796baeb00791a8d7f650305b3415889f7390a12a9
  Stored in directory: /root/.cache/pip/wheels/b9/b2/05/f883527ffcb7f2ead5438a2c23439aa0c881eaa9a4c80256f4
  Building wheel for randomname (setup.py) ... done
  Created wheel for randomname: filename=randomname-0.2.1-py3-none-any.whl size=89194 sha256=2a8d8de7a2f372d8ac778c4ad55d31e5daef208e7982873e5d7f5bbb295a2b37
  Stored in directory: /root/.cache/pip/wheels/10/50/8a/25f3820d26a431ffed1834d72ff2eb349123cf2b44c5a45727
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.7.0-py3-none-any.whl size=114248 sha256=3cf4b49d3da97f56e43a0e48d14bd1a2bf59f6069f285be1a96e9d5de7c53d55
  Stored in directory: /root/.cache/pip/wheels/19/39/2f/2d3cadc408a8804103f1c34ddd4b9f6a93497b11fa96fe738e
Successfully built stable-audio-tools encodec argbind pathtools progressbar wget julius randomname fire
Failed to build pypesq
ERROR: Could not build wheels for pypesq, which is required to install pyproject.toml-based projects

[lyramakesmusic]

when i used the suggested pip install --use-pep517 pypesq, it gave AttributeError: 'dict' object has no attribute '__NUMPY_SETUP__' from setuptools and still failed

[julian-parker]

Hi @lyramakesmusic!

Thanks for reporting. Looks like this is describing the same issue: https://stackoverflow.com/questions/75587486/how-can-i-install-pypesq-properly

They suggest pip installing from the repo instead of the PyPI package:
pip install https://github.com/vBaiCai/python-pesq/archive/master.zip

[lyramakesmusic]

thanks! i will try next run

[roserbatlleroca]

I have encountered a similar issue as @lyramakesmusic and I'd like to share the solution in case someone else faces the same situation. Note that my configuration is Ubuntu 22.04, Python 3.10, and GCC 11.2.0.

In my case, it was gcc that caused the error (see below), and the proposed solution didn't work. It seems that there's a persistent issue with building pypesq as many users have been reporting issues related to this recently. I found a plausible solution at davidmartinrius/speech-dataset-generator#13, where they recommend upgrading numpy: pip install --upgrade numpy.

Then, after executing pip install https://github.com/vBaiCai/python-pesq/archive/master.zip to install pypesq, stable-audio-tools was successfully installed.

Building wheels for collected packages: stable-audio-tools, pypesq
  Building wheel for stable-audio-tools (pyproject.toml) ... done
  Created wheel for stable-audio-tools: filename=stable_audio_tools-0.0.17-py3-none-any.whl size=151299 sha256=1bb31d63ef92c5923679ae9e9f7232f9e39b50336aae8bba6f15081c082f1d03
  Stored in directory: /home/roser/.cache/pip/wheels/3b/c8/df/ee53da1546ec37f2ef214e2d9b30044e96f88987fa61d6581f
Building wheel for pypesq (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      /home/roser/miniconda3/envs/sta-open/lib/python3.10/site-packages/setuptools/__init__.py:94: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!

      
              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************
      
      !!
        dist.fetch_build_eggs(dist.setup_requires)
      running bdist_wheel
      running build
      running build_py
      file numpy.py (for module numpy) not found
      creating build/lib.linux-x86_64-cpython-310/pypesq
      copying pypesq/__init__.py -> build/lib.linux-x86_64-cpython-310/pypesq
      file numpy.py (for module numpy) not found
      running build_ext
      building 'pesq_core' extension
      creating build/temp.linux-x86_64-cpython-310/pypesq
      gcc -pthread -B /home/roser/miniconda3/envs/sta-open/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/roser/miniconda3/envs/sta-open/include -fPIC -O2 -isystem /home/roser/miniconda3/envs/sta-open/include -fPIC -Ipypesq -I/home/roser/miniconda3/envs/sta-open/include/python3.10 -I/tmp/pip-install-98mn3nah/pypesq_3c46df4f449e49f78b973b4d3f6e7ab4/.eggs/numpy-2.2.2-py3.10-linux-x86_64.egg/numpy/_core/include -c pypesq/dsp.c -o build/temp.linux-x86_64-cpython-310/pypesq/dsp.o
      gcc -pthread -B /home/roser/miniconda3/envs/sta-open/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/roser/miniconda3/envs/sta-open/include -fPIC -O2 -isystem /home/roser/miniconda3/envs/sta-open/include -fPIC -Ipypesq -I/home/roser/miniconda3/envs/sta-open/include/python3.10 -I/tmp/pip-install-98mn3nah/pypesq_3c46df4f449e49f78b973b4d3f6e7ab4/.eggs/numpy-2.2.2-py3.10-linux-x86_64.egg/numpy/_core/include -c pypesq/pesq.c -o build/temp.linux-x86_64-cpython-310/pypesq/pesq.o
      pypesq/pesq.c:2:10: fatal error: arrayobject.h: No such file or directory
          2 | #include "arrayobject.h"
            |          ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pypesq
  Running setup.py clean for pypesq
Successfully built stable-audio-tools
Failed to build pypesq
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (pypesq)

[accraze]

I had this issue too and needed to install libpythonX.X-dev on Ubuntu to install pypesq. Seems to work fine now w/ python3.10 and gcc11.3

[pangolin24]

One possible workaround if you're not using the repo for speech (PESQ being a speech quality metric):

1. Comment out pypesq from setup.py).

# 'pypesq==1.2.4',

And then to avoid import issues when using the repo:

2. Comment out the import in the losses init:

# from .metrics import *  # Don't require pypesq

3. Make the import of PESQMetric lazy in stable_audio_tools/training/autoencoders.py (here):

        if eval_loss_config is not None:
            if "pesq" in eval_loss_config:
                from .losses.metrics import PESQMetric  # Add this
                self.eval_losses["pesq"] = PESQMetric(sample_rate)
            if "stft"in eval_loss_config:
                self.eval_losses["stft"] = auraloss.STFTLoss(**eval_loss_config["stft"])
            if "sisdr" in eval_loss_config:
                self.eval_losses["sisdr"] = auraloss.SISDRLoss(**eval_loss_config["sisdr"])
            if "mel" in eval_loss_config:
                self.eval_losses["mel"] = auraloss.MelSTFTLoss(
                    sample_rate, **eval_loss_config["mel"])

4. Remove the import from the top of autoencoders training module.

from .losses import MelSpectrogramLoss, MultiLoss, AuralossLoss, ValueLoss, TargetValueLoss, L1Loss, LossWithTarget, MSELoss, HubertLoss # , PESQMetric