Update pyproject.toml and remove requirements and setup files

[lesander]

Moved dependency definitions to exclusively use pyproject.toml and updated the readme to reflect these changes.

This simplifies the install experience for new Volatility users and ensures users correctly install package tools in their local environment (vol and volshell).

Also bumped the defined minimal CPython version to 3.9.0 since 3.8.0 is EOL. If you have any further questions I will be more than happy to elaborate.

[ikelos]

Thanks, we don't tend to bump the python number at EoL, only when a feature of volatility requires it. Not everybody updates, even when they should, so applying an unnecessary lower bound on the tool doesn't make sense it just creates hurdles for users to jump through.

I'm also wondering about the benefits of the pyproject.toml? requirements.txt has been a supported method for longer, and I believe they're used in our github tests, meaning this would impact the github test files, if you'd included the test/requirements-testing.txt. I'm not specifically against the idea, but I'm also not fully versed in pyproject.toml and I don't like taking away a working mechanism without a clear benefit, see my comments from June when the pyproject.toml was first introduced. You've said it simplifies the installation process, but I'm not sure how reducing the number of methods of achieving a goal simplifies things? At the moment I believe either pyproject.toml or requirements.txt will get the tool properly installled. Is that not the case?

[lesander]

Thanks for the quick response @ikelos. I will take a look at the CI pipeline and do not see any issue in reverting the 3.8 to 3.9 bump. I personally believe that keeping support for end of life python versions should be avoided but I can understand your reasoning. Are there still non-end-of-life LTS unix distributions that support and ship python 3.8?

You've said it simplifies the installation process, but I'm not sure how reducing the number of methods of achieving a goal simplifies things? At the moment I believe either pyproject.toml or requirements.txt will get the tool properly installled. Is that not the case?

Currently that is indeed not the case. Using pip install -r requirements.txt does not install the vol and volshell aliases in the user's environment (it only installs requirements). The main benefit here would be that the installation process is more fool-proof and decreases the possible burden of keeping three installation methods working instead of just one - the python packaging world is confusing enough as it is in my opinion. Using pyproject.toml is indeed opinionated, I think most of the python space is tending to move towards this solution in favor of requirements.txt and setup.py (https://packaging.python.org/en/latest/guides/modernize-setup-py-project/).

[ikelos]

It's not so much what's available from vendors, as what users have on their ancient systems. It would be frustrating for a version of the package to work perfectly fine, but then an upgrade suddenly not work with your system for no reason other than the maintainers trying to be tidy.

It feels like setup.py would probably support adding in the aliases, I'm just not sure we ever bothered? At the moment it's not too much of a burden supporting all the installation mechanisms we do, but I'm happy to improve the pyproject.toml if that would help, and somewhere in the future I'm not against moving to it once it proves to be the prevailing system, but I don't think that situation's been resolved yet (given how eggs became wheels, and setup.py became requirements.txt became pyproject.toml just as poetry and other installation systems began to pick up speed). Happy to revisit this in a year (or less) if you'd like but at the moment I don't feel things have settled just yet... 5:)

[lesander]

It's not so much what's available from vendors, as what users have on their ancient systems. It would be frustrating for a version of the package to work perfectly fine, but then an upgrade suddenly not work with your system for no reason other than the maintainers trying to be tidy.

Thanks for elaborating. I've reverted back to 3.8 in the last commit.

I don't think that situation's been resolved yet [..]

Perhaps I was formatting my words too cautious in the last comment. Pyproject.toml is currently the preferred method for dependency management and has been for some time.

• https://packaging.python.org/en/latest/specifications/pyproject-toml/#history
• https://packaging.python.org/en/latest/guides/modernize-setup-py-project/
• https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#pyproject-toml
• https://peps.python.org/pep-0660/
• https://peps.python.org/pep-0621/
• https://peps.python.org/pep-0518/

I feel that this change would modernize, simplify (and centralize) the dependency management of volatility 3. To illustrate: if you look at the diff of this pull request you can see that three different requirement definitions of pefile are currently used in the project in three different locations.

[ikelos]

Just asking for @npetroni 's perspective as the installation guru to check he's happy with all the changes put forward. As long as we're happy pyproject.toml is a suitable way to go and we're not gonna leave anyone out in the cold, I'm ok with it. 5:)

[ikelos]

Feels like this should be part of full, shouldn't it? I can't see a reason for selecting full and then not getting cloud as well?

[ikelos]

@npetroni said he's happy. Just wondering about dealing with the separate cloud section before this gets merged?