[Question] Bundle vs Dependency JDK in OpenSearch RPM

[peterzhuamazon]

There are several different benefits of bundling a jdk inside RPM vs treat jdk as dependency during installation time.

• Bundle:

  • Pros:
    • No need to configure JDK env, opensearch will use the bundled jdk as default
  • Cons:
    • Problem is, every time when you upgrade/downgrade opensearch, jdk is upgraded alongside the opensearch
    • Many companies likes to use a specific version of JDK themselves

• Dependency:

  • Pros:
    • When upgrade opensearch, JDK will stay unless there are changes in new version dependency
  • Cons:
    • During installation time, if our repo does not contain that jdk (corretto? adoptopenjdk/Adoptium?), installation fail, unless user manually install the specific version we demand, or they add the required repo to their yum.repo.d folder
    • User needs to define OPENSEARCH_JAVA_HOME to change to other jdk and rpm will need to hardcode define it by default to ensure OpenSearch can start
    • This seems not too much different compares to bundled jdk, as it not only complicated things, but also requires manual setups from users if they dont like the default version we bundled.

I want to see if there is any more thoughts on this from you guys.

PS:
Additional Question: Should we include corretto jdk in RPM or use adoptopenjdk/Adoptium like in tarball? If latter, can we directly have their .rpm pkg in our yum repo when released?

Thanks.

@dblock @spotaws

[peterzhuamazon]

Related to opensearch-project/OpenSearch#2302.

[dblock]

I prefer a non-bundled JDK in RPM, but I would only take a dependency on the same version as tested in the suite. See opensearch-project/project-website#670 for a breakdown across versions (note the actual list is in flux).

[spotrh]

Strong preference towards not bundling and locking to a specific jdk version. You should also provide corretto packages in your repo to meet the dependency.

Additionally, OPENSEARCH_JAVA_HOME can be set in the systemd config, which is how the majority of users consuming opensearch via RPM will use it (it is also where they can override it to use a different JDK).

I disagree with your assertion that this is "not too much different compares to bundled jdk". It dramatically decreases the size of the opensearch RPM (which is what will be updating in the majority of transactions). We have Corretto packages at our disposal which are already supported by AWS (and prebuilt) to ensure a running JDK is present at all times and a simple mechanism (1 line change) to have the service use a different JDK.

[peterzhuamazon]

Strong preference towards not bundling and locking to a specific jdk version. You should also provide corretto packages in your repo to meet the dependency.

Additionally, OPENSEARCH_JAVA_HOME can be set in the systemd config, which is how the majority of users consuming opensearch via RPM will use it (it is also where they can override it to use a different JDK).

I disagree with your assertion that this is "not too much different compares to bundled jdk". It dramatically decreases the size of the opensearch RPM (which is what will be updating in the majority of transactions). We have Corretto packages at our disposal which are already supported by AWS (and prebuilt) to ensure a running JDK is present at all times and a simple mechanism (1 line change) to have the service use a different JDK.

Hi @spotrh my question is what about adopt/adoptium jdk distro? As we already bundle them in tarball. Using corretto is nice from the rpm generation perspective and repo maintenance, but it is not the version opensearch core and other component owners have tested with. Can we take an existing adopt* jdk rpm and put it in our repo for distribution? Or we shall also include their repo link and ask user to install them alongside our repo?

Thanks.

[dblock]

@spotrh Maybe the coretto vs. adoptium is a separate discussion? Shall we open an issue and also benchmark it as part of opensearch-project/OpenSearch#1276?

[peterzhuamazon]

Strong preference towards not bundling and locking to a specific jdk version. You should also provide corretto packages in your repo to meet the dependency.

Additionally, OPENSEARCH_JAVA_HOME can be set in the systemd config, which is how the majority of users consuming opensearch via RPM will use it (it is also where they can override it to use a different JDK).

I disagree with your assertion that this is "not too much different compares to bundled jdk". It dramatically decreases the size of the opensearch RPM (which is what will be updating in the majority of transactions). We have Corretto packages at our disposal which are already supported by AWS (and prebuilt) to ensure a running JDK is present at all times and a simple mechanism (1 line change) to have the service use a different JDK.

Also, similar to JDK, we have discussed nodejs10 last week, which seems pretty hard to do the same as JDK, which means either bundle nodejs, or ask use to install nvm themselves.

Would be quite interesting to see JDK is a dependency but nodejs is not.

[spotrh]

@spotrh Maybe the coretto vs. adoptium is a separate discussion? Shall we open an issue and also benchmark it as part of opensearch-project/OpenSearch#1276?

I think the choice of default JDK dependency is a separate discussion, however, I happen to know that Corretto is not patched and should be identical to OpenJDK. One big benefit of Corretto is that it is built by AWS (and distributed by AWS already), so there is no real additional risk to AWS/OpenSearch distributing it. Any other JDK does not provide that measure of trust and for AWS/OpenSearch to distribute it involves additional risk (unless we audit it and build it ourselves).

But yes, separate discussion. :D

[spotrh]

Also, similar to JDK, we have discussed nodejs10 last week, which seems pretty hard to do the same as JDK, which means either bundle nodejs, or ask use to install nvm themselves.

Would be quite interesting to see JDK is a dependency but nodejs is not.

As discussed, Java is architected to permit multiple copies (and multiple versions) of the JDK to reside simultaneously on a system without conflict, and it is packaged in RPM in a way to permit this. You can have the Corretto RPM installed next to the OpenJDK without conflict. This is not the case for nodejs, which is not to say that it is impossible to package up nodejs 10 in such a way that it is used by OpenSearch but does not conflict with the system nodejs (which is almost certainly much much newer), but the work involved is notable (especially if you are making a proper built from source nodejs10 package).

[dblock]

I opened opensearch-project/OpenSearch#2312 to track the proposal to switch for Coretto. Please add your comments/pros/cons.

[peterzhuamazon]

Having a bundled JDK would also allow user to directly download an rpm package, and install, without the need to setup repos on their host.

This is particularly useful and less complicated if we want to use AdoptOpenJDK.

I welcome more conversation on this and will approach with bundled JDK for now.

[bbarani]

Closing this issue as we have released the RPM with bundled JDK. Feel free to re-open if required.