Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds property validation for instance.volumes property #5365

Merged
merged 4 commits into from
Mar 1, 2025

Conversation

keith-turner
Copy link
Contributor

No description provided.

@keith-turner keith-turner added this to the 3.1.0 milestone Feb 27, 2025
@keith-turner
Copy link
Contributor Author

keith-turner commented Feb 27, 2025

This is an initial stab at making the property validation more strict for an important property that currently accepts any string. There are some other properties that could be made more strict, but would probably require introducing more single use property types. This change introduced PropertyType.VOLUMES that is only used by a single property.

@keith-turner
Copy link
Contributor Author

The test are failing because the default value for instance.volumes is empty string which is not a valid value for this property. The property has no valid default but must be set to something that is valid at runtime.

Copy link
Member

@ctubbsii ctubbsii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code could be cleaned up by putting the validation in PropertyType.java (see the Predicate inner classes in there for reference).

By the time we get to the VolumeManager, the property should be guaranteed to be valid, so the code that redundantly checks the validation there could be simplified.

@ctubbsii
Copy link
Member

I'm not sure how best to express the "empty is valid if only empty in the DefaultConfiguration, but not anywhere else" criteria that the tests are failing on. Typically, we have validators for the content of the property, based on the property type, and that's independent from the validation of the overall config, which might check several properties, or that some properties are set that are required, etc. So, for the PropertyType validator, it should probably just check that it is of the proper form or that it is empty (make empty valid) and then check that it's non-empty elsewhere.

@kevinrr888
Copy link
Member

Related thread:
#5348 (comment)

@ctubbsii
Copy link
Member

I think this code could be cleaned up by ...

Okay, after looking more closely, I can see now why it was being done this way... because it's relying on some of the private methods for normalization of URIs to do some of the validation. This is trickier than I first thought.

@keith-turner
Copy link
Contributor Author

keith-turner commented Feb 28, 2025

Okay, after looking more closely, I can see now why it was being done this way... because it's relying on some of the private methods for normalization of URIs to do some of the validation. This is trickier than I first thought.

Another reason I structured the code the way it is was to avoid making assumptions about exceptions thrown in another class. The code calls some private methods in the same class that throw a specific exception that is catches.

I have not figured out a good way to do it, but would like to somehow accomplish the following goals.

  • Be able to validate volumes independently of creating a server process if possible. Like validate config in a stand alone utility.
  • Support the concept that there is no default value but to be valid config for a system it must be set to a list of one ore more URIs.
  • If possible have the validation code reside in Property and PropertyType instead of off at some random place in the code.

@ctubbsii
Copy link
Member

I'm about to add a commit to this PR which I think helps some of these things.

  • Be able to validate volumes independently of creating a server process if possible. Like validate config in a stand alone utility.

I put the parsing code for volumes into ConfigurationTypeHelper and added a unit test to help support anything that needs to parse it, either for use or for validation.

  • Support the concept that there is no default value but to be valid config for a system it must be set to a list of one ore more URIs.

This one is tricky, because the PropertyType validation is tied to the Property enum which is so tightly coupled to our AccumuloConfiguration (including DefaultConfiguration). We have a separate ConfigCheckUtil, and maybe other places in the code that maybe could do validation of the configuration as a whole, like ensuring a property is set because it's a required property. But the PropertyType validation is really supposed to validate the format, if set. Making it required to be set is going to need to be done elsewhere, if it's not already done.

  • If possible have the validation code reside in Property and PropertyType instead of off at some random place in the code.

I moved the validation to PropertyType

* Move parsing code for volume properties to ConfigurationTypeHelper and
  add a unit test
* Make VolumeManager do an additional check to require the property to
  be set (could also be done elsewhere in other overall configuration
  checks)
* Added a validator to the PropertyType to verify the format when it is
  set
* Slightly modified some of the exception messages, so that it does not
  mention the instance.volumes property itself... since the PropertyType
  validation could apply to other properties of the same type

(Added other minor changes to log errors at the error level)
@ctubbsii ctubbsii marked this pull request as ready for review February 28, 2025 22:12
@keith-turner
Copy link
Contributor Author

keith-turner commented Mar 1, 2025

@ctubbsii these changes look good. I added one more test because the code in ConfigurationTypeHelper and VolumeConfigugration work together to handle some of the empty cases. The added test just covers this case.

@keith-turner keith-turner merged commit 46c4c37 into apache:3.1 Mar 1, 2025
8 checks passed
@keith-turner keith-turner deleted the validate-vols branch March 1, 2025 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants