Skip to content

Commit 4eb1738

Browse files
authored
#351: Schema validator updates file with invalid output when behind VPN (#357)
* #351: replace JSON_data_files_validator.py when downloading only if content-type is text/plain to resolve issue * #351: remove previous code used for testing the content type check in schema validator * #351: Fix logger default log level * #351: refactor encapsulate configuration load and schema validator load in a Loader class * #351: remove url returning html for SchemaValidator used for testing * #351: update help message for the `config` argument in LBAF_app * #351: fix bad SchemaValidator url introduced for testing at some step * #351: fix bug with configuration loader * #351: update and add code comments * #351: fix pylint line too long * #351: refactor and ignore pylint logging-fstring-interpolation * #351: fix PR checks * #351: readme badge bad url and requirements add project dir for dev * #351: try enable setup for python 3.9 * #351: restore requirements.txt files and move path logic to common functions * #351: refactor move schema loader logic to a new module and remove some unused functions * #351: fix download path for shema validator * #351: update gh checkout plugin version in acceptance tests job * #351: use actions/checkout@v3 in all workflows * #351: remove duplicated script * #351: fix quotes and pylint
1 parent 3720ef2 commit 4eb1738

16 files changed

+239
-221
lines changed

.github/workflows/acceptance-tests.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ jobs:
2222
test_output: /__w/LB-analysis-framework/LB-analysis-framework/output/imbalance.txt
2323

2424
steps:
25-
- uses: actions/checkout@v2
25+
- uses: actions/checkout@v3
2626

2727
- name: Install packages
2828
run: |
@@ -72,7 +72,7 @@ jobs:
7272
log_output: /__w/LB-analysis-framework/LB-analysis-framework/output/log.txt
7373

7474
steps:
75-
- uses: actions/checkout@v2
75+
- uses: actions/checkout@v3
7676

7777
- name: Install packages
7878
run: |

.github/workflows/action-git-diff-check.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ jobs:
77
name: Run git check
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v2
10+
- uses: actions/checkout@v3
1111
with:
1212
fetch-depth: 0
1313
ref: ${{ github.event.pull_request.head.sha }}

.github/workflows/build-and-publish-to-pypi.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ jobs:
55
name: Build and publish Python Package to PyPi
66
runs-on: ubuntu-latest
77
steps:
8-
- uses: actions/checkout@master
8+
- uses: actions/checkout@v3
99
- name: Set up Python 3.8
1010
uses: actions/setup-python@v4
1111
with:

.github/workflows/check-commit-format.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ jobs:
77
name: Check commit message format
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v2
10+
- uses: actions/checkout@v3
1111
with:
1212
fetch-depth: 0
1313
ref: ${{ github.event.pull_request.head.sha }}

.github/workflows/find-trailing-whitespace.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ jobs:
77
name: Find Trailing Whitespace
88
runs-on: ubuntu-latest
99
steps:
10-
- uses: actions/checkout@v2
10+
- uses: actions/checkout@v3
1111
- uses: DARMA-tasking/find-trailing-whitespace@master
1212
with:
1313
exclude: "doc" # ; separated path to exclude

.pylintrc

+4-2
Original file line numberDiff line numberDiff line change
@@ -85,13 +85,15 @@ disable=raw-checker-failed,
8585
suppressed-message,
8686
useless-suppression,
8787
deprecated-pragma,
88-
use-symbolic-message-instead
88+
use-symbolic-message-instead,
89+
logging-fstring-interpolation
8990

9091
# Enable the message, report, category or checker with the given id(s). You can
9192
# either give multiple identifier separated by comma (,) or put this option
9293
# multiple time (only on the command line, not in the configuration file where
9394
# it should appear only once). See also the "--disable" option for examples.
94-
enable=c-extension-no-member,C0303
95+
enable=c-extension-no-member,
96+
trailing-whitespace
9597

9698

9799
[REPORTS]

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
[![Unit Tests](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/unit-tests.yml)
1+
[![Unit Tests](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/code-quality.yml/badge.svg)](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/code-quality.yml)
22
[![Acceptance Tests](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/acceptance-tests.yml/badge.svg)](https://github.com/DARMA-tasking/LB-analysis-framework/actions/workflows/acceptance-tests.yml)
33
[![Pylint](https://raw.githubusercontent.com/DARMA-tasking/LB-analysis-framework/deploy-badges/pylint.svg)](https://raw.githubusercontent.com/DARMA-tasking/LB-analysis-framework/deploy-badges/pylint.svg)
44
[![Coverage](https://raw.githubusercontent.com/DARMA-tasking/LB-analysis-framework/deploy-badges/coverage.svg)](https://raw.githubusercontent.com/DARMA-tasking/LB-analysis-framework/deploy-badges/coverage.svg)

scripts/upgrade_configuration.py

+22-22
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
try:
77
project_dir = f"{os.sep}".join(os.path.abspath(__file__).split(os.sep)[:-2])
8-
src_dir = f"{os.sep}".join([project_dir, 'src'])
8+
src_dir = f"{os.sep}".join([project_dir, "src"])
99
sys.path.append(src_dir)
1010
except Exception as ex:
1111
print(f"Can not add project path to system path! Exiting!\nERROR: {ex}")
@@ -19,37 +19,37 @@
1919
# get and validate args
2020
parser = argparse.ArgumentParser()
2121
default_pattern = [
22-
'./config/**/*[.yml][.yaml]',
23-
'./tests/config/**/*[.yml][.yaml]'
22+
"./config/**/*[.yml][.yaml]",
23+
"./tests/config/**/*[.yml][.yaml]"
2424
]
25-
parser.add_argument('-a', '--add', type=str, default=None,
26-
help='Key name (tree dot notation) to add')
27-
parser.add_argument('-r', '--remove', type=str, default=None,
28-
help='The key name (tree dot notation) to remove. e.g. `work_model.paramters.foo`. ' +
29-
'Required for a remove operation.')
30-
parser.add_argument('-v', '--value', type=str, default=None,
31-
help='The initial value for a key to add. e.g. `42`. Required for a add operation.')
32-
parser.add_argument('-t', '--type', type=str, default='str',
33-
help='Optional. The type of the initial value to set for a new key. Ex. `int`. Default `str`')
34-
parser.add_argument('-p', '--pattern', nargs='+', type=str, default=default_pattern,
35-
help='The list of patterns indicating which configuration files reside (path must be defined ' +
36-
'as relative to the project directory). Defaults `' + str.join(' ', default_pattern) + '`'
25+
parser.add_argument("-a", "--add", type=str, default=None,
26+
help="Key name (tree dot notation) to add")
27+
parser.add_argument("-r", "--remove", type=str, default=None,
28+
help="The key name (tree dot notation) to remove. e.g. `work_model.paramters.foo`. " +
29+
"Required for a remove operation.")
30+
parser.add_argument("-v", "--value", type=str, default=None,
31+
help="The initial value for a key to add. e.g. `42`. Required for a add operation.")
32+
parser.add_argument("-t", "--type", type=str, default="str",
33+
help="Optional. The type of the initial value to set for a new key. Ex. `int`. Default `str`")
34+
parser.add_argument("-p", "--pattern", nargs="+", type=str, default=default_pattern,
35+
help="The list of patterns indicating which configuration files reside (path must be defined " +
36+
"as relative to the project directory). Defaults `" + str.join(' ', default_pattern) + '`'
3737
)
3838
args = parser.parse_args()
3939

4040
if not args.add and not args.remove:
41-
raise ValueError('Missing either add (-a or --add xxx) or remove arg (-r or --remove xxx)')
41+
raise ValueError("Missing either add (-a or --add xxx) or remove arg (-r or --remove xxx)")
4242
if args.add and args.remove:
43-
raise ValueError('Cannot set both add and remove args')
43+
raise ValueError("Cannot set both add and remove args")
4444
if args.add and not args.value:
45-
raise ValueError('Missing value (-v or --vvv xxx)')
45+
raise ValueError("Missing value (-v or --vvv xxx)")
4646

4747
# init logger
48-
upg_logger = get_logger('upgrade', level='debug')
49-
upg_logger.info('Script to bulk add or remove key to/from LBAF configuration files within the project')
48+
upg_logger = get_logger("upgrade", level="debug")
49+
upg_logger.info("Script to bulk add or remove key to/from LBAF configuration files within the project")
5050
upg_logger.info(
51-
'NOTICE: Remember that the keys must be defined first at the schema level defined in ' \
52-
'the ConfigurationValidator class'
51+
"NOTICE: Remember that the keys must be defined first at the schema level defined in " \
52+
"the ConfigurationValidator class"
5353
)
5454

5555
upgrader = ConfigurationUpgrader(logger=upg_logger)

0 commit comments

Comments
 (0)