Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] [GHA] Transfer Linux and Windows workflows from Azure to GitHub Actions #904

Merged
merged 21 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 176 additions & 0 deletions .github/workflows/linux.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
name: Linux (Ubuntu 20.04, Python 3.11)
on:
workflow_dispatch:
pull_request:
paths-ignore:
- 'modules/nvidia_plugin'
- 'modules/openvino_code'
push:
branches:
- master
- 'releases/**'
paths-ignore:
- 'modules/nvidia_plugin'
- 'modules/openvino_code'

concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-linux
cancel-in-progress: true

env:
PYTHON_VERSION: '3.11'

jobs:
Build_and_test:
name: Build and Test
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: ubuntu-20.04-16-cores
container:
image: ubuntu:20.04
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_BUILD_TYPE: 'Release'
CMAKE_CXX_COMPILER_LAUNCHER: ccache
CMAKE_C_COMPILER_LAUNCHER: ccache
GITHUB_WORKSPACE: /__w/openvino_contrib/openvino_contrib
OPENVINO_REPO: /__w/openvino_contrib/openvino_contrib/openvino
OPENVINO_CONTRIB_REPO: /__w/openvino_contrib/openvino_contrib/openvino_contrib
TEST_DATA: /__w/openvino_contrib/openvino_contrib/testdata
INSTALL_DIR: /__w/openvino_contrib/openvino_contrib/openvino_install
BUILD_DIR: /__w/openvino_contrib/openvino_contrib/openvino_build
GRADLE_VER: '7.1.1'

steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' > /etc/apt/apt.conf.d/80-retries

- name: Install git
run: |
apt-get update
apt-get install --assume-yes --no-install-recommends git git-lfs ca-certificates

- name: Clone OpenVINO
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino'
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
ref: 'master'

- name: Clone OpenVINO Contrib
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'

- name: Clone Testdata
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/testdata'
path: ${{ env.TEST_DATA }}
lfs: 'true'
submodules: 'true'

#
# Dependencies
#

- name: Install build dependencies
run: |
bash ${OPENVINO_REPO}/install_build_dependencies.sh
# default-jdk - Java API; unzip for gradle installation
apt install --assume-yes --no-install-recommends default-jdk libopencv-dev unzip

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v3
with:
gradle-version: ${{ env.GRADLE_VER }}

- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}

- name: Install python dependencies
run: python3 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt

- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
max-size: "2000M"
# Should save cache only if run in the master branch of the base repo
# github.ref_name is 'ref/PR_#' in case of the PR, and 'branch_name' when executed on push
save: ${{ github.ref_name == 'master' && 'true' || 'false' }}
verbose: 2
key: linux-ubuntu
restore-keys: |
linux-ubuntu

#
# Build
#

- name: CMake configure - OpenVINO
run: |
cmake \
-GNinja \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} \
-DBUILD_nvidia_plugin=OFF \
-DENABLE_INTEL_GPU=OFF \
-DENABLE_OV_TF_FRONTEND=OFF \
-DENABLE_OV_PADDLE_FRONTEND=OFF \
-DENABLE_OV_TF_LITE_FRONTEND=OFF \
-DENABLE_OV_PYTORCH_FRONTEND=OFF \
-DOPENVINO_EXTRA_MODULES=${OPENVINO_CONTRIB_REPO}/modules \
-DENABLE_PYTHON=ON \
-DENABLE_WHEEL=ON \
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}

- name: Clean ccache stats
run: ccache --zero-stats --show-config

- name: Cmake build
run: cmake --build ${BUILD_DIR} --parallel

- name: Show ccache stats
run: ccache --show-stats

- name: Cmake install
run: cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -P ${BUILD_DIR}/cmake_install.cmake

- name: Java tests
working-directory: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/java_api
run: |
source ${INSTALL_DIR}/setupvars.sh
gradle clean build --info

for d in CPU HETERO:CPU; do
gradle test -Prun_tests -DMODELS_PATH=${TEST_DATA} -Ddevice=$d --info;
done

- name: Install requirements for custom operations tests
run: |
python3 -m pip install -r ${OPENVINO_CONTRIB_REPO}/modules/custom_operations/tests/requirements.txt
python3 -m pip install ${INSTALL_DIR}/tools/openvino-*.whl

- name: Custom user operation tests
working-directory: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/custom_operations
run: python3 -m pytest -k "not sparse_conv" tests/run_tests.py
env:
CUSTOM_OP_LIB: ${{ env.OPENVINO_REPO }}/bin/intel64/${{ env.CMAKE_BUILD_TYPE }}/libuser_ov_extensions.so

- name: Upload Test Results
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: test-results-java
path: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/java_api/build/test-results
if-no-files-found: 'warn'
193 changes: 193 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
name: Windows (VS 2019, Python 3.11)
on:
workflow_dispatch:
pull_request:
merge_group:
push:
branches:
- master
- 'releases/**'

concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-windows
cancel-in-progress: true

env:
PYTHON_VERSION: '3.11'

jobs:
Build_and_test:
name: Build and Test
timeout-minutes: 150
defaults:
run:
shell: pwsh
runs-on: windows-2019-16-core
env:
CMAKE_BUILD_TYPE: 'Release'
CMAKE_CXX_COMPILER_LAUNCHER: ccache
CMAKE_C_COMPILER_LAUNCHER: ccache
OPENVINO_REPO: ${{ github.workspace }}\\openvino
OPENVINO_CONTRIB_REPO: ${{ github.workspace }}\\openvino_contrib
TEST_DATA: ${{ github.workspace }}\\testdata
INSTALL_DIR: ${{ github.workspace }}\\openvino\\install
BUILD_DIR: ${{ github.workspace }}\\openvino\\build
BUILD_DIR_CONTRIB: ${{ github.workspace }}\\openvino\\build_contrib
GRADLE_VER: '7.1.1'

steps:
- name: git configuration
run: git config --system core.longpaths true

- name: Clone OpenVINO
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino'
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
ref: 'master'

- name: Clone OpenVINO Contrib
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_CONTRIB_REPO }}
submodules: 'true'

- name: Clone Testdata
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/testdata'
path: ${{ env.TEST_DATA }}
lfs: 'true'
submodules: 'true'

#
# Dependencies
#

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v3
with:
gradle-version: ${{ env.GRADLE_VER }}

- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'

- name: Install python dependencies
run: |
# For Python API: build and wheel packaging
python3 -m pip install -r ${env:OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt

- name: Install build dependencies
run: |
Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.11.1/ninja-win.zip -OutFile ninja-win.zip -MaximumRetryCount 10
Expand-Archive -Force ninja-win.zip
# Add it to the GitHub Path so it would be available in the subsequent steps
Add-Content -Path $env:GITHUB_PATH -Value "${{ github.workspace }}/ninja-win"
choco install opencv -y

#
# Build
#

- name: Configure Developer Command Prompt for Microsoft Visual C++
uses: ilammy/msvc-dev-cmd@v1

- name: Setup ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
max-size: "2000M"
# Should save cache only if run in the master branch of the base repo
# github.ref_name is 'ref/PR_#' in case of the PR, and 'branch_name' when executed on push
save: ${{ github.ref_name == 'master' && 'true' || 'false' }}
verbose: 2
key: ccache-windows
restore-keys: |
ccache-windows

- name: CMake configure - OpenVINO
run: |
$env:OpenCV_DIR="C:/tools/opencv/build"
& "C:\tools\opencv\build\setup_vars_opencv4.cmd"
cmake -GNinja `
-DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} `
-DBUILD_nvidia_plugin=OFF `
-DENABLE_OV_TF_FRONTEND=OFF `
-DENABLE_OV_PADDLE_FRONTEND=OFF `
-DENABLE_OV_TF_LITE_FRONTEND=OFF `
-DENABLE_OV_PYTORCH_FRONTEND=OFF `
-DENABLE_INTEL_GPU=OFF `
-DENABLE_CPPLINT=OFF `
-DENABLE_SAMPLES=OFF `
-DENABLE_PYTHON=ON `
-DENABLE_JS=OFF `
-DOPENVINO_EXTRA_MODULES=${{ env.OPENVINO_CONTRIB_REPO }}/modules `
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} `
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} `
-S ${{ env.OPENVINO_REPO }} `
-B ${{ env.BUILD_DIR }}

- name: Clean ccache stats
run: ccache --zero-stats --show-config

- name: Cmake build - OpenVINO
run: cmake --build ${{ env.BUILD_DIR }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} --verbose

- name: Show ccache stats
run: ccache --show-stats

- name: Cmake install - OpenVINO
run: cmake -DCMAKE_INSTALL_PREFIX=${{ env.INSTALL_DIR }} -P ${{ env.BUILD_DIR }}/cmake_install.cmake

- name: CMake configure - OpenVINO Contrib
run: |
$env:OpenCV_DIR="C:/tools/opencv/build"
. "${{ env.INSTALL_DIR }}/setupvars.ps1"
cmake -GNinja `
-DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} `
-S ${{ env.OPENVINO_CONTRIB_REPO }}/modules/custom_operations `
-B ${{ env.BUILD_DIR_CONTRIB }}

- name: Cmake build - OpenVINO Contrib
run: cmake --build ${{ env.BUILD_DIR_CONTRIB }} --parallel --config ${{ env.CMAKE_BUILD_TYPE }} --verbose

- name: Java tests
working-directory: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/java_api
run: |
. "${{ env.INSTALL_DIR }}/setupvars.ps1"
gradle clean build --info
gradle test -Prun_tests -DMODELS_PATH=${{ env.TEST_DATA }} -Ddevice=CPU --info

- name: Custom user operation tests (CMD)
working-directory: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/custom_operations
shell: cmd
run: |
call C:\tools\opencv\build\setup_vars_opencv4.cmd
call ${{ env.INSTALL_DIR }}\setupvars.bat
python3 -m pip install -r ${{ env.OPENVINO_CONTRIB_REPO }}\modules\custom_operations\tests\requirements.txt

python3 -m pytest -k "not sparse_conv" tests\run_tests.py
env:
CUSTOM_OP_LIB: ${{ env.OPENVINO_REPO }}/bin/intel64/${{ env.CMAKE_BUILD_TYPE }}/user_ov_extensions.dll

# - name: Custom user operation tests
# working-directory: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/custom_operations
# run: |
# . "${{ env.INSTALL_DIR }}/setupvars.ps1"
# python3 -m pip install -r ${{ env.OPENVINO_CONTRIB_REPO }}/modules/custom_operations/tests/requirements.txt
#
# python3 -m pytest -k "not sparse_conv" tests/run_tests.py
# env:
# CUSTOM_OP_LIB: ${{ env.OPENVINO_REPO }}/bin/intel64/${{ env.CMAKE_BUILD_TYPE }}/user_ov_extensions.dll

- name: Upload Test Results
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: test-results-java
path: ${{ env.OPENVINO_CONTRIB_REPO }}/modules/java_api/build/test-results
if-no-files-found: 'warn'
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ public void testWrongLayout() {
+ " input"));
}

@Ignore // Ticket: 138994
@Test
public void testWrongElementType() {
String exceptionMessage = "";
Expand Down
Loading