Skip to content

Commit 34984d8

Browse files
committed
Updated GELU Review changes
[Specification] MaxPool-14 and AvgPool-14 - new ceiling mode `CEIL_TORCH` (openvinotoolkit#22930) - Add specification for `MaxPool-14` and `AvgPool-14` - They both introduce a new ceil mode: `ov::op::RoundingType::CEIL_TORCH` - The new ceiling mode does not allow the last pooling in a Dimension to start in the padding area - [Reference and Core](openvinotoolkit#22796) - [Python API](openvinotoolkit#22966) - [PT FE](openvinotoolkit#23027) - [Downgrade transformations](openvinotoolkit#23381) - 131961 openvinotoolkit#18731 --------- Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com> Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com> [TF FE] Support ApproximateEqual operation for TensorFlow (openvinotoolkit#23351) - *Adding operation support for ApproximateEqual operation* - *Addresses issue openvinotoolkit#22082 * --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> [OV JS] Expose export_model()/import_model() (openvinotoolkit#23366) - Expose `compiledModel::export_model()`, a method to export a compiled model to the binary data stream. - Expose `core::import_model(model_file : Buffer, device_name : str)`, a method to import a compiled model from a previously exported one. - *134820* *134818* --------- Co-authored-by: Vishniakov Nikolai <nikolai.vishniakov@intel.com> [core] Low precision element iterator and `u2, u3, u6` types (openvinotoolkit#23279) - Introduce new low precision types `u2`, `u3`, `u6`. - Introduce `ov::element::Iterator` for low precision types like `u1, u2, u3, u4, i4, u6`: - Gives pointer like access to low precision values in Tensor, containers etc. - Can be used by STL algorithms to access data in unified algorithms for data manipulation. - Can be used in Constant, Convert operators to replace duplicate implementations for accessing low precision data (bin-size reduction). - Can be used for operator reference implementation or plugin if there is no hardware specific solution. - [CVS-126998](https://jira.devtools.intel.com/browse/CVS-126998) - Part of [CVS-128024](https://jira.devtools.intel.com/browse/CVS-128024) [DOCS] Updated file (openvinotoolkit#23509) - *item1* - *...* - *ticket-id* Add 'pad' operator support for ov::preprocess::PrePostProcessor (openvinotoolkit#23093) - Add 'pad' preprocessor operator - openvinotoolkit#23068 - [CVS-121548](https://jira.devtools.intel.com/browse/CVS-121548) [API][AUTO] Fail to get PERF_COUNT from compiled_model (openvinotoolkit#23123) - *Fail to get PERF_COUNT from compiled_model* - *CVS-130349* [GPU] Fix dynamic loop's not matched issue during multiple shapes are inferenced (openvinotoolkit#22806) - *Fix the issue which second infer with updated shape in dynamic loop doesn't update sliced layout.* - *Fix the issue that the optimized reshape doesn't reinterpret output memory in update_output_layout()* - *122739* - *131544* [DOCS] Add docs about ignored subgraphs (openvinotoolkit#23435) - Add documentation about `nncf.Subgraph` - 100999 [TRANSFORMATIONS] Fix Optional to match even with no inputs (openvinotoolkit#23471) [TRANSFORMATIONS] Fix Optional to match even with no inputs The Optional pattern type may create a wrong pattern to match if no inputs are provided to the Optional node. If no inputs present to the Optional type, it will not create an alternative branch(es) to check against resulting in the incorrect matching. Fix that by adding a check for the number of inputs being 0. Do a minor refactoring/renaming for the readability purposes. CSV-133523 Signed-off-by: Andrii Staikov <andrii.staikov@intel.com> --------- Signed-off-by: Andrii Staikov <andrii.staikov@intel.com> Enable Paddle FastSpeech2 model (openvinotoolkit#23311) - *Enable Paddle FastSpeech2 model* - *fix issue in 'set_value'* - *add 'round' op* - *CVS-134638* [Conformance Test] Fix cache test case failure for auto plugin (openvinotoolkit#23473) - check if the blob size remains the same as it was during the initial caching of the compiled model, rather than comparing it with a specified number, such as 1 in this case. - count the size of cached blobs after the model compilation is completed on all HW plugin within AUTO plugin. - CVS-130395 [GPU] Remove unused formats (openvinotoolkit#23431) + Most of them are in onednn weights format. - *119476* [CPU][ARM] Make f16 precision as default for CNN (openvinotoolkit#22839) Remove mentioning of compatibility folder in mac docs (openvinotoolkit#23542) - *item1* - *...* - *ticket-id* [TRANSFORMATIONS] Fix ReshapeAMatMul pattern to work with shared node as reshape input (openvinotoolkit#23535) - *`ReshapeAMatMul` worked incorrect in case of using shared nodes as reshape input* - *Fix: to reconnect reshape input to new `shape_of` pattern* - *[CVS-134625](https://jira.devtools.intel.com/browse/CVS-134625)* [TF FE] Support complex tensors for Reciprocal operations (openvinotoolkit#23355) - *Extended loader Reciprocal by propagating ComplexTypeMark from input to output and to represent output complex type tensor as a floating-point type tensor with an auxiliary dimension that concatenates real and imaginary parts of complex tensor.* - *Performed reciprocal for complex numbers.* - *Wrapped the complex result with ComplexTypeMark and returned the result* - openvinotoolkit#23234 --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> [GPU] Fix SIMD for non supporting platforms (openvinotoolkit#23540) - Check is simd 8 is supported - *[CVS-133769](https://jira.devtools.intel.com/browse/CVS-133769)* [PT FE] Fix typo and improve the error info. (openvinotoolkit#23507) - *Fix the typo of the code (then -> than)* - *Improve the error info here, to let developer know the size of output if the assertion fails.* - *No ticket id* [PT FE] Fix sporadic issue in quantized tests (openvinotoolkit#23520) - *Relax quantized tests condition to remove sporadicity.* - *CVS-129734* [GPU] Fixed not to set GATHER_AXIS_SHAPE_INFO_INDEX when input0 is static (openvinotoolkit#23548) - This PR fixes `Gather` not to set GATHER_AXIS_SHAPE_INFO_INDEX when input0 is static. - It enables some functional tests again. Add test for CoreImpl::get_versions() (openvinotoolkit#23336) Closes [23298](openvinotoolkit#23298) - [CVS-132140](https://jira.devtools.intel.com/browse/CVS-132140) --------- Co-authored-by: Oleg Pipikin <oleg.pipikin@intel.com> [PT FE] Add ModuleExtension (openvinotoolkit#23536) - *Continuation of openvinotoolkit#22867* - *CVS-133733* --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> [api conformance] Fix batch/hetero plugins config (openvinotoolkit#23547) - *item1* - *...* - *ticket-id* [Transformations] Added If operation to NMS path propagation for ignore negative indices in Gather (openvinotoolkit#23451) - *127874* [TF FE] Test TextVectorization on white-space string input and Equal on empty string tensor (openvinotoolkit#23572) **Details:** Test `tf.keras.TextVectorization` on white-space string input and Equal on empty string tensor. **Ticket:** 135749 --------- Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com> [PT FE] Make ModuleExtension patching in independent function scope (openvinotoolkit#23584) - *Make ModuleExtension patching in independent function scope* - *ticket-id* [GPU] Increase FC tile_b size for INT4 shape agnostic kernel (openvinotoolkit#23532) - Increased FC tile_B size for INT4 shape agnostic kernel for improving context processing - 133444 [GPU] Enable 8bit compression support on dGPU via oneDNN (openvinotoolkit#22740) - Enable 8bit compression support on dGPU via oneDNN - Update oneDNN version - Enable oneDNN primitives cache Ticket: 124115 [CPU] Add PagedAttention support (openvinotoolkit#23524) - *Support PagedAttention support, depends on:* - openvino_contrib: openvinotoolkit/openvino_contrib#867 - vLLM: ilya-lavrenov/vllm#4 - *TODO* - Models with alibi feature - *[134329](https://jira.devtools.intel.com/browse/CVS-134329)* - *[134327](https://jira.devtools.intel.com/browse/CVS-134327)* [GPU] In gemm_tile_kernel, applied to use block read when N and K byte-size is aligned 4. (openvinotoolkit#23400) - *Element by element read is the bottle-neck in gemm_tiled kernel. Enable block-read when N and K size are aligned 4byte with N and K are leftover*. - *Increasing tile_n_size has performance improvement when m_size and n_size are not shallow and n_size is aligned at 32.* - *Add GEMM_TILE_M/N/K/SIMD environment variables for convenience.* - *134279* --------- Signed-off-by: hyunback <hyunback.kim@intel.com> [CPU] [ARM64] jit eltwise: int8 support (openvinotoolkit#22687) - *int8 support* - *CVS-128643* [ONNX] Extended ReduceMax by opsets 13,18,20 (openvinotoolkit#23475) - Extended ReduceMax by opsets 13,18,20 - Updated a using opset for ONNX to 20 - Added tests for additional supported types - Enabled backend tests - Closes openvinotoolkit#20555 [CPU] Enable concat nspc layout inplace for urlnet model cases (openvinotoolkit#23454) - *enable concat nspc layout inplace for channel only cases, with these concat node use inplace impl, urlnet model gain performance benefits, and this(intermediate concat node is nspc layout but actually is one dimension) could be common case especially for models with 1D input* - *130282* [CPU]Fix GPT-J RoPE fusion (openvinotoolkit#23519) - *Support new RoPE pattern of GPT-J* - *Local test shows 17 % improvement for 2nd token latency for BF16 in `Intel(R) Xeon(R) Platinum 8468`* - *CVS-134949* Torch Compile - New Op Support (openvinotoolkit#23310) New op support for: - torch.export updates - benchmarking model support - chatglm2 support --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com> Co-authored-by: Maxim Vafin <maxim.vafin@intel.com> Co-authored-by: suryasidd <surya.siddharth.pemmaraju@intel.com> [DOCS] Latency highlight for OV devices + update of Optimize Inference for master (openvinotoolkit#23575) Jira: 133389 * Added an indication on Latency being the default use for OV devices * Streamlined the Optimize Inference article for better clarity. [TF FE] Support complex tensors for OnesLike operation (openvinotoolkit#23445) - *Adding support for OnesLike operation on complex type tensor* - Closes openvinotoolkit#22953 --------- Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com> Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> [CI] [GHA] Remove usage of the `SimenB/github-actions-cpu-cores` action (openvinotoolkit#23583) - The action does not have a License. - `cmake` should figure out the # of cores for parallel. [CPU] [ARM64] jit select (openvinotoolkit#23450) - *[CPU] [AARCH64] jit select* - *CVS-135445* New DB schema for GitHub metrics script (openvinotoolkit#23606) Improvements and fixes for the script which sends GitHub Workflow metrics to a database. See also: [23484](openvinotoolkit#23484) [JS API] Extract code from CompiledModel getters (openvinotoolkit#23515) - Extract the same logic structure from `CompileModel::input` and `CompileModel::output` - Add a private `CompileModel::get_node` method that gets the specified input or output node. Note: No changes to argument validation or conversion. - *127617* constraints openvino-dev: Limit mpmath<1.4 (openvinotoolkit#23601) - Limit mpmath because of pytorch/pytorch#120995 and sympy/sympy#26273 [GPU] Re-enable memory reuse for gemm (openvinotoolkit#23600) - Since openvinotoolkit#22726 gemm is derived from multi-stage impl which had memory reuse flag enforced to false for all sub-classes. - This patch enables memory reuse back for gemm kernel to reduce memory consumption. - *135361* [TF FE] Support TensorFlow 2.16 (openvinotoolkit#23562) **Details:** Support TensorFlow 2.16 **Ticket:** TBD --------- Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com> [IE TESTS][OP CONFORMANCE] Move `ConstRanges` range calculation to `InGenData` constructor (openvinotoolkit#23427) - *Move static const range initialization to `InData` structure* - *[125993](https://jira.devtools.intel.com/browse/CVS-125993)* Enable new property model_distribution_policy for CPU inference (openvinotoolkit#23077) - *Enable new property model_distribution_policy for CPU inference* -- *Add C++ interface and test cases* -- *Add Python interface and test cases* - *CVS-127844* [CPU] optimize PagedAttention's shape inference (openvinotoolkit#23603) - *Specific shape inference for PagedAttention* - *...* - *ticket-id* [CPU] [ARM64] jit equal (openvinotoolkit#23266) - *[CPU] [AARCH64] jit eltwise Equal - *CVS-134691* [GPU] Fix count non zero for empty input (openvinotoolkit#23597) - Adds buffer reset to 0 in `count_nonzero` impl in case of empty input tensor as currently we may try to allocate random amount of memory in subsequent `gather_nonzero` call [PyOV] Add Python API for MaxPool-14 and AvgPool-14 (openvinotoolkit#22966) - Extend Python API with`MaxPool-14` and `AvgPool-14` - They both introduce a new ceil mode: `ov::op::RoundingType::CEIL_TORCH` - The new ceiling mode does not allow the last pooling in a Dimension to start in the padding area - openvinotoolkit#22930 - openvinotoolkit#22796 - openvinotoolkit#23027 - openvinotoolkit#23381 - openvinotoolkit#23582 - 131961 openvinotoolkit#18731 --------- Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com> [Spec] Clarify specification for StridedSlice (openvinotoolkit#23039) - Add notes with descriptions of: Out of Bounds, Indexing in Reverse, Negative Indices - Clarified length of masks - Clarified the definition of `-1` value - Described in detail the behavior of masks, aligned with Reference Implementation - Added more latex-like style, add the examples for the missing masks. - 90128 [TRANSFORMATIONS] Remove use of legacy names from transformations (openvinotoolkit#23574) [TRANSFORMATIONS] Remove use of legacy names from transformations API function create_ie_output_name() and get_ie_output_name() are deprecated in a28a000 ("Deprecated functions to operate with legacy port names (openvinotoolkit#22717)") Remove usages of create_ie_output_name() in Transformations CVS-132087 Signed-off-by: Andrii Staikov andrii.staikov@intel.com --------- Signed-off-by: Andrii Staikov andrii.staikov@intel.com [Opset14][Spec] ConvertPromoteTypes-14 specification (openvinotoolkit#23264) - *This PR introduces specification for ConvertPromoteTypes-14 op - conversion op used to align two inputs to common type* - *Operator was introduced for PyTorch Frontend, rules also match Tensorflow https://www.tensorflow.org/guide/tf_numpy_type_promotion* - PR with core implementation: openvinotoolkit#22566 - Draft PR with improvements to core + replacement it PTFe: openvinotoolkit#22770 - *129197* --------- Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com> [CPU][ARM] Upgrade to ACL v24.02.1 (openvinotoolkit#22598) oneDNN PR: openvinotoolkit/oneDNN#227 [API CONFORMANCE] Modify API conformance suite for SW plugins (openvinotoolkit#23557) - *Move some properties from mandatory to optional for sw plugins* - *...* - *[133459](https://jira.devtools.intel.com/browse/CVS-133459)* Calculate model weights hash in parallel (openvinotoolkit#23605) - Calculate model weights hash in parallel in case of reading model from buffer - CVS-134771 [DOCS] improve legacy section formatting (openvinotoolkit#23512) [DOCS] ai legal disclaimer (openvinotoolkit#23587) [TRANSFORMATIONS] Create python binding for pattern::Optional (openvinotoolkit#23558) [TRANSFORMATIONS] Create python binding for pattern::Optional Expose the C++ op::pattern::Optional to Python in order to simplify patterns creation. Cover the functionality with the dedicated tests. CVS-133523 Signed-off-by: Andrii Staikov <andrii.staikov@intel.com> --------- Signed-off-by: Andrii Staikov <andrii.staikov@intel.com> [CPU] Fix SDPA pattern matching (openvinotoolkit#23581) Limit the Concat layer to have maximum 3 children. The third one is allowed to be a ShapeOf op only (to support Mixtral). - 135375 [chore] Use debug loglevel for github metrics script (openvinotoolkit#23633) We can switch log level for GitHub metrics script only when the workflow is restarted with debug logging [TF FE] Enable parallel execution of TensorFlow Layer 2 python tests (openvinotoolkit#23344) Addresses issue: openvinotoolkit#20919 - Enables parallel execution of TensorFlow Layer 2 python tests - Fixes test_tf2_keras_conv_lstm_2d.py and test_tf2_map_fn.py to not fail during parallel execution - Appends args in github workflow to enable parallel execution Errors fixed: - Due to varying Kera activation function addresses causing the workers to get different parameter inputs and thus failing. See [known issue](https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html#order-and-amount-of-test-must-be-consistent) ``` -tensorflow2_keras_tests/test_tf2_keras_conv_lstm_2d.py::TestKerasConvLSTM2D::test_keras_conv_lstm_2d_basic[ ie_device:CPU - precision:FP32 - params:{'params': {'filters': 4, 'kernel_size': (3, 3), 'padding': 'same', 'return_sequences': False, 'activation': <function swish at 0x7f1fadf364d0>}, 'input_shapes': [[2, 5, 20, 30, 2]]} ] -tensorflow2_keras_tests/test_tf2_keras_conv_lstm_2d.py::TestKerasConvLSTM2D::test_keras_conv_lstm_2d_basic[ ie_device:CPU - precision:FP32 - params:{'params': {'filters': 6, 'kernel_size': (2, 3), 'padding': 'valid', 'dilation_rate': 3, 'recurrent_activation': <function elu at 0x7f1fe6a1a830>, 'return_sequences': True, 'use_bias': True, 'data_format': 'channels_first'}, 'input_shapes': [[2, 5, 1, 40, 30]]} ] +tensorflow2_keras_tests/test_tf2_keras_conv_lstm_2d.py::TestKerasConvLSTM2D::test_keras_conv_lstm_2d_basic[ ie_device:CPU - precision:FP32 - params:{'params': {'filters': 4, 'kernel_size': (3, 3), 'padding': 'same', 'return_sequences': False, 'activation': <function swish at 0x7f635e4d24d0>}, 'input_shapes': [[2, 5, 20, 30, 2]]} ] +tensorflow2_keras_tests/test_tf2_keras_conv_lstm_2d.py::TestKerasConvLSTM2D::test_keras_conv_lstm_2d_basic[ ie_device:CPU - precision:FP32 - params:{'params': {'filters': 6, 'kernel_size': (2, 3), 'padding': 'valid', 'dilation_rate': 3, 'recurrent_activation': <function elu at 0x7f6396fa2830>, 'return_sequences': True, 'use_bias': True, 'data_format': 'channels_first'}, 'input_shapes': [[2, 5, 1, 40, 30]]} ] ``` - Due to lambda function definitions giving varying addresses as inputs ``` -tensorflow2_keras_tests/test_tf2_map_fn.py::TestMapFN::test_multiple_inputs_outputs_int32[ ie_device:CPU - precision:FP32 - params:{'fn': <function TestMapFN.<lambda> at 0x7f66c2c63c70>, 'input_type': tf.int32, 'fn_output_signature': (tf.int32, tf.int32, tf.int32), 'back_prop': True, 'input_names': ['x1', 'x2', 'x3'], 'input_shapes': [[2, 1, 3, 4], [2, 1, 3, 4], [2, 1, 3, 4]]} ] -tensorflow2_keras_tests/test_tf2_map_fn.py::TestMapFN::test_multiple_inputs_outputs_int32[ ie_device:CPU - precision:FP16 - params:{'fn': <function TestMapFN.<lambda> at 0x7f66c2c63c70>, 'input_type': tf.int32, 'fn_output_signature': (tf.int32, tf.int32, tf.int32), 'back_prop': True, 'input_names': ['x1', 'x2', 'x3'], 'input_shapes': [[2, 1, 3, 4], [2, 1, 3, 4], [2, 1, 3, 4]]} ] +tensorflow2_keras_tests/test_tf2_map_fn.py::TestMapFN::test_multiple_inputs_outputs_int32[ ie_device:CPU - precision:FP32 - params:{'fn': <function TestMapFN.<lambda> at 0x7f211b56fd00>, 'input_type': tf.int32, 'fn_output_signature': (tf.int32, tf.int32, tf.int32), 'back_prop': True, 'input_names': ['x1', 'x2', 'x3'], 'input_shapes': [[2, 1, 3, 4], [2, 1, 3, 4], [2, 1, 3, 4]]} ] +tensorflow2_keras_tests/test_tf2_map_fn.py::TestMapFN::test_multiple_inputs_outputs_int32[ ie_device:CPU - precision:FP16 - params:{'fn': <function TestMapFN.<lambda> at 0x7f211b56fd00>, 'input_type': tf.int32, 'fn_output_signature': (tf.int32, tf.int32, tf.int32), 'back_prop': True, 'input_names': ['x1', 'x2', 'x3'], 'input_shapes': [[2, 1, 3, 4], [2, 1, 3, 4], [2, 1, 3, 4]]} ] ``` --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> [ IE TESTS ] Update tensor comparation function according plugin requirments (openvinotoolkit#23226) - *Comparation function was changed to compare tensors based on element comparation* - *`std::abs(ref_value - plugin_value) <= abs_threshold + rel_threshold * ref_value`* - *`abs_threshold ` = std::max(std::numeric_limits::eps<plugin_element_type>(), std::numeric_limits::eps<ref_element_type>())* - *`ref_threshold = eps_by_expected_type()`, which is based on half `bit length of mantissa`* - [CVS-133173](https://jira.devtools.intel.com/browse/CVS-133173) - [CVS-135540](https://jira.devtools.intel.com/browse/CVS-135540) --------- Co-authored-by: sbalandi <sofya.balandina@intel.com> [TF FE] Support Angle operation for TensorFlow models (openvinotoolkit#23028) - *Support Angle operation for TensorFlow models* - Closes openvinotoolkit#22083 --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> [GPU] Extend gemm to fuse broadcast and reshape layers (openvinotoolkit#23513) - Fuse `broadcast` and `reshape` layers into `gemm` layer for LLM's 2nd latency optimization - before : [`broadcast`] --> [`reshape`] --> `gemm` - after : `gemm` - `gemm` is extended to have `input0_target_shape`, `input1_target_shape`, `input0_output_pattern` and `input1_output_pattern` from `broadcast` and `reshape` layers - 128343 --------- Signed-off-by: Andrew Park <andrew.park@intel.com> [GPU] Extend pattern for ClampFP16Output (openvinotoolkit#23592) - By PR(openvinotoolkit#22245), `clamp_fp16_output` opt pass was moved to ngraph - Because nodes such as eltwise(`Add`, `Subtract`, `Multiply`, `Divide`) that were fused into target node `gemm` are not supported in pattern, corresponding pattern was extended for this purpose - 135060 Fix the aten::mv for pytorch models openvinotoolkit#22073 (openvinotoolkit#22677) - *item1* - *...* Add aten::mv operator close openvinotoolkit#22073 - *ticket-id* --------- Co-authored-by: Ekaterina Aidova <ekaterina.aidova@intel.com> Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com> Remove NGraphFunctions namespace (openvinotoolkit#23627) - Remove NGraphFunctions namespace - CVS-133379 [PY API] Fix the preoblem that Node.get_attributes() cannot return all attributes (openvinotoolkit#23530) - extend the `util::DictAttributeSerializer::on_adapter()` method, making it compatible with `ov::PartialShape` and `ov::op::util::Variable` types; - add extra tests to test the correctness of `Node.get_attributes()` - openvinotoolkit#23455 --------- Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com> [CPU] Correct type configuration for i8 inner_product with f16 output (openvinotoolkit#23610) - 136298 - 136163 Support aten::bucketize for pytorch models openvinotoolkit#23328 (openvinotoolkit#23527) ](openvinotoolkit#23328) - Support aten::bucketize for pytorch models Move ConvertConvertPromoteTypes transformation from Common to MOC (openvinotoolkit#23630) Move ConvertConvertPromoteTypes transformation from Common to MOC N/A [CPU][ARM] Enable both f16 and f32 kernels for aarch64 and introduce runtime f16 support check (openvinotoolkit#22992) Inherited from openvinotoolkit#22437 --------- Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com> [ONNX] Reduced memory consumption while running tests (openvinotoolkit#23628) - Significantly reduced amount of using RAM while testing - May introduce test regression in multi-worker scenario (-n auto), but it isn't detected while validation - 129958 [TF FE] Add testing StringLower and TextVectorization operations on non-ASCII sentences (openvinotoolkit#23641) **Details:** Add testing non-ASCII sentences for StringLower operation. Needs to be merged after openvinotoolkit/openvino_tokenizers#80. **Ticket:** 135752 --------- Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com> Symbol Tracking API updated and made public (openvinotoolkit#23136) - dev_api `ov::DimensionTracker` and `ov::TableOfEquivalence` classes deleted, logic moved to `ov::Symbol` which is now stored by `ov::Dimension` - new implementation moves responsibility to store and report relations between Symbols directly to the Symbol object. Hence, there is no need for `ov::TableOfEquivalence` and no need for synchronization point anymore. - Equivalence is being tracked by using [Disjoint-set_data_structure](https://en.wikipedia.org/wiki/Disjoint-set_data_structure) which uses less memory than previous implementation. ![image](https://github.com/openvinotoolkit/openvino/assets/55839243/f1266f32-976d-44f9-a6ea-cd04dce07407) ![image](https://github.com/openvinotoolkit/openvino/assets/55839243/3108d1ad-0d30-4041-aa93-c4de1f1fb979) - *CVS-133123* Align friendly names uniqueization (openvinotoolkit#22729) Removed code that makes friendly names unique from Serialization and a name uniqueness check from Deserializator. Enabled the mode of ResolveNameCollisions transformation to uniqueize all friendly names, not only autogenerated in Frontends - *CVS-131567* --------- Co-authored-by: Evgenya Nugmanova <evgeniia.nugmanova@intel.com> Co-authored-by: Andrei Kochin <andrei.kochin@intel.com> [CPU][REFACTORING] Use memory access helper methods where possible (openvinotoolkit#23442) fix coverity issue 1540833 and 1540832 (openvinotoolkit#23635) - *fix coverity scan issue1540833 and issue1540832* - *ticket-id* [CPU] Prohibit fc avx2_vnni_2 decompression for bf16 input (openvinotoolkit#23638) - The FC changes made in scope of openvinotoolkit#20486 were missed when rebasing - The context is: Even the system and the node does support bf16 precision we have to fall back to f32 in/out precision due to lack of support for decompression with bf16 avx2_vnni_2 in oneDNN fork. - To cover this limitation an additional type mapping parameter in form of std::function was introduced for disabling particular type mapping entry using a runtime check (isa support in this case) - 122347 - 136163 Merged master changes Update src/frontends/tensorflow_common/src/op/gelu.cpp updated approximation access
1 parent f5087e5 commit 34984d8

File tree

565 files changed

+13178
-7141
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

565 files changed

+13178
-7141
lines changed

.github/scripts/collect_github_metrics.py

+62-31
Original file line numberDiff line numberDiff line change
@@ -6,37 +6,49 @@
66
import logging
77
import psycopg2
88
import dateutil
9+
import argparse
910

1011
def init_logger():
1112
LOGLEVEL = os.environ.get('LOGLEVEL', 'INFO').upper()
1213
logging.basicConfig(level=LOGLEVEL,
1314
format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
1415
datefmt='%m-%d-%Y %H:%M:%S')
1516

17+
def make_parser():
18+
parser = argparse.ArgumentParser()
19+
parser.add_argument('-r', '--repository-name', type=str, required=True,
20+
help='Repository name in OWNER/REPOSITORY format')
21+
parser.add_argument('--run-id', type=str, required=True,
22+
help='Workflow Run ID')
23+
24+
return parser
25+
1626
def create_db_tables(conn, cur):
17-
cur.execute('''CREATE TABLE IF NOT EXISTS github_workflow_runs_test(
18-
id SERIAL,
19-
run_id BIGINT PRIMARY KEY,
27+
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_runs(
28+
id SERIAL PRIMARY KEY,
29+
run_id BIGINT,
2030
html_url TEXT,
2131
name VARCHAR(255),
2232
run_started_at TIMESTAMP,
33+
created_at TIMESTAMP,
34+
updated_at TIMESTAMP,
2335
triggering_actor_login VARCHAR(255),
2436
conclusion VARCHAR(25),
25-
run_number INT,
2637
event VARCHAR(50),
2738
run_attempt INT,
2839
repository_full_name VARCHAR(255),
2940
head_repository_full_name VARCHAR(255),
3041
head_branch VARCHAR(255),
3142
status VARCHAR(25),
3243
display_title TEXT,
33-
path TEXT
44+
path TEXT,
45+
total_duration_seconds INT
3446
);
3547
''')
36-
cur.execute('''CREATE TABLE IF NOT EXISTS github_workflow_jobs_test(
37-
id SERIAL,
38-
job_id BIGINT PRIMARY KEY,
39-
parent_run_id BIGINT REFERENCES github_workflow_runs_test(run_id),
48+
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_jobs(
49+
id SERIAL PRIMARY KEY,
50+
job_id BIGINT,
51+
parent_run_id BIGINT,
4052
html_url TEXT,
4153
name VARCHAR(255),
4254
created_at TIMESTAMP,
@@ -47,12 +59,14 @@ def create_db_tables(conn, cur):
4759
runner_name VARCHAR(255),
4860
status VARCHAR(25),
4961
conclusion VARCHAR(25),
50-
head_branch VARCHAR(255)
62+
head_branch VARCHAR(255),
63+
run_attempt INT,
64+
workflow_name TEXT
5165
);
5266
''')
53-
cur.execute('''CREATE TABLE IF NOT EXISTS github_workflow_steps_test(
67+
cur.execute('''CREATE TABLE IF NOT EXISTS workflow_steps(
5468
id SERIAL PRIMARY KEY,
55-
parent_job_id BIGINT REFERENCES github_workflow_jobs_test(job_id),
69+
parent_job_id BIGINT,
5670
name VARCHAR(255),
5771
conclusion VARCHAR(25),
5872
number INT,
@@ -65,20 +79,16 @@ def create_db_tables(conn, cur):
6579

6680
def main():
6781
init_logger()
68-
82+
parser = make_parser()
83+
args = parser.parse_args()
6984
logger = logging.getLogger(__name__)
7085

7186
github_token = os.environ.get('GITHUB_TOKEN')
7287
if not github_token:
7388
raise ValueError('GITHUB_TOKEN environment variable is not set!')
7489

75-
run_id = os.environ.get('RUN_ID')
76-
if not run_id:
77-
raise ValueError('RUN_ID environment variable is not set!')
78-
79-
repo_name = os.environ.get('GITHUB_REPOSITORY')
80-
if not repo_name:
81-
raise ValueError('GITHUB_REPOSITORY environment variable is not set!')
90+
run_id = args.run_id
91+
repo_name = args.repository_name
8292

8393

8494
# this should be specified in runner's env
@@ -102,18 +112,31 @@ def main():
102112
repo = g.get_repo(repo_name)
103113

104114
run = repo.get_workflow_run(int(run_id))
105-
106-
workflow_data_query = f'''INSERT INTO github_workflow_runs_test(
115+
logger.debug('Processing run ID %s - %s', run_id, run.name)
116+
if run.status != 'completed':
117+
logger.error('Run %s is not completed! Only completed runs should be in the database', run_id)
118+
raise SystemExit(1)
119+
120+
# We rely on the following assumptions:
121+
# - The workflow run is completed. When run.status != 'completed' we should not add it to the database
122+
# theoretically the second attempt can be triggerred right after the completion of the first one
123+
# or while the runner which executes this script is deploying
124+
#
125+
# - Job's queued duration equals "job.started_at - job.created_at" if started_at > created_at.
126+
# Otherwise the job should not be added to the database
127+
total_duration_seconds = round(run.timing().run_duration_ms / 1000)
128+
workflow_data_query = f'''INSERT INTO workflow_runs(
107129
run_id, html_url, name,
108-
run_started_at, triggering_actor_login, conclusion,
109-
run_number, event, run_attempt, repository_full_name,
110-
head_branch, display_title, path)
130+
run_started_at, created_at, updated_at, triggering_actor_login, conclusion,
131+
event, run_attempt, repository_full_name,
132+
head_branch, display_title, path, total_duration_seconds)
111133
VALUES(
112134
'{run_id}', '{run.html_url}', '{run.name}', '{run.run_started_at}',
135+
'{run.created_at}', '{run.updated_at}',
113136
'{run.raw_data['triggering_actor']['login']}',
114-
'{run.conclusion}', '{run.run_number}', '{run.event}',
137+
'{run.conclusion}', '{run.event}',
115138
'{run.run_attempt}', '{run.raw_data['repository']['full_name']}',
116-
'{run.head_branch}', '{run.display_title}', '{run.path}'
139+
'{run.head_branch}', '{run.display_title}', '{run.path}', '{total_duration_seconds}'
117140
);
118141
'''
119142

@@ -122,10 +145,15 @@ def main():
122145

123146
for job in run.jobs():
124147
job_id = job.id
148+
logger.debug('Processing job %s', job.name)
125149
queued_duration_seconds = 0
126150
duration_seconds = 0
127151

128152
job_created_at_date = dateutil.parser.parse(job.raw_data['created_at'])
153+
if job_created_at_date > job.started_at:
154+
logger.warning('Skipping job %s of run %s - most likely a stub \
155+
job created after workflow restart', job.name, run_id)
156+
continue
129157

130158
queued_duration_timedelta = job.started_at - job_created_at_date
131159
queued_duration_seconds = round(queued_duration_timedelta.total_seconds())
@@ -134,27 +162,30 @@ def main():
134162
duration_seconds = round(duration_timedelta.total_seconds())
135163

136164
job_data_query = f'''
137-
INSERT INTO github_workflow_jobs_test(
165+
INSERT INTO workflow_jobs(
138166
job_id, parent_run_id, html_url, name,
139167
created_at, started_at, completed_at,
140168
queued_duration_seconds, duration_seconds,
141-
runner_name, status, conclusion, head_branch)
169+
runner_name, status, conclusion, head_branch,
170+
run_attempt, workflow_name
171+
)
142172
VALUES(
143173
'{job_id}', '{run_id}', '{job.html_url}', '{job.name}',
144174
'{job.raw_data['created_at']}', '{job.started_at}', '{job.completed_at}',
145175
'{queued_duration_seconds}', '{duration_seconds}',
146176
'{job.raw_data['runner_name']}', '{job.status}', '{job.conclusion}',
147-
'{job.raw_data['head_branch']}'
177+
'{job.raw_data['head_branch']}', '{job.raw_data['run_attempt']}', '{job.raw_data['workflow_name']}'
148178
);
149179
'''
150180
logger.debug('Job query: %s', job_data_query)
151181
cur.execute(job_data_query)
152182
for step in job.steps:
183+
logger.debug('Processing step %s', step.name)
153184
duration_seconds_timedelta = step.completed_at - step.started_at
154185
duration_seconds = round(duration_seconds_timedelta.total_seconds())
155186

156187
step_data_query = f'''
157-
INSERT INTO github_workflow_steps_test(
188+
INSERT INTO workflow_steps(
158189
parent_job_id, name, conclusion,
159190
number, started_at, completed_at,
160191
duration_seconds)

.github/workflows/code_snippets.yml

+1-5
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,5 @@ jobs:
3939
- name: CMake configure
4040
run: cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=SEQ -B build
4141

42-
- name: Get number of CPU cores
43-
uses: SimenB/github-actions-cpu-cores@v2
44-
id: cpu-cores
45-
4642
- name: Build snippets
47-
run: cmake --build build --target openvino_docs_snippets --parallel ${{ steps.cpu-cores.outputs.count }}
43+
run: cmake --build build --target openvino_docs_snippets --parallel

.github/workflows/coverage.yml

-6
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,6 @@ jobs:
5454
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_tf2.txt
5555
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_dev.txt
5656
57-
- name: Get number of CPU cores
58-
uses: SimenB/github-actions-cpu-cores@v2
59-
id: cpu-cores
60-
6157
- name: Build OpenVINO with CMake
6258
uses: ashutoshvarma/action-cmake-build@master
6359
with:
@@ -81,7 +77,6 @@ jobs:
8177
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
8278
-DENABLE_SYSTEM_SNAPPY=ON
8379
build-type: Release
84-
parallel: ${{ steps.cpu-cores.outputs.count }}
8580

8681
- name: Install wheel packages
8782
run: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/install_pkg -P '${{ github.workspace }}/build/cmake_install.cmake'
@@ -129,7 +124,6 @@ jobs:
129124
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
130125
-DCMAKE_C_LINKER_LAUNCHER=ccache
131126
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
132-
parallel: ${{ steps.cpu-cores.outputs.count }}
133127
134128
135129
- name: Print info

.github/workflows/job_cxx_unit_tests.yml

+6
Original file line numberDiff line numberDiff line change
@@ -215,6 +215,12 @@ jobs:
215215
--gtest_filter=*smoke* \
216216
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-TemplateFuncTests.xml
217217
218+
- name: OV utils unit tests
219+
run: |
220+
source ${INSTALL_DIR}/setupvars.sh
221+
${INSTALL_TEST_DIR}/ov_util_tests --gtest_print_time=1 \
222+
--gtest_output=xml:${INSTALL_TEST_DIR}/TEST-ov_util_tests.xml
223+
218224
- name: OpenVINO C API tests
219225
if: fromJSON(inputs.affected-components).C_API.test
220226
run: |

.github/workflows/job_python_unit_tests.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ jobs:
280280
run: |
281281
# requires 'unit_tests' from 'mo'
282282
export PYTHONPATH=${INSTALL_TEST_DIR}/mo
283-
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow2_keras_tests/ -m precommit_tf_fe --junitxml=${INSTALL_TEST_DIR}/TEST-tf2_fe.xml
283+
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/tensorflow2_keras_tests/ -n logical -m precommit_tf_fe --junitxml=${INSTALL_TEST_DIR}/TEST-tf2_fe.xml
284284
env:
285285
TEST_DEVICE: CPU
286286
TEST_PRECISION: FP16

.github/workflows/job_tensorflow_models_tests.yml

+6-1
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,12 @@ jobs:
114114
- name: TensorFlow Models Tests - TF FE
115115
run: |
116116
export PYTHONPATH=${MODEL_HUB_TESTS_INSTALL_DIR}:$PYTHONPATH
117-
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/ -m ${{ inputs.model_scope }} --html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_${{ inputs.model_scope }}.html --self-contained-html -v
117+
python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/test_tf_convert_model.py -m ${{ inputs.model_scope }} \
118+
--html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_${{ inputs.model_scope }}.html --self-contained-html -v
119+
# decouple notebook tests due to GitHub issue in tensorflow_hub https://github.com/tensorflow/hub/issues/903
120+
# and use WA to switch to (legacy) Keras 2
121+
TF_USE_LEGACY_KERAS=1 python3 -m pytest ${MODEL_HUB_TESTS_INSTALL_DIR}/tensorflow/test_tf_hub_api_notebooks.py -m ${{ inputs.model_scope }} \
122+
--html=${INSTALL_TEST_DIR}/TEST-tf_fe_models_notebooks_${{ inputs.model_scope }}.html --self-contained-html -v
118123
env:
119124
TEST_DEVICE: CPU
120125

.github/workflows/linux_arm64.yml

+1
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,7 @@ jobs:
172172
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
173173
-DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
174174
-DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
175+
-DOV_CPU_AARCH64_USE_MULTI_ISA=OFF \
175176
-S ${OPENVINO_REPO} \
176177
-B ${BUILD_DIR}
177178

.github/workflows/send_workflows_to_opentelemetry.yml

+8-1
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ jobs:
4949
- name: Install deps
5050
run: |
5151
pip3 install -r .github/scripts/requirements.txt
52+
# dependency review action has these as an exception
53+
# yet it still complains, so install them here
5254
pip3 install PyGithub==2.2.0 psycopg2-binary==2.9.9
5355
5456
- name: Send metrics to SQL database
@@ -58,6 +60,11 @@ jobs:
5860
PGHOST: ${{ secrets.METRICS_DATABASE_HOST }}
5961
PGUSER: ${{ secrets.METRICS_DATABASE_USERNAME }}
6062
PGPASSWORD: ${{ secrets.METRICS_DATABASE_PASSWORD }}
63+
PGDATABASE: ${{ secrets.METRICS_DATABASE_NAME }}
6164
PGPORT: 5432
65+
# LOGLEVEL: "${{ runner.debug == '1' && 'DEBUG' || 'INFO' }}"
66+
LOGLEVEL: DEBUG
6267
run: |
63-
python3 .github/scripts/collect_github_metrics.py
68+
python3 .github/scripts/collect_github_metrics.py \
69+
--run-id ${{ github.event.workflow_run.id }} \
70+
--repository-name ${GITHUB_REPOSITORY}

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -61,4 +61,5 @@ __pycache__
6161
/tools/mo/*.svg
6262
/src/plugins/intel_cpu/tools/commit_slider/*.json
6363
/src/plugins/intel_cpu/tools/commit_slider/slider_cache/*
64+
/src/plugins/intel_cpu/thirdparty/ComputeLibrary/build/*
6465
.github/GITHUB_OUTPUT

cmake/developer_package/ncc_naming_style/openvino.style

+1-1
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ VariableReference: '^\w+$'
1818

1919
EnumName: '^[A-Z][\w]+$'
2020
# excepts element_type
21-
EnumConstantName: '^([A-Z\d_]+|undefined|dynamic|boolean|bf16|f16|f32|f64|i4|i8|i16|i32|i64|u1|u4|u8|u16|u32|u64|nf4|f8e4m3|f8e5m2|string|asymmetric|align_corners|round_prefer_floor|round_prefer_ceil|floor|ceil|simple|nearest|linear|linear_onnx|cubic|area|scales|sizes|half_pixel|tf_half_pixel_for_nn|pytorch_half_pixel|asymetric)$'
21+
EnumConstantName: '^([A-Z\d_]+|undefined|dynamic|boolean|bf16|f16|f32|f64|i4|i8|i16|i32|i64|u1|u2|u3|u4|u6|u8|u16|u32|u64|nf4|f8e4m3|f8e5m2|string|asymmetric|align_corners|round_prefer_floor|round_prefer_ceil|floor|ceil|simple|nearest|linear|linear_onnx|cubic|area|scales|sizes|half_pixel|tf_half_pixel_for_nn|pytorch_half_pixel|asymetric)$'
2222
# TODO: align
2323
UsingDeclaration: '^.*$'
2424
TypedefName: '^.*$'

docs/articles_en/about-openvino/additional-resources/legal-information.rst

+27-13
Original file line numberDiff line numberDiff line change
@@ -5,31 +5,45 @@ Legal Information
55

66

77
.. meta::
8-
:description: Learn about legal information and policies related to the use
8+
:description: Learn about legal information and policies related to the use
99
of Intel® Distribution of OpenVINO™ toolkit.
1010

1111

12-
Performance varies by use, configuration and other factors. Learn more at `www.intel.com/PerformanceIndex <https://www.intel.com/PerformanceIndex>`__.
13-
14-
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details. No product or component can be absolutely secure.
15-
12+
Performance varies by use, configuration and other factors. Learn more at
13+
`www.intel.com/PerformanceIndex <https://www.intel.com/PerformanceIndex>`__.
14+
15+
Performance results are based on testing as of dates shown in configurations and may not
16+
reflect all publicly available updates. See backup for configuration details. No product or
17+
component can be absolutely secure.
18+
1619
Your costs and results may vary.
17-
20+
1821
Intel technologies may require enabled hardware, software or service activation.
1922

2023
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos.
2124

22-
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
23-
25+
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel
26+
Corporation or its subsidiaries. Other names and brands may be claimed as the property of
27+
others.
28+
2429
OpenVINO™ Logo
2530
###########################################################
26-
To build equity around the project, the OpenVINO logo was created for both Intel and community usage. The logo may only be used to represent the OpenVINO toolkit and offerings built using the OpenVINO toolkit.
27-
31+
32+
To build equity around the project, the OpenVINO logo was created for both Intel and community
33+
usage. The logo may only be used to represent the OpenVINO toolkit and offerings built using
34+
the OpenVINO toolkit.
35+
2836
Logo Usage Guidelines
2937
###########################################################
30-
The OpenVINO logo must be used in connection with truthful, non-misleading references to the OpenVINO toolkit, and for no other purpose.
31-
Modification of the logo or use of any separate element(s) of the logo alone is not allowed.
32-
3338

39+
The OpenVINO logo must be used in connection with truthful, non-misleading references to the
40+
OpenVINO toolkit, and for no other purpose. Modification of the logo or use of any separate
41+
element(s) of the logo alone is not allowed.
3442

43+
Intel Global Human Right Principles
44+
###########################################################
3545

46+
Intel is committed to respecting human rights and avoiding causing or contributing to adverse
47+
impacts on human rights. See `Intel's Global Human Rights Principles <https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf>`__.
48+
Intel's products and software are intended only to be used in applications that do not cause or
49+
contribute to adverse impacts on human rights.

docs/articles_en/about-openvino/release-notes-openvino.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,7 @@ OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Kh
351351

352352
Other names and brands may be claimed as the property of others.
353353

354-
Copyright © 2023, Intel Corporation. All rights reserved.
354+
Copyright © 2024, Intel Corporation. All rights reserved.
355355

356356
For more complete information about compiler optimizations, see our Optimization Notice.
357357

0 commit comments

Comments
 (0)