Skip to content

Commit 00be244

Browse files
committed
fix stateful seq2seq model inference perfomance
1 parent 09a22a3 commit 00be244

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

optimum/intel/openvino/modeling_seq2seq.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -668,7 +668,7 @@ def forward(
668668
logits = torch.from_numpy(self.request.get_tensor("logits").data).to(self.device)
669669
self._past_length += input_ids.shape[1]
670670

671-
out_past_key_values = ()
671+
out_past_key_values = ((),)
672672

673673
if not self.stateful:
674674
# Tuple of length equal to : number of layer * number of past_key_value per decoder layer (2 corresponds to the

0 commit comments

Comments
 (0)