Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPENVINO CODE] Added Fill-in-the-middle(FIM) support #848

Merged
merged 8 commits into from
Feb 6, 2024
Prev Previous commit
Next Next commit
added README for adding Fill in the middle mode support and updated O…
…verviewSection
kumarijy committed Feb 4, 2024
commit a841d512aac9c7664e87717dad6e01662e9b6a7f
18 changes: 18 additions & 0 deletions modules/openvino_code/README.md
Original file line number Diff line number Diff line change
@@ -8,6 +8,7 @@ OpenVINO Code provides the following features:

- Inline Code Completion
- Summarization via Docstring
- Fill in the Middle Mode

## Working with Extension

@@ -48,6 +49,23 @@ You can select the desired type of quotes in the extension settings.
The model can generate docstring in Code Completion mode, but in this case it is impossible to control the result.
In the docstring generation mode, various popular templates are available in the settings that will guide the model output.

### Fill in the Middle Mode


1. Create a new Python file or open an existing one.
1. Type `def main():` or place the cursor where you'd like middle text to be generated.
1. Press the keyboard shortcut `Ctrl+Alt+Space` (`Cmd+Alt+Space` for macOS) or click the `Generate Code Completion` button located in the side panel.
1. You can select the text then generate the related code.
1. You may also right-click on "Generate Inline Code Completion In New Tab" to generate code in a new tab.
1. Use the `Tab` key to accept the entire suggestion or `Ctrl`+`Right Arrow` to accept it word by word. To decline the suggestion, press `Esc`.

You can customize the length of the generated code by adjusting `Max New Tokens` and `Min New Tokens` parameters in the extension settings.
The number of generated tokens is also influenced by the `Server Request Timeout` setting.

Fill in the middle mode brings in advanced code completion capabilities supporting fill-in-the-blank task, supporting project-level code completion and infilling tasks.

To enable fill in the middle mode, check the `Fill In The Middle Mode` checkbox in the extension settings.

### Monitoring Extension Output

To examine the input and output from the code generation API, follow these steps:
Original file line number Diff line number Diff line change
@@ -7,7 +7,8 @@ export function OverviewSection(): JSX.Element {
OpenVINO Code provides the following features:
<ul>
<li>Inline Code Completion</li>
<li>Summarization via docstring</li>
<li>Summarization via docstring</li>
<li>Fill in the Middle Mode</li>
</ul>
To use OpenVINO Code please start the server.
</span>