Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial REST API for ramalama #726

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions restapi/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__pycache__/
21 changes: 21 additions & 0 deletions restapi/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## Running the server

In the server side, run the following:

```console
uvicorn api:app --host 0.0.0.0 --port 8000 --reload
```

## Client side getting information

```console
curl -X GET "http://192.168.82.25:8000/models"

{"models":["NAME MODIFIED SIZE ","ollama://tinyllama:latest 3 minutes ago 608.16 MB"]}%
```

## Posting data

```console
curl -X POST "http://192.168.82.25:8000/run/tinyllama" &
```
78 changes: 78 additions & 0 deletions restapi/api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
from fastapi import FastAPI
from fastapi.responses import RedirectResponse
import subprocess

app = FastAPI()


# Redirect root "/" to "/docs"
@app.get("/", include_in_schema=False)
def root():
return RedirectResponse(url="/docs")


@app.get("/info")
def info_command():
try:
result = subprocess.run(
["ramalama", "info"], capture_output=True, text=True
)
return {"info": result.stdout.strip().split("\n")}
except Exception as e:
return {"error": str(e)}


@app.get("/ps")
def ps_command():
try:
result = subprocess.run(
["ramalama", "ps"], capture_output=True, text=True
)
return {"ps": result.stdout.strip().split("\n")}
except Exception as e:
return {"error": str(e)}


# List available AI models in RamaLama
@app.get("/models")
def list_models():
try:
result = subprocess.run(
["ramalama", "list"], capture_output=True, text=True
)
return {"models": result.stdout.strip().split("\n")}
except Exception as e:
return {"error": str(e)}


# Pull method
@app.post("/pull/{model_name}")
def pull_model(model_name: str):
try:
command = ["ramalama", "pull", model_name]
subprocess.run(command, check=True)
return {"message": f"Model {model_name} is running"}
except subprocess.CalledProcessError as e:
return {"error": str(e)}


# Run an AI model
# (chatbot will open in the server side at this moment)
@app.post("/run/{model_name}")
def run_model(model_name: str):
try:
command = ["ramalama", "run", model_name]
subprocess.run(command, check=True)
return {"message": f"Model {model_name} is running"}
except subprocess.CalledProcessError as e:
return {"error": str(e)}


# Stop a running AI model
@app.post("/stop/{model_name}")
def stop_model(model_name: str):
try:
subprocess.run(["ramalama", "stop", model_name], check=True)
return {"message": f"Model {model_name} stopped"}
except Exception as e:
return {"error": str(e)}
2 changes: 2 additions & 0 deletions restapi/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
fastapi
uvicorn
Loading