Skip to main content

Oobabooga Text Web API Tutorial

Install + Import LiteLLM​

!pip install litellm
from litellm import completion 
import os

Call your oobabooga model​

Remember to set your api_base

response = completion(
  model="oobabooga/WizardCoder-Python-7B-V1.0-GPTQ",
  messages=[{ "content": "can you write a binary tree traversal preorder","role": "user"}], 
  api_base="http://localhost:5000",
  max_tokens=4000
)

See your response​

print(response)

Credits to Shuai Shao, for this tutorial.