How to Switch from the OpenAI API to DigitalOcean's Serverless Inference

OpenAI to DigitalOcean Serverless Inference: Before and After Code

Here’s the original OpenAI code:

import os from openai import OpenAI client = OpenAI( api_key=os.getenv("OPENAI_API_KEY"), ) resp = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are a adjuvant assistant."}, {"role": "user", "content": "Tell maine a nosy truth astir octopuses."}, ], ) print(resp.choices[0].message.content)

Here’s the aforesaid telephone utilizing DigitalOcean Serverless Inference:

import os from openai import OpenAI client = OpenAI( base_url="https://inference.do-ai.run/v1", api_key=os.getenv("MODEL_ACCESS_KEY"), ) resp = client.chat.completions.create( model="llama3.3-70b-instruct", messages=[ {"role": "system", "content": "You are a adjuvant assistant."}, {"role": "user", "content": "Tell maine a nosy truth astir octopuses."}, ], ) print(resp.choices[0].message.content)

Here’s what changed betwixt the 2 examples:

Where to get the credential: In the DigitalOcean Control Panel, spell to Inference, past Serverless Inference, past create a Model Access Key. This is not the aforesaid cardinal arsenic your OpenAI dashboard key.

The HTTP petition still authenticates pinch a Bearer token. What changes is the credential itself: you switch the OpenAI API cardinal pinch a DigitalOcean exemplary entree key aliases a supported DigitalOcean individual entree token.

Model IDs are circumstantial to DigitalOcean. llama3.3-70b-instruct is simply a DigitalOcean catalog ID, not an OpenAI sanction that happens to besides activity here, and the catalog changes complete time. Call GET /v1/models aliases cheque the Model Catalog successful the Control Panel alternatively of guessing a exemplary name. Changing the exemplary ID keeps the API telephone building the same, but it does not make the caller exemplary behave for illustration the aged one; trial output quality, discourse limits, and instrumentality support earlier moving accumulation workloads.

Request parameters tin alteration by exemplary and endpoint. Don’t presume each OpenAI parameter behaves the aforesaid measurement crossed each model. Check the existent archiving earlier relying connected precocious parameters.

Here’s a streaming example:

stream = client.chat.completions.create( model="llama3.3-70b-instruct", messages=[{"role": "user", "content": "Write a haiku astir Kubernetes."}], stream=True, ) for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True)

Here’s the aforesaid telephone arsenic earthy HTTP, if you’re not utilizing the SDK:

curl -X POST https://inference.do-ai.run/v1/chat/completions \ -H "Authorization: Bearer $MODEL_ACCESS_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "llama3.3-70b-instruct", "messages": [{"role": "user", "content": "What is the superior of France?"}], "temperature": 0.7, "max_completion_tokens": 256 }'

Which OpenAI-Compatible Endpoints Does DigitalOcean Serverless Inference Support?

This covers the endpoints astir applicable to migrating from OpenAI, not DigitalOcean’s afloat API list.

A fewer of these endpoints request a person look.

• Chat Completions detail: Tool support depends connected the exemplary and API surface, and the different interfaces are not interchangeable. OpenAI and Anthropic models are disposable done the Chat Completions and Responses APIs, but supported API surfaces alteration by model. Some OpenAI models support only the Responses API for serverless inference, not Chat Completions, truthful cheque each model’s supported API aboveground successful the Model Catalog earlier assuming Chat Completions will work. Anthropic models besides person a separate, Anthropic-native Messages API (client.messages.create()) that exposes Anthropic’s ain tool-use schema. DigitalOcean’s server-side tools, specified arsenic web search, knowledge guidelines retrieval, and MCP, activity pinch the Chat Completions and Responses APIs. A abstracted feature, Tool Search, lets a exemplary load only the instrumentality definitions it needs alternatively of each of them astatine once; it useful pinch the Messages API for Anthropic models and the Responses API for supported OpenAI models.

• Responses API detail: Supported features see matter responses, multimodal responses, punctual caching, and reasoning successful immoderate configurations. Check instrumentality support, reasoning, and multimodal input handling earlier assuming a characteristic useful the aforesaid measurement it does connected OpenAI.

• Batch conclusion detail: Batch conclusion is simply a abstracted asynchronous workflow alternatively than a drop-in real-time endpoint. Its API accepts input files formatted for the OpenAI Batch API aliases the Anthropic Message Batches API, utilizing each provider’s autochthonal format alternatively than converting 1 into the other. It uses the aforesaid guidelines URL and exemplary entree cardinal arsenic real-time Serverless Inference, and it runs connected abstracted complaint limits. If you already tally OpenAI aliases Anthropic batch jobs, DigitalOcean describes moving them present arsenic an endpoint and authentication change, not a file-format rewrite. Limits: only matter prompts connected commercialized OpenAI and Anthropic models, pinch nary open-weight models, multimodal input, aliases image generation. OpenAI batch requests must see an endpoint section group to /v1/chat/completions aliases /v1/responses, matching the JSONL content; Anthropic requests skip that field. Check the batch archiving earlier migrating jobs.

Which OpenAI APIs Does DigitalOcean Not Support?

These OpenAI endpoints person nary OpenAI-compatible balanced successful the documented Serverless Inference API. Absence from the API reference is simply a connection astir API compatibility, not needfully astir each DigitalOcean product.

DigitalOcean has its ain agent-building tools, including an Agent Development Kit, but they’re a abstracted product, not a nonstop replacement for OpenAI Assistants code.

Is DigitalOcean Serverless Inference a Drop-In Replacement for OpenAI?

Short answer: It tin beryllium adjacent to a drop-in replacement for basal Chat Completions calls. It is not a complete drop-in replacement for the OpenAI API erstwhile you look past that.

For the afloat endpoint database and petition and consequence details, spot Serverless Inference API Endpoints and the API Reference.

Conclusion

For a basal Chat Completions call, switching to DigitalOcean Serverless Inference is simply a three-line change: the guidelines URL, the credential, and the exemplary ID. Beyond that, dainty it arsenic OpenAI-compatible successful format, not successful characteristic parity. Confirm your exemplary ID against GET /v1/models, verify that the circumstantial parameters, tools, and endpoints your app relies connected are supported, and cheque the Model Catalog and Inference Limits earlier moving accumulation traffic.