API Integrations
Build reliable Python clients for model APIs and other AI services.
Use a provider adapter
Put authentication, base URLs, request construction, timeouts, streaming, retries, and response normalization behind a small adapter. The rest of the application should depend on an internal contract rather than a vendor-specific dictionary. This makes provider changes and local testing safer. Keep model selection in configuration and expose provider details in controlled metadata for debugging.
Timeouts and retries
Set a connection timeout and a total operation deadline so a stalled provider cannot consume a worker forever. Retry only transient failures and operations that are safe to repeat. Use bounded exponential backoff with jitter, respect rate-limit guidance, and stop when the deadline is reached. Never retry invalid input, authorization failures, or a tool action that may already have succeeded.
import time
for attempt in range(3):
try:
response = call_provider(timeout=10)
break
except TemporaryError:
if attempt == 2:
raise
time.sleep(2 ** attempt)Normalize errors
Convert provider-specific failures into categories such as invalid request, authentication, rate limit, timeout, unavailable, and malformed response. Return a useful user-facing message while keeping diagnostic details in protected logs. Include a correlation ID so support can find the trace. Do not expose raw provider payloads, stack traces, keys, or private prompts in an error response.
Streaming responses
Streaming reduces perceived wait time but requires a parser for partial events, terminal markers, disconnects, cancellation, and incomplete output. Accumulate text safely and validate the final result before treating it as complete. Close the response and release resources when a client disconnects. Test slow, empty, malformed, and interrupted streams rather than only a happy-path example.
Validate inputs and outputs
Enforce request size, file type, token, and concurrency limits before sending data. Parse responses into a schema and reject unexpected or incomplete values. Escape model text for its destination, especially when it becomes HTML, SQL, shell input, or a browser command. Validation belongs in application code because a prompt instruction cannot enforce a security or business rule.
Protect data and logs
Decide what data the provider receives, how long it retains it, and whether it may use it for training. Minimize personal and confidential content, redact logs before storage, and restrict trace access. Do not log complete request bodies by default. If a support workflow needs original content, make that access deliberate, audited, and temporary.
Testing the integration
Use mocked responses for deterministic unit tests and a small controlled integration test for authentication, schema, streaming, timeout, rate-limit, and provider-outage behavior. Record the model and API version used by live tests. Exercise retries and cancellation without causing duplicate side effects. A reliable adapter makes provider behavior boring for the rest of the application.