diff options
| author | dev <dev@localhost> | 2026-10-21 22:02:22 +0000 |
|---|---|---|
| committer | dev <dev@localhost> | 2026-10-21 22:02:22 +0000 |
| commit | 710f0daff34e4ae02cf906841117449bcf114198 (patch) | |
| tree | c542c044e8e20f416b25280a43309fe67a037736 /config.example.yaml | |
| parent | 9d40159cad3b55c753ed02e7a4dee8cc7b6d15eb (diff) | |
| download | aichat-710f0daff34e4ae02cf906841117449bcf114198.tar.gz | |
Add idempotency support to aichat HTTP clients
- Add boolean to client config. When enabled, a
fresh (UUID) header is sent with each logical request
attempt so the outer retry loop reuses the same submission instead of
starting duplicate work on providers that support it (e.g. Sail Research).
The key is generated once per logical request (before the retry loop) and
reused across all retry attempts via the cloned ChatCompletionsData/request
builders, so providers debounce duplicate generations.
- Wire the header in request_builder (macro-based clients) and Bedrock's
chat_completions_builder.
- Document the option in config.example.yaml.
Diffstat (limited to 'config.example.yaml')
| -rw-r--r-- | config.example.yaml | 9 |
1 files changed, 6 insertions, 3 deletions
diff --git a/config.example.yaml b/config.example.yaml index 24e4cf9..7101f3b 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -118,6 +118,7 @@ clients: # connect_timeout: 10 # Set timeout in seconds for connect to api # timeout: 600 # Set total request timeout in seconds (0 or omitted disables) # retries: 3 # Number of retries after the first failed attempt (0 disables) + # idempotency: true # Send an Idempotency-Key header so retries reuse the same submission # retry_delay_ms: 1000 # Base delay between retries in milliseconds (default 1000) # retry_max_delay_ms: 60000 # Cap exponential backoff delay in milliseconds (default uncapped) @@ -125,9 +126,11 @@ clients: # A non-streaming request may hold one HTTP connection for minutes. If the # connection is dropped by an intermediary, the client otherwise waits # forever. Set `timeout` to bound each attempt, and `retries` to re-submit - # when an attempt fails. Be aware that retrying a non-idempotent request - # may resubmit the same work; prefer streaming (stream: true) for - # long-running requests so data flows continuously over the wire. + # when an attempt fails. Set `idempotency: true` (and `retries` > 0) so each + # retry reuses the same Idempotency-Key and the provider does not start + # duplicate work — required for providers such as Sail Research. Prefer + # streaming (stream: true) for long-running requests so data flows + # continuously over the wire. # See https://platform.openai.com/docs/quickstart - type: openai |
