Skip to content

Fixes float32->float64 precision widening a.k.a. float bloat #3743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
binarymax wants to merge 5 commits into
base: main
Choose a base branch
from
+46 −0
Open

Fixes float32->float64 precision widening a.k.a. float bloat #3743

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ff8ebc7
Removes float32->base64 precision widening a.k.a. float bloat
binarymax Aug 21, 2026
bbe6593
Merge branch 'main' into maxirwin/retain-float32-embeddings
binarymax Aug 27, 2026
74607ee
Update custom_code_report.py
binarymax Aug 27, 2026
dde0f48
Update custom_code_report.py
binarymax Aug 27, 2026
42b61a3
Update test_custom_code_report.py
binarymax Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions src/openai/resources/embeddings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,29 @@ def create(
if not is_given(encoding_format):
params["encoding_format"] = "base64"

def parser(obj: CreateEmbeddingResponse) -> CreateEmbeddingResponse:
if is_given(encoding_format):
Comment on lines +113 to +114

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wire the precision-normalizing parser into requests

For default-format embedding requests, this newly defined callback is never invoked: the request still passes partial(_parse_embedding_response, ...) as its post_parser on line 144, so responses continue through the unchanged decoder and retain the widened representations this commit is intended to remove. The asynchronous implementation has the same disconnect; implement the normalization in the shared handwritten parsing helper and keep this generated resource delegating to it.

AGENTS.md reference: AGENTS.md:L3-L8

Useful? React with 👍 / 👎.

# don't modify the response object if a user explicitly asked for a format
return obj

if not obj.data:
raise ValueError("No embedding data received")

for embedding in obj.data:
data = cast(object, embedding.embedding)
if not isinstance(data, str):
continue
if not has_numpy():
# use array for base64 optimisation
values = array.array("f", base64.b64decode(data)).tolist()
else:
values = np.frombuffer( # type: ignore[no-untyped-call]
base64.b64decode(data), dtype="float32"
).tolist()
embedding.embedding = [float(f"{value:.9g}") for value in values]

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid per-coordinate string round trips

For default-format embedding requests, this now formats and reparses every coordinate in Python, adding a temporary string allocation per value. In a local check with one 3,072-dimensional vector, this conversion took roughly 1.67 ms versus 0.05 ms for the previous tolist() conversion, and the cost scales across every vector in a batch; the mirrored async parser has the same issue. This can add seconds of SDK-side CPU work to large embedding batches, counteracting the performance goal of this change, so the shortening should be implemented without a Python string round trip for every coordinate.

Useful? React with 👍 / 👎.


return obj

return self._post(
"/embeddings",
body=maybe_transform(params, embedding_create_params.EmbeddingCreateParams),
Expand Down Expand Up @@ -212,6 +235,29 @@ async def create(
if not is_given(encoding_format):
params["encoding_format"] = "base64"

def parser(obj: CreateEmbeddingResponse) -> CreateEmbeddingResponse:
if is_given(encoding_format):
# don't modify the response object if a user explicitly asked for a format
return obj

if not obj.data:
raise ValueError("No embedding data received")

for embedding in obj.data:
data = cast(object, embedding.embedding)
if not isinstance(data, str):
continue
if not has_numpy():
# use array for base64 optimisation
values = array.array("f", base64.b64decode(data)).tolist()
else:
values = np.frombuffer( # type: ignore[no-untyped-call]
base64.b64decode(data), dtype="float32"
).tolist()
embedding.embedding = [float(f"{value:.9g}") for value in values]

return obj

return await self._post(
"/embeddings",
body=maybe_transform(params, embedding_create_params.EmbeddingCreateParams),
Expand Down