Skip to content

fix: data race in convert_utf8_bytes plugin - #1005

Merged
DmitryRomanov merged 3 commits into
masterfrom
fix/convert-utf8-bytes-data-race
Aug 24, 2026
Merged

fix: data race in convert_utf8_bytes plugin#1005
DmitryRomanov merged 3 commits into
masterfrom
fix/convert-utf8-bytes-data-race

Conversation

@DmitryRomanov

Copy link
Copy Markdown
Collaborator

Description

The plugin reused a single p.buf buffer across events, causing a race
when concurrent processors modified it while outputs were encoding.
Fixed by using per-event event.Buf instead of plugin-shared p.buf.

WARNING: DATA RACE
Read at 0x00c00512a000 by goroutine 72:
  github.com/ozontech/insane-json.escapeString()
      github.com/ozontech/insane-json@v0.1.9/insane.go:2026 +0x3f2
  github.com/ozontech/insane-json.(*Node).Encode()
      github.com/ozontech/insane-json@v0.1.9/insane.go:660 +0x7ee
  github.com/ozontech/file.d/pipeline.(*Event).Encode()
      github.com/ozontech/file.d/pipeline/event.go:167 +0x75
  github.com/ozontech/file.d/encoder.(*JSONEncoder).Encode()
      github.com/ozontech/file.d/encoder/json.go:14 +0x3b
  github.com/ozontech/file.d/plugin/output/http.(*Plugin).out.func1()
      github.com/ozontech/file.d/plugin/output/http/http.go:392 +0x6a5
  github.com/ozontech/file.d/pipeline.(*Batch).ForEach()
      github.com/ozontech/file.d/pipeline/batch.go:106 +0x980
  github.com/ozontech/file.d/plugin/output/http.(*Plugin).out()
      github.com/ozontech/file.d/plugin/output/http/http.go:388 +0x473
  github.com/ozontech/file.d/plugin/output/http.(*Plugin).out-fm()
      <autogenerated>:1 +0x3e
  github.com/ozontech/file.d/pipeline.(*RetriableBatcher).Out()
      github.com/ozontech/file.d/pipeline/backoff.go:58 +0x248
  github.com/ozontech/file.d/pipeline.(*RetriableBatcher).Out-fm()
      <autogenerated>:1 +0x3e
  github.com/ozontech/file.d/pipeline.(*Batcher).work()
      github.com/ozontech/file.d/pipeline/batch.go:207 +0x268
  github.com/ozontech/file.d/pipeline.(*Batcher).Start.gowrap1()
      github.com/ozontech/file.d/pipeline/batch.go:189 +0x2e

Previous write at 0x00c00512a000 by goroutine 74:
  runtime.slicecopy()
      runtime/slice.go:392 +0x0
  github.com/ozontech/file.d/plugin/action/convert_utf8_bytes.(*Plugin).convert()
      github.com/ozontech/file.d/plugin/action/convert_utf8_bytes/convert_utf8_bytes.go:170 +0x157
  github.com/ozontech/file.d/plugin/action/convert_utf8_bytes.(*Plugin).Do()
      github.com/ozontech/file.d/plugin/action/convert_utf8_bytes/convert_utf8_bytes.go:155 +0x14b
  github.com/ozontech/file.d/pipeline.(*processor).doActions()
      github.com/ozontech/file.d/pipeline/processor.go:206 +0x236
  github.com/ozontech/file.d/pipeline.(*processor).processEvent()
      github.com/ozontech/file.d/pipeline/processor.go:170 +0x6b
  github.com/ozontech/file.d/pipeline.(*processor).processSequence()
      github.com/ozontech/file.d/pipeline/processor.go:149 +0x34
  github.com/ozontech/file.d/pipeline.(*processor).dischargeStream()
      github.com/ozontech/file.d/pipeline/processor.go:141 +0x44
  github.com/ozontech/file.d/pipeline.(*processor).process()
      github.com/ozontech/file.d/pipeline/processor.go:129 +0x64
  github.com/ozontech/file.d/pipeline.(*processor).start.gowrap1()
      github.com/ozontech/file.d/pipeline/processor.go:118 +0x2e

@DmitryRomanov
DmitryRomanov merged commit 1dad7f7 into master Aug 24, 2026
8 checks passed
@DmitryRomanov
DmitryRomanov deleted the fix/convert-utf8-bytes-data-race branch August 24, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants