Skip to content

AVRO-4236: [doc] Update security model for Avro IPC #3681

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RyanSkraba wants to merge 1 commit into
base: main
Choose a base branch
from
Open

AVRO-4236: [doc] Update security model for Avro IPC #3681

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions doc/content/en/project/Security/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ Security Model
The Avro library implementations are designed to read and write any data conforming
to a schema. Transport is outside the scope of the Avro library: applications using
Avro should be surrounded by security measures that prevent attackers from writing
random data and otherwise interfering with the consumers of schemas.
random data and otherwise interfering with the consumers of schemas. In addition,
the Avro IPC mechanism should not be exposed on a public network to untrusted actors.
Copy link
Member

@raboof raboof Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe:

Suggested change
the Avro IPC mechanism should not be exposed on a public network to untrusted actors.
the Avro IPC mechanism should not be exposed on a public network or to untrusted actors.

Copy link
Contributor

@opwvhk opwvhk Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a simple addition, and will at the very least warn people about the IPC mechanism.

I wonder though, would it be feasible/useful to create a more elaborate paragraph, mentioning the things one would need to do to make IPC safer?

Copy link
Contributor Author

@RyanSkraba RyanSkraba Mar 10, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about this! I don't want to suggest that adding a layer of input validation or logging in users would solve the problem (and I don't particularly know how to document how to do that!)

But we could definitely be a bit more categorical:

Avro IPC is intended for use in inter-process communication between trusted components within a controlled and secure network environment. It does not implement input validation, authorization, or authentication, and therefore must not be deployed on public networks or made accessible to untrusted clients or services.

What do you think?

Copy link

@lf- lf- Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i actually think that what separates gRPC from avro ipc is not any of these things. it's simply that it's implemented in a more hardened way.

avro ipc could work on untrusted networks if:

  • protocol canonicalization actually worked (maybe by converting to a strict avro schema rather than JSON to remove language differences?)
    • clients couldn't send arbitrary length nonsense as protocol or non canonical things as protocol: less detail is admissible to send to servers (or it's squashed when it's received)
    • spec needs a lot more detail on how to canonicalize 100% consistently across languages
  • protocol caches couldn't be totally filled forever by one client: eviction mechanisms with sharding
  • protocol hashing were implemented properly as server side to prevent cache poisoning
    • or maybe you use a confluent schema registry but for protocols?
  • auditing were done for "allocating user controlled buffer lengths" and similar bugs

these aren't intractable, and avro ipc looks very interesting to me! but the implementations just aren't designed for untrusted input today.

Copy link
Contributor

@opwvhk opwvhk Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the Avro SASL profile there actually is authentication, though the documentation is very sketchy (to put it mildly). And when reading the code, I see that implementing an authentication method requires non-trivial work. Also, it is not immediately clear how to add encryption.

All in all, Avro IPC would benefit from the hardening that @lf- mentions and more documentation. Ideally with a detailed example that uses an SSL connection with DIGEST or OAUTHBEARER authentication.


Although the Avro library will not read or write data except as directed to by
invoking it, avoiding leaking data into a side channel like log files is a non-goal
Expand All @@ -55,7 +56,6 @@ additional validation is strongly advised. Such validation can use the parsed sc
as schema parsing itself is safe: the parser allows SPIs, but is not otherwise
configurable.


Summary
-------

Expand Down