Multi encoder

[trumpetinc]

Co-authored-by: yvasyliev ye.vasyliev@gmail.com

Continuing work started in: #3476

Note

Target branch: 14.x

Important

Please squash and merge this PR if it's approved.

Important

Please add Co-authored-by: yvasyliev <ye.vasyliev@gmail.com> footer to the commit message if this PR is approved.

Summary

This PR introduces multi-encoder support in Feign, allowing users to register multiple encoders that are automatically selected at request time. The Encoder.encode() method now returns boolean (true if the encoder handled the object, false otherwise), and a new MultiEncoder class that executes multiple registered encoders until one is successful. EncoderException is now thrown only when there are true encoding exceptions.

Motivation

Previously, Feign only supported a single Encoder registered via Feign.builder().encoder(...). For APIs that mix different content types (JSON, XML, form data, etc.), users had to manually wrap encoders (e.g., new FormEncoder(new JacksonEncoder())). This was brittle and didn't scale well.

Changes

New API

• Encoder.encode() now returns boolean - return true when encoding succeeds, false if the encoder does not handle the request. Note that this is a semantic change of the encode() method - previously, an EncodeException was thrown in two scenarios: the encoder could not handle the encoding request and when there was a failure in handling the encoding request. Now, a return type of 'false' is used to indicate that the encoder could not handle the encoding request. EncodeException throw is reserved for true failures to encode.

• MultiEncoder - a composite encoder that delegates to a list of encoders, using the first one whose encode() returns true. If none of the delegates can handle the request, MultiEncoder.encode() returns false.

• PredicateEncoder - Conditionally delegates to a delegate encoder if the provided EnocerPredicate passes. Useful for conditionally processing a given encoder (for example, if the user wishes to limit JsonEncoder based on a content-type header value)

Encoder Updates

All built-in encoders now return boolean from encode() and gate on Content-Type where applicable:

Encoder	Strategy
DefaultEncoder	Type-based: accepts String, byte[], File, Path, InputStream, Request.Body, and null
FormEncoder	Content-Type-based: accepts multipart/form-data and application/x-www-form-urlencoded
GraphqlEncoder	Metadata-based: accepts requests matched by GraphqlContract; delegates to wrapped encoder otherwise
MeteredEncoder (all variants)	Delegates to the wrapped encoder
All JSON encoders	Any encode request
All XML encoders	Any encode request

Example Usage

Before:

// Only one encoder — had to manually wrap
Feign.builder()
    .encoder(new FormEncoder(new JacksonEncoder()))
    .target(MyApi.class, "https://api.example.com");

After:

Feign.builder()
    .encoder(MultiEncoder.of(
        new FormEncoder(),          // handles multipart/form-urlencoded by Content-Type
        new JacksonEncoder()       // handles JSON encoding
    ))
    .target(MyApi.class, "https://api.example.com");

Breaking Changes

Breaking changes are documented in MIGRATION-v14.md (sections 14–17):

1. Encoder.encode() now returns boolean instead of void. All custom Encoder implementations must update the return type and return true on success or false when the encoder can not handle the request (instead of throwing EncodeException). EncodeException should still be thrown for encoding failures.
2. Single encoder usage is unchanged — Feign.builder().encoder(new JacksonEncoder()) still works.

[trumpetinc]

that could target main, and it would be amazing to have decoder following the same principle

@velo I actually looked at Decoder yesterday. It will probably be less impactful than on the encoder side, but I would prefer to use the same syntax for configuring encoders and decoders, so I think it is worth consideration.

The question becomes: "How do we differentiate between the 'decoder does not handle this return type' case vs the 'decoding error was encountered' case".

Here are my initial thoughts (if this looks promising, we should probably break this out into a separate PR/issue):

decode() has a return type. We could have decode() return a sentinel value (Decoder.NOT_SUPPORTED) to indicate that the type isn't supported.

One issue I see here is that this will need to be a semantic change, but we don't have a clean way of forcing implementers to rewrite their code. This could result in really ugly runtime failures after a user migrates. I'm not sure what to do about that - normally this would be done by changing the method signature in a breaking way, but it's not obvious how to do that here.

Do you have any thoughts?

Once we have the semantic issue figured out, creating a MultiDecoder and PredicateDecoder will be simple.

[trumpetinc]

@yvasyliev @velo one question: I was trying to keep the scope of this PR small (as small as possible, anyway), so I did not include any of the methods/classes related to creating EncoderPredicate of various types. My intent is to add those in a future PR - is that OK, or would it be better to include those now?

Basically, I really, really want to get this PR accepted and merged so other dependent PRs (forms overhaul for example) can proceed.

[yvasyliev]

that could target main, and it would be amazing to have decoder following the same principle

@velo I actually looked at Decoder yesterday. It will probably be less impactful than on the encoder side, but I would prefer to use the same syntax for configuring encoders and decoders, so I think it is worth consideration.

The question becomes: "How do we differentiate between the 'decoder does not handle this return type' case vs the 'decoding error was encountered' case".

Here are my initial thoughts (if this looks promising, we should probably break this out into a separate PR/issue):

decode() has a return type. We could have decode() return a sentinel value (Decoder.NOT_SUPPORTED) to indicate that the type isn't supported.

😊 I apologize for moving conversations in circles. May I ask you to revisit the Encoder#canEncode approach at this stage?

There's a broken single responsibility principle in Feign's Encoder. By providing direct access to the RequestTemplate, the Encoder acts less like a body encoder and more like a request interceptor, despite having a dedicated RequestInterceptor interface in the library.

If it were a pure encoder, the encode method would have either a non-void return type (Request.Body encode(Object object, Type bodyType)) or direct access to the OutputStream (void encode(Object object, Type bodyType, OutputStream out)), effectively disabling the mutation of an underlying RequestTemplate. And it's okay to add another Map<String, List<String>> method parameter for reading request headers in both scenarios.

Just a side note

If we need to reduce the number of method parameters, we can pack everything into a container: Request.Body encode(RequestContext ctx)

To be explicit, I don't mind having a mutating Encoder, but I want to address the missing symmetry between the Encoder and the Decoder, which I hadn't considered previously, but now see as a strong argument for Encoder#canEncode.

The only sentinel value I'm aware of in Java is null. Any Util.NOT_SUPPORTED value would be Java-unfriendly and confusing to users. Even true/false return values are not that intuitive despite being adopted by some JDK APIs. This suggests that it would be nice if the encode method had a return type like Request.Body. If the Encoder#encode method had a return type, (a) a null value could indicate an unsupported type and (b) the logic could be the same for Decoder#decode.

The alternative is to have explicit canEncode and canDecode methods in the Encoder and Decoder interfaces, respectively. The benefit is obvious: we can keep all existing encode/decode implementations as-is while requiring users to define canEncode and canDecode explicitly - without the need to read Javadocs to understand why encode now needs to return a boolean or what a hypothetical Util.NOT_SUPPORTED even means. And I'm quite confident it's a good practice to separate business (encoding/decoding) logic from validation logic.

[velo]

> Feign.builder()
>   .encoder(
>           PredicateEncoder.forJsonContentType(new JacksoneEncoder()),
>           PredicateEncoder.forEmptyBody(new EmptyEncoder()),
>           new DefaultEncoder()
>   )
>   ...

This syntax looks nice.... but would need to be the other way around with DefaultEncoder first.

public Builder encoder(Encoder defaultEncoder, PredicatedEncoder...encoders) {
  return this.encoder(MultiEncoder.of( defaultEncoder, encoders );
}

major gain in terms of functionality, and don't need to break compatibility

love the recommended Predicates, just make sure PredicatedEncoder is a functionaly type so people can easily expand on this

I still see value on a easy syntax to provide PredicatedEncoder/Encoder pairs to build a MultiEncoder/Decoder

There is a lot happening, I can't keep up

[trumpetinc]

@velo

Do see an issue (backwards compatibility, etc...) with just changing the Builder.encoder(Encoder) method to be Builder.encoder(Encoder...) (just change the argument to vararg) ?

This change is source compatible with previous code.

This change is not compile time compatible, but we are already making breaking changes because of the semantic change, so breaking changes are absolutely mandatory - no way around that. So unless I am missing something, making a source compatible change should be fine to do.

I also do not want to assume that users will want to always add PredicateEncoder as their additional encoders - that class is very useful for some scenarios, but certainly not all.

[trumpetinc]

@yvasyliev I am fine with discussing this until we get it right. I have had a very long day today and tomorrow, so it will be a couple of days before I can respond. Apologies for the delay!

[trumpetinc]

@yvasyliev finally had a chance to read your response ( #3485 (comment) ) and give it some careful thought.

Attempting to find symmetry between Encoder and Decoder

For background, during one of my earlier attempts at adding streaming support to Feign, I actually did a lot of work attempting to unify the concepts of a request body and response body. The understanding I gained from that effort is informing my opinions on this (including the fact that trying to unify request and response bodies was a terrible idea!).

Here are my opinions:

1. I completely agree that Encoder is fundamentally a request interceptor that has a strict stage in the request processing lifecycle. In addition, there is an implied contract that only a single Encoder should operate on a given request flow (this was not part of the original Feign lifecycle b/c only one encoder was allowed).
2. I would argue that the response decoding phase has nothing in common with the encoding phase. It might be tempting to view them as mirror images, and that their designs should reflect that. But when you really get into the specifics, you will find that they have almost nothing in common except having names ending in 'coder'.

Decoder produces a Java object (return type of the service interface method), and that's pretty much it.

Encoder is a completely different animal. It is not just about taking a Java object and converting it into a stream of bytes that are sent to the server. Having the ability to transform the RequestTemplate is fundamental to the Feign encoding lifecycle. In fact, it is helpful to think about RequestTemplate as being a builder interface (mutable), and Encoder.encode() is intended to make changes to that builder.

Based on what I've seen in the architecture and code, the Feign designers almost certainly intended for the Encoder to be able to manipulate the request body and the request headers and other aspects of the RequestTemplate as needed. I probably wouldn't have designed it this way myself - but there are advantages to the current approach.

I am not completely opposed to attempting to change this (i.e. make the encoder so it focuses only on the bodies) - but I think this would be a massive, massive challenge with all sorts of unexpected fallout.

How to indicate that a Decoder did not work

(I'm sure you guys are sick of hearing me talk about semantic differences! But I'm really serious that needs to be taken into account when we make breaking changes... I've lost months and a ton of money when vendors made semantic changes that didn't force compile time errors, so I am coming at this from a place of a lot of painful experience).

I see six approaches (I realize I'm partially restating what yvasyliev wrote - but I want to summarize all options) for dealing with this semantic change:

1. introduce a DecodeResult return type - this would force Decoder implementers to change. However, it would NOT force errors on any code that calls decode(). This may not be an important consideration because all code that called Decoder.decode() is feign core code, and we can find instances and fix them. But I want to point out that the method signature from the callers perspective won't change. They are expecting an Object. They will get an Object. This will cause downstream failures when Feign attempts to case to the return type of the method - but that is a runtime error, not compile time.
2. Introduce a checked new DecodeNotSupportedException (important that this NOT derive from FeignException, though). This would cause compile time errors for callers.
3. Sentinel value (and it can't be null because there are scenarios where a decode operation produces a null)
4. Add a separate canDecode(Type, Response) method to the interface
5. Change return type to Optional
6. Change the decode() method name or argument list (and return a DecodeResult)

Each of the above has pros/cons:

• DecodeResult - Pro: Compile time errors for implementers. Con: runtime errors for callers.
• DecodeNotSupportedException - Pro: Compile time errors for implements and callers. Con: Exception for control flow - yuck.
• Sentinel value - Pro: None. Con: Runtime errors for implementers and callers
• canDecode(Type, Response) - Pro: Compile time errors for implementers. Con: Runtime errors for callers.
• Optional return type - Pro: Compile time errors for implementers. Con: Runtime errors for callers.
• Change method name/arguments - Pro: Compile time errors for implements and callers. Con: seems like it would be weird to rename such an obvious method name...

I think that narrowing this list down will require determining whether we care about "Runtime errors for callers". If this is something we have to avoid at all costs, then the exception and change method name may be the only ways to do this.

Personally, I think accepting the risk of runtime exceptions for callers is an acceptable compromise. These methods truly have no value except by Feign core (maybe in some unit test scenarios, but runtime errors are fine in a test situation).

A comment about separation of concerns

I think that a lot of your view on separation of concerns is being driven by the idea that Encoder's concern is to take a request body argument from the method call and convert it into a request Body object.

I do not view that as Encoder's concern. I view that Encoder's concern is "Manipulate a RequestTemplate to ensure that it is ready to be sent to the server". In that context, the boolean return type is not a matter of "can the encoder handle the request?", it is "did the encoder actually handle the request?"

This is a subtle thing, but is why I believe that canEncode() isn't appropriate.

Fundamentally, in a chain of responsibility design, the goal is to keep trying until you have success - then stop.

I could be convinced of the need for a canEncode method - but only if there were use cases where that method would be called by itself. In the context of the Feign request processing cycle, every single call to canEncode() would be paired with a subsequent call to encode(). This is a clear sign that the purpose of the returned boolean is not to provide information about whether an Encoder can encode (along with weird side effects of actually doing the encoding!). The purpose of the returned boolean is to inform the caller that the RequestTemplate was actually updated - thus telling the chain of responsibility that it's work is done.

I hope that makes sense. And for sure, if you have an example of where Encoder.canEncode() would be called standalone for a reason that is truly answering the question ("Can you encode this request?"), I am very happy to consider it. I just don't see it in a chain of responsibility design like we have here.

Maybe have an EncodeResult?

If we wanted to preserve symmetry between Encoder.encode() and Decoder.decode(), we could go with having them return an EncodeResult / DecodeResult. The EncodeResult would pretty much just have an encodingWasPerformed boolean - so not much advantage - but it would allow for some cosmetic symmetry between the calls.

Side Note: EncodeResult can not include a summary of the changes made to the RequestTemplate, unless we are going to try to make the RequestTemplate passed to the encode() method immutable, then introduce a RequestTemplateMutator that the encode() method returns and is then applied to the RequestTemplate. This just feels like a huge undertaking for pretty minimal gains.

Would a simple EncodeResult like I'm describing help with "it's not symmetrical" feeling a bit? One thing that I like this approach is that the method names would make really clear what was intended. Returning 'true' or 'false' is not nearly as informative as returning EncodeResult.encodingSuccessful() vs EncodeResult.encodingNotSupported().

[velo]

1. we clearly split the semantic change (return false if encoding not supported, continue throwing EncodeException if there is an actual failure during encoding) - this is necessary to properly implement encoder chaining like we are in MultiExtractor.

we can control support in the predicate, so encoder stays as is.

[trumpetinc]

The current Encoder.encode() method uses a thrown EncodeException to indicate TWO outcomes: a) Encoder does not support encoding of the request; and b) an error happened during encoding.

This means that callers have no way of knowing whether an exception thrown by encode() means failure, or just that the Encoder doesn't support the operation.

It is not possible to construct a MultiEncoder with that ambiguity. If you feel there is, we will need to see some code showing how it would work.

I think that we need to get clear on this before we have any discussion about adding this to a BaseBuilder.encode() method.

[velo]

@trumpetinc your own comment has a the best solution
#3485 (comment)

When making a multi encoder (or decoder) need to have a predicate that will decide which encoder will be used.

[trumpetinc]

@trumpetinc your own comment has a the best solution #3485 (comment)

When making a multi encoder (or decoder) need to have a predicate that will decide which encoder will be used.

Before we get too bogged down on this, can we agree that it's ok to target this PR for the 14.X branch, and accept that it is necessary to make a breaking change to the Encoder.encode() method? @yvasyliev and I are both quite convinced that is the case, would it be ok to just go with our judgement?

If so, then the only remaining question is whether we want to enhance BaseBuilder.encoder() as part of this PR.

I am OK doing this - but if we want to enhance the BaseBuilder.encoder() method to easily handle registering multiple encoders, I will need to move MultiEncoder into the api module.

BaseBuilder lives in api. MultiEncoder is currently in core. If we want to reference MultiEncoder, I need to have MultiEncoder be in the api module.

Are you ok with that?

Thanks.

[yvasyliev]

Hey @trumpetinc, all your points make sense. I'd say that the boolean return type for the Encoder#encode method is more reasonable, and EncodeResult would be overengineering.

I only have a few small questions/requests:

1. Request to revisit the PR description.

I'd suggest placing Co-authored-by... on the first line of the PR description so that GitHub will copy it into the commit message body automatically when merging the PR.

Please add Co-authored-by: trumpetinc <6618744+trumpetinc@users.noreply.github.com> and yvasyliev <ye.vasyliev@gmail.com> footer to the commit message if this PR is approved.

No need to add yourself as a co-author, because you're the author. 😊

and a new Encoder.of() factory combines encoders into a DelegatingEncoder

I can't find either the Encoder.of() factory or the DelegatingEncoder in the PR changes...

This replaces the need for a separate canEncode() method.

There was no canEncode() method in the first place. 😅

2. I can still see multiple irrelevant dependency upgrades. Let's remove them from the PR scope if possible.

3. I went through the PR one more time and added some new comments, most of which are minor.

[yvasyliev]

Could you please help me to understand why we need this class to be exposed publicly? Was Encoder#of factory method a bad approach?

[trumpetinc]

Yes - I should have explained that.

Right now, MultiEncoder is in the core module. Encoder is in the API module. I cannot add Encoder.of() without moving MultiEncoder to the api module.

I'm still not clear on why the core->api module split is happening (I can't find any Issue or PR about it that might explain what's going on). It looks to me like almost nothing is left in core, so I'm not sure what the point is. Other projects I've worked on that have an api module, that module is reserved for interfaces and exception classes only.

This is also why I am hesitating about making BaseEncoder.encoder() support varargs, etc... I can't add the dependency to MultiEncoder without moving MultiEncoder into api.

If I can get a thumbs up on placing MultiEncoder into api, then I absolutely would hide MultiEncoder behind Encoder.of() - and make the change to BaseEncoder. I just feel like I need some sort of agreement from a maintainer first.

[yvasyliev]

I'd suggest initially submitting a PR with whichever approach is better for the overall user experience. If it doesn't align with the maintainers' expectations, they can always request changes or move the files themselves.

[yvasyliev]

Not critical; the isJsonContentType & isXmlContentType static methods share the same business logic. Is it possible to extract a private static method with two parameters (RequestTemplate template, String regex)?

[trumpetinc]

Yes, certainly possible - quick question, though: Right now, I do not have a predicate for json or xml (in fact, these two methods theoretically should be removed from this PR).

I think that adding XmlContentTypeEncoderPredicate and JsonContentTypeEncoderPredicate would be fine - but should that maybe be in a separate PR? I was trying to keep this PR focused on raw capability...

Also, for another PR ( https://github.com/OpenFeign/feign/pull/3494/changes#diff-3d5ee4752b168285974eb09fc4782f489edeadba936b5c71dc59ff6a043d779d ), I have introduced a dedicated content-type header parser that handles charset sub-elements, etc...). It may be better to use that, then compare the actual extracted content-type to the regex.