p3(http): Consider specifying the list of definitely forbidden field names

[saulecabrera]

Field operations such as from_list or set may return a header-error. It is currently not specified which field names might cause these operations to return header-error.forbidden.

Wasmtime's implementation defines a list of default forbidden headers, which according to bytecodealliance/wasmtime#7538 (comment) that list is an evolution from what I believe is an original design document of the WASI HTTP spec.

[badeend]

Side note on the subject of forbidden headers; fields.clone currently mentions that it is semantically equivalent to copy-all + from-list. Which implies that fields.clone should return an error if the original fields instance contains any forbidden headers. I assume this is not actually the case. clone is not fallible, so wouldn't even be possible.

[pchickey]

I'd rephrase the issue described by @badeend above as being essential: individual headers can be forbidden in fields.from-list and fields.set, but neither the spec nor implementations filters any of those headers out from being readable to the importer, as immutable values, by the request.headers method.

So, a common problem a proxy runs into is that if it just wants to get {request, response}.headers and that returned headers contains a forbidden header (and then, optionally clone, and mutate non-forbidden headers), and then pass that headers to create a new request (or response) with request.new, there's 1. no way for request.new to give an error related to being passed in a forbidden header, and 2. there is no description for what behavior should happen instead of reporting an error.

However, I still think its desirable (maybe even necessary for some applications?) for forbidden headers to be present in the headers returned from {request, response}.headers

So, in addition to specifying a minimum set of headers that are always forbidden, I believe we need to specify what happens in this case - the most sensible thing, since we cannot report an error without a breaking interface change, is for all forbidden headers to be erased from the set. That still has the pitfall of being potentially "surprising" behavior to a user that cannot be debugged entirely in userland, especially if implementations diverge in what headers are forbidden.

[lukewagner]

(sorry for slow reply) Agreed with @pchickey that we probably want a minimum set of forbidden headers which hosts can superset. The last time I looked into this question and interviewed some HTTP server folks, the set of minimum set I came up with was: Connection, Keep-Alive, Proxy-Authenticate, Proxy-Authorization, Proxy-Connection, TE (except ‘trailers’ is allowed), Transfer-Encoding, Upgrade, HTTP2-Settings, Host.