Skip to content

into_inner on Decompressor discards data due to buffering #84

@jsha

Description

@jsha

Steps to reproduce:

  1. Decompress a Brotli stream from some input data.
  2. Call into_inner(), then read() the result to check if there is any remaining unread input data (for instance, extra junk at the end of the compressed stream).

Expected result:

If there is unread input data, it is readable.

Actual result:

If there is unread input data, it may or may not be available after calling into_inner(), depending on whether it was in the buffer or not.

Proposed fix:

If a fix for #83 is implemented, Decompressor will have the property that reading Ok(0) from it means the entire input data was consumed. In that situation, into_inner() can safely be called. Decompressor's into_inner() could be documented to only be safe after reading Ok(0) from Decompressor.

This would be useful in the ureq HTTP client, where the inner reader might be a PoolReturnRead that returns a connection to a connection pool once a response body has been fully read. In that situation, we need to check for any unread bytes at the end of the response body, which could indicate a desynchronized HTTP connection that we need to drop.

Test code to reproduce:

    #[test]
    fn test_no_vanishing_bytes() {
        // Output from this command:
        // (brotli -c <<<"hello"; echo goodbye) | xxd -p | sed 's/../\\x&/g'
        let compressed_with_extra = b"\x8f\x02\x80\x68\x65\x6c\x6c\x6f\x0a\x03\x67\x6f\x6f\x64\x62\x79\x65\x0a";
        let mut cursor = Cursor::new(compressed_with_extra);
        let mut reader = brotli_decompressor::Decompressor::new(cursor, 8000);
        std::io::read_to_string(&mut reader).unwrap();
        let unwrapped_cursor = reader.into_inner();
        assert_eq!(std::io::read_to_string(unwrapped_cursor).unwrap(), "goodbye".to_string());
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions