Skip to content

Comments

Fix sse size mismatch 15094702859884413816#216

Open
TheSpaceGod wants to merge 30 commits intonextcloud:mainfrom
TheSpaceGod:fix-sse-size-mismatch-15094702859884413816
Open

Fix sse size mismatch 15094702859884413816#216
TheSpaceGod wants to merge 30 commits intonextcloud:mainfrom
TheSpaceGod:fix-sse-size-mismatch-15094702859884413816

Conversation

@TheSpaceGod
Copy link

Fixes cURL Error 26 (SSE Size Mismatch) in Nextcloud Context Chat.

When SSE is enabled, fstat() on a file handle returns the encrypted size, while reading the stream returns the decrypted content. This mismatch caused uploads to fail with cURL error 26 because Guzzle uses fstat to calculate the request Content-Length.

This fix:

  1. Explicitly passes the decrypted file size (obtained via $file->getSize()) to the Source object.
  2. In LangRopeService, wraps the file stream in a Guzzle FnStream and overrides getSize() to return the correct decrypted size. This forces Guzzle to advertise the correct length, matching the data streamed.

Includes comprehensive reproduction scripts validating the fix against Nextcloud VFS encryption.

Test output before on commit: [f7c85cd]

nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 
❯ ./run_test.sh 
WARN[0000] No services to build                         
[+] up 2/2
 ✔ Container reproduction-context_chat_backend-1 Created                                                                                                0.2s 
 ✔ Container reproduction-nextcloud-1            Created                                                                                                0.2s 
Waiting for container to accept commands...
Checking Nextcloud status...
Nextcloud is not installed - only a limited number of commands are available
Nextcloud is not installed. Installing...
Nextcloud was successfully installed
Waiting for Nextcloud to be fully ready...
Nextcloud is ready.
Configuring Nextcloud...
encryption 2.20.0 enabled
Encryption enabled

Default module: OC_DEFAULT_MODULE
Master key already enabled
context_chat 5.2.0 enabled
app_api already enabled
Cleaning up previous registrations...
ExApp context_chat_backend not found. Failed to unregister.
Daemon config manual_install not found.
Registering Mock Backend...
Daemon successfully registered.
ExApp context_chat_backend deployed successfully.
ExApp context_chat_backend successfully registered.
Enabling Context Chat Backend...
ExApp context_chat_backend already enabled.
Listing AppAPI apps...
ExApps:
context_chat_backend (Context Chat Backend): 1.0.0 [enabled]
Config value 'backend_init' for app 'context_chat' is now set to 'true', stored as mixed in fast cache
Creating test file via VFS (Encrypted)...
[+] copy 1/1
 ✔ reproduction-nextcloud-1 Copied create_test_file.php to reproduction-nextcloud-1:/var/www/html/create_test_file.php                                  0.0s 
Created encrypted test.txt successfully.
Verifying file existence in Nextcloud VFS...
SUCCESS: test.txt found in Nextcloud VFS.
DEBUG: Checking file sizes...
[+] copy 1/1
 ✔ reproduction-nextcloud-1 Copied debug_sizes.php to reproduction-nextcloud-1:/var/www/html/debug_sizes.php                                            0.0s 
File::getSize() for test.txt: 1048576
fstat()['size'] for test.txt: 1069248
Actual Read Size for test.txt: 1048576
Mismatch for test.txt: 0
File::getSize() for Nextcloud Manual.pdf: 22209301
fstat()['size'] for Nextcloud Manual.pdf: 22209301
Actual Read Size for Nextcloud Manual.pdf: 22209301
Mismatch for Nextcloud Manual.pdf: 0
Running Scan (Direct Indexing) on test.txt...

In LangRopeService.php line 132:
                                                                                                                                                            
  Error during request to Context Chat Backend (ExApp): cURL error 26: client read function EOF fail, only 1048919/1069591 of needed bytes read (see https  
  ://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://context_chat_backend:23000/loadSources                                                          
                                                                                                                                                            

context_chat:scan [-m|--mimetype MIMETYPE] [-d|--directory DIRECTORY] [--] <user_id>


nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 took 38s 
❯ 

Test output after on commit: [f7c85cd]

nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 
❯ ./run_test.sh 
WARN[0000] No services to build                         
[+] up 2/2
 ✔ Container reproduction-nextcloud-1            Created                                                                                                0.3s 
 ✔ Container reproduction-context_chat_backend-1 Created                                                                                                0.3s 
Waiting for container to accept commands...
Checking Nextcloud status...
Nextcloud is not installed - only a limited number of commands are available
Nextcloud is not installed. Installing...
Nextcloud was successfully installed
Waiting for Nextcloud to be fully ready...
Nextcloud is ready.
Configuring Nextcloud...
encryption 2.20.0 enabled
Encryption enabled

Default module: OC_DEFAULT_MODULE
Master key already enabled
context_chat 5.2.0 enabled
app_api already enabled
Cleaning up previous registrations...
ExApp context_chat_backend not found. Failed to unregister.
Daemon config manual_install not found.
Registering Mock Backend...
Daemon successfully registered.
ExApp context_chat_backend deployed successfully.
ExApp context_chat_backend successfully registered.
Enabling Context Chat Backend...
ExApp context_chat_backend already enabled.
Listing AppAPI apps...
ExApps:
context_chat_backend (Context Chat Backend): 1.0.0 [enabled]
Config value 'backend_init' for app 'context_chat' is now set to 'true', stored as mixed in fast cache
Creating test file via VFS (Encrypted)...
[+] copy 1/1
 ✔ reproduction-nextcloud-1 Copied create_test_file.php to reproduction-nextcloud-1:/var/www/html/create_test_file.php                                  0.0s 
Created encrypted test.txt successfully.
Verifying file existence in Nextcloud VFS...
SUCCESS: test.txt found in Nextcloud VFS.
DEBUG: Checking file sizes...
[+] copy 1/1
 ✔ reproduction-nextcloud-1 Copied debug_sizes.php to reproduction-nextcloud-1:/var/www/html/debug_sizes.php                                            0.0s 
File::getSize() for test.txt: 1048576
fstat()['size'] for test.txt: 1069248
Actual Read Size for test.txt: 1048576
Mismatch for test.txt: 0
File::getSize() for Nextcloud Manual.pdf: 22209301
fstat()['size'] for Nextcloud Manual.pdf: 22209301
Actual Read Size for Nextcloud Manual.pdf: 22209301
Mismatch for Nextcloud Manual.pdf: 0
Running Scan (Direct Indexing) on test.txt...
[admin] Scanned files/test.txt
Checking backend logs...
Collecting flask
  Downloading flask-3.1.2-py3-none-any.whl (103 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.3/103.3 kB 2.4 MB/s eta 0:00:00
Collecting click>=8.1.3
  Downloading click-8.1.8-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.2/98.2 kB 3.1 MB/s eta 0:00:00
Collecting importlib-metadata>=3.6.0
  Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB)
Collecting itsdangerous>=2.2.0
  Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Collecting markupsafe>=2.1.1
  Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB)
Collecting jinja2>=3.1.2
  Downloading jinja2-3.1.6-py3-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.9/134.9 kB 10.5 MB/s eta 0:00:00
Collecting werkzeug>=3.1.0
  Downloading werkzeug-3.1.5-py3-none-any.whl (225 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.0/225.0 kB 6.6 MB/s eta 0:00:00
Collecting blinker>=1.9.0
  Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB)
Collecting zipp>=3.20
  Downloading zipp-3.23.0-py3-none-any.whl (10 kB)
Installing collected packages: zipp, markupsafe, itsdangerous, click, blinker, werkzeug, jinja2, importlib-metadata, flask
Successfully installed blinker-1.9.0 click-8.1.8 flask-3.1.2 importlib-metadata-8.7.1 itsdangerous-2.2.0 jinja2-3.1.6 markupsafe-3.0.3 werkzeug-3.1.5 zipp-3.23.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 26.0.1
[notice] To update, run: pip install --upgrade pip
 * Serving Flask app 'mock_server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:23000
 * Running on http://172.18.0.3:23000
Press CTRL+C to quit
172.18.0.2 - - [07/Feb/2026 00:02:16] "GET /heartbeat HTTP/1.1" 200 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "POST /init HTTP/1.1" 404 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "GET /heartbeat HTTP/1.1" 200 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "PUT /enabled?enabled=1 HTTP/1.1" 404 -
172.18.0.2 - - [07/Feb/2026 00:02:25] "PUT /loadSources HTTP/1.1" 200 -
Test completed successfully.

nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 took 38s 
❯ 

google-labs-jules bot and others added 30 commits February 5, 2026 01:40
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size).
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/` including `debug_sizes.php`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/` including extensive size debugging tools.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans, and correctly handle single file inputs.
- Updated `LangRopeService` to use the `$size` property for the `Content-Length` header in the API request, overriding the auto-detection which uses `fstat` (returning encrypted size). Casts size to string to satisfy Guzzle requirements.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added reproduction scripts in `tests/reproduction/`.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans, and correctly handle single file inputs.
- Updated `LangRopeService` to wrap the content stream in a Guzzle `FnStream` and override the `getSize()` method to return the logical size. This ensures Guzzle calculates the correct `Content-Length` for the multipart request body, preventing `cURL error 26` (EOF fail) caused by `fstat` returning the encrypted size.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added robust reproduction scripts in `tests/reproduction/` including encryption verification and size debugging.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans, and correctly handle single file inputs.
- Updated `LangRopeService` to wrap the content stream in a Guzzle `FnStream` and override the `getSize()` method to return the logical size. This ensures Guzzle calculates the correct `Content-Length` for the multipart request body, preventing `cURL error 26` (EOF fail) caused by `fstat` returning the encrypted size.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added robust reproduction scripts in `tests/reproduction/` including encryption verification and size debugging.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
This commit fixes a bug where uploading files with Server-Side Encryption (SSE) enabled failed due to a size mismatch between the decrypted stream and the physical file size.

Changes:
- Modified `Source` class to accept an optional `$size` property.
- Updated `IndexerJob` to pass the logical (decrypted) file size to the `Source`.
- Updated `ScanService` to pass the logical file size to the `Source` during manual scans, and correctly handle single file inputs.
- Updated `LangRopeService` to wrap the content stream in a Guzzle `FnStream` and override the `getSize()` method to return the logical size. This ensures Guzzle calculates the correct `Content-Length` for the multipart request body, preventing `cURL error 26` (EOF fail) caused by `fstat` returning the encrypted size.
- Added `getAppApiFunctions` to `LangRopeService` to facilitate testing.
- Added unit tests in `tests/unit/Service/LangRopeServiceTest.php` to verify the fix.
- Added robust reproduction scripts in `tests/reproduction/` including encryption verification and size debugging.

Co-authored-by: TheSpaceGod <17182063+TheSpaceGod@users.noreply.github.com>
Copy link
Contributor

@kyteinsky kyteinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello,
thanks for the PR! And sorry for the delayed review.
it seems there is some issue in my local setup so I can't verify the changes properly, so far I can reproduce the difference in size of the files as in debug_sizes.php.
would you mind copy pasting those lines in the PR description and dropping the tests/reproduction/ folder? The tests in there are better suited in the github actions similar to .github/workflows/integration-test.yml. Simlarly the LangRopeServiceTest and mock_server.log can be dropped too.

// todo: app_api is always available now (composer update)
try {
$appApiFunctions = \OCP\Server::get(\OCA\AppAPI\PublicFunctions::class);
$appApiFunctions = $this->getAppApiFunctions();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for this change, the method getAppApiFunctions() can be dropped too

$file->getMtime(),
$file->getMimeType(),
ProviderConfigService::getDefaultProviderKey(),
(int)$fileSize,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change should be done in lib/BackgroundJobs/SubmitContentJob.php too

Comment on lines +42 to +50
if ($userFolder instanceof File) {
$source = $this->getSourceFromFile($mimeTypeFilter, $userFolder);
if ($source !== null) {
$this->langRopeService->indexSources([$source]);
yield $source;
}
return [];
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be in a separate PR

public int|string $modified,
public string $type,
public string $provider,
public ?int $size = null,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public ?int $size = null,
public int $size,


$params = array_map(function (Source $source) {
$contents = $source->content;
if ($source->size !== null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this check would not be required after the size is always available in the Source class

@kyteinsky
Copy link
Contributor

btw, does the change let the documents that failed to index previously pass now, with a real (dev) setup?

@TheSpaceGod
Copy link
Author

TheSpaceGod commented Feb 19, 2026

btw, does the change let the documents that failed to index previously pass now, with a real (dev) setup?

If the "nextcloud:latest" docker image counts as a "real dev setup", then yes. This code went from:

Running Scan (Direct Indexing) on test.txt...

In LangRopeService.php line 132:
                                                                                                                                                            
  Error during request to Context Chat Backend (ExApp): cURL error 26: client read function EOF fail, only 1048919/1069591 of needed bytes read (see https  
  ://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://context_chat_backend:23000/loadSources                                                          
                                                                                                                                                            

context_chat:scan [-m|--mimetype MIMETYPE] [-d|--directory DIRECTORY] [--] <user_id>


nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 took 38s 
❯ 

To:

Running Scan (Direct Indexing) on test.txt...
[admin] Scanned files/test.txt
Checking backend logs...
Collecting flask
  Downloading flask-3.1.2-py3-none-any.whl (103 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.3/103.3 kB 2.4 MB/s eta 0:00:00
Collecting click>=8.1.3
  Downloading click-8.1.8-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.2/98.2 kB 3.1 MB/s eta 0:00:00
Collecting importlib-metadata>=3.6.0
  Downloading importlib_metadata-8.7.1-py3-none-any.whl (27 kB)
Collecting itsdangerous>=2.2.0
  Downloading itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Collecting markupsafe>=2.1.1
  Downloading markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB)
Collecting jinja2>=3.1.2
  Downloading jinja2-3.1.6-py3-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.9/134.9 kB 10.5 MB/s eta 0:00:00
Collecting werkzeug>=3.1.0
  Downloading werkzeug-3.1.5-py3-none-any.whl (225 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.0/225.0 kB 6.6 MB/s eta 0:00:00
Collecting blinker>=1.9.0
  Downloading blinker-1.9.0-py3-none-any.whl (8.5 kB)
Collecting zipp>=3.20
  Downloading zipp-3.23.0-py3-none-any.whl (10 kB)
Installing collected packages: zipp, markupsafe, itsdangerous, click, blinker, werkzeug, jinja2, importlib-metadata, flask
Successfully installed blinker-1.9.0 click-8.1.8 flask-3.1.2 importlib-metadata-8.7.1 itsdangerous-2.2.0 jinja2-3.1.6 markupsafe-3.0.3 werkzeug-3.1.5 zipp-3.23.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 26.0.1
[notice] To update, run: pip install --upgrade pip
 * Serving Flask app 'mock_server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:23000
 * Running on http://172.18.0.3:23000
Press CTRL+C to quit
172.18.0.2 - - [07/Feb/2026 00:02:16] "GET /heartbeat HTTP/1.1" 200 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "POST /init HTTP/1.1" 404 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "GET /heartbeat HTTP/1.1" 200 -
172.18.0.2 - - [07/Feb/2026 00:02:16] "PUT /enabled?enabled=1 HTTP/1.1" 404 -
172.18.0.2 - - [07/Feb/2026 00:02:25] "PUT /loadSources HTTP/1.1" 200 -
Test completed successfully.

nextcloud_context_chat/tests/reproduction on  fix-sse-size-mismatch-15094702859884413816 via 🐘 v8.5.2 via 🐍 v3.14.2 took 38s 
❯ 

So it shows on that image that it went from a state of not working with encryption to working, via a test performed by the run_test.sh script.

@TheSpaceGod
Copy link
Author

Please note that this PR was mostly a proof of concept and not necessarily production ready code to merge. I am not a PHP developer and you can see that all of this code was developed by Google Jules AI with my guidance as a developer of other software languages.

I think the main thing this PR shows is that LangRopeService usage of Guzzle before was not correctly accounting for encrypted/decrypted file sizes using the Nextcloud VFS.

@TheSpaceGod
Copy link
Author

@kyteinsky,

If you agree with the results of this PR, I would ask you to please take the "ideas" from it and formalize them into a production ready release. I don't really have the project background or the PHP experience to do so. Please let me know if you need any more information or data from this debugging trial.

Thanks!

@kyteinsky
Copy link
Contributor

If the "nextcloud:latest" docker image counts as a "real dev setup", then yes. This code went from:

yes, it does but "Running Scan (Direct Indexing) on test.txt..." and other logs are from the test files which use a dummy context_chat_backend, not the real one. Setup instructions for which can be found here: https://github.com/nextcloud/context_chat_backend?tab=readme-ov-file#dev-install-with-docker
I really would wish to not ask you of all this but it just is not reproducible on my system or even in the github CI.

on a different note, to cover our bases, can you post the output to this from the real instance?
from your observations here: nextcloud/context_chat_backend#176 (comment) it seems to be related to timeouts if one file would exceed the threshold.

occ config:list context_chat | grep request_timeout
docker inspect <docker-socket-proxy-container-name/id> | grep TIMEOUT_SERVER

@github-actions
Copy link

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants