[Bug]: Chunked upload with S3 primary storage records size 0 in filecache

[nicolassp]

Chunked upload to S3 primary storage records size = 0 in oc_filecache

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on GitHub (I've searched it).
• Nextcloud Server is up to date.
• I agree to follow Nextcloud's Code of Conduct.

---

Bug description

With S3 primary storage, every chunked upload writes the object to the bucket
correctly but records size = 0 in oc_filecache.

The data is intact — the S3 object has the right byte count and the chunks are
assembled properly. Only the filecache entry is wrong. As a result:

• the file shows as empty in the web UI
• it is not counted toward the user's quota
• WebDAV clients abort with corrupted on transfer: sizes differ … vs dst 0
• MOVE returns 201 Created, so nothing signals a failure
• nothing is written to nextcloud.log at any log level

A single (non-chunked) PUT records the size correctly. Only the chunked path
is affected.

Steps to reproduce

Four plain curl calls are enough — no client involved:

USER=<user>
PASS=<app password>
BASE=https://<host>/remote.php/dav
UP=$BASE/uploads/$USER/test
DST=$BASE/files/$USER/out.bin

dd if=/dev/urandom of=/tmp/p1 bs=1M count=10
dd if=/dev/urandom of=/tmp/p2 bs=1M count=10

curl -X MKCOL -u "$USER:$PASS" "$UP"
curl -T /tmp/p1 -u "$USER:$PASS" "$UP/00000000000000000000-00000000010485759"
curl -T /tmp/p2 -u "$USER:$PASS" "$UP/00000000010485760-00000000020971519"
curl -X MOVE  -u "$USER:$PASS" -H "Destination: $DST" "$UP/.file"

All four return 201.

Expected behaviour

oc_filecache.size for out.bin is 20971520.

Actual behaviour

$ psql -c "SELECT fileid, size FROM oc_filecache WHERE name='out.bin'"
 fileid | size
--------+------
  59233 |    0

$ aws s3api head-object --bucket <bucket> --key "urn:oid:59233" --query ContentLength
20971520

The object is complete in S3. The filecache says zero.

Root cause

lib/private/Files/ObjectStore/ObjectStoreStorage.php, writeStream():

public function writeStream(string $path, $stream, ?int $size = null): int {
    if ($size === null) {
        $stats = fstat($stream);
        if (is_array($stats) && isset($stats['size'])) {
            $size = $stats['size'];
        }
    }
    …
    $stat['size'] = (int)$size;

On the final MOVE of a chunked upload, the caller passes $size = 0, not
null. The strict === null check does not match, fstat() is never called,
and (int)0 is written to the filecache.

fstat() would have returned the correct value. AssemblyStream implements
stream_stat() and reports the summed size of the chunks.

Instrumentation that shows it

Adding a log line at the top of writeStream():

@file_put_contents('/tmp/ncdbg.log', sprintf(
    "%s path=%s in=%s out=%s fstat=%s\n",
    date('H:i:s'), $path, var_export($dbgIn, true),
    var_export($size, true), json_encode(@fstat($stream))
), FILE_APPEND);

produces:

path=uploads/test/00000000000000000000-00000000010485759  in=10485760  out=10485760  fstat=false
path=files/out.bin                                        in=0  out=0
    fstat={"0":0,…,"7":10485760,…,"size":10485760,…}

The chunk PUT passes a correct size. The final assembly passes 0, while
fstat() on the same stream reports 10485760 — it is simply never consulted.

Proposed fix

Treat 0 as "size unknown" and fall back to the stream, trusting it only when
it reports a positive value:

-        if ($size === null) {
-            $stats = fstat($stream);
-            if (is_array($stats) && isset($stats['size'])) {
+        if ($size === null || $size === 0) {
+            $stats = @fstat($stream);
+            if (is_array($stats) && isset($stats['size']) && $stats['size'] > 0) {
                 $size = $stats['size'];
             }
         }

With this applied, the same reproduction records 20971520 and WebDAV clients
complete normally. Verified at 64 MB, 256 MB, 500 MB and 1 GB.

I am aware this is a symptom-level fix — the real question is why the caller
passes 0 instead of null, or why the assembled size is not propagated. But
the guard is safe: a stream that genuinely holds zero bytes reports
size => 0, fails the > 0 test, and $size stays 0 as before.

Impact

Before the fix, a 1 GB chunked upload took 2 m 29 s and ended with a corrupted
filecache entry; the browser retried repeatedly. After the fix the same upload
completes in one pass. Effective web UI upload throughput on this instance went
from ~6.5 MB/s to ~59 MB/s — the earlier figure was mostly failed retries.

Server configuration

Nextcloud version: 34.0.3.2
Operating system: Ubuntu 26.04
Web server: Caddy → PHP-FPM (official nextcloud:fpm image)
Database: PostgreSQL 16
PHP version: 8.5.10
Primary storage: S3 object storage (Backblaze B2, s3.eu-central-003.backblazeb2.com)

Relevant config:

files.chunked_upload.max_size          104857600
files.chunked_upload.max_parallel_count       10
objectstore.arguments.uploadPartSize   209715200
objectstore.arguments.concurrency             10
memcache.local                         \OC\Memcache\APCu
memcache.distributed                   \OC\Memcache\Redis
encryption                             disabled

Not AIO. Server-side encryption is off, no antivirus app, no SSE-C.

Logs

nextcloud.log is empty for the failing request at every log level,
including loglevel = 0. The MOVE returns 201, so there is no error path
to log. This is part of what makes the bug hard to find: the only visible
symptom is a client-side size mismatch.

Additional notes

Chunking v2 (ChunkingV2Plugin) is not affected by the bug above — it uses S3
multipart uploads directly and never reaches that code path.

Secondary observation: chunking capability is hardcoded to 1.0

apps/dav/lib/Capabilities.php:29:

'dav' => [
    'chunking' => '1.0',

The developer manual states that "Version 2 is the recommended version to use",
ChunkingV2Plugin is registered unconditionally in apps/dav/lib/Server.php,
and OC\Files\ObjectStore\S3 implements IObjectStoreMultiPartUpload — yet
the advertised capability is 1.0, so clients default to v1 and hit the bug
described above.

Driving v2 manually (adding a Destination header to MKCOL, each PUT and
the MOVE, with numeric chunk names) works and is dramatically faster on the
assembly step:

200 MB upload, same instance	MOVE	total
v1	12.1 s	~20 s
v2	0.4 s	~3.4 s

With v1 the chunks are read back from the object store and rewritten; with v2
the final MOVE is a CompleteMultipartUpload.

Is 'chunking' => '1.0' intentional, or should it follow what the storage
backend actually supports? If it is deliberate — e.g. because not every client
handles v2's constraints (5 MB minimum part size, numeric chunk names,
Destination on every request) — it would help to document that, since the
current combination means the recommended path is never taken by default.

I changed it to '2.0' locally. Clients fall back to v1 gracefully when
checkPrerequisites() fails, so nothing broke, but I have not measured a
throughput gain for browser uploads: those are network-bound here and the
parallel chunk count dominates. The gain is in server-side work, not wall
clock, for that particular workload.