Configurable `SftpFileStream.MaxPendingReads`

[GMMan]

Is it possible to make SftpFileStream.MaxPendingReads a configurable value? I'm currently dealing with a SFTP implementation by a third party integration that applies some special behavior when the file has been read, and it seems like it might be doing so when the read hits EOF. Since this library spams read requests at the server when the file is being downloaded and the size is known, I'm suspecting the server is applying the behavior early due to a race condition, since there's no requirement in the protocol to process requests in the same order as received (actually there is for the same file, but it also mentions something about parallelization that makes this slightly ambiguous) or reads to occur linearly. I don't think the third party will be able to change their implementation soon, so second best thing would be to configure it in this library so there's only one concurrent request, ideally without having to fork and maintain my own copy.

[mus65]

There already is an open PR for this: #1796

[Rob-Hague]

I would be fine with it but I would like to see some evidence that it is actually the problem. OpenSSH client also sends lots of requests in parallel, and I don't personally see any ambiguity in the spec (I am looking at https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02#section-6.1)

[GMMan]

I don't think it has any conflicts with the spec, but sometimes real world applications don't behave well and it would be useful to have some control over parameters that might help alleviate symptoms. Said third-party replied to me today that they're using a vendor-provided implementation that should be protocol compliant, and they see it working fine with most clients. (Basically they move files to an archive folder after they've been read, and most of the time I'm seeing the last chunk missing, so my guess is they are applying the action on EOF read and an out of order read request would cause this.)

#1795 is another issue that demonstrates adjusting the parameter helps in their case.

[mus65]

The only behaviour difference I see is that OpenSSH seems to always start with 1 request:

https://github.com/openssh/openssh-portable/blob/0ef0f5a839831c213f24e3f2ae434765c607fb50/sftp-client.c#L1666

SSH.NET on the other hand may start with a lot, depending on file size since #1705 .

SSH.NET/src/Renci.SshNet/Sftp/SftpFileStream.cs

Line 312 in f7d2ac1

var initialPendingReads = (int)Math.Max(1, Math.Min(MaxPendingReads, 2 + (attributes.Size / readBufferSize)));

Also, OpenSSH maxes out at 64 (DEFAULT_NUM_REQUESTS) by default instead of 100 .

Not saying SSH.NET has a bug, but maybe that explains why only SSH.NET triggers it.

@GMMan Can you also reproduce it if you use OpenAsync and copy the stream manually instead of using DownloadAsync? That would skip the the mentioned initialPendingReads calculation and start with 1 like OpenSSH does.