Optimize parallel workers start-up

[ilevkivskyi]

This is a micro-optimization that gives fixed 50-100ms performance improvement. It is mostly important for "simple" runs, where one only modifies few files. Such runs are often sub-second, where this will be a visible win.

Two ideas here:

• Start workers in background, but don't wait for them to become ready until they are actually needed.
• Broadcast graph/SCCs data to all workers in parallel.

I am actually not 100% sure about second one, but I guess it should help in big code-bases. IIUC sockets are I/O, so it should be possible to speed it up using threads. I am also adding a bit more logging, so that we get more insight into communication overhead.

cc @JukkaL

[ilevkivskyi]

I decided to also increase our read size limit (just in case). Btw this makes me think that maybe we should set the same socket buffer size on Mac and Linux. On Linux it is fixed to 212992 bytes. While on Mac it is flexible/floating between 8192 bytes (checked using s.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF) on a fresh socket), and 8388608 bytes (value of sysctl kern.ipc.maxsockbuf). What may be happening is that Mac gets seriously confused trying to adjust buffer size, since we periodically sent very small (few bytes) and very large (few megabytes) payloads. So we may want to limit this behavior by setting some large lower limit with setsockopt(). Anyway, I will probably try this later in a separate PR.

[ilevkivskyi]

I played a bit with the buffer size, and didn't see much improvement on Mac. However it definitely didn't get worse, so I decided to set same buffer size just so that it is easier to reason about.

I also discovered some timeout inconsistency when worker receives data from coordinator. Worker should be allowed to be idle for the same time we allow other worker to work (in case there is a bottleneck SCC). This adds an extra syscall, but I propose to start with correctness and optimize for performance later.

I add a bunch of perf stats here, so I am going to merge this soon, and rebase the splitting PR.

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅