Opt-in API that has a cached `ThreadpoolController`

[itamarst]

Creating a controller is still expensive on Linux at least, so for libraries that do this a lot a cached version that notices when libraries change might be useful.

My thought is a top-level API get_cached_controller(force_new=False), that checks if e.g. the length of sys.modules has changed. If yes, it creates (and caches) a new controller, otherwise it returns a cached one. With force_new=True it always invalidates.

Application authors would presumably know that there's no other mechanism where new libraries might get loaded.

I would like to use this for joblib, on the theory that:

1. Joblib never had this oversubscription mitigation before.
2. This will catch the vast majority of cases where new libraries get added.
3. In the rare cases it doesn't catch a loaded library, it's not a bug, it's just a performance hit... and again, joblib never solved this before anyway so those users are no worse off.

[itamarst]

For FlexiBLAS one could have custom logic invalidation if it's loaded.

[itamarst]

Benchmarks should indicate if this is necessary, or if already done optimizations are sufficient to make threadpoolctl unnecessary in cases where it's used a lot. Benchmarks should be done on Linux, Windows, macOS though.