Epic: List unification

[robert3005]

All lists should be lists. Distinction between FixedSizedList and List was a mistake.

Goal

For certain operations it's more efficient to convert FixedSizedList to a ListView, however, because of dtypes we cannot do it right now. FixedSizedList is just a bit of metadata that shouldn't preclude us from doing these optimisations. By removing FixedSizedList dtype and instead looking at the physical encoding information we would be able to achieve it.

This is particularly painful right now in take operations where we would like all types to be freely reorderable but FixedSizedList requires copying all values in order to do it.

[connortsui20]

I feel like this needs much more justification. Maybe there is good reason for this, but just from reading the issue description, we are saying we need to make this massive change for the sake of take optimizations? I would say removing a DType is probably THE most breaking and destructive change we could possibly make in Vortex, and it is not clear to me why this specific operation is so important and unsolvable unless we completely remove FSL?

IIUC the issue here is that the canonical encoding of FSL is fully contiguous, and so any intermediate destructive operations like take and filter would have to make lots of memcpys. The solution this issue proposes is that we completely remove FSL and replace it with ListView so that we can reorder and overlap data internally in the type. But I guess my question is: why not just keep a DictArray on top of the FixedSizeList array tree and do our best not to apply the compute unless absolutely necessary? AFAICT that does the exact same thing (and really I think we want this functionality regardless of this specific issue, as it is generally useful to keep things encoded as Dict).

One of the other reasons I have such a visceral reaction to this proposal is that I like to compare Vortex's type system to how a standard type system in a statically-typed programming language is. To me, this proposal is akin to saying "we are going to remove fixed-length arrays ([T] in Rust) from this language because variable-size lists (Vec<T>) do a similar thing".

Removing FSL seems like a huge oversimplification of other issues in Vortex (lazy execution into specific physical encodings probably being the main one), and I would like to see some more justification and reasoning before we decide to move forwards with this massive change.