feat(format): add generic metadata API

[lidavidm]

• Add a generic API for fetching metadata, modeled on ADO.NET's GetSchema
• Define the standard metadata collections
• Revert the current revisions to GetObjects in favor of not changing it

Related:

• format: provide way to escape wildcards in GetObjects and other functions #3997: define a way to escape search pattern arguments
• discussion: ConnectionGetObjects vs ConnectionGetTableSchema #1704 (comment): define a way to provide pagination tokens

For consideration:

• Should filters be (optionally) named arguments? (Pointer to struct) The justification: we could more easily implement/define "standard" arguments to handle things like the pagination token above without having to replicate it into every single definition
• Should filters be typed? (Pointer to struct containing union)
• Should filters be replaced by just leveraging SetOption(Int,...)? (This would obviate the former two points but is rather clunky, and precludes a hypothetical driver that could handle these operations concurrently)

Closes #4400.

[lidavidm]

Also CC @iconara

I've also been thinking about your request for stateless pagination, and I think it's reasonable enough to define a way to get/pass a pagination token. I think there are enough systems that could use it: Athena, BigQuery, Databricks, Snowflake, Iceberg REST catalog, etc.

[lidavidm]

And also CC @mullinsms, who kicked all this off by listing all the types of metadata we didn't support 🙂

In particular, Curt's suggestion means we can support database-specific metadata (though, I'm still of the opinion that something truly database-specific is probably better handled by the application; if the application has a very specific metadata query it wants to issue I'm not sure how useful it is to build it into the driver).

[lidavidm]

At the risk of overcomplicating things: similar to #3623, it might be nice to have a way to request that certain extra fields be included/omitted, e.g. table properties (#3995), where it may be efficient to fetch the data at the same time as the "standard" fields, but where some (many) clients also may not want the field. Maybe it could optionally take an Arrow schema as input for that.

Similarly, maybe the application wants to opt in to run-length-encoding certain response columns to save memory. (Or is that not really a concern so long as things are properly streamed/paginated?)