feat: add __arrow_c_stream__ function

[jules-ch]

Description

Add pyarrow capsule method to quickly convert datarray to polars

The function is mostly zero copy, only the coordinates grid need to be computed.

I wanted to implement the __arrow_c_array__ function to return a fixed_shape_tensor but somehow polars prioritize this over __arrow_c_stream__ method.

So for convenience I leave this here for now.

Feel free to close this PR and discuss this further in a dedicated issue if you want.

We can go one step further to save memory with pa.DictionaryArray to use the indice encoding that pyarrow supports out of the box we just need to create the indices using numpy before.

This enable :

>>> import polars as pl
>>> import pyarrow as pa
>>> ds = xr.tutorial.load_dataset("air_temperature")
>>> df = pl.from_arrow(ds.air)
>>> df
shape: (3_869_000, 4)
┌─────────────────────┬──────┬───────┬────────┐
│ time                ┆ lat  ┆ lon   ┆ air    │
│ ---                 ┆ ---  ┆ ---   ┆ ---    │
│ datetime[ns]        ┆ f32  ┆ f32   ┆ f64    │
╞═════════════════════╪══════╪═══════╪════════╡
│ 2013-01-01 00:00:00 ┆ 75.0 ┆ 200.0 ┆ 241.2  │
│ 2013-01-01 00:00:00 ┆ 75.0 ┆ 202.5 ┆ 242.5  │
│ 2013-01-01 00:00:00 ┆ 75.0 ┆ 205.0 ┆ 243.5  │
│ 2013-01-01 00:00:00 ┆ 75.0 ┆ 207.5 ┆ 244.0  │
│ 2013-01-01 00:00:00 ┆ 75.0 ┆ 210.0 ┆ 244.1  │
│ …                   ┆ …    ┆ …     ┆ …      │
│ 2014-12-31 18:00:00 ┆ 15.0 ┆ 320.0 ┆ 297.39 │
│ 2014-12-31 18:00:00 ┆ 15.0 ┆ 322.5 ┆ 297.19 │
│ 2014-12-31 18:00:00 ┆ 15.0 ┆ 325.0 ┆ 296.49 │
│ 2014-12-31 18:00:00 ┆ 15.0 ┆ 327.5 ┆ 296.19 │
│ 2014-12-31 18:00:00 ┆ 15.0 ┆ 330.0 ┆ 295.69 │
└─────────────────────┴──────┴───────┴────────┘

Checklist

• Closes Add a .to_polars_df() method (very similar to .to_dataframe(), which implicitly uses pandas) #10135
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst
• New functions/methods are listed in api.rst

AI Disclosure

• This PR contains AI-generated content.
  • I have tested any AI-generated content in my PR.
  • I take responsibility for any AI-generated content in my PR. Tools: {e.g., Claude, Codex, GitHub Copilot, ChatGPT, etc.}

[jules-ch]

I think we can only use values.ravel down there to ensure contiguous array.

[jules-ch]

Thought

For xr.Dataset I think we can loop over dataarrays convert them to pyarrow tables and join them together in one big pyarrow Table.

It will be sparse if dataarrays does not have the same coords, but that's another PR altogether.

Edit:

Well that worked better than I thought it would:

pl.DataFrame(pa.table(ds))
shape: (2_082_966, 7)
┌───────┬───────┬──────────┬───────────┬───────────────┬──────┬──────┐
│ month ┆ level ┆ latitude ┆ longitude ┆ z             ┆ u    ┆ v    │
│ ---   ┆ ---   ┆ ---      ┆ ---       ┆ ---           ┆ ---  ┆ ---  │
│ i32   ┆ i32   ┆ f32      ┆ f32       ┆ f64           ┆ f64  ┆ f64  │
╞═══════╪═══════╪══════════╪═══════════╪═══════════════╪══════╪══════╡
│ 1     ┆ 200   ┆ 90.0     ┆ -180.0    ┆ 106837.512109 ┆ null ┆ null │
│ 1     ┆ 200   ┆ 90.0     ┆ -179.25   ┆ 106839.237136 ┆ null ┆ null │
│ 1     ┆ 200   ┆ 90.0     ┆ -178.5    ┆ 106837.512109 ┆ null ┆ null │
│ 1     ┆ 200   ┆ 90.0     ┆ -177.75   ┆ 106839.237136 ┆ null ┆ null │
│ 1     ┆ 200   ┆ 90.0     ┆ -177.0    ┆ 106837.512109 ┆ null ┆ null │
│ …     ┆ …     ┆ …        ┆ …         ┆ …             ┆ …    ┆ …    │
│ null  ┆ 200   ┆ null     ┆ null      ┆ null          ┆ null ┆ null │
│ null  ┆ 500   ┆ null     ┆ null      ┆ null          ┆ null ┆ null │
│ null  ┆ 850   ┆ null     ┆ null      ┆ null          ┆ null ┆ null │
│ 1     ┆ null  ┆ null     ┆ null      ┆ null          ┆ null ┆ null │
│ 7     ┆ null  ┆ null     ┆ null      ┆ null          ┆ null ┆ null │
└───────┴───────┴──────────┴───────────┴───────────────┴──────┴──────┘

Will make an another PR if this one get merged.