实现状态#
下表总结了各官方 Arrow 库中可用的功能。所有库目前均遵循 Arrow 格式 1.0.0 版本，或与 1.0.0 兼容的更高次要版本。有关版本控制的详细信息，请参阅 格式版本控制与稳定性。除非另有说明，Python、R、Ruby 和 C/GLib 库均遵循 C++ Arrow 库。
数据类型#
数据类型 (原始类型) |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Null |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Boolean |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Int8/16/32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
UInt8/16/32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Float16 |
✓ |
✓ (1) |
✓ |
✓ |
✓ (2) |
✓ |
✓ |
✓ |
|
Float32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
十进制数32 |
✓ |
✓ |
✓ |
✓ |
✓ |
||||
十进制数64 |
✓ |
✓ |
✓ |
✓ |
✓ |
||||
Decimal128 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
十进制数256 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Date32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Time32/64 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Timestamp |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
持续时间 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
间隔 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
||
定长二进制 (Fixed Size Binary) |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Binary |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
大型二进制 |
✓ |
✓ |
✓ |
✓ |
(4) |
✓ |
✓ |
✓ |
|
Utf8 字符串 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
Large Utf8 |
✓ |
✓ |
✓ |
✓ |
(4) |
✓ |
✓ |
✓ |
|
二进制视图 |
✓ |
✓ |
✓ |
✓ |
✓ |
||||
Utf8 View |
✓ |
✓ |
✓ |
✓ |
✓ |
数据类型 (嵌套类型) |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
定长列表 (Fixed Size List) |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
列表 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
大型列表 |
✓ |
✓ |
✓ |
(4) |
✓ |
✓ |
✓ |
||
列表视图 |
✓ |
✓ |
✓ |
✓ |
✓ |
||||
Large List View |
✓ |
✓ |
✓ |
✓ |
|||||
结构体 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
Map |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
密集联合体 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
稀疏联合类型 (Sparse Union) |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
数据类型 (特殊类型) |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
字典 |
✓ |
✓ (3) |
✓ |
✓ |
✓ |
✓ (3) |
✓ |
✓ |
|
扩展 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
游程编码 (Run-End Encoded) |
✓ |
✓ |
✓ |
✓ |
规范扩展类型 (Canonical Extension types) |
C++ |
Java |
Go |
JavaScript |
C# |
Rust |
Julia |
Swift |
|---|---|---|---|---|---|---|---|---|
定型张量 (Fixed shape tensor) |
✓ |
✓ |
||||||
变型张量 (Variable shape tensor) |
✓ |
|||||||
JSON |
✓ |
✓ |
✓ |
|||||
不透明类型 (Opaque) |
✓ |
✓ |
✓ |
✓ |
||||
UUID |
✓ |
✓ |
✓ |
|||||
8位布尔值 |
✓ |
✓ |
✓ |
|||||
Parquet 变体 (Parquet Variant) |
✓ |
备注
(1) Java 不支持 Float16 的转换。
(2) C# 中的 Float16 支持仅在针对 .NET 6+ 时可用。
(3) 不支持嵌套字典
(4) 提供 C# 大数组类型是为了帮助与其他库进行互操作，但它们不支持大于 2 GiB 的缓冲区，如果尝试导入过大的数组，将会引发异常。
另请参阅
Arrow 列式格式 和 规范扩展类型 规范。
IPC 格式#
IPC 功能 |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|
Arrow 流格式 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ (4) |
Arrow 文件格式 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|
记录批次 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
字典 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
||
替换字典 |
✓ |
✓ |
✓ |
✓ |
✓ |
||||
增量字典 |
✓ (1) |
✓ (1) |
✓ |
✓ |
✓ |
✓ |
|||
张量 |
✓ |
||||||||
稀疏张量 |
✓ |
||||||||
缓冲区压缩 |
✓ |
✓ (3) |
✓ |
✓ |
✓ |
✓ |
|||
字节序转换 |
✓ (2) |
✓ (2) |
✓ (2) |
||||||
自定义模式元数据 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
备注
(1) 嵌套字典不支持增量字典
(2) 读取非原生字节序的数据时，可以自动进行字节交换。
(3) LZ4 编解码器目前效率较低。ARROW-11901 负责跟踪性能改进。
(4) nanoarrow IPC 实现仅实现了 IPC 流的读取功能。
另请参阅
序列化与进程间通信 (IPC) 规范。
Flight RPC#
Flight RPC 传输 |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
|---|---|---|---|---|---|---|---|---|
gRPC 传输 (grpc:, grpc+tcp:) |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
gRPC 域套接字传输 (grpc+unix:) |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
gRPC + TLS 传输 (grpc+tls:) |
✓ |
✓ |
✓ |
✓ |
✓ |
gRPC 传输中的支持功能
Flight RPC 功能 |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
|---|---|---|---|---|---|---|---|---|
所有 RPC 方法 |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
身份验证处理程序 |
✓ |
✓ |
✓ |
✓ (1) |
✓ |
|||
调用超时 |
✓ |
✓ |
✓ |
✓ |
||||
调用取消 |
✓ |
✓ |
✓ |
✓ |
||||
并发客户端调用 (2) |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
自定义中间件 |
✓ |
✓ |
✓ |
✓ |
||||
RPC 错误代码 |
✓ |
✓ |
✓ |
✓ |
✓ |
备注
(1) 支持使用 AspNetCore 身份验证处理程序。
(2) 单个客户端是否支持多个并发调用。
另请参阅
Arrow Flight RPC 规范。
Flight SQL#
注意
Flight SQL 目前仍处于实验阶段。
功能支持仅指客户端/服务器库；实现 Flight SQL 协议的数据库本身可能会也可能不会支持个别功能。
功能 |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
|---|---|---|---|---|---|---|---|---|
BeginSavepoint |
✓ |
✓ |
||||||
BeginTransaction |
✓ |
✓ |
||||||
CancelQuery |
✓ |
✓ |
||||||
ClosePreparedStatement |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
CreatePreparedStatement |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
CreatePreparedSubstraitPlan |
✓ |
✓ |
||||||
EndSavepoint |
✓ |
✓ |
||||||
EndTransaction |
✓ |
✓ |
||||||
GetCatalogs |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetCrossReference |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetDbSchemas |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetExportedKeys |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetImportedKeys |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetPrimaryKeys |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetSqlInfo |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetTables |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetTableTypes |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
GetXdbcTypeInfo |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
PreparedStatementQuery |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
PreparedStatementUpdate |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
StatementSubstraitPlan |
✓ |
✓ |
||||||
StatementQuery |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
StatementUpdate |
✓ |
✓ |
✓ |
✓ |
✓ |
另请参阅
Arrow Flight SQL 规范。
C 数据接口#
功能 |
C++ |
Python |
R |
Rust |
Go |
Java |
C/GLib |
Ruby |
Julia |
C# |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Schema 导出 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
||
数组导出 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
||
Schema 导入 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
||
数组导入 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
另请参阅
C 数据接口 规范。
C 流接口#
功能 |
C++ |
Python |
R |
Rust |
Go |
Java |
C/GLib |
Ruby |
Julia |
C# |
Swift |
nanoarrow |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
流导出 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
|||
流导入 |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
✓ |
另请参阅
C 流接口 规范。
第三方数据格式#
格式 |
C++ |
Java |
Go |
JS |
C# |
Rust |
Julia |
Swift |
|---|---|---|---|---|---|---|---|---|
Avro |
R |
R |
读/写 |
|||||
CSV |
读/写 |
R (2) |
读/写 |
读/写 |
读/写 |
|||
ORC |
读/写 |
R (1) |
||||||
Parquet |
读/写 |
R (2) |
读/写 |
读/写 |
备注
R = 支持读取
W = 支持写入
(1) 通过 JNI 绑定。（由
org.apache.arrow.orc:arrow-orc提供）(2) 通过 JNI 绑定到 Arrow C++ Datasets。（由
org.apache.arrow:arrow-dataset提供）