Thread

What if Jev spoke Arrow?

44 points · 5 comments · ianmcook

Loading the complete thread in the background. This saved snapshot is available now. Refresh

orf · 1790708140 · focus · HN ↗
[delayed]
1. mihevc · 1790708848 · focus · HN ↗
  grpc is an RPC framework and Arrow IPC is a serialization format. You can use Arrow Flight which uses grpc to send Arrow IPC. But I suppose TypeSafe doesn't provide grpc endpoints yet?
  1. orf · 1790708974 · focus · HN ↗
    [delayed]
  2. billsmithaustin · 1790952095 · focus · HN ↗
    gRPC uses Protocol Buffers for serialization. You can serialize with Protocol Buffers without making remote procedure calls with gRPC.
2. mihevc · 1790709733 · focus · HN ↗
  If I read the example correctly they are using long living http/2 connection to send arrow IPC. Which seems pretty pragmatic.
3. ianmcook · 1790710702 · focus · HN ↗
  Agreed. With the API as it is today, the bottleneck is the thousands of round trips (one per state). The post is a bit of a "what if" where we're imagining (and hoping!) that TypeSafe adds a batch endpoint, which would take the round trips out of the picture. Then serialization would start to matter, and that's where Arrow would come in
  It would be straightforward to wrap this as an Arrow Flight (Arrow over gRPC) endpoint. But it would make it heavier and less browser-compatible.

Unofficial Hacker News client; not affiliated with Y Combinator.