大网数据广告活动
服务器运维问答

IPoIB (IP over InfiniBand) vs. RDMA performance

· 52CE 我爱测

I have partly inherited a Linux HA cluster at the center of which currently serves a connection with DRBD 8 over IPoIB (IP over InfiniBand) between two Debian hosts. It ain't broken, so I won't fix it. I have also noticed that DR…

问题原文

原帖链接 · rookie09 · CC BY-SA 3.0

I have partly inherited a Linux HA cluster at the center of which currently serves a connection with DRBD 8 over IPoIB (IP over InfiniBand) between two Debian hosts. It ain't broken, so I won't fix it.

I have also noticed that DRBD 9 supports RDMA, so the question may come up whether to replace the connection with DRBD 9 over RDMA (i.e. "native" InfiniBand) in the future.

Since I don't want to run performance tests on a production system I am wondering: Are there published performance comparisons for IPoIB vs. RDMA/InfiniBand. For instance, could I expect bandwidth/latency gains from switching away from IPoIB in the orders of magniute of either 10%, 50%, or 100%, say? What could be expected?

已采纳解答

原帖链接 · Tux_DEV_NULL · CC BY-SA 3.0

have you seen these presentations? https://www.linbit.com/en/drbd-9-over-rdma-with-micron-ssds/ http://downloads.openfabrics.org/Media/Monterey_2015/Tuesday/tuesday_09_ma.pdf

InfiniBand is just a specific network architecture offering RDMA but your performance will depend on what type of applications your are running. My experience is based on academic/research systems mostly using MPI based applications. In certain cases I have seen RDMA performing 20% better than IPoIB. But I am not aware of any such benchmarking but there are plenty of academic papers written and also vendor white papers. If you are just thinking about I/O then consider: file sizes, number of reads vs. writes. RDMA usually provided a big benefit for random small reads but only a small benefit for writes. You might want to read up on RoCE (RDMA over Converged Ethernet) and InfiniBand native RDMA.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2017-10-02)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。