大网数据广告活动
服务器运维问答

Ceph or Gluster for implementing big NAS

· 52CE 我爱测

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as a block storage for virtual m…

问题原文

原帖链接 · prema · CC BY-SA 4.0

We are planning to build NAS solution which will be primarily used via NFS and CIFS and workloads ranging from various archival application to more “real-time processing”. The NAS will not be used as a block storage for virtual machines, so the access really will always be file oriented.

We are considering primarily two designs and I’d like to kindly ask for any thoughts, views, insights, experiences.

Both designs utilize “distributed storage software at some level”. Both designs would be built from commodity servers and should scale as we grow. Both designs involve virtualization for instantiating "access virtual machines" which will be serving the NFS and CIFS protocol - so in this sense the access layer is decoupled from the data layer itself.

First design is based on a distributed filesystem like Gluster or CephFS. We would deploy this software on those commodity servers and mount the resultant filesystem on the “access virtual machines” and they would be serving the mounted filesystem via NFS/CIFS.

Second design is based on distributed block storage using CEPH. So we would build distributed block storage on those commodity servers, and then, via virtualization (like OpenStack Cinder) we would allocate the block storage into the access VM. Inside the access VM we would deploy ZFS which would aggregate block storage into a single filesystem. And this filesystem would be served via NFS/CIFS from the very same VM.

Any advises and insights highly appreciated. I should also say that we are internally inclined towards the "monster VM" approach due to seemingly simpler architecture (data distribution on block layer rather than on file system layer).

Cheers, Prema

已采纳解答

原帖链接 · dario · CC BY-SA 4.0

First design:

Gluster + (NFS OR GaneshaNFS) in cluster

No access VM. In this case Gluster has simpler architecture than CephFS. Gluster has some rules regarding adding nodes and capacity. It's ok, just plan for it from start.

Second design:

If your goal is to have single access VM to provide NFS/CIFS, Linux can mount Ceph as block device. So you have stack like this:

NFS/CIFS in Linux -- Ceph RBD

If you require HA for access VM, then add HA cluster:

NFS/CIFS in Linux HA cluster -- Ceph RBD

Or, instead of Ceph RBD you can use Ceph iSCSI gateway.

Things to consider:

  1. scaling up
  2. data protection: 2 or 3 copies, erasure/sharding
  3. for decent performance use enterprise SATA and SSD disks
  4. online/offline upgrade
  5. other solutions: eg. DRBD

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2018-11-12)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。