问题原文
原帖链接 · Barmi · CC BY-SA 3.0
What is the best way to synchronize huge data of a running production server?
Our server has over 20 million files (small files with 10k and bigger files up to 50MB) stored in 1 millon directories. The size of all data is about 5 TB (steadily increasing).
Is it possible to synchronize the data with lsyncd and what are the limits (especially of inotify)? How much additional space does lsyncd need? What about the load (cpu and memory) and the live time?
Another solution would be GlusterFS. Is it possible to use GlusterFS on a production with none or minimal downtime? GlusterFS stores a lot of magic data in x-attributes files and the storage volume is about 15 to 20% bigger than systems with non GlusterFS. Seems like a huge amount of waste...? What about the load?
And at least rsync and cronjobs could do the job. rsync would only run on the slave... So no additional space is needed on the primary server, but rsync must read the full directory tree every time the cron runs...
社区高分解答
原帖链接 · ewwhite · CC BY-SA 3.0
I'd seriously recommend using something like ZFS for the filesystem.
Built-in tools like the ZFS snapshot and ZFS send/receive allow you to take block-level snaps of the filesystem and ship it to a second server.
Some third party tools like sanoid/syncoid can set automatic management/pruning and synchronization of your filesystem from one host to another.
This is done at the block device level, so you avoid the rsync checksum/inventory process.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
