大网数据广告活动
服务器运维问答

How do "correctly" sync millions of files with rsync and inotify?

· 52CE 我爱测

Assume I have 1 million files, the file list would be something like 20MB. If I add just one file, inotify will tell rsync to transfer the file list and the new file. My network is not good and I might have to limit the bandwidth…

问题原文

原帖链接 · seanlook · CC BY-SA 3.0

Assume I have 1 million files, the file list would be something like 20MB. If I add just one file, inotify will tell rsync to transfer the file list and the new file. My network is not good and I might have to limit the bandwidth (--bwlimit).

Multiple files might be added at once and, given the network transfer speed, there may be multiple rsyncs running at once.

Is it worth it? Is there a better way?

已采纳解答

原帖链接 · faker · CC BY-SA 4.0

Your case sounds exactly what lsyncd was made for.
It watches the directory with inotify and spawns an rsync with only the changed file(s).
It aggregates changes for a few seconds.
So when you make 5 changes in one second it will only spawn one new sync process to the destination server(s).

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2015-05-05)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0 / CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。