大网数据广告活动
服务器运维问答

Weekly RAID check affecting my system - any way to mitigate?

· 52CE 我爱测

I recently got my webhost (Hetzner) to add a paid of 16TB SATA drives to my webserver. Currently using 2.5TB of them. They're RAID 1 mirrored. I also have two 4TB nVME drives with 700GB currently on them, also RAID 1 mirrored. Ev…

问题原文

原帖链接 · Codemonkey · CC BY-SA 4.0

I recently got my webhost (Hetzner) to add a paid of 16TB SATA drives to my webserver. Currently using 2.5TB of them. They're RAID 1 mirrored.

I also have two 4TB nVME drives with 700GB currently on them, also RAID 1 mirrored.

Every week CentOS kicks off a cronjob to run a "check" on both of my md arrays. They happen concurrently, with the nVME one finishing after 5 hours. The SATA one takes a painful 18hours, at 200MB/sec the whole time.

# Run system wide raid-check once a week on Sunday at 1am by default
0 1 * * Sun root /usr/sbin/raid-check

My server is plenty powerful, with a 32core EPYC and 128GB of ram, but I do notice an IO slowdown when this check is running.

  1. is it necessary to run these weekly?

  2. 200MB/sec * 18 hours means it's doing the whole 16TB, not just the occupied space. Can this be made smarter/lazier in any way, so it only runs on the occupied space?

  3. could this job be niced or similar? I appreciate it would take longer, but that might be preferable. see edit below

  4. would scripting pauses into this be a bad idea? So instead of 18 hours in one hit I could do (say) 3 hours per night?

  5. is this a problem everyone suffers, or have I made some poor decisions? Would getting a hardware raid card installed make me much happier, for example?

Edit

I have now discovered /etc/sysconfig/raid-check and changed NICE=low to NICE=idle. I guess I won't know what difference that makes until next week.

已采纳解答

原帖链接 · Nikita Kipriyanov · CC BY-SA 4.0

No, MD RAID can't be smarter than this. If you want to only check used areas, use ZFS, or perhaps BTRFS.

Weekly check is too often. Do this on monthly basis, or even every other month.

I don't know what this NICE really does. If it's setting the I/O nice of the [mdX_resync] kernel process, that's good and use idle. What you can limit is the bandwidth of the check: it's set in the /sys/block/mdX/md/sync_speed_max file in kB/s. This is a virtual file, e.g. it'll be reset after system restart.

By the way, it's limited at 200 MB/s by default and you seem to hit that limit. You may increase speed for SSDs (set 5000000 and and see in what time they will be checked). And instead "pausing" it for HDDs, I'd play with limits (e.g. during periods of high load I'd set lower limit, during idle time I'd set 600000 — SATA 6 Gb/S interface maximum bandwidth).

I doubt HW RAID card will make things much better.

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2022-05-12)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。