问题原文
原帖链接 · Seth · CC BY-SA 4.0
I'm trying to figure out the lowest hassle way to provision 24x locally attached SSDs as a large logical volume with low-value data. I'm using them as a hot-set cache for data who's master state (about a petabyte) resides in S3, so I care more about performance, complexity of maintenance, and downtime more than lost data. Nothing will linger in the hot data set for more than a couple days, and its all easy to recreate from S3 anyway.
- Medium large instance: 32x vCPUs, 120GB RAM, Skylake
- 24x locally attached SSDs @ 375GB each = 9TB total
- Hosted on Google Cloud (GCP)
- Debian 10 (Buster)
- Access is ~4x heavier on read than write
- High number of concurrent users (human and machine) with pretty random access patterns, and very hungry for I/O.
- 90% of files are larger than 10MB
I'm thinking RAID 5 is out of the question, no chance I'm going to wait for manual rebuilds. I'm inclined toward either RAID 0, RAID 10, or.... maybe this is actually a case for a simple LVM pool with no RAID at all? Do I really lose anything by going that relatively simpler route in this case?
My ideal solution would have each subdir (I have one self contained dataset per subdir) of / completely contained on a single disk (I can fit maybe 10 subdirs on each drive). If a drive failed, I'd have a temporary outage of the subdirs/datasets on that drive, but an easy to reason about set of "these data sets are redownloading and not available". Then I'd just rebuild the missing data sets from S3 on a new drive. I suspect LVM jbods (not sure of exactly the right word for this?) might come closest to replicating this behavior.
社区高分解答
原帖链接 · davidgo · CC BY-SA 4.0
You appear to be contradicting your needs - "My ideal solution would have each subdir (I have one self contained dataset per subdir) of / completely contained on a single disk" tells you that you don't want RAID, LVM or any abstraction technology - *surely the solution to this is would be to simply mount each disk individually. The disadvantage here is you are likely to waste disk space and if the data set grows you will need to spend more time juggling it. (I expect you know Unix can mount drives in arbitrary places of a filesystem tree, so with a bit if thought it should be easy enough to make the drives visible as a logical tree structure)
You talk about JBOD or RAID0. If you do decide for a combined disk solution, RAID0 will give you better read performance in most cases, as data is broken up over the disks easily. RAID10 would buy you redundancy you said you don't need. JBOD is only useful to you if you have disks of different sizes, and you would be better off using LVM instead, as it can behave the same way but give you flexibility to move data around.
I can see edge cases where LVM would help over individual disk, but in general, any scenario is likely to add more complexity then it gives useful flexibility here - particularly bearing in mind the initial statement about data sets being bound to disks.
Where you might want to spend some effort is looking at the most appropriate file system and tuning parameters.
摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。
