大网数据广告活动
服务器运维问答

Do file systems exist which can intelligently handle redundant blocks for saving disk space and instantaneous copying?

· 52CE 我爱测

If I have a very large contiguous file in the gigabytes that I want to copy, my disk has to allocate all of the necessary space and write a duplicate of every block. Why can't a copy be "fast" in a sense that it instead copies re…

问题原文

原帖链接 · Zhro · CC BY-SA 4.0

If I have a very large contiguous file in the gigabytes that I want to copy, my disk has to allocate all of the necessary space and write a duplicate of every block.

Why can't a copy be "fast" in a sense that it instead copies references to the blocks and writes new blocks only when a change is made?

I understand that this would lead to a decoupling of how much data is on disk (due to block references) with the potential of a disk appearing to contain data in excess of its actual capacity. This could also result in a large amount of space being taken up on writes due to entirely new blocks having to be written when they change from their source block.

There would certainly be unique penalties to such a file system but it sounds like an interesting use case.

Are there any file systems existing today which exploit a similar way of handling data?

Note that I am not an expert on file systems so some of my assumptions may be embarrassingly wrong. I welcome any corrections in the comments.

已采纳解答

原帖链接 · Andrew Henle · CC BY-SA 4.0

What you are referring to is a reflink. Per the Linux cp man page:

When --reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails, or if --reflink=auto is specified, fall back to a standard copy. Use --reflink=never to ensure a standard copy is performed.

On Linux, this is implemented by the FICLONE ioctl() call:

If a filesystem supports files sharing physical storage between multiple files ("reflink"), this ioctl(2) operation can be used to make some of the data in the src_fd file appear in the dest_fd file by sharing the underlying storage, which is faster than making a separate physical copy of the data. Both files must reside within the same filesystem. If a file write should occur to a shared region, the filesystem must ensure that the changes remain private to the file being written. This behavior is commonly referred to as "copy on write".

Reflinks are supported for BTRFS, and XFS in Linux kernel 4.8:

Linux kernel 4.8 in August 2016 added a new feature, "reverse mapping". This is the foundation for a large set of planned features: snapshots, copy-on-write (COW) data, data deduplication, reflink copies, online data and metadata scrubbing, highly accurate reporting of data loss or bad sectors, and significantly improved reconstruction of damaged or corrupted filesystems. This work required changes to XFS's on-disk format.

cp -z ... and the reflink() function are available on Solaris 11.4 for ZFS. ZFS reflink support will presumably be available in OpenZFS and in ZFSonLinux at some time. See https://github.com/zfsonlinux/zfs/issues/405

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2019-09-25)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。