大网数据广告活动
服务器运维问答

ZFS: Does anyone know how to roll a zvol back in time?

· 52CE 我爱测

I have a ZFS based iSCSI SAN that serves ZVOLs to a bunch of VM servers. Today a network glitch caused all the iSCSI volumes mounted on the clients to go RO. The only way out of that is to shut them all down and restart, often ru…

问题原文

原帖链接 · tlum · CC BY-SA 3.0

I have a ZFS based iSCSI SAN that serves ZVOLs to a bunch of VM servers. Today a network glitch caused all the iSCSI volumes mounted on the clients to go RO. The only way out of that is to shut them all down and restart, often running fsck to get the iSCSI volumes back on-line. Well, fsck decided to thoroughly destroy one of the volumes. So, it does not look like I'm going to be able to fix the mess fsck made.

I've read quite a bit about recovering files on ZFS, however, in this case I'm dealing with ZVOLs, which in a sense is a lot simpler, but I haven't seen anything dealing with trying to roll back the contents of a block device. Any suggestions?

-TIA-

Some dataset details:

Dataset zpool1/vm3 [ZVOL], ID 59, cr_txg 12078, 44.6G, 2 objects, rootbp DVA[0]=<6:6c2c4b1e00:200> DVA[1]=<7:487aa4b200:200> [L0 DMU objset] fletcher4 lz4 LE contiguous unique double size=800L/200P birth=7736596L/7736596P fill=2 cksum=4c78779ec:2049fb2de6c:6f2f6c4a44e9:1042484aee3ded

    Deadlist: 1K (512/512 comp)

mintxg 0 -> obj 48
mintxg 1 -> obj 4157

    Object  lvl   iblk   dblk  dsize  lsize   %full  type
         0    7    16K    16K  7.00K    16K    6.25  DMU dnode
        dnode flags: USED_BYTES
        dnode maxblkid: 0

    Object  lvl   iblk   dblk  dsize  lsize   %full  type
         1    5    16K     8K  44.6G   200G   36.45  zvol object
        dnode flags: USED_BYTES
        dnode maxblkid: 26214399

    Object  lvl   iblk   dblk  dsize  lsize   %full  type
         2    1    16K    512      0    512  100.00  zvol prop
        dnode flags: USED_BYTES
        dnode maxblkid: 0
        microzap: 512 bytes, 1 entries

                size = 214748364800

The system is CentOS 7.1

Linux san1srvp01 3.10.0-514.6.1.el7.x86_64 #1 SMP Wed Jan 18 13:06:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

There isn't a relevant snapshot; I figured that went without saying.

The reason that I'm asking the question, and what I'm after relates to articles like this one from Max Burning which delves into object recovery through forensic techniques. This of course relies on no shortage of knoledge of ZFS internals. Most of what I've seen though deals with walking back over file objects which is very different from the objects that implement raw block storage and I have seen next to nothing on the internals related to ZVOLs.

Even if I can't technically "roll back" the changes that fsck made it would at least be helpful to walk back and find some of the key original blocks. This should be possible given ZFS's COW behavior... and sufficient knoledge which I lack, but I don't usually let that stop me.

已采纳解答

原帖链接 · jkhilmer · CC BY-SA 4.0

Yes, it can be done. It will be messy.

https://gist.github.com/jshoward/5685757

Including the original file now that the link has gone dead.

zfs_revert-0.1.py


#!/usr/bin/python
# -*- coding: utf-8 -*-

#Script for reverting ZFS changes by destroying uberblocks
#Author: Martin Vool
#E-mail: [email omitted]
#Version: 0.1
#Date: 16 November 2009


import time
import subprocess
import sys
import os
#Default blocksize
bs=512
#default total blocks (sorry programming in estonian :-/)
suurus=None

if len(sys.argv) > 2:
    for arg in sys.argv:
        arg=arg.split('=')
        if len(arg) == 1:
            file=arg[0]
        elif arg[0] == '-bs':
            bs=int(arg[1])
        elif arg[0] == '-tb':
            suurus=int(arg[1])
else:
    print 'Usage: zfs_revert.py [-bs=n default:n=512 blocksize] \\n [-tb=n total block size in blocks] [file/device] You have to set -tb'
    exit(1)
print int(bs)
if suurus == None:
    print 'Total block size in blocks is undefined'
    exit(1)
#make solaris use gnu grep.
if os.uname()[0] == 'SunOS':
    grep_cmd='ggrep'
else:
    grep_cmd='grep'


#to format program output
def formatstd(inp):
    inp=inp.split('\n')
    ret=[]
    for line in inp:
        columns=line.split(' ')
        nc=[]
        for c in columns:
            if c != '':
                nc.append(c)
        ret.append(nc)
    return ret


#read blocks from beginning(64mb)
a_count=(256*bs)
#read blocks from end (64mb)
l_skip=suurus-(256*bs)


print 'Total of %s blocks'%suurus
print 'Reading from the beginning to %s blocks'%a_count
print 'Reading from %s blocks to the end'%l_skip

#get the uberblocks from the beginning and end
yberblocks_a=formatstd(subprocess.Popen('sync && dd bs=%s if=%s count=%s | od -A x -x | %s -A 2 "b10c 00ba" | %s -v "\-\-"'%(bs,file, a_count,grep_cmd,grep_cmd), shell=True, stdout=subprocess.PIPE).communicate()[0])
yberblocks_l=formatstd(subprocess.Popen('sync && dd bs=%s if=%s skip=%s | od -A x -x | %s -A 2 "b10c 00ba" | %s -v "\-\-"'%(bs,file, l_skip,grep_cmd,grep_cmd), shell=True, stdout=subprocess.PIPE).communicate()[0])


yberblocks=[]

for p in yberblocks_a:
    if len(p) > 0:
        #format the hex address to decmal so dd would eat it.
        p[0]=(int(p[0], 16)/bs)
        yberblocks.append(p)

for p in yberblocks_l:
    if len(p) > 0:
        #format the hex address to decmal so dd would eat it and add the skipped part.
        p[0]=((int(p[0], 16)/bs)+int(l_skip)) #we have to add until the place we skipped so the adresses would mach.
        yberblocks.append(p)
print '----'
#here will be kept the output that you will see later(TXG, timestamp and the adresses, should be 4, might be less)
koik={}
i=0
for p in yberblocks:
    if len(p) > 0:
        if i == 0:#the first output line
            address=p[0]
        elif i == 1:#second output line
            #this is the output of od that is in hex and needs to be reversed
            txg=int(p[4]+p[3]+p[2]+p[1], 16)
        elif i == 2:#third output line
            timestamp=int(p[4]+p[3]+p[2]+p[1], 16)
            try:
                aeg=time.strftime("%d %b %Y %H:%M:%S", time.localtime(timestamp))
            except:
                aeg='none'
            if koik.has_key(txg):
                koik[txg]['addresses'].append(address)
            else:
                koik[txg]={
                    'txg':txg,
                    'timestamp':timestamp,
                    'htime': aeg,
                    'addresses':[address]
                }
        if i == 2:
            i=0
        else:
            i+=1
    keys = koik.keys()
    keys.sort()
    
while True:
    keys = koik.keys()
    keys.sort()
    print 'TXG\tTIME\tTIMESTAMP\tBLOCK ADDRESSES'
    for k in keys:
        print '%s\t%s\t%s\t%s'%(k, koik[k]['htime'],koik[k]['timestamp'],koik[k]['addresses'])
    try:
        save_txg=int(input('What is the last TXG you wish to keep?\n'))
        keys = koik.keys()
        keys.sort()
        for k in keys:
            if k > save_txg:
                for adress in koik[k]['addresses']:
                    #wrtie zeroes to the unwanted uberblocks
                    format=formatstd(subprocess.Popen('dd bs=%s if=/dev/zero of=%s seek=%s count=1 conv=notrunc'%(bs, file, adress), shell=True, stdout=subprocess.PIPE).communicate()[0])
                del(koik[k])
        #sync changes to disc!
        sync=formatstd(subprocess.Popen('sync', shell=True, stdout=subprocess.PIPE).communicate()[0])
    except:
        print ''
        break

摘自公开归档,保留原文语言,版式经整理,外部媒体与邮箱地址已移除。

资料来源与适用范围

资料来源
Server Fault
适用范围
以原帖系统、软件版本与发布日期为准(2017-04-12)
内容语言
中文;命令、路径和错误标识保留技术原文
本地收录日期
2026-11-15
授权说明
CC BY-SA 3.0 / CC BY-SA 4.0

正文与所需资源由本站本地存储。查看来源及更新记录 ↗

请按适用版本和实际日志确认原因;本站没有对每一种系统或设备逐项执行文中的操作。