{"id":517,"date":"2018-03-07T11:28:10","date_gmt":"2018-03-07T06:58:10","guid":{"rendered":"http:\/\/www.enhisecure.com\/isecureblog\/?p=517"},"modified":"2018-03-09T10:19:16","modified_gmt":"2018-03-09T05:49:16","slug":"troubleshooting-a-linux-partition-with-corrupted-metadata","status":"publish","type":"post","link":"https:\/\/www.enhisecure.com\/isecureblog\/2018\/03\/07\/troubleshooting-a-linux-partition-with-corrupted-metadata\/","title":{"rendered":"Troubleshooting a Linux Partition with Corrupted Metadata"},"content":{"rendered":"<p>A corruption in the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Linux\">Linux<\/a> file system causes the system to boot into emergency mode by default.<\/p>\n<p>The following error message is displayed on boot up<\/p>\n<p><strong>Welcome to emergency mode! After logging in, type \u201cjournalctl -xb\u201d to view system log, \u201csystemctl reboot\u201d to reboot, \u201csystemctl default\u201d or ^D to try again to boot into default mode.<\/strong><\/p>\n<p>Filesystems can be corrupted by<\/p>\n<ul>\n<li>Hardware Errors\n<ul>\n<li>Media errors are common<\/li>\n<li>Disks are getting bigger and bigger<\/li>\n<\/ul>\n<\/li>\n<li>To a much lesser degree, bugs in the filesystem<\/li>\n<\/ul>\n<p>Filesystems are able to \u201crepair\u201d themselves since they consist of lists, links and reference counts that can be validated<\/p>\n<ul>\n<li>But not all information is always recovered, inodes that do not have a parent directory is common due to the directory structure being corrupted<\/li>\n<\/ul>\n<h1>Detection<\/h1>\n<p>The OS shows the following error:<\/p>\n<p><strong>Corruption detected. Unmount and run xfs_repair.<\/strong><\/p>\n<p><strong>Corruption of in-memory data detected. Shutting down filesystem(s)<\/strong><\/p>\n<p><strong>Please unmount the filesystem and rectify the problem(s)<\/strong><\/p>\n<h1>Solution<\/h1>\n<p>Enter <a href=\"https:\/\/linux.die.net\/man\/8\/lvdisplay\"><strong>lvdisplay<\/strong><\/a>. This command would bring up the logical volumes present in the Linux machine, the common logical volumes (assuming no changes have been made) are <strong><em>root<\/em><\/strong>, <strong><em>home<\/em><\/strong> and <strong><em>swap<\/em><\/strong>.<\/p>\n<p>To mount a logical volume the command <strong>mount \/dev\/ol\/logical_volume_name <\/strong>needs to be entered.<\/p>\n<p>If a logical volume\u2019s metadata is corrupted, the following error is observed after trying to mount it.<\/p>\n<p><strong>XFS(dm-2) Metadata corruption detected at xfs_inode_buf_verify 0x75\/0xd0 [xfs]<\/strong><\/p>\n<p>For the course of this blog it is assumed that the <strong>home<\/strong> logical volume is corrupted, so the error is encountered when the following command is executed<\/p>\n<p><strong>mount \/dev\/ol\/home<\/strong><\/p>\n<p>To fix this enter the command<\/p>\n<p><strong><a href=\"http:\/\/xfs.org\/docs\/xfsdocs-xml-dev\/XFS_User_Guide\/tmp\/en-US\/html\/xfs-repair.html\">xfs_repair<\/a> -L \/dev\/mapper\/ol-home <\/strong><\/p>\n<p>Where <strong>ol-home<\/strong> is the default partition created by <a href=\"https:\/\/wiki.debian.org\/LVM\"><strong>Logical Volume Manager (LVM) <\/strong><\/a>on the <strong>home <\/strong>logical volume. To view the list of partitions the command <strong><a href=\"https:\/\/en.wikipedia.org\/wiki\/Fdisk\">fdisk<\/a> \u2013l <\/strong>can be used.<\/p>\n<p>The <strong>\u2013L <\/strong>option specifies Force Log Zeroing.<\/p>\n<p>Forces xfs_repair to zero the log even if it is dirty (contains metadata changes).<\/p>\n<p><strong>It is important to understand that this option should be used only if data of that partition has been backed up before, using this in a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mission_critical\">mission-critical<\/a> environment without prior testing would spell trouble as in certain cases, the inode tree could end up with even more corrupted metadata. <\/strong><\/p>\n<p>With fresh metadata, the inode tree of the filesystem is rebuilt and the <strong>\/home <\/strong>directory can now be mounted by using:<\/p>\n<p><strong><a href=\"http:\/\/www.linfo.org\/mounting.html\">mount<\/a> dev\/ol\/home<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>The changes in the filesystem can be observed by checking the <strong>df-l<\/strong><\/p>\n<p><strong>Reboot the OS for changes to show effect. <\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>A corruption in the Linux file system causes the system to boot into emergency mode by default. The following error message is displayed on boot up Welcome to emergency mode! After logging in, type \u201cjournalctl -xb\u201d to view system log, \u201csystemctl reboot\u201d to reboot, \u201csystemctl default\u201d or ^D to try again to boot into default [&hellip;]<\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[],"class_list":["post-517","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/posts\/517","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/comments?post=517"}],"version-history":[{"count":5,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/posts\/517\/revisions"}],"predecessor-version":[{"id":522,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/posts\/517\/revisions\/522"}],"wp:attachment":[{"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/media?parent=517"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/categories?post=517"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.enhisecure.com\/isecureblog\/wp-json\/wp\/v2\/tags?post=517"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}