• 26 Posts
  • 27 Comments
Joined 6 months ago
cake
Cake day: March 2nd, 2024

help-circle












  • To accomplish this task, we can use a combination of Bash scripting and various Linux utilities such as find, mv, du, and sort. Here's a step-by-step approach:

    1. Create a bash script file, let's call it reorganize.sh.
    #!/bin/bash
    
    # Define the mount points and the desired free space threshold
    declare -a MOUNT_POINTS=("/mnt/hdd0" "/mnt/hdd1" "/mnt/hdd2" ...)
    FREE_SPACE_THRESHOLD=20  # In gigabytes
    
    # Function to get the available space on a file system
    get_available_space() {
        local mount_point="$1"
        df --output=avail "$mount_point" | tail -n 1 | awk '{print $1}'
    }
    
    # Create an associative array to keep track of the available space on each file system
    declare -A AVAILABLE_SPACE
    for mount_point in "${MOUNT_POINTS[@]}"; do
        AVAILABLE_SPACE["$mount_point"]=$(get_available_space "$mount_point")
    done
    
    # Function to find the mount point with the most available space
    find_mount_point_with_most_space() {
        local max_space=0
        local max_mount_point=""
    
        for mount_point in "${!AVAILABLE_SPACE[@]}"; do
            if [ "${AVAILABLE_SPACE[$mount_point]}" -gt "$max_space" ]; then
                max_space="${AVAILABLE_SPACE[$mount_point]}"
                max_mount_point="$mount_point"
            fi
        done
    
        echo "$max_mount_point"
    }
    
    # Function to redistribute files based on alphabetical order
    redistribute_files() {
        local source_dir="$1"
        local target_dir="$2"
    
        # Get a list of files sorted alphabetically by relative path
        local files=$(find "$source_dir" -type f | sort)
    
        # Move files to the target directory
        while read -r file; do
            local relative_path=$(realpath --relative-to="$source_dir" "$file")
            local file_size=$(du -k "$file" | cut -f 1)
    
            # Update available space on source and target directories
            AVAILABLE_SPACE["$source_dir"]=$((AVAILABLE_SPACE["$source_dir"] + file_size))
            AVAILABLE_SPACE["$target_dir"]=$((AVAILABLE_SPACE["$target_dir"] - file_size))
    
            mkdir -p "$target_dir/$(dirname "$relative_path")"
            mv "$file" "$target_dir/$relative_path"
        done <<< "$files"
    }
    
    # Function to move files from one file system to another if the free space is below the threshold
    move_files_if_needed() {
        local source_dir="$1"
        local temp_dir=$(mktemp -d --tmpdir="$(find_mount_point_with_most_space)")
    
        if [ "${AVAILABLE_SPACE[$source_dir]}" -lt "$((FREE_SPACE_THRESHOLD * 1024 * 1024))" ]; then
            # If the available space on the source directory is below the threshold,
            # move the files to a temporary directory on the mount point with the most available space
            redistribute_files "$source_dir" "$temp_dir"
            rm -rf "$source_dir"/*
            # Move the files back to the source directory, preserving the alphabetical order
            redistribute_files "$temp_dir" "$source_dir"
            rmdir "$temp_dir"
        fi
    }
    
    # Main script
    for ((i=0; i<${#MOUNT_POINTS[@]}; i++)); do
        local mount_point="${MOUNT_POINTS[$i]}"
        # Check if the current mount point needs to be cleared to meet the free space threshold
        move_files_if_needed "$mount_point"
        # Redistribute files from the current mount point to the remaining mount points in alphabetical order
        for ((j=i+1; j<${#MOUNT_POINTS[@]}; j++)); do
            redistribute_files "$mount_point" "${MOUNT_POINTS[$j]}"
        done
    done
    
    1. Save the script file and make it executable:
    chmod +x reorganize.sh
    
    1. Run the script:
    ./reorganize.sh
    



  • More critically, the proof-of-concept so far appears to lack any real work on moderation tools or implementing a web of trust system. These would be absolutely vital components for a federated encyclopedia to have any chance of controlling quality and avoiding descending into a sea of misinformation and edit wars between conflicting "truths." Centralized oversight and clear enforced guidelines are key reasons why Wikipedia has been relatively successful, despite its flaws.

    Without a robust distributed moderation system in place, a federated encyclopedia runs the risk of either devolving into siloed echo chambers pushing various agendas, or becoming an uncoordinated mess making it impractical as a general reference work. The technical obstacles around federating content policies, privileges and integrated quality control across instances are immense challenges that aren't obviously addressed by this early proof-of-concept.

    While novel approaches like federation are worth exploring, straying too far from Wikipedia's principles of neutral point-of-view and community-driven policies could easily undermine the entire premise. Lofty goals of disrupting Wikipedia are admirable, but successfully replacing its dominance as a general reference work seems extremely unlikely without solving these fundamental issues around distributed content governance first.


  • To recover the file system, files, and directory structure from an ext4 drive that you formatted to NTFS, you'll need to use data recovery tools designed for this purpose. Here are the general steps you can follow:

    1. Stop using the drive immediately: Once you realize you need to recover data from the formatted drive, stop using it and avoid writing any new data to it. This will increase your chances of successful recovery.

    2. Create an image of the drive: It's recommended to create a byte-by-byte image of the entire drive using a tool like ddrescue or dd. This will ensure that you're working with an exact copy of the drive, preserving all existing data.

    3. Replicate the loss of data on a smaller drive (optional): If you're working with a large-capacity drive and want to speed up the recovery process for testing purposes, you can replicate the loss of data on a smaller drive. This will allow you to work with a more manageable drive size while testing the recovery process.

    4. Use a data recovery tool: There are several data recovery tools available for Linux that can scan the drive image and attempt to recover files and directory structures from the previously existing ext4 file system. Some popular options include:

    • TestDisk: A powerful command-line tool that can recover lost partitions and repair file systems.
    • PhotoRec: A data recovery tool designed to recover lost files from hard disks, CD-ROMs, and other media.
    • extundelete: A command-line tool specifically designed to recover deleted files from ext4 file systems.
    • Foremost: A data recovery tool that can recover files based on their headers, footers, and data structures.
    • ext4magic: A tool to recover deleted or overwritten files on ext3 and ext4 file systems.
    • Scalpel: A file carving and indexing application that allows you to specify headers and footers to recover file types from a storage media.
    • ddrutility: A complement to GNU ddrescue that finds which files are related to bad sectors and provides special tools for NTFS (no longer actively supported).
    • dvdisaster: A tool that provides additional error protection for CD/DVD media.
    • xfs_undelete: A tool that traverses the inode B+trees of each allocation group and tries to recover all files marked as deleted on an XFS file system.
    1. Scan the drive image: Use the chosen data recovery tool to scan the drive image. Most tools will allow you to specify the type of file system you're trying to recover (in this case, ext4).

    2. Recover files and directories: After the scan is complete, the tool should present you with a list of recoverable files and directories. You can then select the ones you want to recover and specify a location to save them.

    3. Verify recovered data: Once the recovery process is complete, verify that the recovered files and directories are intact and usable.

    It's worth noting that the success of data recovery greatly depends on the extent of the formatting process and whether any new data was written to the drive after formatting. The sooner you attempt recovery, the better your chances of success.

    Additionally, if the recovered data is particularly important or valuable, you may want to consider consulting a professional data recovery service, as they often have more advanced tools and expertise to handle complex recovery scenarios.


  • The error message "The disk structure is corrupted and unreadable" indicates that there is a problem with the file system or the disk itself, which is preventing Windows from accessing the drive. The Master File Table (MFT) is a critical component of the NTFS file system, and if it's corrupt, the system cannot access files on the drive. You've already attempted to use the chkdsk utility, which is the right first step, but it has failed to repair the MFT.

    To recover the files from the drive, you can try the following methods:

    Use Data Recovery Software

    Since chkdsk was unable to fix the MFT, you can use data recovery software to try and recover your files. EaseUS Data Recovery Wizard is recommended by one of the sources and is known for its ability to recover data from corrupted drives[2]. Follow these steps:

    1. Download and install EaseUS Data Recovery Wizard.
    2. Launch the software and select the drive with the corrupted MFT.
    3. Click "Scan" to start the scanning process.
    4. Once the scan is complete, preview the recoverable files.
    5. Select the files you want to recover and save them to a different drive.

    Use the FixMbr Command

    Another approach is to use the bootrec.exe command with the /FixMbr parameter to repair the Master Boot Record, which might indirectly help with MFT issues[2]. To do this:

    1. Boot from a Windows installation media.
    2. Choose "Repair your computer" and then "Command Prompt".
    3. Type bootrec.exe /FixMbr and press Enter.

    Format the Drive

    If the above methods do not work and you cannot recover your files, the last resort is to format the drive, which will erase all data. Before doing this, ensure that you have recovered as much data as possible using data recovery software. To format the drive:

    1. Open Disk Management by pressing Windows + X and selecting it from the list.
    2. Right-click on the drive and select "Format".
    3. Choose NTFS as the file system and complete the format process[4].

    Additional Tips

    • Before attempting recovery, it's crucial to stop using the drive to avoid overwriting any recoverable data.
    • If the drive is an external one, try unplugging and replugging it into a different port or computer to rule out connection issues[1].
    • Running hardware and device troubleshooter might help if the issue is related to drivers or hardware[1].
    • If you are not comfortable with these steps or if they do not work, consider contacting a professional data recovery service[1].

    Remember, these methods are not guaranteed to recover all your data, and there is a risk of data loss. If the data is extremely important, it's often best to consult with a professional data recovery service before proceeding with any recovery attempts.

    Citations: [1] https://www.salvagedata.com/fix-disk-structure-is-corrupted-and-unreadable/ [2] https://www.easeus.com/data-recovery/fix-corrupt-master-file-table-error-without-losing-data.html [3] https://forums.tomshardware.com/threads/master-file-table-corrupt-chkdsk-fails.3712756/ [4] https://www.stellarinfo.com/blog/disk-structure-is-corrupted-and-unreadable/ [5] https://www.partitionwizard.com/disk-recovery/corrupt-master-file-table.html [6] https://www.anyrecover.com/hard-drive-recovery-data/fix-corrupt-master-file-table-error/ [7] https://www.reddit.com/r/datarecovery/comments/fo9o5f/the_disk_structure_is_corrupted_and_unreadable/?rdt=48629 [8] https://recoverit.wondershare.com/file-recovery/fix-corrupt-master-file-table-error.html [9] https://superuser.com/questions/688367/external-hard-disk-is-not-accessible-the-disk-structure-is-corrupted-and-unrea [10] https://answers.microsoft.com/en-us/windows/forum/all/windows-cannot-recover-master-file-table-chkdsk/ecb68215-7329-4006-9f70-2d51f610a27f [11] https://www.youtube.com/watch?v=qKQ5EejHarU [12] https://www.techrepublic.com/forums/discussions/master-file-table-recovery/ [13] https://7datarecovery.com/blog/disk-structure-corrupted-and-unreadable/ [14] https://www.diskpart.com/articles/windows-cannot-recover-master-file-table-0310.html [15] https://www.partitionwizard.com/partitionmagic/disk-structure-corrupt-unreadable.html [16] https://www.stellarinfo.com/blog/fix-corrupt-master-file-table-error/ [17] https://4ddig.tenorshare.com/hard-drive/fix-the-disk-structure-is-corrupted-and-unreadable.html [18] https://7datarecovery.com/blog/corrupt-master-file-table/


  • The error messages you're receiving indicate that the NTFS file system on the drive has become corrupted, and the master file table (MFT) – which is a critical component that stores metadata about files and directories – is damaged. Unfortunately, when the MFT is severely corrupted, Windows' built-in chkdsk utility is often unable to repair it.

    However, there are third-party data recovery tools that may be able to recover your files from the corrupted NTFS drive. These tools use advanced algorithms and techniques to scan the drive sector by sector, searching for file signatures and reconstructing the file system structure.

    Here are a few recommended data recovery tools you could try:

    1. TestDisk and PhotoRec (Free, open-source): TestDisk is a powerful tool that can attempt to repair damaged file systems, including NTFS. If it fails to repair the file system, you can use its companion tool PhotoRec to recover files directly from the drive.

    2. Stellar Data Recovery (Paid): Stellar Data Recovery is a commercial data recovery suite that includes tools for recovering data from corrupted NTFS drives. It has a solid reputation and decent success rates.

    3. EaseUS Data Recovery Wizard (Paid): Another popular commercial data recovery tool that can recover data from corrupted NTFS drives.

    4. R-Studio (Paid): A professional-grade data recovery tool that can handle various types of file system corruption and data loss scenarios.

    Before using any data recovery tool, it's crucial to create a byte-by-byte copy (image) of the corrupted drive first. This will ensure that the tool works on a copy rather than the original drive, reducing the risk of further data loss. You can use tools like ddrescue (Linux/macOS) or ImageX (Windows) to create a drive image.

    Once you have a drive image, you can attempt data recovery using the tool of your choice. Keep in mind that the success rate of data recovery depends on the extent of the corruption and the tool's capabilities.


  • I accidentally formatted an ext4 partition to NTFS in my Ubuntu 16.04 recently and was able to recover the full partition successfully by running a file system check.

    sudo fsck.ext4 -vy /dev/sda10
    

    I recorded the steps in this blog post. However note that the scenario is a bit different. Hope this helps someone else.

    https://superuser.com/a/1204121

    Recovering Accidentally Formatted ext4 Partition / Fixing Superblock

    rajind.dev

    Recovering Accidentally Formatted ext4 Partition / Fixing Superblock

    Published by Rajind Ruparathna 5–6 minutes

    Today, I made the silly mistake of accidentally formatting one of the ext4 partitions in my Ubuntu 16.04 machine to NTFS instead of formatting the pen drive, which I was hoping to format. So if you are reading this most probably you have done something similar or perhaps someone you know has gone down that path and you maybe trying to help him/her.

    Fortunately I was able to recover my partition completely and in this post I’ll go through the few things that helped me recover my partition. There is still hope my friend. 🙂

    First I must thank Shane for this askubuntu answer, the author of this blog post for giving pointers and my friend Janaka for helping me out. Have a look at those two posts as they are very helpful.

    If you accidentally formatted your partition (or in any case of lost partitions or data), the most important thing is avoiding any data writes on that partition. Another important thing is creating a backup image of the messed up disk.

    Quoting Shane,

    If your messed up drive is sda, and you wanted to store the image in yourname’s home directory for instance: dd if=/dev/sda of=/home/yourname/sda.img.bak bs=512
    
    to restore the image after a failed recovery attempt: dd if=/home/yourname/sda.img.bak of=/dev/sda bs=512
    
    You could of course use /dev/sda1 if you are only interested in the first partition, but as some of these utilities alter the partition table, it is perhaps a better idea to image the whole disk..
    
    Also, if you are using dd for a large operation, it is very helpful to see a progress bar, for which you can use a utility called pv which reports progress of data through a pipeline
    
    for instance: pv -tpreb /dev/sda | dd of=/home/yourname/sda.img.bak bs=512
    

    First of all I tried TestDisk tool. I was able to get some files recovered but I wasn’t able to find a way to recover the whole partition using TestDisk tool.

    Then I started following the other blog post I shared above. In their first thing was identifying the affected partition. I already knew mine, however if you want to get info on that you can run sudo fdisk -l command. Output for me was as follows.

    fdisk-output

    Now the idea for the next step is that, since I did not write anything on the formatted disk, my previous ext file system data should be still there. In ext, file system data are kept in a record called Superblock which keeps the characteristics of a filesystem, including its size, the block size, the empty and the filled blocks and their respective counts, the size and location of the inode tables, the disk block map and usage information, and the size of the block groups. (You can read more about it here if you are interested). So what what we are trying to do here is to fix the ext file system.

    In my case, I was able to do it with the following file system check command. (Note that the same command is there for ext2 and ext3 as well). Before you run the command, make sure the partition is unmounted.

    sudo fsck.ext4 -v /dev/xxx

    First part of the output for me was as follows. I was able to see the original partition name (“Personal”) which I had previously given to my ext4 partition and it gave me a slight relief. So if you are able to see the same, hopefully things will turn out better.

    fcsk-output-part1

    At the bottom of the above screen capture, you could see a prompt asking it it is okay to fix certain blocks count. I went with yes here and it was actually the only option for me to go as well. There were few more prompts of the similar manner and seemed like a lot more is coming. So I stopped the command and went with the following which basically says yes to all prompts.

    sudo fsck.ext4 -vy /dev/sda10

    Then after a number of fix prompts, I got the following output.

    fsck-output-success

    When I mount the partition everything was back to normal. Hopefully you will be able to recover yours as well. Please note that this might not be the recovery method for all the scenarios. I’m just noting this down hoping that this will help someone else like me. Make sure you understand the steps well before doing these.

    EDIT (03/09/2017):

    If you have a dual boot set-up, once you boot up the other OS (e.g. Windows) you might lose the partition again since in the other OS the partitioning might be different. So it’s better to make sure to get a backup once the recovery is done.

    Further even if you lose the partition again once you login to the other OS, you can still recover using the fcsk.ext4 command.

    Cheers! 🙂

    ~ Rajind Ruparathna





  • The idea of a federated, decentralized Wikipedia alternative is intriguing, but implementing it successfully faces major hurdles. Federating moderation policies and privileges across different instances seems incredibly complex. I believe it would also require some kind of web of trust system. Quality control is also a huge challenge without centralized oversight and clear guidelines enforced universally.

    While it could potentially replace commercial wiki farms like Wikia/Fandom for niche topics, realistically replacing Wikipedia's dominance as a general reference work seems highly ambitious and unlikely, at least in the short term. But as they say - shoot for the stars, and you may just land on the moon.

    That said, ambitious goals can spur innovation. Even if Ibis falls short of usurping Wikipedia, it could blaze new trails and pioneer federated wiki concepts that feed back into Wikipedia and other platforms. The federated model allowing more perspectives and focused communities is worth exploring, despite the technical obstacles around distributed moderation and content integration. The proof-of-concept shows the core pieces are in place as a starting point.






  • I did read the links, and I still strongly feel that no automated mechanical system of weights and measures can outperform humans when it comes to understanding context.

    But this is not a way to replace humans; it's just a method to grant users moderation privileges based on their tenure on a platform. Currently, most federated platforms only offer moderator and admin levels of moderation, making setting up an instance tedious due to the time spent managing the report inbox. Automating the assignment of moderation levels would streamline this process, allowing admins to simply adjust the trust level of select users to customize their instance as desired.





  • A system like this rewards frequent shitposting over slower qualityposting. It is also easily gamed by organized bad faith groups. Imagine if this was Reddit and T_D users just gave each other a high trust score, valuing their contributions over more “organic” posts.

    You are just assuming that this would work similarly to Reddit based on karma. I don't know why you would assume the worst possible implementation just so you can complain about this. If you had read the links, you would know that shitposting wouldn't help much because what contributes most to Trust Levels in Discourse is reading posts.