update 2: The Linux community has suggested that I use a tar file to backup, as this preserves symlinks. With that, the home directory now takes up just 290 ish GiB, as it should. Now I will be distro hopping, wish me luck!

update: I was able to copy it! There are still some folders that are really big (as many have said, it is probably because symlinks aren’t supported in exFAT. When I transfer these files over to btrfs, will the symlinks come back or are they permanently gone?) but, with the uninstallation of Steam and Kdenlive (each taking a ridiculous amount of storage), removing a couple games I don’t really play, and removing old folders that lingered around from already uninstalled programs means I now have enough space to fit my home folder in the SSD (like 23 GiB left, so the lack of symlinks still hurts, but still, it fits!)

When running

rsync -Paz /home/sbird "/run/media/sbird/My Passport/sbird"

As told by someone, I run into a ran out of storage error midway. Why is this? My disk usage is about 385 GiB for my home folder, and there is around 780 GiB of space in the external SSD (which already has stuff like photos and documents). Does rsync make doubly copies of it or something? That would be kind of silly. Or is it some other issue?

Note that the SSD is from a reputable brand (Western Digital) so it is unlikely that it is reporting a fake amount of storage.

EDIT: Wait, is it because my laptop SSD is BTRFS and the external SSD is exFAT? Could that be the issue? That would be kind of weird, why would files become so much more bigger with the external SSD?

Thanks everyone for your help to troubleshoot! It was super helpful! Now I need to go to bed, since I’ve been up so late it’s already tomorrow!

  • [object Object]@lemmy.world
    6·
    5 days ago

    The simplest explanation for the size difference could be if you have a symlink in your home folder pointing outside it. Idk if rsync traverses symlinks and filesystems by default, i.e. goes into linked folders instead of just copying the link, but you might want to check that. Note also that exFAT doesn’t support symlinks, dunno what rsync does in that case.

    It would be useful to run ls -R >file.txt in both the source and target directories and diff the files to see if the directory structure changed. (The -l option would report many changes, since exFAT doesn’t support Unix permissions either.) Apps like Double Commander can diff the directories visually (be sure to uncheck ‘compare by content’).

    As others mentioned, if you have hardlinks in the source, they could be copied multiple times to the target, particularly since exFAT, again, doesn’t have hardlinks. But the primary source of hardlinks in normal usage would probably be git, which employs them to compact its structures, and I doubt it that you have >300 GB of git repositories.

    • Wildmimic@anarchist.nexusEnglish
      3·
      5 days ago

      A second possibility is the deduplication feature of BTRFS. If he made copies of files on his SSD, they only take up space there when changing something - thats how i keep 5 differently modded Cyberpunk 2077 installations on my drive while only taking up a fraction of space that would be needed - I wouldn’t be able to copy this drive 1:1 onto a different filesystem.

      • [object Object]@lemmy.world
        1·
        5 days ago

        Ah, I knew the mention of btrfs heebied my jeebies a little, but forgot about the CoW thing.

        I’m guessing some btrfs-specific utils are necessary to figure out how much it cow’ed.

    • bleistift2@sopuli.xyzEnglish
      3·
      5 days ago

      Idk if rsync traverses symlinks and filesystems by default,

      From the man page:

      Beginning with rsync 3.0.0, rsync always sends these implied directories as real directories in the file list, even if a path element is really a symlink on the sending side. This prevents some really unexpected behaviors when copying the full path of a file that you didn’t realize had a symlink in its path.

      That means, if you’re transferring the file ~/foo/bar/file.txt, where ~/foo/bar/ is a symlink to ~/foo/baz, the baz directory will essentially be duplicated and end up as the real directory /SSD/foo/bar and /SSD/foo/baz.