For fun, convenience and a chance to learn shell scripting for myself, I’ve been working on a collection of scripts to semi-automate the backup of my computer, two network connected Raspberry Pi’s and my Android phone with Termux.

The main script basically runs a bunch of remote backup scripts, then copies those remote backups to a dedicated partition on my computer and finally creates a copy of that partition to an external drive connected to my computer. I use rsync and it’s dry run feature so I am able to examine any file changes which has been super useful to me for catching mistakes and issues as I’ve been learning how to self-host over the past half year.

I have a simplified version of those backup scripts that makes a copy of my /home directory:

#!/bin/sh



# VARIABLES
## ENABLE FILE TRANSFER: Change variable `DRY_RUN` to `#DRY_RUN` to enable file tranfers
DRY_RUN="--dry-run" # Disables all file transfers

## PATHS (USER DEFINED)
SOURCE_DIRECTORY_PATH="/home"
SOURCE_DIRECTORY_PATH_EXCLUSIONS="--exclude=lost+found --exclude=.cache/*"
BACKUP_NAME="home"
BACKUP_BASE_PATH="/backup"

## PATHS (SCRIPT DEFINED/DO NOT TOUCH)
SOURCE_DIR="${SOURCE_DIRECTORY_PATH}/"
DESTINATION_DIR="${BACKUP_BASE_PATH}/${BACKUP_NAME}/"

## EXCLUSIONS (SCRIPT DEFINED/DO NOT TOUCH)
EXCLUDE_DIR="${SOURCE_DIRECTORY_PATH_EXCLUSIONS}"

## OPTIONS (SCRIPT DEFINED/DO NOT TOUCH)
OPTIONS="--archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress"
OPTIONS_EXTRA="--delete --numeric-ids"



# FUNCTIONS
## SPACER
SPACER() {
    printf "\n\n\n\n\n"
}

## RSYNC ERROR WARNINGS
ERROR_WARNINGS() {
    if [ "$RSYNC_STATUS" -eq 0 ]; then

        # SUCCESSFUL
        printf "\nSync successful"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"

    else
        # ERRORS OCCURED
        printf "\nSome error occurred"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"
    fi
}

## CONFIRMATION (YES/NO)
CONFIRM_YESNO() {
    while true; do
        prompt="${1}"
        printf "%s (Yes/No): " "${prompt}" >&2 # FUNCTION CALL REQUIRES TEXT PROMPT ARGUMENT
        read -r reply
        case $reply in
            [Yy]* ) return 0;; # YES
            [Nn]* ) return 1;; # NO
            * ) printf "Options: y / n\n";;
        esac
    done
}



##### START

# CHECK FOR ROOT
if ! [ "$(id -u)" = 0 ]; then

    # EXIT WITH NO ACTIONS TAKEN
    printf "\nRoot access required\n\n"
    return

else
    printf "\nStarting backup process..."

    # ${SOURCE_DIR} TO ${DESTINATION_DIR} DRY RUN
    SPACER
    printf "\nStarting %s dry run\n" "${SOURCE_DIR}"
    rsync --dry-run ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
    RSYNC_STATUS=$?
    ERROR_WARNINGS

    # CONFIRM ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP
    SPACER
    if CONFIRM_YESNO "Proceed with ${SOURCE_DIR} backup?"; then

        # CONTINUE ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP & EXIT
        printf "\nContinuing %s backup\n" "${SOURCE_DIR}"
        rsync ${DRY_RUN} ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
        RSYNC_STATUS=$?
        ERROR_WARNINGS

        printf "\n%s backup completed\n\n" "${SOURCE_DIR}"
        return

    else
        # SKIP ${SOURCE_DIR} TO ${DESTINATION_DIR} BACKUP & EXIT
        printf "\n%s backup skipped\n\n" "${SOURCE_DIR}"
        return
    fi
fi

##### FINISH

I would like to adapt this script so that I can add multiple copies of the following variables:

## PATHS (USER DEFINED)
SOURCE_DIRECTORY_PATH="/home"
SOURCE_DIRECTORY_PATH_EXCLUSIONS="--exclude=lost+found --exclude=.cache/*"
BACKUP_NAME="home"
BACKUP_BASE_PATH="/backup"

without having to make multiple copies of the following commands within the running script:

    # ${SOURCE_DIR} TO ${DESTINATION_DIR} DRY RUN
    SPACER
    printf "\nStarting %s dry run\n" "${SOURCE_DIR}"
    rsync --dry-run ${OPTIONS} ${OPTIONS_EXTRA} ${EXCLUDE_DIR} "${SOURCE_DIR}" "${DESTINATION_DIR}"
    RSYNC_STATUS=$?
    ERROR_WARNINGS

I’m mainly just looking for a way to avoid touching the script commands itself so I don’t have to change the variable names for each additional directory I want to add. I’m not sure what that would be called or where to look. Any help would be greatly appreciated.

  • NullPointer@programming.devEnglish
    2·
    1 month ago

    try storing the paths to backup in a dot-file; one path per line. then, in the script, iterate that file line by line backing up the paths.

    • confusedpuppy@lemmy.dbzer0.comOPEnglish
      1·
      1 month ago

      I ended up using dot-files like you suggested. Instead of multiple source/destination locations within one dot-file, I’ve set it up to read multiple dot-files that contains one source/destination location each.

      Now I can use the command dir-sync /path/to/directory/.sync1 /path/to/directory/.sync2 or dir-sync /path/to/directory/* which is close enough to how I imagined it working.

      The challenge is that POSIX shell only has one array to work with. I may be able to inefficiently use more arrays, and I might try just for fun in the future but for now I have something that functions the way I want it to. I would have to use something like Bash if I intend to work more in depth with arrays though.

      .sync-0 example 
      SOURCE_DIRECTORY_PATH="/home"
SOURCE_DIRECTORY_PATH_EXCLUSIONS="--exclude=.sync* --exclude=lost+found --exclude=.cache/*"
DESTINATION_DIRECTORY_PATH="/media/archive/home"
      dir-sync script 
      #!/bin/sh



# VARIABLES
## ENABLE FILE TRANSFER: Change variable `DRY_RUN` to `#DRY_RUN` to enable file tranfers
DRY_RUN="--dry-run" # Disables all file transfers

## OPTIONS (SCRIPT DEFINED/DO NOT TOUCH)
OPTIONS="--archive --acls --one-file-system --xattrs --hard-links --sparse --verbose --human-readable --partial --progress --compress"
OPTIONS_EXTRA="--delete --numeric-ids"



# FUNCTIONS
## SPACER
SPACER() {
    printf "\n\n\n\n\n"
}

## RSYNC ERROR WARNINGS
ERROR_WARNINGS() {
    if [ "$RSYNC_STATUS" -eq 0 ]; then

        # SUCCESSFUL
        printf "\nSync successful"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"

    else
        # ERRORS OCCURED
        printf "\nSome error occurred"
        printf "\nExit status(0): %s\n" "$RSYNC_STATUS"
    fi
}

## COPY INPUT STRING ARRAY
COPY_ARRAY()
{
    for i do
        printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/"
    done
    printf " "
}

## CONFIRMATION (YES/NO)
CONFIRM_YESNO() {
    while true; do
        prompt="${1}"
        printf "%s (Yes/No): " "${prompt}" >&2 # FUNCTION CALL REQUIRES TEXT PROMPT ARGUMENT
        read -r reply
        case $reply in
            [Yy]* ) return 0;; # YES
            [Nn]* ) return 1;; # NO
            * ) printf "Options: y / n\n";;
        esac
    done
}

## DRY RUN
DRY_RUN() {
    # ARRAY SETUP
    eval "set -- ${1}" # ${ARRAY}

    for i do
        # SET CURRENT VARIABLE FROM ARRAY
        current_array="${1}"

        # SET VARIABLES FROM SOURCE FILE
        . "${current_array}"
        source_dir="${SOURCE_DIRECTORY_PATH}"
        exclusions="${SOURCE_DIRECTORY_PATH_EXCLUSIONS}"
        destination_dir="${DESTINATION_DIRECTORY_PATH}"

        # RUN RSYNC DRY RUN COMMAND
        SPACER
        printf "\nStarting %s dry run\n" "${source_dir}"
        rsync --dry-run ${OPTIONS} ${OPTIONS_EXTRA}  ${exclusions} "${source_dir}"/ "${destination_dir}"/
        RSYNC_STATUS=$?
        ERROR_WARNINGS

        # SHIFT TO NEXT VARIABLE IN ARRAY
        shift 1
    done
}

## TRANSFER
TRANSFER() {
    # ARRAY SETUP
    eval "set -- ${1}" # ${ARRAY}

    for i do
        # SET CURRENT VARIABLE FROM ARRAY
        current_array="${1}"

        # SET VARIABLES FROM SOURCE FILE
        . "${current_array}"
        source_dir="${SOURCE_DIRECTORY_PATH}"
        exclusions="${SOURCE_DIRECTORY_PATH_EXCLUSIONS}"
        destination_dir="${DESTINATION_DIRECTORY_PATH}"

        # RUN RSYNC TRANSFER COMMAND
        SPACER
        printf "\nContinuing %s transfer\n" "${source_dir}"
        rsync ${DRY_RUN} ${OPTIONS} ${OPTIONS_EXTRA} ${exclusions} "${source_dir}"/ "${destination_dir}"/
        RSYNC_STATUS=$?
        ERROR_WARNINGS

        # SHIFT TO NEXT VARIABLE IN ARRAY
        shift 1
    done
}



##### START

# CHECK FOR ROOT
if ! [ "$(id -u)" = 0 ]; then

    # EXIT WITH NO ACTIONS TAKEN
    printf "\nRoot access required\n\n"
    return

else
    # ARRAY SETUP
    ARRAY=$(COPY_ARRAY "$@")

    printf "\nStarting transfer process..."

    # SOURCE TO DESTINATION DRY RUN
    DRY_RUN "${ARRAY}"

    # CONFIRM SOURCE TO DESTINATION TRANSFER
    SPACER
    if CONFIRM_YESNO "Proceed with transfer(s)?"; then

        # CONTINUE SOURCE TO DESTINATION TRANSFER & EXIT
        TRANSFER "${ARRAY}"

        printf "\nBackup(s) completed\n\n"
        return

    else
        # SKIP SOURCE TO DESTINATION TRANSFER & EXIT
        printf "\nBackup(s) skipped\n\n"
        return
    fi
fi

##### FINISH

      I now have a bunch of other ideas on how I want to improve the script now. Adding ssh/network options, a check to make sure there are no empty command arguments, an option to create a blank template .sync- file in a chosen directory and a reverse direction transfer option are a few things I can think of off the top off my head. Hopefully this array was the most difficult part to learn and understand.