I’ve been homelabbing for a few years now, and recently I’ve really been focusing in on learning how to use gnu/linux. I thought it might be fun to periodically share the things I’ve been learning. The stand outs for me this past week were:
- Use the full path when referencing files and directories in bash scripts (Edit: when it makes sense, which is something I’m also still learning. This mkaes sense when the files will always be located in the same place.)
- In a bash script, the variable
${file##*/}will get you the name of the file your script is handling (example, when looping over files in a directory. I believe that’s a shell/bash standard variable, but I need to learn where it came from and how it works) - Ubuntu gets a ton of justifiable criticism, but I find Canonical’s Multipass to be a great tool for spinning up Linux virtual machines. Especially on Apple silicon macs.
- Piping the output of
lstogrepas a variable in a path is a great way to change to a directory you know exists but can’t remember the exact name of. (Example:cd ~/movies/“$(ls ~/movies | grep movie-name)”) - The reason Mac cli utilities have syntax variations compared to the standard gnu/linux utilities is because macOS and its cli utilities are BSD based. This was information I knew at a high level, but had never really understood the implications of until this week.
- Related to point 5, if you’re on macOS trying to learn and you’re annoyed by the syntax differences between bsd and gnu utilities, you can run this script from darksonic37 on github to remove the bsd utilities from macOS and replace them with their gnu counterparts. (I have not run or reviewed the script. I found mulitpass first, and so far I’m happy using the Ubuntu virtual machine)
Well, if you didn’t replace grep with gnu/grep then you should call it belllabs/gnu/linux. Oh and don’t forget canonical for consistency: canonical/belllabs/gnu/linux
Keep in mind to sort the complete list by cpu cycles used by each of the projects on your specific system in ascending order. Maybe you can write a canonical/belllabs/gnu/linux script to automatically keep track and output an up to date string for easy proper nomenclature.
/s
I generally agree with the message behind this sarcasm, but in this specific case OP really is learning the GNU utilities in particular (via Linux) so I don’t mind the extra nomenclature.
except for grep and multipass, which aren’t gnu, and amount to half of the utilities mentioned in the post if I read it correctly 🤷
You’re certainly right about Multipass, but the grep included in Ubuntu does seem to be from GNU.
Regarding some of your points:
- Depends on the case. A script operating on stuff in its subdirectories will probably be better with a relative path, especially if they get moved somewhere. Same logic goes for symbolic links.
- This is called string manipulation.
#deletes the longest match from the beginning, so it deletes everything to the last slash, as the asterisk expands as far as possible. If you wanted the directory the file is in using this method, you’d use${file%/*}. This deletes the shortest match from the end. You also have the dedicated commandsbasenameanddirnamefor this. - Can’t comment
- I guess this works as long as you get exactly one match.
- Agree. Always was too strong a statement. I’ll edit my post to reflect that.
- Thank you! I’m going to read up on string manipulation
- True. This has been working for me because I know enough of the directories I’m looking for to insure a single match, but not the necessarily the exact, specific directory name.
To expand a little on @Laser ‘s point 2:
In bash (and other programming languages) # is used at the start of the line to notate comments.
When writing percentages, you write the symbol after the number, e.g. 50%
That’s how I keep them apart, lol
Theres a section in the bash manual with these and a whole bunch of more expansion tricks.
One I find useful is
echo "${myvar@A}"That’s a super useful way to remember. Thank you!
Instead of cd into output of ls | grep I’d suggest you to install fzf and fuzzy find directories to cd into.
I didn’t know that tool existed I will give fzf a try!
Thanks for putting this out for public benefit! I haven’t messed around with MacOS much but the things you’ve mentioned are nice to know.
I believe that’s a shell/bash standard variable, but I need to learn where it came from and how it works
You may know this already, but I’ve found the
man(as in manual) utility to be one of the most useful things in GNU/Linux user space. I don’t have much insight into ‘${file##*/}’ off the cuff, but I can tell you there’s manual entries forfile,sh, andbashthat may help you track it down.# simply type man [some-command] man file man sh man bash man man # very useful for getting started!Manpages are local to your system so they’re extremely fast to pull up and searchable!
Here’s some online info on
manif you’re interested:Yes, thank you! The man pages have been a huge help as I’m working through things. Sometimes I don’t know enough to understand what the man pages are telling me, and then I usually end up on stack exchange looking at a command example that someone has helpfully broken down.
It’s definitely a skill that I haven’t mastered either! That being said I think it’s one of the pillars of being a bonafide “super user” and I’d like to set there one day :)
Maybe I’ll take inspiration from this post and write something up about what I learn in the future about manpages.
Cheers and happy tinkering!
I agree, and I’d like to be there as well so that I can easily read and understand a man page.
This is really great info! I never knew Multipass existed, thanks for sharing.
For macOS, Homebrew can be used to selectively replace certain parts of the coreutils with the GNU versions
Edit: On reviewing the script you mentioned, that’s exactly what it does. It uses Homebrew to replace all the coreutils in one go
You’re welcome! I stumbled across Multipass when I was looking for virtual machine options for the m1 mac mini I’m working on. I specifically was trying to get away from using the mac coreutils for a consistent syntax experience, and Multipass has been working perfectly for that.
It was only after I’d been using Multipass already that I stumbled across that script, and planned to take a look at it to possibly implement on my machine. I didn’t realize that Homebrew allowed for replacing the coreutils with the GNU versions. Another thing learned!
- why
It’s definitely an edge case by say you’re in ~/ and you run a script like ./code/script.sh then it thinks the current working direct is ~/ rather than what is probably intended which is ~/code/. If your bash script uses full paths like /home/$USER/code/ then it will still run correctly regardless of the current working directory that the scrip was run from.
To insure the script works on the files you want it to work on.
If you use a relative path in a script (is relative the right word?), with something like:
mv *.txt /place/to/move/to/The script will act on the files in the current working directory, which may not be the files you intend for the script to act on.
If you put the absolute path to where the files reside in the filesystem, then you can run the script from any location, and know that it will act on the files in the specified directory.
Maybe always is too strong a statement. For the scripts I was working on, it was a learning moment when I ran the script without the full path and the script errored out because the files were not in the working (home) directory. (I had ssh’d into the machine to run the script)
I’d never use absolute paths unless it is a file that will always be there, e.g. system files, icons, etc.
That’s fair. Always was probably too strong.
