Bash Commands For Effective Software Engineers

Photo by Mars on Unsplash

Bash is a very important tool that programmers ignore most of the time.

Git is another topic that programmers miss out on, and I have covered it in one of my newsletter posts.

Another topic I would love to cover in my future posts is How to document your codebase. Stay tuned for that!

There are several commands I use to make my developer life easy. Either they help me with navigating faster on my home computer. Other times, it helps me ssh into my prod server and reliably messes it up! (Just kidding, I don’t mess things up, 😉)

Shells

So what is a shell? Shell is a program that helps a user communicate with the kernel. The user can pass commands to the kernel and the kernel can respond to them. Shell seems to be a very powerful tool when you look at it from this perspective, right?

There are multiple shells available and you can choose which one to use. Following are the most popular ones:

1. Bourne shell (sh)
2. GNU Bourne-Again shell (bash)
3. Korn shell (ksh)
4. Z shell (zsh)

Most linux/unix systems come with a default shell program. For e.g. Ubuntu has bash installed by default while the new Macs have zsh by default.

I personally use zsh since it’s quite powerful. I will talk a bit about it towards the end of the issue. You can swap your shell to whichever one you want. It’s quite easy!

No matter which shell you use, most of the commands shown here will work the same way.

Why should you learn shell programming?

If you learn to do shell interaction, a lot of things become easy for you. Many of the operations such as changing file permissions which takes multiple clicks through GUI can be done in single command execution. If you have to search through a file for keywords, you can do it without having to open a code editor.

The best advantage I feel that learning shell programming offers is you will be able to ssh into remote servers and perform tasks that you cannot do with GUI, since your remote server on cloud won’t provide you one.

It’s a daily thing for me to ssh into our servers and search through the server logs or check for permissions assigned to various files, or create and edit files on it.

File permissions and how to change them

When we take a file or a directory in Linux, there are 3 levels of permission you will have to define for it.

1. First is on a user level, the owner of the file (denoted by u)
2. Second is on a group level, the users who belong to the group of which the owner is part of (denoted by g)
3. The third is for other users outside the owner’s group (denoted by o)

Now, for each of these levels we can make the access permissions more granular like:

1. Whether the user can read
2. Whether the user can write
3. Whether the user can execute a file (applicable for executable files like .sh files)

Let’s look at how to figure out file permissions in Linux.

If I got to a directory and enter ls -alh

It gives me the following output. Let’s focus on the last two entries for now.

The left-most column shows the permissions and file type.

1. The first entry, d shows that it’s a directory. If the first entry is a - (dash). It means it’s a file. There are a few other types as well such as a socket file, block device file, etc.
2. The next 3 letters (rwx) shows you the permission the owner has, the 3 letters after that show you the permission the group has, and the rest of the 3 letters show the permission other users have.

Here, for the file hello.txt, the owner (that’s me!) has permissions to read and write but the group and other users don’t have write or executable permissions.

Now, if I want to give write permissions to the group users, we will have to change the permission of the file.

The command used to change permissions is chmod

chmod g+w hello.txt

Here, I am adding (+) the permission to write (w) for the group (g) users.

Similarly, if I want to remove the read permission for anyone outside the group, then:

chmod o-r hello.txt

Here, I am removing (-) the permission to read (r) for the others (o)

Now that we have changed the permissions, let’s see the output of ls -alh now

Quite what we wanted, isn’t it?

You can do this with directories as well. But if you do chmod for a directory then it will only affect the directory itself, and not the files and directories inside it. To recursively change the permissions inside a directory, you will have to add the -R flag, like this:

chmod o-r -R graphic-assets

Creating and writing files

This is quite useful when you are working on a remote machine.

To create a file you can use the touch command

touch hello.txt

Now let’s enter some text in it using the cat command

cat > hello.txt
hello from the other sideeeee!

After this, you can press ctrl + C to exit the prompt

Now, if you check the contents of hello.txt, you will be able to see the text you entered above.

cat command is versatile in a way that it also lets you read the contents of a file

cat hello.txt

This will show the contents of the file

Read and stream a file

tail command is similar to cat in a way that it lets you read a file.

But unlike cat you can define how you want to view it.

For e.g. if you want to see only the last 100 lines of a log file (log files contain thousands of lines of text) you can use the following command

tail -n100 log_filename

An added advantage of tail command is that you can use it to stream a file as well

tail -f filename

This command will give you a streaming output that keeps updating as the log file is updated

Filtering the output of a command

The above commands to read through a file can produce hundreds of lines of output. But that will make it difficult for you to go through them.

You can filter out the output so any line that contains a substring you want to see will be shown in the output.

Let me introduce you to pipe in unix. A pipe is used to send the output of a certain command to a different command. A pipe is represented by | .

grep is the command you use to filter out an output. If you pass the output of tail to the grep command via a pipe, that should do the trick of filtering lines.

Let’s say you only want to view only the lines in a file that contain the word gunicorn in it then this is how you do it

tail -n100 log_filename | grep 'gunicorn'

A much more powerful tool for filtering is awk

awk follows the pattern

awk options 'selection _criteria {action }' input-file

An example of awk command is like this

awk '/gunicorn/ { print $1 }' log_filename

Here, the command will print out the first word (indicated by $1) of lines that contain the word “gunicorn”

awk has a lot of options that cannot be included in this article for brevity purposes. You can refer to this blog post for more options.

Input and output redirection

In Linux, the input, output, and error are given their own numbers.

Standard input from the terminal is 0

Standard output from the terminal is 1

And the standard error is 2

So if your command produces an error, then you can explicitly write it an error file like this

error_command 2>error_file

You can also merge the output of your error with the standard output like this:

error_command 2>&1

Why you should install zsh

zsh makes you a really fast developer. zsh supports all the commands that bash supports. On top of that it has some really cool features. When you install it with ohmyzsh, it becomes super powerful .

My personal favorite feature is not having to type cd to change directories. In fact, you don’t even have to complete the names of the directories either

Before zsh:

cd /home/denny/directory

After zsh:

/ho/de/d

zsh associates the characters to the existing directory structure and figures out what directory you want to cd into

Also, the command history is super easy to browse through when you have zsh.

Take a scenario where you entered a wget command a couple of days back but you don’t remember it now. Going through the history of two days of commands would be tiresome. If you have zsh, all you have to do is type wget and press the up arrow key. It will only show you the commands which start with wget!

Apart from this, you can add custom themes and plugins using ohmyzsh which makes working on the terminal a pretty cool thing.

Wrapping up

There are a few other commands that help me a lot and I would like to give them a mention.

htop: Gives you the system usage of CPU, memory and other parameters. Really helpful while debugging system loads.

vim: I was never a fan of vim but once you learn it, it becomes easy for you to browse through files on remote servers.

crontab -e: If you want to run periodic tasks then this is what you should go for.

du -sh: This command gives you the space consumed by your directories and files.

find: Akin to the search functionality in your GUI filesystem, this one does a great job in looking up files on remote servers.

Software Engineering Weekly

I write about Software Engineering and how to scale your applications on my weekly newsletter. To get such stories directly in your inbox, subscribe to it! 😃

If you like my content and want to support me to keep me going, consider buying me a coffee ☕️ ☕️