Category: Bash

Quick way to make HDD light blink

Damien

A quick and dirty way to make a specific hard drive LED blink so that it can be identified.

while true; do smartctl -a /dev/<device>; done

The simple trick is to run smartctl in a while loop. The command will run forever until you Ctrl + C. It make the LED blink rather obviously, which made things much easier.

Other method :

To start: ledctl locate=/dev/sdd
To stop: ledctl off={/dev/sdd}

This lives in the ledmon package.

Find files containing a specific text string

Damien 
grep -rnlw '/path/to/dir/' -e 'pattern'
  • -r or -R is recursive,
  • -n is line number, and
  • -w stands for match the whole word.
  • -l can be added to just give the file name of matching files.
  • -e is the pattern used during the search

Along with these, --exclude, --include, --exclude-dir flags could be used for efficient searching:

  • This will only search through those files which have .c or .h extensions:
    grep --include=\*.{c,h} -rnw '/path/to/search/' -e "pattern"
  • This will exclude searching all the files ending with .o extension:
    grep --exclude=\*.o -rnw '/path/to/search/' -e "pattern"
  • For directories it’s possible to exclude one or more directories using the --exclude-dir parameter. For example, this will exclude the dirs dir1/, dir2/ and all of them matching *.dst/:
    grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/search/' -e "pattern"

Find all the symlinks in a directory tree

Damien

This will recursively traverse the /path/to/folder directory and list only the symbolic links:

ls -lR /path/to/folder | grep '^l'

If your intention is to follow the symbolic links too, you should use your find command but you should include the -L option; in fact the find man page says:

   -L     Follow symbolic links.  When find examines or prints information
          about files, the information used shall be taken from the  prop‐
          erties  of  the file to which the link points, not from the link
          itself (unless it is a broken symbolic link or find is unable to
          examine  the file to which the link points).  Use of this option
          implies -noleaf.  If you later use the -P option,  -noleaf  will
          still  be  in  effect.   If -L is in effect and find discovers a
          symbolic link to a subdirectory during its search, the subdirec‐
          tory pointed to by the symbolic link will be searched.

          When the -L option is in effect, the -type predicate will always
          match against the type of the file that a symbolic  link  points
          to rather than the link itself (unless the symbolic link is bro‐
          ken).  Using -L causes the -lname and -ilname predicates  always
          to return false.

Then try this:

find -L /var/www/ -type l

This will probably work: I found in the find man page this diamond: if you are using the -type option you have to change it to the -xtype option:

          l      symbolic link; this is never true if the -L option or the
                 -follow option is in effect, unless the symbolic link  is
                 broken.  If you want to search for symbolic links when -L
                 is in effect, use -xtype.

Then:

find -L /var/www/ -xtype l

Difference Between .bashrc and .bash_profile

Damien

Interactive Login and Non-Login Shell

When invoked, Bash reads and executes commands from a set of startup files. What files are read depends upon whether the shell is invoked as an interactive login or non-login shell.

A shell can be interactive or non-interactive.

In simple terms, an interactive shell is a shell that reads and writes to a user’s terminal, while a non-interactive shell is a shell that is not associated with a terminal, like when executing a script.

An interactive shell can be either login or non-login shell.

A login shell is invoked when a user login to the terminal either remotely via ssh or locally, or when Bash is launched with the --login option. An interactive non-login shell is invoked from the login shell, such as when typing bash in the shell prompt or when opening a new Gnome terminal tab.

Bash Startup Files

When invoked as an interactive login shell, Bash looks for the /etc/profile file, and if the file exists , it runs the commands listed in the file. Then Bash searches for ~/.bash_profile, ~/.bash_login, and ~/.profile files, in the listed order, and executes commands from the first readable file found.

When Bash is invoked as an interactive non-login shell, it reads and executes commands from ~/.bashrc, if that file exists, and it is readable.

Difference Between .bashrc and .bash_profile

.bash_profile is read and executed when Bash is invoked as an interactive login shell, while .bashrc is executed for an interactive non-login shell.

Use .bash_profile to run commands that should run only once, such as customizing the $PATH environment variable .

Put the commands that should run every time you launch a new shell in the .bashrc file. This include your aliases and functions , custom prompts, history customizations , and so on.

Typically, ~/.bash_profile contains lines like below that source the .bashrc file. This means each time you log in to the terminal, both files are read and executed.

if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

Most Linux distributions are using ~/.profile instead of ~/.bash_profile. The ~/.profile file is read by all shells, while ~/.bash_profile only by Bash.

Find & delete files

Damien 
find /path/to/dir -args

Delete empty directories

find /path/to/dir -empty -type d -delete

Delete empty files

find /path/to/dir -empty -type f -delete

Count all empty files or directories

Count empty dirs only 

find /path/to/dir/ -empty -type d | wc -l

Count empty files only 

find /path/to/dir/ -empty -type f | wc -l

Where :

  • -empty : Only find empty files and make sure it is a regular file or a directory.
  • -type d : Only match directories.
  • -type f : Only match files.
  • -delete : Delete files. Always put -delete option at the end of find command as find command line is evaluated as an expression, so putting -delete first will make find try to delete everything below the starting points you specified.

This is useful when you need to clean up empty directories and files in a single command.

Cleanup unnecessary files and Desktop OSses clutter

cd "$SOURCEDIR"
find ./ \( -name ".DS_Store" -or -name "._*" -or -name "Thumbs.db" -or -name "*.tmp" -or -name "*.lnk" -or -name "TemporaryItems" -or -name "folders.501" -or -name ".TemporaryItems" -or -name "__MACOSX" \) -ls -delete

Change ownership of files owned by a specific username

find /path/to/dir/ -user username -exec chown newuser:newgroup {} +

How can I permanently add my SSH private key to Keychain so it is automatically available to ssh?

Damien

On OSX, the native ssh-add client has a special argument to save the private key’s passphrase in the OSX keychain, which means that your normal login will unlock it for use with ssh. On OSX Sierra and later, you also need to configure SSH to always use the keychain (see Step 2 below).

Alternatively you can use a key without a passphrase, but if you prefer the security that’s certainly acceptable with this workflow.

Step 1 – Store the key in the keychain

Just do this once:

ssh-add -K ~/.ssh/[your-private-key]

Enter your key passphrase, and you won’t be asked for it again.

(If you’re on a pre-Sierra version of OSX, you’re done, Step 2 is not required.)

Step 2 – Configure SSH to always use the keychain

It seems that OSX Sierra removed the convenient behavior of persisting your keys between logins, and the update to ssh no longer uses the keychain by default. Because of this, you will get prompted to enter the passphrase for a key after you upgrade, and again after each restart.

The solution is fairly simple, and is outlined in this github thread comment. Here’s how you set it up:

  1. Ensure you’ve completed Step 1 above to store the key in the keychain.
  2. If you haven’t already, create an ~/.ssh/config file. In other words, in the .ssh directory in your home dir, make a file called config.
  3. In that .ssh/config file, add the following lines:
    Host *
    UseKeychain yes
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_rsa
    Change ~/.ssh/id_rsa to the actual filename of your private key. If you have other private keys in your ~.ssh directory, also add an IdentityFile line for each of them. For example, I have one additional line that reads IdentityFile ~/.ssh/id_ed25519 for a 2nd private key. The UseKeychain yes is the key part, which tells SSH to look in your OSX keychain for the key passphrase.
  4. That’s it! Next time you load any ssh connection, it will try the private keys you’ve specified, and it will look for their passphrase in the OSX keychain. No passphrase typing required.

Useful Linux commands

Damien

1- Save man-page as pdf

man -t awk | ps2pdf - awk.pdf

2- Duplicate installed packages from one machine to the other (RPM-based systems)

ssh root@remote.host "rpm -qa" | xargs yum -y install

3- Stamp a text line on top of the pdf pages to quickly add some remark, comment, stamp text, … on top of (each of) the pages of the input pdf file

echo "This text gets stamped on the top of the pdf pages." | enscript -B -f Courier-Bold16 -o- | ps2pdf - | pdftk input.pdf stamp - output output.pdf

4- Display the number of connections to a MySQL Database

Count the number of active connections to a MySQL database.
The MySQL command “show processlist” gives a list of all the active clients.
However, by using the processlist table, in the information_schema database, we can sort and count the results within MySQL.

mysql -u root -p -BNe "select host,count(host) from processlist group by host;" information_schema

5- Create a local compressed tarball from remote host directory

ssh user@host "tar -zcf - /path/to/dir" > dir.tar.gz

This improves on #9892 by compressing the directory on the remote machine so that the amount of data transferred over the network is much smaller. The command uses ssh(1) to get to a remote host, uses tar(1) to archive and compress a remote directory, prints the result to STDOUT, which is written to a local file. In other words, we are archiving and compressing a remote directory to our local box.

6- tail a log over ssh

This is also handy for taking a look at resource usage of a remote box.

ssh -t remotebox "tail -f /var/log/remote.log"

7- Print diagram of user/groups

Parses /etc/group to “dot” format and pases it to “display” (imagemagick) to show a usefull diagram of users and groups (don’t show empty groups).

awk 'BEGIN{FS=":"; print "digraph{"}{split($4, a, ","); for (i in a) printf "\"%s\" [shape=box]\n\"%s\" -> \"%s\"\n", $1, a[i], $1}END{print "}"}' /etc/group|display

8- Draw kernel module dependancy graph.

Parse `lsmod’ output and pass to `dot’ drawing utility then finally pass it to an image viewer

lsmod | perl -e 'print "digraph \"lsmod\" {";<>;while(<>){@_=split/\s+/; print "\"$_[0]\" -> \"$_\"\n" for split/,/,$_[3]}print "}"' | dot -Tpng | display -

9- Create strong, but easy to remember password

Why remember? Generate!
Up to 48 chars, works on any unix-like system

read -s pass; echo $pass | md5sum | base64 | cut -c -16

10- Find all files larger than 500M and less than 1GB

find / -type f -size +500M -size -1G

11- Find size of directories and files in current directory

du -h -d 1

12- Limit the cpu usage of a process

This will limit the average amount of CPU it consumes.

sudo cpulimit -p pid -l 50

13- Show the difference between two dirs

diff -qr /var/some/dir1 var/some/dir2