Had to figure out how to do string replacement (Scott wanted to convert WMA files to similarly named MP3 files) and pass a single parameter that has spaces into a shell script.
${original_string/thing_to_find/thing_to_replace_there} does string replacement. And $@ is the unexpanded parameter set. So this wouldn’t work if we were trying to pass in more than one parameter (well, it *would* … I’d just have to custom-handle parameter expansion in the script)
Yes, I know md5sum has a “-c” option for checking the checksum in a file … but, if I was going to screw with a file, I’d have the good sense to edit the checksum file in the archive!
#!/bin/bash #STRFILE=dead.letter #STRCHECKMD5=5f3748d9c653b78c9ee7559acd423652 #STRMD5=`md5sum $STRFILE` STRFILE=$1 STRCHECKMD5=$2 STRMD5=`md5sum $STRFILE` diff -s <( printf '%s\n' "$STRCHECKMD5 $STRFILE" ) <( printf '%s\n' "$STRMD5" )
It’ll either output the file hash and the hash to match (a problem) or indicate the files are identical (a good thing)
A quick one-line Bash command to output all containers and the startup policy:
docker container ls -aq | xargs docker container inspect --format '{{ .Name }}: {{.HostConfig.RestartPolicy.Name}}'
For Docker on Windows, the following PowerShell command produces similar results:
$jsonData = docker container ls -aq |%{docker container inspect --format "{{json .}}"$_}
[System.Collections.ArrayList]$arrayContainerConfig = @()
foreach($jsonContainerConfig in $jsonData ){
$psobjContainerConfig = ConvertFrom-JSON $jsonContainerConfig
$arrayContainerConfig.add(@{Name=$psobjContainerConfig.Name;Hostname=$psobjContainerConfig.Config.Hostname;CurrentlyRunning=$psobjContainerConfig.State.Running;RestartPolicy=$psobjContainerConfig.HostConfig.RestartPolicy.Name})
}
$arrayContainerConfig | ForEach {[PSCustomObject]$_} | Format-Table -AutoSize
We occasionally have to re-home our shell scripts, which means updating any static path values used within scripts. It’s quick enough to build a sed script to convert /old/server/path to /new/server/path, but it’s still extra work.
The dirname command works to provide a dynamic path value, provided you use the fully qualified path to run the script … but it fails spectacularly whens someone runs ./scriptFile.sh and you’re trying to use that path in, say, EXTRA_JAVA_OPTS. The “path” is just . — and Java doesn’t have any idea what to do with “-Xbootclasspath/a:./more/path/goes/here.jar”
Voila, realpath gives you the fully qualified file path for /new/server/path/scriptFile.sh, ./scriptFile.sh, or even bash scriptFile.sh … and the dirname of a realpath is the fully qualified path where scriptFile.sh resides:
#!/bin/bash
DIRNAME=`dirname $(realpath "$0")`
echo ${DIRNAME}
Hopefully next time we’ve got to re-home our batch jobs, it will be a simple scp & sed the old crontab content to use the new paths.
One of the web servers at work uses a refspec in the “git pull” command to map the remote development branch to the local remote-tracking master branch. This is fairly confusing (and it looks like the dev server is using the master branch unless you dig into how the pull is performed), but I can see how this prevents someone from accidentally typing something like “git checkout master” and really messing up the development environment. I can also see a dozen ways someone can issue what is a completely reasonable git command 99% of the time and really mess up the development environment.
While it is simple enough to just checkout the development branch, doing so does open us up to the possibility that someone will erroneously deliver the production code to the development server and halt all testing. While you cannot create shell aliases for multi-word commands (or, more accurately, alias expansion is performed for the first word of a simple command is checked to see if it has an alias … so you’ll never get the multi-word command), you can define a function to intercept git commands and avoid running unwanted commands:
function git() {
case $* in
"checkout master" ) command echo "This is a dev server, do not checkout the master branch!" ;;
"pull origin master" ) command echo "This is a dev server, do not pull the master branch" ;;
* ) command git "$@" ;;
esac
}
Or define the desired commands and avoid running any others:
function git(){
if echo "$@" | grep -Eq '^checkout uat$'; then
command git $@
elif echo "$@" | grep -Eq '^pull .+ uat$'; then
command git $@
else
echo "The command $@ needs to be whitelisted before it can be run"
fi
}
Either approach mitigates the risk of someone incorrectly using the master branch on the development server.
Ran into a problem using Sphinx to document some Python modules and scripts that Scott put together for OpenHAB. They’re making some changes to the files to get Sphinx to process them, thus making copies of the original code. Problem is, some of the folders just weren’t showing up in the copy. Needed to change IFS in order to tokenize the find results into full paths that don’t break on spaces in file or folder names.
SAVEDIFS=$IFS
IFS=$(echo -en "\n\b")
for DIRNAME in $(find "$COMMUNITY_DIR" -maxdepth 1 -type d 2>/dev/null); do
echo $DIRNAME
done
IFS=$SAVEDIFS
I’ve used sed to replace file content — use a regex to replace the sendmail.cf line that routes mail directly with a smarthost directive
sed -i -e 's/^DS/DS\\\[mailTWB.example.com\\\]/' $strSendmailDirectory/etc/mail/sendmail.cf
But I’ve needed to prepend text to a file. Turns out sed acn do that. In fact, you can insert strings at any line number. Using “sed -i ‘5s;^;StringsToInsert\n;’ filename.xtn will insert “StringsToInsert\n” at line 5. To prepend text to a file, use “1s”
[lisa@fedora tmp]# cat test.txt;sed -i ‘5s;^;NewLine1\nNewLine2\n;’ test.txt;cat test.txt
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
**********
Line 1
Line 2
Line 3
Line 4
NewLine1
NewLine2
Line 5
Line 6
**********
I’ve also come across an oddity in the Win32 sed — the method I usually use to blow away everything after a newline for some reason blows away everything after the first line. Works fine on RHEL7 and Fedora29, so the quick solution is “run it from the Linux box”.
C:\temp>cat input.txt
line 1
line 2
line 3
line 4
line 5
C:\temp>sed -i ‘/^$/q’ input.txt&cat input.txt
line 1
We occasionally have to re-home our shell scripts, which means updating any static path values used within scripts. It’s quick enough to build a sed script to convert /old/server/path to /new/server/path, but it’s still extra work.
The dirname command works to provide a dynamic path value, provided you use the fully qualified path to run the script … but it fails spectacularly whens someone runs ./scriptFile.sh and you’re trying to use that path in, say, EXTRA_JAVA_OPTS. The “path” is just . — and Java doesn’t have any idea what to do with “-Xbootclasspath/a:./more/path/goes/here.jar”
Voila, realpath gives you the fully qualified file path for /new/server/path/scriptFile.sh, ./scriptFile.sh, or even bash scriptFile.sh … and the dirname of a realpath is the fully qualified path where scriptFile.sh resides:
#!/bin/bash
DIRNAME=`dirname $(realpath "$0")`
echo ${DIRNAME}
Hopefully next time we’ve got to re-home our batch jobs, it will be a simple scp & sed the old crontab content to use the new paths.
Determining if a specific file exists within a shell script is straight-forward:
if [ -f filename.txt ]; then DoSomething; fi
The -f verifies that a regular file exists. You might want -s (exists and size is greater than zero), -w (exists and is writable), -e (a regular or special file exists), etc. But the test comes from the “CONDITIONAL EXPRESSIONS” section of the bash man page and is simply used in an if statement.
What if you don’t know the exact name of the file? Using the text “if [ -f *substring*.xtn ]” seems like it works. If there is no matching file, the condition evaluates to FALSE. If there is one matching file the condition evaluates to TRUE. But when there are multiple matching files, you get an error because there are too many parameters
[lisa@fc test]# ll total 0 [lisa@fc test]# if [ -f *something*.txt ]; then echo "The file exists"; fi [lisa@fc test]# touch 1something1.txt [lisa@fc test]# if [ -f *something*.txt ]; then echo "The file exists"; fi The file exists [lisa@fc test]# touch 2something2.txt [lisa@fc test]# if [ -f *something*.txt ]; then echo "The file exists"; fi -bash: [: 1something1.txt: binary operator expected
Beyond throwing an error … we are not executing the code-block meant to be run when the condition is TRUE. In a shell script, execution will continue past the block as if the condition evaluated to FALSE (i.e. the script does not just abnormally end on the error, making the failure more obvious).
To test for the existence of possibly multiple files matching a pattern, we can evaluate the number of files returned from ls. I include 2>/dev/null to hide the error which will otherwise be displayed when there are zero matching files.
[lisa@fc test]# ll total 0 [lisa@fc test]# if [ $(ls *something*.txt 2>/dev/null | wc -l) -gt 0 ]; then echo "Some matching files are found."; fi [lisa@fc test]# touch 1something1.txt [lisa@fc test]# if [ $(ls *something*.txt 2>/dev/null | wc -l) -gt 0 ]; then echo "Some matching files are found."; fi Some matching files are found. [lisa@fc test]# touch 2something2.txt [lisa@fc test]# if [ $(ls *something*.txt 2>/dev/null | wc -l) -gt 0 ]; then echo "Some matching files are found."; fi Some matching files are found. [lisa@fc test]#
Now we have a test that evaluates to TRUE when there are one or more matching files in the path.