Category: Coding

Shell Script: Path To Script

We occasionally have to re-home our shell scripts, which means updating any static path values used within scripts. It’s quick enough to build a sed script to convert /old/server/path to /new/server/path, but it’s still extra work.

The dirname command works to provide a dynamic path value, provided you use the fully qualified path to run the script … but it fails spectacularly whens someone runs ./scriptFile.sh and you’re trying to use that path in, say, EXTRA_JAVA_OPTS. The “path” is just . — and Java doesn’t have any idea what to do with “-Xbootclasspath/a:./more/path/goes/here.jar”

Voila, realpath gives you the fully qualified file path for /new/server/path/scriptFile.sh, ./scriptFile.sh, or even bash scriptFile.sh … and the dirname of a realpath is the fully qualified path where scriptFile.sh resides:

#!/bin/bash 
DIRNAME=`dirname $(realpath "$0")`
echo ${DIRNAME}

Hopefully next time we’ve got to re-home our batch jobs, it will be a simple scp & sed the old crontab content to use the new paths.

Modifying Shared PHP Function

We needed to modify a shared function to include additional information … but didn’t want to coordinate changing all of the calls to the function as one change. Simplest way to accomplish that was to set a default value for the new parameter — either to NULL and just not do the new thing when the parameter is NULL or some value that indicates that we’re not yet gathering that data.

<?php

function testFunction($strOldParameter, $strNewParameter=NULL){
     echo "The old parameter is |$strOldParameter|\n";
     if($strNewParameter){
          echo "The new parameter is |$strNewParameter|\n";
     }
}

testFunction("first", "second");
testFunction("justFirst");

?>

Preventing erronious use of the master branch on development servers

One of the web servers at work uses a refspec in the “git pull” command to map the remote development branch to the local remote-tracking master branch. This is fairly confusing (and it looks like the dev server is using the master branch unless you dig into how the pull is performed), but I can see how this prevents someone from accidentally typing something like “git checkout master” and really messing up the development environment. I can also see a dozen ways someone can issue what is a completely reasonable git command 99% of the time and really mess up the development environment.

While it is simple enough to just checkout the development branch, doing so does open us up to the possibility that someone will erroneously  deliver the production code to the development server and halt all testing. While you cannot create shell aliases for multi-word commands (or, more accurately, alias expansion is performed for the first word of a simple command is checked to see if it has an alias … so you’ll never get the multi-word command), you can define a function to intercept git commands and avoid running unwanted commands:

function git() { 
     case $* in 
         "checkout master" ) command echo "This is a dev server, do not checkout the master branch!" ;; 
         "pull origin master" ) command echo "This is a dev server, do not pull the master branch" ;; 
         * ) command git "$@" ;; 
     esac
}

Or define the desired commands and avoid running any others:

function git(){
     if echo "$@" | grep -Eq '^checkout uat$'; then
          command git $@
     elif echo "$@" | grep -Eq '^pull .+ uat$'; then
          command git $@
     else
          echo "The command $@ needs to be whitelisted before it can be run"
     fi
}

Either approach mitigates the risk of someone incorrectly using the master branch on the development server.

Handling PHP Execution Timeout

There’s no straight-forward way to handle execution timeout in PHP 5.x — it’s not like you can try/except or something. The execution time-limit is exceeded, the program terminates. Which, thinking from the perspective of the person who maintains the server, is a Good Thing … bugger up the ‘except’ component and now that becomes an infinite loop.

But I’m looking to throw a “pretty” error to the end user and have them try again with a data set that will take less time to process. Turns out, you can use a shutdown function to display something other than the generic PHP time limit exceeded page.

<?php

function runOnShutdown(){
     $arrayError = error_get_last();

     if( substr($arrayError['message'], 0, strlen("Maximum execution time of")) === "Maximum execution time of"   ){
          echo "<P>Maximum execution time";
     }
     elseif($arrayError){
          print_r($arrayError);
     }
}

function noOp($iInteger){
     for($z = 0; $z < $iInteger; $z++){
          $a = $iInteger * $iInteger;
     }
     return $iInteger;
}

register_shutdown_function('runOnShutdown');
ini_set('display_errors', '0');
ini_set('max_execution_time', 2);

// for($i = 0; $i < 10; $i++){
for($i = 0; $i < 10000; $i++){
     $j = noOp($i);
     print "<P>$j</P>\n";
}
print "<P>Done</P>\n";

?>

And the web output includes a customized message because the max execution time has been exceeded.

 

Reverting a Single File with Git

Git revert is great for resetting the entire project to a particular state – I went down a bad path, really don’t want to do this, and resetting to the state I was in this morning is exactly what I want to do. Sometimes, though … that’s not the case. I added a couple of debugging lines to a file that I don’t really need. Or I’ve gone down a bad path here but have good work in a few other files too. In those cases, you can revert a single file to the latest committed version. Run “git status” and “git diff” to confirm that it is an uncommited change.

To revert a single file to its latest committed state, use “git checkout – filename” – you can see the added line has disappeared.

 

Git Log

Git log can be used to get a quick summary of the differences between two branches. The three dots between the branch names indicates you want a “symmetric difference”. That is the set of commits that are in the branch on the left or the branch on the right but not in both branches.

The –left-right option prefixes each commit with an ‘<’ or ‘>’ indicating which “side” has the commit. The –oneline option prints the abbreviated commit ID and the first line of the commit message.

Showing the differences between your local uat branch and the remote uat branch:

D:\git\gittest>git log –left-right –oneline origin/uat…uat

> 961f53a (uat) Merge branch ‘ossa-123’ into uat

> 803096b (origin/ossa-123, ossa-123) Added additional files

> cf9c419 Added initial code to branch

The top line is the most recent commit, the bottom line is the oldest commit that does not exist in both branches. I can see that the uat branch in my local repo is not missing anything from the remote (there are no commits with “<” indicating changes in the remote that do not exist in my local copy) but I have local changes which have not yet been pushed: two code commits plus the merge commit which incorporated the code commits to my local repo’s uat branch. The head of the local and remote ossa-123 branch are at the commit just prior to the merge, so on my local repo that branch has been fully merged into UAT and I just need to push uat up to the remote.

Additional options to enhance output:

–cherry-pick will omit any changes that represent the same changes in both branches (or –cherry-mark to mark those commits with an “=” flag)

–graph uses an ASCII chart to depict branch relationships.

* The three dots mean something different in git diff than in git log. In git diff, mean “what are the differences between the right-hand branch and the common ancestor shared by both the right and left-hand branches”.

Two dots in git diff mean is the differences that are in the branch on the left or the branch on the right but not in both branches.

In git log, two dots displays only commits unique to the second branch. Since commits and differences are not exactly the same thing, two and three dots don’t exactly have the opposite meaning between diff and log. But the meaning is not logically consistent.

 

Testing Procedural Code with PHPUnit

You can use PHPUnit to test procedural code — in this case, I’m testing the output of a website. I have some Selenium tests for UI components but wanted to use the shell executor for functional testing. In the test code, you can populate the _SERVER and _POST (or _GET) arrays and simulate the web environment.

<?php
    namespace phpUnitTests\CircuitSearch;
    class CircuitExportTest extends \PHPUnit_Framework_TestCase{
        private function _execute(array $paramsPost = array(), array $paramsServer = array() ) {
            $_POST = $paramsPost;
            $_SERVER = $paramsServer;
            ob_start();
            include "../../myWebSitePage.php";
            return ob_get_clean();
        }
        public function testUsageLogging(){
            $argsPost = array('strInput'=>'SearchValue', 'strReportFormat'=>'JSON');
            $argsServer = array("DOCUMENT_ROOT" => '/path/to/website/code/html/', "HOSTNAME" => getHostByName(),
                                "SERVER_ADDR" => getHostByName(php_uname('n')), "PWD" => '/path/to/website/code/html/subcomponent/path');
            $this->assertEquals('{}', $this->_execute($argsPost, $argsServer));
        }
    }
?>

Running the test, my web output is compared to the static string in assertEquals. In this case, I am searching for a non-existent item, nothing is returned, and I expect to get empty braces. I could use AssertsRegExp or or AssertsStringContainsString to verify the specifics of a real result set.

Python f-strings

I’m accustomed to using variables directly in strings — most of my scripting experience was with Bash, Perl, and PHP. It’s something I have to think about when I would write Java or C++ code, and it’s one of the few things that struck me as a step backward in Python. This is basically the advantage that kwargs have over args, except changing a string common where changing a function definition is pretty rare.

I’ve taken to using the arg identifier numbers in the print statement.

>>> first=”Lisa”
>>> last=”Rushworth”
>>> print(“Hi {0}, you are {0} {1}.”.format(first, last))
Hi Lisa, you are Lisa Rushworth.

This allows me to append a new variable to the format argument list and use it anywhere in the string
>>> salutation=”Hello”
>>> print(“{2} {0}, you are {0} {1}.”.format(first, last, salutation))
Hello Lisa, you are Lisa Rushworth.

Python 3 introduced f-strings, which go beyond the functionality I was missing. You *can* use variable names within the string:

>>> print(f”{salutation} {first}, you are {first} {last}”)
Hello Lisa, you are Lisa Rushworth

You can have multi-line strings — notice that newlines aren’t being magically added. If you *want* newlines, add \n!

>>> strString = (
… f”{salutation} {first}, ”
… f”You are {first} {last} ”
… f”And I have more ”
… f”data down here”
… )
>>> print(strString)
Hello Lisa, You are Lisa Rushworth And I have more data down here

But you can also perform operations:
>>> print(f”{2 ** 3}”)
8

And use the f-string in classes
>>> class MyThing:
… def __init__(self, thing1, thing2, thing3):
… self.thing1 = thing1
… self.thing2 = thing2
… self.age = thing3
…
… def __str__(self):
… return f”{self.thing1} has 2 {self.thing2} and 3 {self.age}.”
…
>>>
>>> thingSample = MyThing(“Something1”, “Something2”, “Something3″)
>>> print(f”{thingSample}”)
Something1 has 2 Something2 and 3 Something3.

What if you need to have a literal curly brace?
>>> print(f”Hi {first}, this output needs a {{curly brace}}”)
Hi Lisa, this output needs a {curly brace}

VS Code Keyboard Shortcut: Commenting

We’ve been watching Microsoft’s Python programming course series — only the first couple so far, so it’s a lot of “yeah, knew that”. *But* there’s the occasional cool tip. You can comment line(s) of code with a keyboard shortcut:

Ctrl k,c comment current line (or selection)
Ctrl k,u uncomment current line (or selection)

 

Highlight the section of code that you want to comment out.

Hit ctrl-k and then ctrl-c … voila, a whole section of commented code. Denoting a comment is language specific, so this does not do anything if VS Code has not yet identified the language for your file.

To remove comments, use ctrl-k then ctrl-u. This works with code that already includes comments – you’ll get an additional comment in front of the commented line, and that additional comment will be removed leaving a commented line.