Tag: git

Git Log

Git log can be used to get a quick summary of the differences between two branches. The three dots between the branch names indicates you want a “symmetric difference”. That is the set of commits that are in the branch on the left or the branch on the right but not in both branches.

The –left-right option prefixes each commit with an ‘<’ or ‘>’ indicating which “side” has the commit. The –oneline option prints the abbreviated commit ID and the first line of the commit message.

Showing the differences between your local uat branch and the remote uat branch:

D:\git\gittest>git log –left-right –oneline origin/uat…uat

> 961f53a (uat) Merge branch ‘ossa-123’ into uat

> 803096b (origin/ossa-123, ossa-123) Added additional files

> cf9c419 Added initial code to branch

The top line is the most recent commit, the bottom line is the oldest commit that does not exist in both branches. I can see that the uat branch in my local repo is not missing anything from the remote (there are no commits with “<” indicating changes in the remote that do not exist in my local copy) but I have local changes which have not yet been pushed: two code commits plus the merge commit which incorporated the code commits to my local repo’s uat branch. The head of the local and remote ossa-123 branch are at the commit just prior to the merge, so on my local repo that branch has been fully merged into UAT and I just need to push uat up to the remote.

Additional options to enhance output:

–cherry-pick will omit any changes that represent the same changes in both branches (or –cherry-mark to mark those commits with an “=” flag)

–graph uses an ASCII chart to depict branch relationships.

* The three dots mean something different in git diff than in git log. In git diff, mean “what are the differences between the right-hand branch and the common ancestor shared by both the right and left-hand branches”.

Two dots in git diff mean is the differences that are in the branch on the left or the branch on the right but not in both branches.

In git log, two dots displays only commits unique to the second branch. Since commits and differences are not exactly the same thing, two and three dots don’t exactly have the opposite meaning between diff and log. But the meaning is not logically consistent.

 

Using File System As A Git Repository

I’ve used GitLab for quite some time, and as a full featured CI/CD platform that also provides git functionality … it’s awesome. But it’s also serious overkill for someone who wants to coordinate code with another developer or two. Or just keep history for their code. Or a backup. To accomplish this, all you need is drive space. If you’ve got a file server, any folder on the share can be a git repository.

On the server, create a git repository and add something to it.

Z:\Temp\test>git init
Initialized empty Git repository in Z:/Temp/test/.git/

Z:\Temp\test>notepad testfile.txt

Z:\Temp\test>git add testfile.txt

Z:\Temp\test>git commit -m "Initial file upload"
[master (root-commit) 9a3ebe7] Initial file upload
1 file changed, 1 insertion(+)
create mode 100644 testfile.txt

Then on your client, either clone the repo from the drive path

C:\Users\lisa>git clone file://z:/temp/test
Cloning into 'test'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

Or from the UNC path

C:\Users\lisa>git clone file://server01.example.com/data/temp/test
Cloning into 'test'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

I prefer to use the UNC paths – if my drive letter fails to map for some reason, the UNC path is still available.

If you’ve got pre-existing code, there’s a bit of a different process. On the server, create an empty folder and use “git init” to initialize the empty repo. On the client where the code exists, run:

git init
git add *
git commit -m “Initial code upload”
git remote add origin git clone file://server01.example.com/data/temp/test
git push origin master

 

Git: Listing Conflicting Files

To list the unmerged files — where you’ve got merge conflicts to resolve:

git diff --name-only --diff-filter=U

Filters are:

  • Added (A)
  • Copied (C)
  • Deleted (D)
  • Modified (M)
  • Renamed (R)
  • Type changed (T)
  • Unmerged (U)
  • Unknown (X)
  • pairing Broken (B)

(use lower case letters to exclude)

Renaming a Branch in Git

I finally had a situation where I needed to rename a branch in git. When I was the only one involved in a development effort (or even looking at it!), it didn’t really matter if I typo’d something. Exchange and Exchagne … I know what I meant. But working under a more formal development process, I started naming my branch after the issue ID. And managed to typo the first one. Sigh!

# Check out the incorrectly named branch

git checkout OSSA166

# Rename it with the correct name

git branch -m OSSA163

# See what you’ve got — the local one is right now, but the remote is still incorrectly named

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA166
remotes/origin/master
remotes/origin/uat

# Push a change to rename the remote one too

git push origin :OSSA166 OSSA163

Total 0 (delta 0), reused 0 (delta 0)

To ssh://git.example.com/path/to/my/repo.git

– [deleted]         OSSA166
* [new branch]      OSSA163 -> OSSA163

# And see what you’ve got again

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA163
remotes/origin/master
remotes/origin/uat

 

Sparse Checkout With Git

I’ve encountered a few repositories that are huge. Unwieldy huge, and stuffed with files that aren’t relevant to what I need. The straight-forward solution is to use multiple repositories — that’s what I do at work with my code samples. There’s a different repo for each language because the PHP developers really don’t care what the C# code looks like. The Java developers don’t need a copy of the Python code. But there are advantages to having a single repository that may preclude you from taking the simple solution. Git sub-modules are an interesting approach — combining multiple repositories into a single functional unit. But that’s a pretty big change to an existing repo. And, if you participate in open source projects, it may not be your decision anyway.

There’s another option for selectively cloning when you’re working with a large repo — an option that doesn’t require any changes to the repository. An end user can perform a sparse checkout — essentially use a filter like .gitignore to select or deselect certain files/folders from being pulled into the local working directory. The file is named sparse-checkout and is located in .git\info — unlike a .gitignore file which indicates what shouldn’t get included, sparse-checking controls what is included (if you want an entire repo except one folder, use !path/to/folder/**)

The sparse-checkout file used to get just the core components of Scott’s OpenHAB helper libraries plus the OpenWeatherMap community scripts is:

.github/**
Core/**
Community/OpenWeatherMap/**

To use sparse checkout, set the core.sparseCheckout config value to true. You can add sparse checkout to a repo you’ve already cloned and use

git read-tree -mu HEAD

to “clean up” unwanted files. Or you can set up sparse checkout before you clone the repo

D:\tmp>mkdir ljrtest

D:\tmp>cd ljrtest

D:\tmp\ljrtest>git init
Initialized empty Git repository in D:/tmp/ljrtest/.git/

D:\tmp\ljrtest>git remote add origin https://github.com/openhab-scripters/openhab-helper-libraries

D:\tmp\ljrtest>git config core.sparseCheckout true

D:\tmp\ljrtest>copy ..\sparse-checkout .git\info\
1 file(s) copied.

D:\tmp\ljrtest>git pull origin master
remote: Enumerating objects: 3591, done.
remote: Total 3591 (delta 0), reused 0 (delta 0), pack-reused 3591R ), 7.00 MiB | 6.95 MiB/s
Receiving objects: 100% (3591/3591), 9.26 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1786/1786), done.
From https://github.com/openhab-scripters/openhab-helper-libraries
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master

D:\tmp\ljrtest>dir
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> .github
07/03/2019 09:07 AM <DIR> Community
07/03/2019 09:07 AM <DIR> Core
0 File(s) 0 bytes
5 Dir(s) 386,515,042,304 bytes free

D:\tmp\ljrtest>dir .\Community
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest\Community

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> OpenWeatherMap
0 File(s) 0 bytes
3 Dir(s) 386,515,042,304 bytes free

Using sparse checkout, no one else has to do anything. Configure your client to get the files you want, and you’re set.

 

Git Commands

There are a few git commands that we use when working with the OpenHAB and helper library repositories. The OpenHAB Eclipse project sets uses a split push/pull repository where the *fetch* repo is the organization and the *push* repo is your personal repo. This is reasonable because you do not have permissions to write to the organizational repository. You can use the same split-repository setup for other projects. Clone the project either from the organization’s repo, and then change the push URL to your personal repository.

# Show list of remotes
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/openhab-scripters/openhab-helper-libraries (push)

# Set push remote to PERSONAL repository
[lisa@linux ~]# git remote set-url –push origin https://github.com/ljr55555/openhab-helper-libraries

# Show list of remotes — verification step
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

While the split repository setup prevents accidentally attempting to push changes to a repo to which you lack write access, I find it a little confusing. Instead, I add specific repos for ORG (the organizational repo) and my personal repo.
The drawback to this configuration is that you *can* attempt to push changes directly to the organization repo — which will either yield an error because you lack access or will inadvertently publish code in the org repo because you don’t lack access.

# Add ORG repo with organizational repo URL
[lisa@linux ~]# git remote add ORG https://github.com/openhab-scripters/openhab-helper-libraries
# Add LJR repo with personal fork URL
[lisa@linux ~]# git remote add LJR https://github.com/ljr55555/openhab-helper-libraries
[lisa@linux ~]# git remote -v
LJR https://github.com/ljr55555/openhab-helper-libraries (fetch)
LJR https://github.com/ljr55555/openhab-helper-libraries (push)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (push)
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

# Scenario: Someone has updated ORG master branch
# I want to incorporate those changes into PERSONAL master branch and push them into my repo
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git push –force LJR master # Overwrite personal repo master with updated info

# Scenario: Someone has updated ORG master branch.
# I want to incorporate those changes in PERSONAL lucid-migration branch
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git checkout lucid-migration # Switch back to your local lucid-migration branch
[lisa@linux ~]# git rebase –preserve-merges master # Rebase your local lucid-migration (checked out branch) onto local master
[lisa@linux ~]# git push –force-with-lease LJR lucid-migration # Overwrite personal repo lucid-migration branch with updated info

 

And a few misc commands that I want to remember
# Check username and email
[lisa@linux ~]# git config –list

# Set username and e-mail address
[lisa@linux ~]# git config –global user.name “FIRST_NAME LAST_NAME”
[lisa@linux ~]# git config –global user.email “MY_NAME@example.com”

# merge gone bad, bail!
[lisa@linux ~]# git merge –abort

# Forgot to add sign-off on commit
[lisa@linux ~]# git commit –amend

Off Label Use: Git

I store a lot of code in Git-based systems, and I read through parts of my state’s Revised Code on occasion. But it never occurred to me to store the Revised Code in a Git-based system. Washington, DC has started doing exactly that.

First, some basic terminology from GitHub — a “commit” is a change. In this case, a change to the revised code text. An “Issue” is where someone points out a problem with the existing code or proposes an enhancement — in this case, someone either pointing out a typo or suggesting a new law. And a “Pull Request” (aka a “PR”) is used to accept a commit into the code — in this case, applying the change to the revised code text.

Imagine being able to comment on a pull request and have your commentary considered during the public hearings for the proposed change. Sure, there’s elitism to assuming everyone’s got a computer and time to peruse proposed changes. But not more than assuming everyone’s got an hour to head over to Town Hall and attend the public hearing. And resident-initiated changes could be proxied through a third party — city government staff, for instance.

The process could increase transparency — GitHub users can subscribe to get notifications when issues and PRs are entered into the system. An issue could be created when discussion over a proposed change begins. Links to public meetings and even discussion about the proposed changes could be available online. A pull request created by government officials finalizes the changes and commits them into the online revised code.

The git storage system retains history — a resident who is convinced an auxiliary building was totally in compliance of zoning regs when it was built could run through the history and identify exactly when the setback requirements changed.

Clicking into a specific “commit” will show you what changed — the left-hand column is the old version and the right-hand column is the new version. Something that was removed is highlighted in red; something added is highlighted in green.

From a procedural standpoint, there are some challenges. Their code is now maintained in XML, which isn’t necessarily something that will edit well in your favorite word processing program (Word, as an example, has some limitations when it comes to editing XML and, in my experience, generates really sloppy code — multiple style elements that all cancel each other out). Build automation is now a requirement of legal documents — a process that ingests changes to the XML and spits out an HTML web site or a PDF file. These challenges require someone with technical knowledge to maintain the revised code. 

Logistically, it might be challenging to convince residents — especially those who don’t work as software developers — to use what is a software development tool to interact with local government. Even without citizen interactions, I think having a change history is an incredible advantage to using GitHub to store the revised code. It would be interesting to chat with the people who implemented this in DC and determine if (1) they get a lot of malicious submissions and (2) actually see citizens propose changes through GitHub.

Document retention implications would need to be considered as well — user comments, maliciously entered issues or PRs, etc may be undesired. It’s possible to restrict “interactions” (comments, issue creation, PR creation) to an approved list of individuals. But there’s an advantage to allowing residents to copy edit the code and to propose changes to code.

New Process Police

As an operational support group, we did not have a software development methodology. Doesn’t mean we didn’t develop software — one of the great things within operational support is the ability to automate day-to-day tasks to reduce workload. Why have someone check for application patches when a process can watch an RSS feed or file repository and notify us when there’s an update. Why have someone clickity-click provisioning users into groups when the user can make a web request, the group owner can approve the request, and an automated process can add the user into the group? The end result of our automation programming is, well, quite a bit of software.

And with a small number of people, informal application development worked. Wasn’t ideal, but it worked. If you want to write in Java while I use C# … not ideal, but the alternative is that one of us needs to learn a new programming language. Problem is the next guy we hired uses VBS, the next guy uses PowerShell … and I’ll use perl for simpler processes. Then someone starts tweaking my code and buggers it up … and we’ve got to figure out what happened and roll back based on some tape backup.

To get our internal software development processes organized, I developed a process. And ran a training session so everyone was familiar with both the process and the tools. Some of us have used the process well — don’t edit production code, clone the repo locally, make a branch for your edits, test it and have another group member sign off on the changes, merge your branch back into master, test more, then pull the code into production. The majority, it seems, have not followed the process at all. Changes are made to the code running in production, not incorporated into the Git repo. Six months after the new development process went in place, half of our code has improperly made changes!

To an extent, I consider this a management problem … if the department doesn’t want software development to be a free-for-all, then the department managers need to ensure their staff follows the process. If the department wants everyone to do their own thing — then get rid of the process and declare our methodology as “do whatever you want”. The challenge for managers, though, is that they don’t know that someone has edited code in production and failed to commit their changes into the repo. If only there were some way to watch for improperly edited code and alert us promptly.

Other scripts I’ve found to perform a similar function attempt to parse ‘git status’ to identify all sorts of issues — but that doesn’t address the specific problem that I’ve got. To facilitate identifying offenders, I wrote a quick Python script that searches a directory tree for git repositories and alerts us when changes have not been staged for commit. If you’ve staged the changes for commit, that won’t be identified. But the particular problem we encounter frequently … there are alerts for that.