Category: Technology

July 3, 2019

Sparse Checkout With Git

I’ve encountered a few repositories that are huge. Unwieldy huge, and stuffed with files that aren’t relevant to what I need. The straight-forward solution is to use multiple repositories — that’s what I do at work with my code samples. There’s a different repo for each language because the PHP developers really don’t care what the C# code looks like. The Java developers don’t need a copy of the Python code. But there are advantages to having a single repository that may preclude you from taking the simple solution. Git sub-modules are an interesting approach — combining multiple repositories into a single functional unit. But that’s a pretty big change to an existing repo. And, if you participate in open source projects, it may not be your decision anyway.

There’s another option for selectively cloning when you’re working with a large repo — an option that doesn’t require any changes to the repository. An end user can perform a sparse checkout — essentially use a filter like .gitignore to select or deselect certain files/folders from being pulled into the local working directory. The file is named sparse-checkout and is located in .git\info — unlike a .gitignore file which indicates what shouldn’t get included, sparse-checking controls what is included (if you want an entire repo except one folder, use !path/to/folder/**)

The sparse-checkout file used to get just the core components of Scott’s OpenHAB helper libraries plus the OpenWeatherMap community scripts is:

.github/**
Core/**
Community/OpenWeatherMap/**

To use sparse checkout, set the core.sparseCheckout config value to true. You can add sparse checkout to a repo you’ve already cloned and use

git read-tree -mu HEAD

to “clean up” unwanted files. Or you can set up sparse checkout before you clone the repo

D:\tmp>mkdir ljrtest

D:\tmp>cd ljrtest

D:\tmp\ljrtest>git init
Initialized empty Git repository in D:/tmp/ljrtest/.git/

D:\tmp\ljrtest>git remote add origin https://github.com/openhab-scripters/openhab-helper-libraries

D:\tmp\ljrtest>git config core.sparseCheckout true

D:\tmp\ljrtest>copy ..\sparse-checkout .git\info\
1 file(s) copied.

D:\tmp\ljrtest>git pull origin master
remote: Enumerating objects: 3591, done.
remote: Total 3591 (delta 0), reused 0 (delta 0), pack-reused 3591R ), 7.00 MiB | 6.95 MiB/s
Receiving objects: 100% (3591/3591), 9.26 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1786/1786), done.
From https://github.com/openhab-scripters/openhab-helper-libraries
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master

D:\tmp\ljrtest>dir
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> .github
07/03/2019 09:07 AM <DIR> Community
07/03/2019 09:07 AM <DIR> Core
0 File(s) 0 bytes
5 Dir(s) 386,515,042,304 bytes free

D:\tmp\ljrtest>dir .\Community
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest\Community

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> OpenWeatherMap
0 File(s) 0 bytes
3 Dir(s) 386,515,042,304 bytes free

Using sparse checkout, no one else has to do anything. Configure your client to get the files you want, and you’re set.

July 2, 2019

Two Approaches to Using PIP Through an Integrated Authenticated Proxy

The proxy at work uses integrated authentication. While BASIC auth prompts happily let you use a proxy of http://uid:pass@proxy:port, ours does not. There are two ways I’ve managed to use pip to install packages.

Proxied Proxy

The easiest approach is to use something that can handle the authenticated proxy, like Fiddler, as an intermediary. I do the same thing with perl’s PPM, docker pull … basically anywhere I’ve got to use a proxy that wants to pass through a proxy username.

Select Tools => Options to open the configuration dialog. Make sure you are handling SSL traffic — if not, check the box to “Capture HTTPS CONNECTs, “Decrypt HTTPS traffic”, and “Ignore server certificate errors” (it’s only unsafe if you don’t understand what you’re doing … don’t log into your bank account bouncing traffic through this config!)

On the “Connections” tab, check the port on which Fiddler is listening. If you cannot install Fiddler on the same box where you want to use pip, you’ll need to check off “Allow remote computers to connect” (and you won’t use localhost as the proxy hostname). Click OK, start capturing traffic (F12), and we’re ready to go.

Use the PIP command line to install the package but proxy the request through your Fiddler instance. In this example, Fiddler is installed on the local box and uses port 8888.

pip –trusted-host pypi.org –trusted-host files.pythonhosted.org –proxy http://localhost:8888 install SomePackage

This is nice because pip will automatically resolve dependencies. Not great if you’re not allowed to install your own software and cannot get Fiddler installed.

Dependency Nighmare

Back in the early days of Linux (think waaaay before package managers working against online repositories), we called this “dependency hell” — navigating dependency chains to get all of the required “stuff” on the box so you can install the thing you actually wanted.

Make a folder for all these wheels we’re going to end up downloading so it’s easy to clean up once we’re done. Search PyPi for the package you want. On the package page, select ‘Download Files’ and then download the whl

Use “pip install something.whl” to attempt installing it. If you’re lucky, you’ve got all the dependencies and the package will install. If you don’t have all of the dependencies, you’ll get an error telling you the first missing one. Go back to the pypi website & get that. Use “pip install somethingelse.whl” to install it and maybe get a dependency error here too. Once you get a dependency installed, try the package you want again. Got another error? Start downloading and installing again. Eventually, though, you’ll get all of the dependencies satisfied and be able to install the package you actually want.

July 1, 2019

Did you know … Windstream’s Teams usage statistics through 2Q2019

Windstream will replace Skype for Business with Teams by the end of 3Q2019. That’s only three months away, so I wanted to provide another update on our progress toward this goal. There are just under 14,000 IM-enabled accounts (this includes both employees and contractors). About 3,800 accounts (about 27% of Windstream’s IM users) have been upgraded to Teams Only. Two third of the company should have already received communication letting them know when their accounts will be upgraded. The remainder of the company will see messages throughout July and August. If you are ready to upgrade before your scheduled date (visit Stream for more info on what to expect when your account is upgraded), use <redacted> to upgrade your account.

Teams accounts for around 80% of Windstream’s IM activity. This is a significant change from the same time last year when under 10% of our IM traffic was in Teams.

From 20 May onward, there have been more people logging into Teams than Skype each day. About 80% of our IM enabled accounts have logged into Teams at least once; and over 8,000 people, about 60% of our IM enabled accounts, are logging into Teams each day. If you aren’t one of these people who are already using Teams, check it out.

In the past year, the number of chat messages sent in Teams has increased 20-fold – from under 10,000 messages a day in July 2018 to around 200,000 messages in June 2019. We’ve seen a reduction in the number of instant messages sent through Skype – from 100,000 daily messages last year to under 50,000 daily messages this month. Some messages that would have been sent in Skype are now being sent in Teams, but the number of IMs sent across the company has doubled in the past year too – from 110,000 messages each weekday this time last year to 220,000 messages each weekday today.

How are people accessing Teams? Teams is predominantly accessed with the Windows desktop app. About 80% of the people using Teams each day use the Windows desktop app (OSX users haven’t been left out, but there are only 20 people using the Mac desktop app). About 5% of the people using Teams each day use the web client. Most of the desktop features are available in the web app, and you can use the web app from a computer that isn’t managed by Windstream. While screen sharing isn’t yet available in private chats, screen sharing is available in meetings when using the Chrome browser. We’ve seen mobile app usage increase from about 10% at the beginning of the year to 15% today, and the mobile app is used by about 30% of Teams users over weekends. The Teams mobile app makes it easy to check in at work while you’re working away from the office, and setting ‘quiet time’ keeps work from intruding on you time.

At the beginning of the year, mobile app usage was split about 50/50 between Android and iOS. By the end of 1Q2019, the percent of Android users dropped to 40%; and, at the end of 2Q2019, iOS accounts for 75% of the mobile app usage.

The Teams: Teams is more than just a replacement for Skype. In addition to private chats, Teams offers collaboration spaces too. The Teams spaces include conversations, shared files, tabs – even a SharePoint Online site. More than 7,000 messages a day are posted in Teams channel conversations.

There are over 3,000 Teams groups – over 1,000 of which were created this quarter. You can search public Teams groups at <redacted>. Microsoft is currently testing a setting which will allow private Teams to be searchable too. Follow our Stream space – we’ll post information on how to mark your private Team as searchable when the feature becomes available.

Fifteen percent of these groups are public – meaning anyone can join the group. Public teams are a great way to solicit end-user feedback, organize local events, provide mentorship, or even discuss “fun stuff”. You can currently search public Teams groups at <redacted>. Microsoft is currently testing a setting which will allow specially-configured private Teams to be searchable too. Follow our Stream space – we’ll post information on how to mark your private Team as searchable when the feature becomes available.

82% of our Teams have between 2 and 25 members. For anyone wondering what the point of a Team with one person is … Teams doesn’t let you send messages to yourself, but you can send messages to a Team that is just you. You can add tabs to provide quick access to your frequently used sites, use connectors to make external data readily available (as an example, I use my Teams space as an RSS aggregator), and play around with Teams functionality without annoying your colleagues.

59% of our Teams have had conversation activity in the past week. Most of the Teams with no conversation activity in the past year have been archived. Archived Teams keep information accessible – visible and searchable – without risking individuals starting conversations in a Teams space that is no longer watched by members.

If you want to see more information about Windstream’s Teams usage, current Teams usage information is available at <redacted>.

June 28, 2019

Adding Python to Fedora Alternatives

Gimp installed Python 2.7). Which, of course, took over my system so nothing was using Python 3 anymore. We’ve used ‘alternatives’ to manage the Java installation, and I thought that might be a good solution in case I ever need to use Python 2

Add both Python versions to alternatives:

[lisa@fedora ~]# sudo alternatives –install /usr/bin/python python /usr/bin/python3.7 1
[lisa@fedora ~]# sudo alternatives –install /usr/bin/python python /usr/bin/python2.7 2

Select which one you want to use:

[lisa@fedora ~]# sudo alternatives –config python

There are 2 programs which provide ‘python’.

Selection Command
———————————————–
1 /usr/bin/python3.7
+ 2 /usr/bin/python2.7

Enter to keep the current selection[+], or type selection number: 1

And, of course, repeat the process for PIP:

[lisa@fedora ~]# sudo alternatives –install /usr/bin/pip pip /usr/bin/pip2.7 2
[lisa@fedora ~]# sudo alternatives –install /usr/bin/pip pip /usr/bin/pip3.7 1

[lisa@fedora ~]# sudo alternatives –config pip

June 27, 2019

Did you know … you can add a “Share to Teams” button to your web content?

If you can add script tags to the page head, you can add a “Share to Teams” button on your web site. This can be used to allow employees to share internal sites to Teams, but it can also be used on public sites to allow visitors to post links to their Teams organization.

How? There are two steps – add “<script async defer src=”https://teams.microsoft.com/share/launcher.js” ></script>” in HEAD. The post that is made to Teams is *prettier* if you have meta properties for title, description, and image within the linked page.

Then add a div with class “teams-share-button”. The “data-href” value is the URL to be shared. If you don’t want a page preview to render, you can set “data-preview” to false.

Sample page content:

<head>
    <title>Teams Share Test</title>
    <meta property="og:title" content="Lisa Rushworth Home Page">
    <meta property="og:description" content="Lisa Rushworth's Home Page">
    <meta property="og:image" content="https://www.rushworth.us/lisa/RedR.png" />
    <script async defer src="https://teams.microsoft.com/share/launcher.js" ></script>
</head>
<body bgcolor="black" text="white">
    <div class="content">
        <P>Here is the really cool information contained on this web site. It is so interesting that you want to share it to Teams.</P>
        <P>Click the Teams button at the bottom and you'll see a form that allows you to share the URL as a thread in a Teams channel.</P>
    </div>
    <div class="teams-share-button" data-href="https://www.rushworth.us/lisa/teamstest.php" data-preview="true">
    </div>
</body>

Visitors will see a small Teams logo in the teams-share-button div. To share the URL in Teams, they just need to click on the Teams logo.

A new window will load. If the user is not logged into the Team web site, they will be prompted to log in. Once logged in, the share dialogue will be displayed. If your site has title, description, and icon meta tags, a preview card will be included at the bottom.

Click in the “Share to” field and type a Team or Channel name – Teams and Channels from the user’s organization will be displayed.

The user can add text to the thread. Click “Share” to share the link to the selected Teams channel.

A confirmation page will be displayed.

In Teams, a new thread will be created. This is the thread for my shared URL.

The URL used in the teams-share-button DIV doesn’t have to match the page on which it is used — I can add a ‘share to Teams’ button that posts any URL to Teams.

June 27, 2019

Did you know … Bing can search your Teams chats, SharePoint Online, and OneDrive?

Searching for documents can be a time-consuming process – querying different locations for the same search string. Microsoft Search is beginning to unify searching across the Office365 environment. From one location, you can search SharePoint online content, OneDrive for Business files, and Microsoft Teams chats and channel conversations.

Head out to Bing, Microsoft’s public search engine, and search for something you’ve discussed recently in Teams … there are results, but nothing from your chats or files. Our information is private!

Click “Sign in” in the upper right-hand corner of the search page.

Once you’ve signed in, Bing looks basically the same … but you’ll see your avatar in the upper right-hand corner of the screen.

Search for that recent discussion topic again – you’ll see the same public search results, but you’ll also have a link to “See work results”.

Voila – work results 😊 Results are grouped by where the information is stored – “Sites” for SharePoint Online sites, Files for SharePoint or OneDrive files (Teams “Files” are stored in SharePoint Online, so will be found in here too), and Teams for Teams chats and channel conversations. Search results only include documents and conversations to which you have access. And no, random people on the Internet aren’t seeing your files and conversations.

Clicking on a search result will open it in the appropriate application.

In addition to providing “one stop shopping” for my searches, I’ve found this particularly useful when researching programming or telecommunication topics – if someone has public OneDrive or SharePoint information on the topic, I supplement the generic Internet search results with information specific to Windstream.

Unfortunately, there’s not a UserVoice site for Microsoft Search. If anyone else wants to suggest they incorporate Exchange Online data into the search, there’s a “Feedback” button in your search results 😊

June 27, 2019

Did you know … you can customize Teams channel notifications?

Microsoft has created additional notification options in Microsoft Teams – and you can now customize those settings for individual channels to ensure you’re getting notifications when and were you want to see them.

To access your general notification settings, click on your avatar in the upper right-hand corner of the Teams window. Select “Settings”.

Click on “Notifications”. You will be presented with a long list of activities and can select the type of notification you wish to receive for each activity. Notice “Show only in feed” is now an option – this means new activity will appear in the “Activity” section of Teams but not display the big purple notification box (a.k.a. “the banner”) in the lower right-hand corner of your screen.

You can also customize notifications for individual channels – maybe you want a banner notification for a really important channel, or you don’t want to be notified about every reply to a message. Click the ellipsis next to the channel name and select “Channel notifications”

Here you can customize the notification you will receive for channel posts and at-mentions (you’ll still get notified about Team or individual at-mentions). You can also un-check the “Include all replies” box to get notified about new threads.

All of the channels that you followed previously will use your Teams-wide notification settings. Channels that you weren’t following will have notifications off. Newly created channels will have new post notifications turned off and channel mentions set to “Only show in feed”.

** If you still see “Follow” and not “Channel Notifications”, click on your avatar in the upper right-hand corner of the Teams app and select “Check for updates”. Once the updates have been applied, click the tip to “refresh” Teams.

June 21, 2019

Updating ll alias

Fedora and Red Hat Enterprise Linux have an alias “ll” which uses the long listing format. It’s a quick little tweak that I love, but I generally want to list hidden files too. Seemed easy enough to tweak the alias … but I never had any luck overriding the system setting or finding the source of the alias. Typing “ll -a” gave me what I wanted, although that’s not appreciably easier than typing “ls -al” …

The ll alias is defined in /etc/profile.d/colorls.sh (or colorls.csh if you use the C shell). Add the ‘a’ and “ll” produces a long list format of all files.

lisa@fedora123 ~]# grep “alias ll” /etc/profile.d/colorls.sh
alias ll=’ls -la’ 2>/dev/null
alias ll=’ls -la –color=auto’ 2>/dev/null

June 20, 2019

Did you know … Microsoft Teams private chats can include 100 people?

In January, Microsoft expanded Teams Chats to 50 people. I’ve heard from a few individuals who wanted to be able to chat with more people — essentially to use Teams to send broadcast messages to a lot of people. Last week, Microsoft upped the limit for private chats to 100. Hopefully they’ll extend the Graph API to allow applications to initiate those chats because adding 100 people to a chat seems like it would take a while!