Month: May 2017

Chicken Food

I’ve found a few good ideas for things chickens will feed themselves — include the compost area in the chicken pen, the chickens will turn the compost for you, eat fresh veggie scraps, and eat bugs they find.

Put a board or rubber mat down on the ground – let it sit there for a day or two, and a bunch of bugs will move in under it. Flip the board/mat into a new location & the chickens will go after the bugs that have been uncovered.

I also plan to grow a “chicken garden” in their coop — buckwheat, millet, flax, red clover, and forage peas. Hopefully they won’t confuse their garden with our garden. I want to try allowing them to roam in our raised bed garden to eat bugs … but I may end up fencing that off so they don’t eat our veggies!

Russia Returns

Russia has a decent play at undermining the American government without actually colluding with Trump’s campaign. Do something that benefits any part and there is suspicion. Do something that benefits someone who has been suspected of shady dealings with your country (money laundering, loans to someone American banks consider too risky) and the suspicion is even deeper. Someone who has used obstruction and intimidation routinely in business using the same tactics in their political misadventure … not exactly shocking.

Trump’s administration seems hopelessly unable to do anything but help the Russians undermine our government. Firing Comey looks bad no matter what happened during the election. Sharing code-word classified information with the same country suspected of interfering with the election … outright silly.

An “independent” investigation or one run by the House / Senate / FBI led by whomever Trump puts in charge – there’s no good outcome.

The investigation finds nothing illegal – half the country things the investigation was tainted, but we continue down this path. Allies withhold intel because they cannot trust Trump not to use the latest intercepts to brag about how great his intel briefings are. Reasonable policies are overturned along with the unreasonable because the Executive branch leadership doesn’t understand the “benefit” part of cost/benefit analysis. Taxes are lowered and deficits explode.

The investigation finds something – half the country thinks it’s fake evidence to go along with their fake news. But something has to be done. It isn’t like there’s a do-over election clause in the Constitution (even if there were, half of the country objects to the do-over election). Trump is impeached and Pence takes over – Democrats object – we’d have almost been better off with the ignorant guy who didn’t heap religious fundamentalism on top of his deregulation, tax cuts, and environmental destruction. Trump voters who are not traditional Republicans object — they didn’t vote for Pence’s policies either. Trump is impeached and Pence goes down too — Ryan takes over. See previous.

 

Git, Version Management, Branches, and Sub-modules

As we have increased in staff, we’ve gained a few new programmers. While it was easy enough for us to avoid stepping on each other’s toes, we have experienced several production problems that could be addressed by rethinking our repository configuration.

Current state: We have a monolithic repository for different batch servers. Each server has a clone of the repository, and the development equivalent has a clone of the same repository. The repository has top-level folders for each independent script. There is a SharedTools top-level folder for reusable functions.

Changes are made on forks located both on the development server and individuals’ computers, tested on the development server, then pushed to the repo. Under a CRQ, a pull is performed from the production server to elevate the new code. Glomming dozens of scripts into a single repository was simple and quick; but, with new people involved with development efforts, we have experienced challenges with changes being lost, unintentional elevation of code, and having UAT run against under-development code.

Pitfalls: Four people working on four different scripts are working in the same repository. We have had individuals developing on their laptop overwrite changes (force push is dangerous, even if force-with-lease is used), we have had individuals developing on the dev server commit other people’s edits (git add * isn’t a good idea in a shared environment – specifically add changed files to your commit), and we’ve had duplication of effort (which is certainly a problem outside of development efforts, and one that can be addressed outside of git).

We could address the issues we’ve seen through training and communication – ensure anyone contributing code to the repository adequately understands what force push means, appreciates what wildcards include, and generally have a more nuanced understanding of git than the one-hour training I provided last year. But I think we should consider the LOE and advantages of using a technical solution to ensure less experienced git users are able to successfully use our repositories.

Proposal – Functional Splits:

While we have a few more individuals with development experience, they are quite specifically Windows script developers (PowerShell, VBScript, etc). We could just stop using the Windows batch server and let the two or three Microsoft guys figure it out for themselves. This limits individual growth – I “don’t do” PowerShell development, the Windows guys don’t learn Linux. And, as the group changes over time, we have not addressed the underlying problem of multiple people working on the same codebase.

Proposal – Git Changes:

We can begin using branches for development efforts and reserve “master” for ready-for-deployment code. Doing so, we eliminate the possibility of inadvertently elevating code before it is ready – only commands targeted to “origin master” will be run on production servers.

Using descriptive branch names (Initials-ScriptFolderName-SummaryOfChange) will help eliminate duplicated efforts. If I notice we need to send a few mass mails with inline images, seeing “TJR-sendMassMail-AddInlineImages” in the branch list lets me know you’ve got it covered. And “TJR-sendMassMail-RecipientListFromLiveLDAPQuery” lets me know you’re working on something else and I’m setting myself up for merge challenges by working on my change right now. If both of our changes are high priority, we might choose to work through a merge operation. This would be an informed, upfront decision instead of a surprise message indicating that fast-forward merging is not possible.

In large development projects, branch management can become a full-time pursuit. I do not think that will be an issue in our case. Minimizing the number of branches used, and not creating branches based on branches, makes branch management a simpler task. We should be able to perform fast-forward merges to push code into master because our branches modify different files in the repository.

To begin a development effort, create a branch and push it to the git server. Make your changes within that branch, and ensure you keep your branch in sync with master – you cannot merge branches that are “behind” into master without force. Once you are finished with your development, merge your branch into master and delete your branch. This approach will require additional training to ensure everyone understands how to create, rebase, merge, and delete branches (and not to just force operations because it lets you complete your task).

Instead of using ‘master’ for production code, the inverse is equally viable: create a “stable” branch that is for production code and only pull that branch to PROD servers. I believe this approach is done to prevent accidental changes to prod code – you’ve got to intentionally target “origin stable” with an operation to impact production code.

Our single repository configuration is a detriment to using branches if development is performed on the DEV server. To illustrate the issue, create a BranchTesting repo and add a single file to master. Create a Branch1 branch in one command window and check it out. Create a Branch2 in a second command window and check it out. In your first command window, add a file and commit it. In your second command window, add a file and commit it. You will find that both files have been committed to Branch2.

How can we address this issue?

Develop on our individual workstations instead of the DEV server. Not sharing a file set for our development efforts eliminates the branch context switching problem. If you clone the repo to your laptop, Gary clones the repo to his laptop, and I clone the repo to my laptop … you can create TJR-sendMassMail-AddInlineImages on your computer, write and test the changes locally, commit the changes and pull them to the DEV server for more robust testing, and then merge your changes into master when you are ready to elevate the code. I can simultaneously create LJR-monitorLDAPReplication-AddOUD11Servers, do my thing, commit changes and pull them to the DEV server (first using “git branch” to determine if someone else is already testing their branch on the DEV server), and merge my stuff into master when I’m ready to elevate. Other than remembering to ensure you verify that DEV has master checked out (i.e. no one else is testing, so the resource is free), we do not have resource contention.

While it may not be desirable to fill up our laptop drives with the entire code set from six different application servers, sparse-checkout allows you to select the specific folders that will come down to your fork.

The advantage of this approach is that it has no initial LOE beyond training and process change. The repositories are left as-is, and we start using them differently.

Unfortunately, this approach may not be viable in some instances – when access to data sources is restricted by IP ACL, you may not be able to do more than linting on your laptop. It may not even be possible to configure a Windows laptop to run some of our code – some Linux requirements are difficult to address in Windows (the PKI website’s cert info check, for instance), and testing code on Windows may not ensure successful operation on the Linux hosts.

Break the monolithic repositories into discrete repositories and use submodules allow the multiple independent repositories to be “rolled up” into a top-level repository. Development is done in the submodule repositories. I can clone monitorLDAPReplication, you can clone sendMassMail, etc. changes can be made within our branches of these completely different repositories and merged into the individual repository’s master branch for release to the production environment. Release can be done for the superset (“–recurse-submodules”) or individual sub-modules.

This would require splitting a repository into its individual components and configuring the sub-module relationships. This can be a scripted operation, and it is an incremental change to the script I used to create the repositories and ingest code; but the LOE for implementation is a few days of script writing / testing. Training will be required to ensure individuals can register their submodules within the top-level repo, and we will need to accustom ourselves to maintaining individual repos.

Or just break monolithic repositories into discrete repositories. The level of effort is about the same initially, but no one needs to learn how to set up a new submodule. We lose single-repo conveniences, but there’s literally no association between our different script folders where someone working in X could inadvertently impact Y.

Homemade Dog Food

In case we do get a dog, I wanted to have a few recipes for homemade dog food because I really don’t want to feed a dog fat coated grain nuggets.

Liver Brown Rice

  • 2 lbs raw chicken livers (3 containers) – remember, you can also use beef liver
  • 2 cup of cooked brown rice, set aside
  • 1 cup of chopped carrots
  • 1 chopped broccoli,
  • 2 cup of water
  • 1 teaspoon olive oil for pan

Chop veggies and liver into bite sized pieces. Saute liver. Add water and simmer until liver is cooked. Add veggies and cook for a few more minutes. Cool and refrigerate/freeze.

Beef Sweet Potato

  • 1 pound of beef
  • 1 small sweet potato
  • 1/2 cup of carrots, diced
  • 1/2 cup of green beans, diced
  • 1/2 cup of flour
  • 1/2 cup of water
  • 1 tablespoon of vegetable oil for frying

Microwave the sweet potato until mostly cooked and chop into bite sized pieces. Dice beef and saute in olive. Remove meat from pan. Add flour and water to make a gravy. Add in veggies, sweet potato, and meat. Cook over medium-low heat until carrots are cooked (5-10 minutes). Cool and refrigerate/freeze.

Lots of Veggies (my own)

  • 3 lbs beef offal
  • 1 cups brown rice
  • 3 cups spinach, shredded
  • 1 zucchini, chopped
  • 1 cup peas
  • 1 tablespoon oil
  • 1/4 cup flour
  • 2 eggs
  • 1 cup meat stock
  • 1 1/2 teaspoon ground egg shells

Dice everything into bite sized pieces. Saute meat for a few minutes until mostly cooked. Add in flour, egg, and stock to make a gravy. Stir in veggies and cook for another 3-4 minutes. Cool and refrigerate/freeze.

Apples!

  • 3 lbs turkey/chicken
  • 1 cups brown rice
  • 2 apple
  • 2 cup carrots
  • 1 tablespoon oil
  • 2 tablespoons tapioca powder
  • 2 eggs
  • 1 cup meat stock
  • 1 1/2 teaspoon ground egg shells

Dice everything into bite sized pieces. Saute meat for a few minutes until mostly cooked. Add in flour, tapioca powder, and stock and heat make a gravy. Stir in fruit/veggies and cook for another 3-4 minutes. Cool and refrigerate/freeze.

 

GoFCCYourself(.com)

You know what you find when you drain a swamp? A whole bunch of rotting detritus. I’m not going to pretend astonishment that a former Associate General Counsel from Verizon thinks net neutrality is a terrible idea. I remember getting an e-mail message from my employer, another network provider, detailing how this terrible proposal was going to drive us all out of business. Or something similarly over-dramatic.

Facilitating public comment on Executive branch proceedings, such as GoFCCYourself.com, is an interesting idea. Take a circuitous government web site that ostensibly allows individuals to post comments on issues and circumvent the terrible user interface by getting your own URL and I assume including the appropriate POST headers to get individuals in exactly the right place to submit their comments.

I’ve used this short-cut to submit my opinion to the FCC, but I also forwarded the same message to my rep in the House and my two state Senators:

I have submitted this to the FCC for Docket 17-108 but wanted to include you as well. If the FCC does roll back net neutrality, as their chairman indicates is his desire, I beseech you to ready legislative controls to prevent ISPs from using speed controls to essentially censor Internet content.

I am writing to express my support for “net neutrality” — while you want to claim it reduces carrier investment or innovation, customer acquisition and retention drives carrier investment and innovation. Lowered cost of operations, creating a service that allows a higher price point, or offering a new service unavailable through a competitor drive innovation. Allowing a carrier to create a new revenue stream by charging content providers for faster access is not innovation – QoS has been around for decades. And it isn’t like the content is being delivered to the Internet for free. Content providers already pay for bandwidth — and a company like Netflix probably paid a LOT of money for bandwidth at their locations. If Verizon didn’t win a bid for network services to those locations, that’s Verizon’s problem. Don’t create a legal framework for every ISP to profit from *not* providing network services for popular sites; the network provider needs to submit a more competitive bid.

What rolling back net neutrality *does* is stifle customers and content providers. If I, as a customer, am paying 50$ a month for my Internet service but find the content that I *want* is de-prioritized and slowed … well, in a perfect capitalist system, I would switch to the provider who ‘innovates’ and goes back to their 2017 configurations. But broadband access – apart from some major metro areas – is not a capitalist system. Where I live, outside of the Cleveland suburbs, I have my choice of the local cable company or sat – sat based Internet introduces a lot of latency and is quite expensive for both the customer and the operator (and has data limits, which themselves preclude a lot of network-intensive traffic that ISPs wish to de-prioritize). That’s not a real choice — pay 50$ to this company who is going to de-prioritize anyone who doesn’t pay their network bandwidth ransom or pay 100$ to some other company that is unable to provide sufficiently low latency to allow me to work from home. So add a hour of commute time, fuel, vehicle wear, and reduced family time to that 100$ bill.

Rolling back net neutrality stifles small businesses — it’s already difficult to compete with large corporations who have comparatively unlimited budgets for advertising and lawyers. Today, a small business is able to present their product online with equal footing. In 1994, I worked at a small University. One of my initiatives was to train departmental representatives on basic HTML coding so the college would have an outstanding presence on the Internet. First hour of the first day of the training session included a method for checking load times off campus without actually having to leave the campus network. On campus, we were 10 meg between buildings and the server room and anything loaded quite quickly. At home, a prospective student was dialing in on a 28.8 modem. If your content is a web page for MIT, a prospective engineering student may be willing to click your site, go eat dinner, and come back. Load time isn’t as much of a problem for an organisation with a big name and reputation. Unknown little University in Western PA? Click … wait … wait, eh, never mind. The advent of DSL was amazing to me because it provided sufficient bandwidth and delivered content with parity that allowed an unknown Uni to offer a robust web site with videos of the exciting research opportunities available to students and the individual attention from professors that small class sizes allow. No longer did we need to restrict graphics and AV on our site because we weren’t a ‘big name’ University. That there ever was a debate about removing this parity astonished me.

Aside from my personal opinion, what is the impact of non-neutral networks on free speech? Without robust legal controls, ISPs engage in a form of quasi-censorship. How do you intend to prevent abuse of the system? Is a large corporation going to be able to direct “marketing” dollars to speeding up their page to the harm of their competitors? Can the Coca-Cola Company pay millions of dollars to have their content delivered faster than PepsiCo’s? Is the ISP then the winner in a bidding war between the two companies? What about political content? Does my ISP now control the speed at which political content is delivered? What happens when Democrats raise more money in the Cleveland metro area and conservative views are relegated to the ‘slow’ lane? What happens when the FCC gets de-prioritized because ISPs want even less regulation??

I would still worry about the legal controls to prevent quasi-censorship, but I would object less if the FCC were to implement the net neutrality requirements like some of the telco regulations for CLEC’s where there were no ILEC’s had been — where there is no or limited competition, net neutrality is a requirement. Where there are a dozen different ISP options, they can try selling the QoS’d packages. Polls and voting aside, the ISP will find out exactly how many customers or content providers support non-neutral networks.

Hack The Vote

There is a component to the ongoing story of Russian involvement in the 2016 election that seems to have gone missing from public discourse. The RNC was hacked too. Information from DNC hacks were released – embarrassing information that questioned the legitimacy of Clinton’s primary victory and that substantiated the worst view people have of her (and politicians in general). What was released from the RNC hack? Zilch.

One possibility is that the RNC hack yielded nothing embarrassing or subversive to release. But do you really think the RNC wasn’t trying to undermine Trump the same way the DNC undermined Sanders? Do you really think the entirety of the RNC is completely upfront about every facet of political discourse?

Immediately after the election, Russians were said to have a kompromat dossier on Trump. Whatever was gathered from the RNC hack is good insurance should Russia’s choice of American leaders backfire on them in some way. Personally damaging information on Trump and RNC information that compromises the integrity of the party.

Repeal and Replace

There’s a television show where a group of people go around to auctions and buy ‘stuff’ to resell. They’ll “bid up” the price to screw other people out of money (I expect this is a strategy to prevent competition for upcoming items?) but sometimes get stuck with a high price on something they didn’t actually want because the competition backs out of bidding prior to expectations. I’m worried the AHCA is the guy who overpaid for junk … it started out as a marketing ploy than actual legislation. Pass a repeal and go to their constituents with “*I* got this passed for you (vote for me again), but the bloody rest of the HR stopped your will from being enacted. We don’t have enough Reps, donate NOW and get more R’s in here. Oh, the cursed President said not to worry because he’ll veto the bill — donate NOW and vote for the R. Oh, wait, this didn’t pass the Senate – send money NOW so we can get a super-majority in 2018.

Except it passes the Senate and the incumbents have to live with the results of their legislation. And, yeah, this country has a policy where hospitals need to provide emergency care to anyone regardless of means (they can also bill the person a few hundred thousand dollars, slowly drain away that person’s assets, and file a lien against the estate). Which is great for a relatively health person who suffers a sudden calamity — car crash, fall down a mountain, etc. May not even be terrible for someone who experiences a heart attack. Town halls with Tea Partiers going on about abstract death panels are going to seem like nothing. Wait until the people slowly dying with access only to emergency interventions that extend their suffering start popping up in the town halls — no coverage for the cancer relapse, but you’ll stabilize me and send me home to suffer a few more weeks. People who realize that, sure, an insurance plan *is* available to someone who had a stroke a few years back but how does this state high risk pool with a 250k annual premium help ME? Seniors who lose their subsidies and can no longer afford heath care. People stuck in terrible work situations because losing coverage means the condition will become pre-existing.

Wait until women see premiums quadruple after having a child. My local rep couldn’t tell me if the insurance company would be disallowed from raising my premiums if I self-funded sterilization, provided a doctors note attesting to menopause, swapped over to female partners, or otherwise precluded future pregnancies … and he then he got all annoyed with my expectation that he would have read and, ya know, *understood* the full text of a bill for which he was voting. 

And Republicans will free insurance companies from ACA’s requirement to spend 80-85% of premiums on health services … so all of these sad stories will be coupled with record profits and stock buy-backs within the insurance sector.

Owntracks Stuck In “Connecting” To MQTT When Using WebSockets

Our home automation presence is maintained through an Android app, OwnTracks, which updates a Mosquitto server via a WebSockets reverse proxy. Mosquitto runs on a Fedora 25 server and was installed from the default RPM repository.

Recently, we stopped receiving location updates – both of our Android clients were stuck “Connecting” to the MQTT server. Nothing appeared in the Apache access or error logs, and capturing network traffic only got a small number of packets (TCP session overhead ‘stuff’). Even bypassing the reverse proxy and using the internal network to communicate directly to the Mosquitto server only created a couple of packets. Using a test client (http://www.hivemq.com/demos/websocket-client/), I saw strange connection failures — so I knew the problem was not specific to the OwnTracks client.

It seems there was a bug in libwebsockets v2.1.1 (and possibly others) — when we updated our Fedora installation, the new libwebsockets broke our MQTT over WebSockets. Currently, the Fedora repository still contains an impacted version of libwebsockets. To resolve the issue, I built the latest stable libwebsockets and built mosquitto against this updated library.

Process: The first step is to remove the dnf managed packages (rpm -e libwebsockets libwebsockets-devel mosquitto). Then build libwebsockets and mosquitto.

To Build LibWebSockets:

wget https://github.com/warmcat/libwebsockets/archive/master.zip
unzip master.zip
cd libwebsockets-master/
mkdir build
cd build
cmake ..
make
make install
cp libwebsockets.pc /usr/lib/
cp lws_config.h /usr/include/
cp ../lib/libwebsockets.h /usr/include/
cp ./lib/libwebsockets.so /usr/lib/

To Build Mosquitto:

wget https://github.com/eclipse/mosquitto/archive/master.zip
unzip master.zip
cd mosquitto-1.4.11
vi config.mk # Line 68, change to “WITH_WEBSOCKETS:=yes”
make
make install

Start the Mosquitto server and try again. Voila, presence works again!

Middle East Peace And Avoiding The Civil War

Donald Trump thinks the Civil War could have been avoided — well, yeah it could have been. If slave owners had voluntarily ceeded their economic advantage (i.e. capitulated to the abolitionists position)  or if the nation had continue granting individual states the “right” to allow their citizens to enslave other humans … voila, no war. But there’s no middle ground between “I can subjugate and exploit other people based on some aesthetic aspect” and “humans are humans”. His ignorant musings worry me – if for no other reason than a great deal of study has been done into the proximal and distal causes of the Civil War (i.e. the exact opposite of “People don’t ask that question, but why was there the Civil War?”).

It is simply terrifying when the same individual is willing to “do anything” to broker a peace deal in the Middle East. I guess he’ll be learning that (just like health care, free trade, and South East Asian affairs) Middle Eastern discord is harder than he thought.