We’ve got all the bits and pieces to assemble our maple reverse osmosis system — four 150 gallon per day filters, a pre-filter, pump, pressure gauge, and a valve to control how much pressure we allow to build up in the system. Plus some tubing and connectors, of course!
When we are decommissioning a website (or web server), I always watch the log files to ensure there aren’t a lot of people still accessing it. Sometimes there are and it’s worth tracking them down individually to clue them into the site’s eminent demise. Usually there aren’t, and it’s just a confirmation that our decommissioning efforts won’t be impactful.
This python script looks for IP addresses in the log files and outputs each IP & it’s access count per log file. Not great if you’ll see a bunch of IP addresses in the recorded URI string, but it’s good enough for 99% of our log data.
import os
import re
from collections import Counter
def parseApacheHTTPDLog(strLogFile):
regexIPAddress = r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'
with open(strLogFile) as f:
objLog = f.read()
listIPAddresses = re.findall(regexIPAddress,objLog)
counterAccessByIP = Counter(listIPAddresses)
for strIP, iAccessCount in counterAccessByIP.items():
print(f"{strLogFile}\t{str(strIP)}\t{str(iAccessCount)}")
if __name__ == '__main__':
strLogDirectory = '/var/log/httpd/'
for strFileName in os.listdir(strLogDirectory):
if strFileName.__contains__("access_log"):
#if strFileName.__contains__("hostname.example.com") and strFileName.__contains__("access_log"):
parseApacheHTTPDLog(f"{strLogDirectory}{strFileName}")
The proto-ducks are in the incubator at March 13th — at 21:13. April 10th will be 28 days, so there should be tiny little ducklings.
We’re about to start incubating eight duck eggs, so I wanted to record the temperature and humidity settings that I’ve found for the chicken, duck, and turkey eggs (well, future turkey eggs! We managed to get five male turkeys last year)
| DUCKS | |||
| Start | End | Temp | Humidity |
| 1 | 25 | 99.5 | 55-58% |
| 26 | 28 | 98.5 | 65% |
| 28 | hatching | 97 | 70-80% |
| CHICKENS | |||
| Start | End | Temp | Humidity |
| 1 | 18 | 99.5-100.5 | 45-55% |
| 19 | Hatching | 99.5 | 65-70% |
| TURKEYS | |||
| Start | End | Temp | Humidity |
| 1 | 24 | 99-100 | 50-60% |
| 25 | Hatching | 99 | 65-70% |
I’ve been successfully reverse proxying MQTT over WebSockets via Apache HTTPD for quite some time now. The last few weeks, my phone isn’t able to connect. There’s no good rational presented (and manually clicking the “send data” button on my client successfully connects). It was time to upgrade the server anyway. Once I got the latest Linux distro on the server, I couldn’t connect at all to my MQTT server. The error log showed AH00898: Error reading from remote server returned by /mqtt
Evidently, httpd 2.4.47 added functionality to upgrade and tunnel in accordance with RFC 7230. And that doesn’t work at all in my scenario. Haven’t dug in to the why of it yet, but adding ProxyWebsocketFallbackToProxyHttp Off to the reverse proxy config allowed me to successfully communicate with the MQTT server through the reverse proxy.
We’ve spent a lot of time trying to get a log file from the rygel server … setting the log level in the config file didn’t seem to do anything useful. And I cannot even find a log file. The only output we’re able to find is formed by running the binary from the command line. Where is that log file?!? Hey — there isn’t one. All of this log level setting has to do with what’s written to STDOUT and STDERR. You can either modify the unit file to tee the output off to a file or run it from the command line
To get debugging output, use
G_MESSAGES_DEBUG=all rygel -g 5
To tee the output off to a file,
rygel -g *:5 2>&1 | tee -a /path/to/rygel.log
Some systems were found to be responding to mDNS requests (5353/udp). Linux hosts were running the avahi-daemon which provides this service. As the auto-discovery service is not used for service identification, the avahi-daemon was disabled.
Confirm response is seen on 5353/udp prior to change:
nmap -P0 -p 5353 -sU hostname.example.net
SSH to host identified as responding to mDNS requests. Disable the avahi-daemon then stop the avahi-daemon:
systemctl disable avahi-daemon systemctl stop avahi-daemon
Verify that 5353/udp is no longer open by repeating the nmap command.
Fin.
To automatically update files from your repository on your server, use a release pipeline. For convenience, I use deployment groups to ensure all of the servers are updated.
Creating a deployment group
Under the Project, navigate to Pipelines and “deployment groups”. Click “New” and provide a name for the deployment group.
Now click into the deployment group and select “Register”
Since I have a Linux server, I changed the “Type of target to register” drop-down to “Linux”. Copy the command and run it on your server (I don’t run literally what MS provides – I break it out into individual commands so I can make a folder named what I want it to be named and just run the part of the command that registers a service with systemctl.
Run the agent – for demonstration purposes, I am using the run.sh script to launch the agent. This outputs details to my console instead of a log file.
If you have multiple servers to which you want to deploy the files, install and run an agent on each one.
Create the release pipeline
Now we will build the pipeline that actually copies files over to the agent. Under Pipelines, navigate to “Releases”. Select “New” and create a “New release pipeline”. Start with an empty job.
You’ll be asked to name the first stage of the deployment pipeline – here, I’m calling it “Deploy Files to Servers”. Close out of the Stage window to see the pipeline.
Click the “+ Add” next to Artifacts to link an artifact to the deployment
If you have a build pipeline, you can link that as the artifact. Since I am just copying files, I selected the “Azure Repo” and configured the test project that contains the files I wish to deploy to my servers.
Click “Add”
Back in the pipeline, click the “1 job, 0 task” hyperlink to create a file deployment task.
We don’t need the “Agent job”, so click on it and click “Remove”
Select the hamburger menu next to “Deploy files to servers” and select “Add a deployment group job”
Click the “Deployment group” dropdown and select the deployment group that contains the servers to which you want to deploy files. You can add tags to limit deployment to a subset of the deployment group – I don’t do this, but I have seen instances where “prod” and “dev” tags were used and all servers in both the prod and dev environment were part of the same deployment group.
Click the “+” on the “Deployment group job” item to add a task.
Find the “Copy files” task and click “Add” to create a task to copy files.
Click on the “Copy files to:” item to configure the task. The source folder is the Azure repo, and the target folder is the path on the server.
Click “Save” to save the task, then click “OK” to save the task.
Now create a release – click the “Create a release” button
ok
When the deployment runs, the agent will show the job running.
Once the deployment completes, the files are on the server.
Scheduling Release
In the pipeline, you can click on “Schedule set” to schedule new releases.
Enable the schedule, set a time – I select to only schedule the release if the source or pipeline has changed … if I’ve not updated files in the repo, there’s no need to redeploy the files. Remember to save your pipeline when you add the schedule.
Ideally, the definitions for Kubernetes objects are all safely stored in your code repository — you can easily revert back to the previous, working iteration, you can see who changed what, and you’ve got a copy of it all available if super electromagnet man takes a stroll through the data center. Ideally.
Here, in the real world, we took over management of a k8s implementation that’s been in service for about a year now. And, fortunately, the production YAMLs are all in the repo. The development system, on the other hand, isn’t. Logic dictates that the config would be similar, but it’s always good to check.
I wrote a quick script to dump YAML files for all of the configmaps, cron jobs, deployments, horizontal pod autoscaling, jobs, persistent volumes, persistent volume claims, secrets, service accounts, services, and stateful sets.
#!/bin/bash
nsbase="namespace/"
for ns in $(kubectl get namespaces -o=name)
do
ns=${ns#${nsbase}}
for n in $(kubectl get --namespace=$ns -o=name configmap,cronjob,deployment,hpa,job,pv,pvc,secret,serviceaccount,service,statefulset)
do
yamlfile="${ns}/${n}.yaml"
mkdir -p $(dirname ${ns}/${n})
kubectl get --namespace=$ns -o=yaml $n > $yamlfile
done
done
Summary: Playlist items are not returned from searches initiated on my uPNP client. The playlist is visible when browsing the Gerbera web UI under Playlists->All Playlists->Playlist Name and Playlists->Directories->Playlists->Playlist Name
Action: In a uPNP client, search using the criteria
upnp:class = "object.container.playlistContainer" and dc:title = "Playlist Name",Expected Results: Playlist matching search criteria is returned
Actual Results: No results are returned
Investigation:
From the Gerbera debug log, the search being executed is:
SELECT DISTINCT "c"."id", "c"."ref_id","c"."parent_id", "c"."object_type", "c"."upnp_class", "c"."dc_title",
"c"."mime_type" , "c"."flags", "c"."part_number", "c"."track_number",
"c"."location", "c"."last_modified", "c"."last_updated"
FROM "mt_cds_object" "c"
INNER JOIN "mt_metadata" "m" ON "c"."id" = "m"."item_id"
INNER JOIN "grb_cds_resource" "re" ON "c"."id" = "re"."item_id"
WHERE (LOWER("c"."upnp_class")=LOWER('object.container.playlistContainer'))
AND (LOWER("c"."dc_title")=LOWER('Playlist Name'))
ORDER BY "c"."dc_title" ASC;
The playlists do not have a row in the grb_cds_resource table, so the “INNER JOIN” means the query returns no records.
I am able to work around this issue by manually inserting playlist items into the grb_cds_resource table
INSERT INTO grb_cds_resource (item_id, res_id, handlerType) VALUES (1235555,0,0);If I have some time, I want to test changing join2 to be a left outer join and see if that breaks anything.