Category: System Administration

Android Outlook Message Sending Failure

I’ve been getting a strange error when trying to send pictures within e-mail messages from my Android phone. I say a strange error because there’s literally one entry that comes back when you search for MessageDeliveryFailedException f5f0 — and no pointer at all as to what might have gone wrong. Just a non-delivery report popping into the Inbox on my phone:

     Technical details
     MessageDeliveryFailedException: Could not deliver the message [len=70, data=50005…C090005] sent at 8/25/2020 10:06:28 PM.Failure code: f5f0

I’ve got a reverse proxy with an application firewall and suspected that was the source of my problems. Mostly because errors caused within the Microsoft Exchange system are generally easy to find online. An oddball error is going to come from an oddball source. And I was right — my application proxy log shows an error each time I attempt to send one of the failed messages.

Edited /etc/httpd/conf.d/mod_security.conf and upped the SecRequestBodyNoFilesLimit. Once Apache HTTPD was restarted, I was able to send my messages without problem.

Discourse Censored Words List

It took an unexpectedly long time to find the censored word list in Discourse. I finally resorted to searching the PRs until I located one where the censored word list was replaced with ‘watched words’ … although there wasn’t any readily apparent watched word list in the configuration either. I was able to locate the meta post regarding the watched word implemented under that PR. It’s hiding under logs?! Under each action (block, censor, require approval, flag), there is a “show words” checkbox that displays the configured words.

Discourse in Docker on Fedora 32

I had to make a few tweaks in order to run the Discourse base Docker image. First, I got the following very clear error:

discourse docker this version of runc doesn't work on cgroups v2: unknown

I had to switch from cgroupv2 to cgroup

grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"

At which point I was at least able to run through the configuration. This yielded an access denied error attempting to create /shared/postgres:

Configuration file at updated successfully!

Updates successful. Rebuilding in 5 seconds.
Building app
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
cd /pups && git pull && /pups/bin/pups --stdin
Already up to date.
I, [2020-08-11T18:15:03.664400 #1] INFO -- : Loading --stdin
I, [2020-08-11T18:15:03.672609 #1] INFO -- : > locale-gen $LANG && update-locale
I, [2020-08-11T18:15:03.746912 #1] INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2020-08-11T18:15:03.747838 #1] INFO -- : > mkdir -p /shared/postgres_run
mkdir: cannot create directory ‘/shared/postgres_run’: Permission denied
I, [2020-08-11T18:15:03.754890 #1] INFO -- :

FAILED
--------------------
Pups::ExecError: mkdir -p /shared/postgres_run failed with return #<Process::Status: pid 21 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params "mkdir -p /shared/postgres_run"
d98ee8471413ad77ab27ed3506f12c5c94a2b6902622faf4d88d5dbb51a10f63
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Gut was that I encountered an SELinux problem. Turns out I was right. There’s a lot of reading you can do about SELinux and Docker — this, for one — but the quick and simple solution is to run the docker container in privileged mode (note: this may not be a good idea in your specific scenario. understand what privileged mode is and the risks it entails). To do so, edit the launcher script (/var/discourse/launcher in my case) and add  “–privileged” to user_args:

And finally (and this may well be a RTFM thing) — you’ve got to have your public DNS set up & whatever firewall rules to get traffic to the http:// website you are trying to build in order to use the LetsEncrypt SSL cert and configure HTTPS. It uses the file-based verification (i.e. create a file named xyz in /path/to/xyz.whatever on your web server, lets encrypt grabs the file and verifies it exists) which fails quite spectacularly when the Internet at large cannot access your about-to-be-a-discourse-server.

Building LIB_MYSQLUDF_SYS On Fedora 31

I moved my MariaDB server to a new host and could not follow my previously working instructions to build lib_mysqludf_sys. The error indicated that my_atomic.h was not found.

[lisa@server03 lib_mysqludf_sys]# make
gcc -fPIC -Wall -I/usr/include/mysql/server -I. -shared lib_mysqludf_sys.c -o /usr/lib64/mariadb/plugin//lib_mysqludf_sys.so
In file included from /usr/include/mysql/server/my_sys.h:34,
from lib_mysqludf_sys.c:41:
/usr/include/mysql/server/my_pthread.h:26:10: fatal error: my_atomic.h: No such file or directory
26 | #include <my_atomic.h>
| ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:4: install] Error 1

The missing file is located in /usr/include/mysql/server/private … so I had to include that file in the gcc command as well. My new Makefile reads as follows:

[lisa@server03 lib_mysqludf_sys]# cat Makefile
LIBDIR=/usr/lib64/mariadb/plugin/

install:
gcc -fPIC -Wall -I/usr/include/mysql/server -I/usr/include/mysql/server/private -I. -shared lib_mysqludf_sys.c -o $(LIBDIR)/lib_mysqludf_sys.so

I was then able to make and use install.sh to load it into MariaDB.

What Can I sudo?

Some 90% of my Linux experience is on servers where I have root or root-equivalent access (i.e. I can sudo anything). In those cases, ‘what can I run under sudo’ was never a question. And I’d use something like “sudo less /etc/sudoers” to inspect what someone else was able to run when they questioned their access. In my new position, we have a lot of servers that we own too — the Engineering IT support group lets us spin up our own VMs, do whatever we want (within reason). But we have a few IT-managed servers with very restricted rights. And the commands I would use to perform functions (think systemctl restart httpd) aren’t in my sudoers access list. Luckily you can list out what you can run under sudo:

$ sudo -l
[sudo] password for useraccount:
Matching Defaults entries for useraccount on this host:
syslog=auth, loglinelen=0, syslog_goodpri=info, syslog_badpri=err,
logfile=/var/log/sudo.log

User useraccount may run the following commands on this host:
(ALL) /opt/lampp/lampp start, (ALL) /opt/lampp/lampp stop, (ALL)
/opt/lampp/lampp restart, (ALL) /usr/sbin/apachectl

And that is how I know to use apachectl instead of systemctl.

NVIDIA Driver Installation Issue – Fedora 30

NVIDIA finally released an updated driver for Scott’s laptop — one that should be compatible with the 5.x kernel. Ran through the normal process and got the following error:

     Unable to load the nvidia-drm kernel module

Which … was at least new. Tried running through the installation again but not registering the driver with the kernel. Installation completed successfully, and he’s able to boot the 5.8.100 kernel.

SCCM Shows “No items found”

The Windows 10 1909 upgrade was rolled out at work, and I got the “if you don’t get this installed, I’m gonna tell your manager” e-mail. Which is odd since all of this ‘stuff’ is supposed to be doing its thing in the background. But whatever. So I opened the “Software Center” and was told there were no items found under applications. Which … possible, I guess. I don’t use IT-deployed software that isn’t part of the stock image. But clicking over to “Operating Systems” (where the update should be found) also yielded “No items found”.

I know enough about Microsoft applications & AD to know I’m on cached credentials when I initiate the VPN connection. No idea what the refresh period is like, so I lock and unlock my workstation to ensure I’ve got an active authentication token. But that didn’t help — still no items found. I had to go into the “Control Panel”, open “Configuration Manager” as an administrative user, and select the ‘Actions’ tab. There were two — “Machine Policy Retrieval & Evaluation Cycle” and “User Policy Retrieval & Evaluation Cycle”. I ran both of them. A few minutes later, I went back into the Configuration Manager utility & found a bunch of things on the actions tab.

I ran all of them — nothing changed. Then let the computer sit for a few hours (I’m certain less than a few hours would have sufficed, but I had other things to do). Ran all of the actions again, and a notice popped up that I have new software available. Sigh! Now I’m downloading the six gig update — a process that should be done in a few hours. But at least I’ll have the update installed before the deadline.

In the process, I also discovered that the CCM logs have been moved from SYSTEM32/SYSWOW64 and are now located at %WINDIR%\CCM\logs

Apache HTTPD: SSL Virtual Hosts

For quite some time, you couldn’t bind multiple SSL web sites to a single IP:Port combination — this had to do with the mechanics of negotiating an SSL session — the client and server negotiated encryption based on a specific certificate before the server really knew what the client was trying to retrieve. The quick/easy solution was to just add a virtual IP to the box and bind each individual web site to a unique IP address. While this was quite effective in a corporate environment or purely internal network, it was a terrible solution for a set of home-hosted personal web servers — I don’t want to buy four public IP addresses to host four differently named websites. My workaround was to off-port sites no one else would be using (the MQTT WebSockets reverse proxy) and use a reverse proxy to map paths within the family website to the remaining web servers. This page, for instance, is rushworth.us/lisa … which the reverse proxy re-maps to https://lisa.rushworth.us behind the scenes.

With Apache HTTPD 2.2.12 or later built against OpenSSL v0.9.8g or later, you can use Server Name Indication (SNI) to serve multiple SSL websites from a single IP:Port just like you have been able to do with non-SSL sites. Using SNI, the client includes “what they’re looking for” in first message of the SSN negotiation process so the server knows which cert to serve.

In your httpd.conf, indicate that you want to use SNI on an IP:Port combo

# Listen for virtual host requests on all IP addresses
NameVirtualHost *:443

And, optionally, configure one of the named virtual hosts as the default for non-SNI browsers:

SSLStrictSNIVHostCheck off

Now the configuration for your SSL sites can include a ServerName directive. Restart Apache HTTPD, and you’ll be able to access the proper SSL-enabled website without adding virtual IP addresses.