Category: System Administration

Listing Modules In Dynamically Linked Shared Object Libraries

We had to rebuild a server over the weekend — it’s a lot harder to get Apache and PHP set up when you don’t have root access to just install things from the yum repository. And, unlike the servers where I built httpd and php from source … we basically relayed requests to the Unix admin to have packages installed. One of the confusions during the whole process was that we didn’t know what to use as the module name for PHP to load in the httpd.conf file. The line from our old server (LoadModule php5_module /etc/httpd/modules/libphp5.so) produced an error that there was no such thing to load.

When a library fails to load with some error, I know to use ldd … but I didn’t know there was a way to list out the modules in a library. Fortunately, one of my coworkers had already run nm and listed out the modules — nm -D –defined-only sharedLibraryFile | grep module — and we were able to identify that the libphp5.so that we had wasn’t anything like the one on the old server. By listing the modules for each of the shared object libraries installed by the php package, we got the proper module name for httpd.conf

Testing A New Web Server Without DNS Changes

When migrating to a new server, it’s good to validate site functionality before redirecting users to the new host. i.e. I have anya.rushworth.us set up in the httpd config on both server1 and server2. DNS currently points traffic to server1, but I need to test the site on server2.

Approach #1 – With administrative access to the host

Edit your hosts file – open an administrative command prompt

Edit %SYSTEMROOT%\system32\drivers\etc\hosts and add lines with the IP address WHITESPACE and the hostname(s). E.G.
127.0.0.1 lisatest lisatest.rushworth.us lisatest2 lisatest2.rushworth.us
10.1.2.3 otherhost otherhost.rushworth.us
10.2.3.4 anya anya.rushworth.us

Clear your DNS cache (ipconfig /flushdns) and navigate to the URL. You’ll be directed the IP address from your hosts file instead of the DNS registered address.

Approach #2 – No admin access

Install ModHeader in your Chrome browser and click the extension to modify the headers or install ModHeader in your Firefox browser. Click on the extension icon to set a header value.

Add a “Host” header with the value of the virtual host name you need to test

Navigate to the hostname of the new server – https://server2.rushworth.us – but the web server will receive the Host header you configured in ModHeader and serve the web site based on that host header.

 

Android Outlook Message Sending Failure

I’ve been getting a strange error when trying to send pictures within e-mail messages from my Android phone. I say a strange error because there’s literally one entry that comes back when you search for MessageDeliveryFailedException f5f0 — and no pointer at all as to what might have gone wrong. Just a non-delivery report popping into the Inbox on my phone:

     Technical details
     MessageDeliveryFailedException: Could not deliver the message [len=70, data=50005…C090005] sent at 8/25/2020 10:06:28 PM.Failure code: f5f0

I’ve got a reverse proxy with an application firewall and suspected that was the source of my problems. Mostly because errors caused within the Microsoft Exchange system are generally easy to find online. An oddball error is going to come from an oddball source. And I was right — my application proxy log shows an error each time I attempt to send one of the failed messages.

Edited /etc/httpd/conf.d/mod_security.conf and upped the SecRequestBodyNoFilesLimit. Once Apache HTTPD was restarted, I was able to send my messages without problem.

Discourse Censored Words List

It took an unexpectedly long time to find the censored word list in Discourse. I finally resorted to searching the PRs until I located one where the censored word list was replaced with ‘watched words’ … although there wasn’t any readily apparent watched word list in the configuration either. I was able to locate the meta post regarding the watched word implemented under that PR. It’s hiding under logs?! Under each action (block, censor, require approval, flag), there is a “show words” checkbox that displays the configured words.

Discourse in Docker on Fedora 32

I had to make a few tweaks in order to run the Discourse base Docker image. First, I got the following very clear error:

discourse docker this version of runc doesn't work on cgroups v2: unknown

I had to switch from cgroupv2 to cgroup

grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=0"

At which point I was at least able to run through the configuration. This yielded an access denied error attempting to create /shared/postgres:

Configuration file at updated successfully!

Updates successful. Rebuilding in 5 seconds.
Building app
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
cd /pups && git pull && /pups/bin/pups --stdin
Already up to date.
I, [2020-08-11T18:15:03.664400 #1] INFO -- : Loading --stdin
I, [2020-08-11T18:15:03.672609 #1] INFO -- : > locale-gen $LANG && update-locale
I, [2020-08-11T18:15:03.746912 #1] INFO -- : Generating locales (this might take a while)...
Generation complete.

I, [2020-08-11T18:15:03.747838 #1] INFO -- : > mkdir -p /shared/postgres_run
mkdir: cannot create directory ‘/shared/postgres_run’: Permission denied
I, [2020-08-11T18:15:03.754890 #1] INFO -- :

FAILED
--------------------
Pups::ExecError: mkdir -p /shared/postgres_run failed with return #<Process::Status: pid 21 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params "mkdir -p /shared/postgres_run"
d98ee8471413ad77ab27ed3506f12c5c94a2b6902622faf4d88d5dbb51a10f63
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.
./discourse-doctor may help diagnose the problem.

Gut was that I encountered an SELinux problem. Turns out I was right. There’s a lot of reading you can do about SELinux and Docker — this, for one — but the quick and simple solution is to run the docker container in privileged mode (note: this may not be a good idea in your specific scenario. understand what privileged mode is and the risks it entails). To do so, edit the launcher script (/var/discourse/launcher in my case) and add  “–privileged” to user_args:

And finally (and this may well be a RTFM thing) — you’ve got to have your public DNS set up & whatever firewall rules to get traffic to the http:// website you are trying to build in order to use the LetsEncrypt SSL cert and configure HTTPS. It uses the file-based verification (i.e. create a file named xyz in /path/to/xyz.whatever on your web server, lets encrypt grabs the file and verifies it exists) which fails quite spectacularly when the Internet at large cannot access your about-to-be-a-discourse-server.

Building LIB_MYSQLUDF_SYS On Fedora 31

I moved my MariaDB server to a new host and could not follow my previously working instructions to build lib_mysqludf_sys. The error indicated that my_atomic.h was not found.

[lisa@server03 lib_mysqludf_sys]# make
gcc -fPIC -Wall -I/usr/include/mysql/server -I. -shared lib_mysqludf_sys.c -o /usr/lib64/mariadb/plugin//lib_mysqludf_sys.so
In file included from /usr/include/mysql/server/my_sys.h:34,
from lib_mysqludf_sys.c:41:
/usr/include/mysql/server/my_pthread.h:26:10: fatal error: my_atomic.h: No such file or directory
26 | #include <my_atomic.h>
| ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:4: install] Error 1

The missing file is located in /usr/include/mysql/server/private … so I had to include that file in the gcc command as well. My new Makefile reads as follows:

[lisa@server03 lib_mysqludf_sys]# cat Makefile
LIBDIR=/usr/lib64/mariadb/plugin/

install:
gcc -fPIC -Wall -I/usr/include/mysql/server -I/usr/include/mysql/server/private -I. -shared lib_mysqludf_sys.c -o $(LIBDIR)/lib_mysqludf_sys.so

I was then able to make and use install.sh to load it into MariaDB.

What Can I sudo?

Some 90% of my Linux experience is on servers where I have root or root-equivalent access (i.e. I can sudo anything). In those cases, ‘what can I run under sudo’ was never a question. And I’d use something like “sudo less /etc/sudoers” to inspect what someone else was able to run when they questioned their access. In my new position, we have a lot of servers that we own too — the Engineering IT support group lets us spin up our own VMs, do whatever we want (within reason). But we have a few IT-managed servers with very restricted rights. And the commands I would use to perform functions (think systemctl restart httpd) aren’t in my sudoers access list. Luckily you can list out what you can run under sudo:

$ sudo -l
[sudo] password for useraccount:
Matching Defaults entries for useraccount on this host:
syslog=auth, loglinelen=0, syslog_goodpri=info, syslog_badpri=err,
logfile=/var/log/sudo.log

User useraccount may run the following commands on this host:
(ALL) /opt/lampp/lampp start, (ALL) /opt/lampp/lampp stop, (ALL)
/opt/lampp/lampp restart, (ALL) /usr/sbin/apachectl

And that is how I know to use apachectl instead of systemctl.

NVIDIA Driver Installation Issue – Fedora 30

NVIDIA finally released an updated driver for Scott’s laptop — one that should be compatible with the 5.x kernel. Ran through the normal process and got the following error:

     Unable to load the nvidia-drm kernel module

Which … was at least new. Tried running through the installation again but not registering the driver with the kernel. Installation completed successfully, and he’s able to boot the 5.8.100 kernel.

SCCM Shows “No items found”

The Windows 10 1909 upgrade was rolled out at work, and I got the “if you don’t get this installed, I’m gonna tell your manager” e-mail. Which is odd since all of this ‘stuff’ is supposed to be doing its thing in the background. But whatever. So I opened the “Software Center” and was told there were no items found under applications. Which … possible, I guess. I don’t use IT-deployed software that isn’t part of the stock image. But clicking over to “Operating Systems” (where the update should be found) also yielded “No items found”.

I know enough about Microsoft applications & AD to know I’m on cached credentials when I initiate the VPN connection. No idea what the refresh period is like, so I lock and unlock my workstation to ensure I’ve got an active authentication token. But that didn’t help — still no items found. I had to go into the “Control Panel”, open “Configuration Manager” as an administrative user, and select the ‘Actions’ tab. There were two — “Machine Policy Retrieval & Evaluation Cycle” and “User Policy Retrieval & Evaluation Cycle”. I ran both of them. A few minutes later, I went back into the Configuration Manager utility & found a bunch of things on the actions tab.

I ran all of them — nothing changed. Then let the computer sit for a few hours (I’m certain less than a few hours would have sufficed, but I had other things to do). Ran all of the actions again, and a notice popped up that I have new software available. Sigh! Now I’m downloading the six gig update — a process that should be done in a few hours. But at least I’ll have the update installed before the deadline.

In the process, I also discovered that the CCM logs have been moved from SYSTEM32/SYSWOW64 and are now located at %WINDIR%\CCM\logs