Category: Technology

Create 2 With a Pi

Scott has a friend who is linking a Pi up to a Create 2 — using the Ubiquity Pi image. Unfortunately, it doesn’t seem to communicate with the robot. Tx but to Rx. There are a few issues in the create_autonomy and libcreate repos that report the same issue and have subsequently merged in a fix. But the Ubiquity image is from June 2019 … which pre-dates the fix. I’ve run through the process of installing ROS and create_autonomy on a base Ubuntu. Thought I’d save the process for later 🙂

 

# Get Ubuntu 16.4 environment to match what Ubiquity Robotics is using … obviously, this would all be run on the Pi instead
docker run -dit –name UbuntuSandbox ubuntu:xenial

# Shell into running container
docker exec -it UbuntuSandbox bash

# Update software catalog
apt-get update

# Install Prerequisities
apt-get install python-rosdep python-rosinstall-generator python-wstool python-rosinstall build-essential cmake git libblkid-dev e2fslibs-dev libboost-all-dev libaudit-dev vim python-pip python-rosdep wget unzip

# Create a directory for this project
mkdir /roomba
cd /roomba

# Install ROS
apt-get install lsb-release
sh -c ‘echo “deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main” > /etc/apt/sources.list.d/ros-latest.list’
apt-key adv –keyserver hkp://ha.pool.sks-keyservers.net:80 –recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
apt-get update
apt-get upgrade
apt-get install -y python-rosdep python-rosinstall-generator python-wstool python-rosinstall build-essential cmake
mkdir /roomba/ros
cd /roomba/ros
rosdep init
rosdep update
mkdir -p /roomba/ros_catkin_ws
cd /roomba/ros_catkin_ws
rosinstall_generator ros_comm –rosdistro kinetic –deps –wet-only –tar > kinetic-ros_comm-wet.rosinstall
wstool init src kinetic-ros_comm-wet.rosinstall

mkdir -p /roomba/ros_catkin_ws/external_src
cd /roomba/ros_catkin_ws/external_src
wget http://sourceforge.net/projects/assimp/files/assimp-3.1/assimp-3.1.1_no_test_models.zip/download -O assimp-3.1.1_no_test_models.zip
unzip assimp-3.1.1_no_test_models.zip
cd assimp-3.1.1
cmake .
make
make install

cd /roomba/ros_catkin_ws
rosdep install -y –from-paths src –ignore-src –rosdistro kinetic -r

./src/catkin/bin/catkin_make_isolated –install -DCMAKE_BUILD_TYPE=Release –install-space /opt/ros/kinetic

# Source in the ROS bash setup stuff
source /opt/ros/kinetic/setup.bash

# Build libcreate
git clone https://github.com/AutonomyLab/libcreate.git
cd libcreate
mkdir build
cd build
cmake ..
make

# Build create_autonomy
apt-get install python-rosdep python-catkin-tools
mkdir -p /roomba/create_ws/src
cd /roomba/create_ws
catkin init
cd /roomba/create_ws/src
git clone https://github.com/AutonomyLab/create_autonomy.git

cd /roomba/create_ws
rosdep update
rosdep install –from-paths src -i

# Setup so CMake will find our libcreate when we build create_autonomy
mv /opt/ros/kinetic/lib/libcreate.so /opt/ros/kinetic/lib/libcreate.so.orig
cp /roomba/libcreate/build/libcreate.so /opt/ros/kinetic/lib/libcreate.so

# Continue building create_autonomy
catkin build

# Source in the autonomy bash environment
source /roomba/create_ws/devel/setup.bash

# Create config yaml
vi /roomba/config.yaml
## File content:
# Create config YAML
# The device path for the robot
dev: “/dev/ttyUSB0”

# Baud rate. Passing this parameter overwrites the inferred value based on the robot_model
baud: 115200

# Base frame ID
base_frame: “base_footprint”

# Odometry frame ID
odom_frame: “odom”

# Time (s) without receiving a velocity command before stopping the robot
latch_cmd_duration: 0.2

# Internal loop update rate (Hz)
loop_hz: 10.0

# Whether to publish the transform between odom_frame and base_frame
publish_tf: true
## End file content

# Run it
roslaunch ca_driver create_2.launch config:=/roomba/config.yaml desc:=false

# Logs at ~/.ros/log/latest

Network Manager GUI

You can use nmcli to configure network interfaces controlled by NetworkManager. But, honestly, I don’t see any advantage to learning the cryptic CLI instead of using the cryptic config file stuff I’ve already learned. And yet … I need my server to have /etc/resolv.conf populated when it reboots. So I figured out how to launch the KDE system settings (assuming you’ve got the X display redirected to your host) — systemsettings5

Mine takes a few minutes to render, during which time the window is black and a handful of errors are written out to the console. But it got there eventually, and I was able to edit the network interface.

Mosquitto 1.6.8 With Websockets Becomes Unresponsive

When I installed software on our new server, I got the “latest and greatest”. And have found that my mosquitto server hangs after about 20 minutes. No errors. Even strace doesn’t show anything beyond it not doing anything. I am able to use the command line utilities to confirm that the service isn’t there even though it looks like it is working.

Subscribe to the subtree of a topic:

mosquitto_sub -h mqtt.example.com -t testtopic/#

Publish a message to a topic:

mosquitto_pub -h mqtt.example.com -t testtopic/data/lisa -m "Test4"

To test WebSockets, I’ve put together a Python script that subscribes to a topic. Changing the commented lines switches between the WebSocket reverse proxy, the old and new MQTT servers via WebSockets, and the old and new MQTT servers directly in an attempt to isolate what is wrong.

import sys
import paho.mqtt.client as mqtt

def on_connect(mqttc, obj, flags, rc):
    print(f"rc: {str(rc)}")

def on_message(mqttc, obj, msg):
    print(f"{datetime.datetime.now()}: {msg.topic} {str(msg.qos)} {str(msg.payload)}")

def on_publish(mqttc, obj, mid):
    print(f"mid: {str(mid)}")

def on_subscribe(mqttc, obj, mid, granted_qos):
    print(f"Subscribed at {datetime.datetime.now()}: {str(mid)} {str(granted_qos)}")
def on_log(mqttc, obj, level, string):
    print(string)
# Client uses websockets
mqttc = mqtt.Client(transport='websockets', client_id="ljrTestingPythonScript", clean_session=False)
# Client uses MQTT directly
#mqttc = mqtt.Client(client_id="ljrTestingPythonScript", clean_session=False)

mqttc.username_pw_set("whateveruser", "wh@t3v3rP@s5w0rd")
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
mqttc.on_subscribe = on_subscribe

#mqttc.connect("oldmosquitto.example.com", 80, 60)
mqttc.connect("newmosquitto.example.com", 80, 60)
#mqttc.connect("newmqtt.example.com", 1883, 60)
#mqttc.connect("newmqtt.example.com", 9001, 60)
#mqttc.connect("oldmqtt.example.com", 1883, 60)
#mqttc.connect("oldmqtt.example.com", 9001, 60)

mqttc.subscribe("owntracks/#", 0)

mqttc.loop_forever()

None of this helped, unfortunately. The reverse proxy couldn’t communicate with the MQTT server because it was unresponsive. Attempting to communicate with the server directly fails too.

I see a few issues in the Mosquitto repository that are similar — and the current discussion indicates that libwebsockets 3.2.0 introduced some incompatibility that they’ve addressed in Mosquitto 1.6.7 … since my problem relates to WebSockets, I wanted to try running the iteration before whatever changed.

git clone --branch v3.1.0 https://libwebsockets.org/repo/libwebsockets
cd libwebsockets
mkdir build
cd build
cmake ..
make
make install

git clone --branch v1.6.2 https://github.com/eclipse/mosquitto.git
cd mosquitto
vi config.mk # Change WITH_WEBSOCKETS:=no to :=yes
make
make install

ln -s /usr/local/lib/libwebsockets.so.14 /lib64/
ln -s /usr/local/lib/libmosquitto.so.1 /usr/lib64/libmosquitto.so.1

/usr/local/sbin/mosquitto -v -c /etc/mosquitto/mosquitto.conf

We’ll see if this is more reliable. It’s been up for 30 minutes … which is longer than the 1.6.8 iteration managed to run.

MariaDB Strangeness

I had to remove and reinstall MariaDB to upgrade from Fedora 29. I assumed Fedora 31 would have a more recent version, so I just installed the database and attempted to start it. Nope.

In journalctl

Dec 28 23:37:49 fedora123 mysql-prepare-db-dir[3977]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done.
Dec 28 23:37:49 fedora123 mysql-prepare-db-dir[3977]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir.
Dec 28 23:37:49 fedora123 mysqld[4013]: 2019-12-28 23:37:49 0 [Note] /usr/libexec/mysqld (mysqld 10.3.20-MariaDB) starting as process 4013 …
Dec 28 23:37:49 fedora123 systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE

In mariadb.log

2019-12-28 23:23:53 0 [ERROR] Fatal error: Can’t open and lock privilege tables: ‘mysql.user’ is not of type ‘TABLE’

Turns out the Fedora repo has 10.3, but the MariaDB repo that I’d used with Fedora 29 had MariaDB 10.4. After enabling the MariaDB repo and installing 10.4, the service without error.

QNAP – SMB1

We’ve got an old NAS, and after the most recent Windows Update … I lost access to it. Now the NAS is really just a backup, so I never really use the NAS. I don’t generally waste my time fixing things I don’t need, but Scott wasn’t able to use the NAS from Fedora 31 either. So I tried to map the drive again and got told the NAS could only use SMB1 but Windows required 2 or 3.

I ran smb2status and, yeah, the max protocol is SMB 1.0. Simply running smb3enable or smb21enable configures Samba for SMB3 or SMB2.1 and restarts the services. We can use the QNAP Samba share from both Windows 10 and Fedora 31 now.

The end of password changes?

I knew Microsoft was publishing recommendations against forced password expiry, but it was still surprising to see this banner in my Azure admin portal. It would be nice if their message was clearer on the nuances here — especially that enabling MFA (preferably not SMS-based MFA that is just asking for someone important’s number to get hijacked) is an important component of this recommendation.

In one of my first jobs, I was a sys admin for call-center systems. As such, I interacted with a lot of the call center management and staff … and, when you know someone in IT, you ping them when the proper support route isn’t as responsive as you’d like. Which is to say I did a good bit of end-user support as well. The number of people whose password was written on a post-it note under the keyboard astonished me. This particular call center didn’t have floating seating, but two or three people would share a cube because they worked different schedules. If I’ve got to come up with a new thing I need to remember every 90 days … well, that’s how you end up with Winter19, Spring20, Summer20, Autumn20 or Maggie12, Maggie13, Maggie14 passwords. That then get posted under the keyboard so I can remember that I’m up to “14” now. Couple that with the overhead of supporting password resets for those who didn’t write it down and happened to forget the password. I’d been a proponent of long password expiry coupled with increased complexity requirements. Maybe !Maggie-19? is good for all of 2019. It’s nice to see a major IT vendor starting to realize the real-world impact of IT policies.

Reading VMWare and Hyper-V Disk Files

libguestfs-tools-c, which was installed when I set up the Linux VM server, can mount VMWare and Hyper-V disks.

[root@server /data/Virtual Machines/_Backup/VMWare/Win - Copy/]# guestmount -a WinHardDisk.vhd -m /dev/sda1 --ro /mnt/vhdx/
[root@server /data/Virtual Machines/_Backup/VMWare/Win - Copy/]# ls /mnt/vhdx/
AUTOEXEC.BAT COMMAND.COM FRUNLOG.TXT MSDOS.SYS 'Program Files' SETUPXLG.TXT VIDEOROM.BIN
BOOTLOG.PRV CONFIG.SYS IO.SYS 'My Documents' RECYCLED SUHDLOG.DAT WINDOWS
BOOTLOG.TXT DETLOG.TXT MSDOS.--- NETLOG.TXT SETUPLOG.TXT SYSTEM.1ST WUTemp
[root@server /data/Virtual Machines/_Backup/VMWare/Win - Copy/]# umount /mnt/vhdx

Screen Mouse Scrolling Sends Arrows

I’m finding all sorts of things we researched and sorted on the old server without documenting 🙁 Today’s discovery is that screen uses alternate screens where the mouse wheel sends arrow keys. Which means you end up going through your history instead of scrolling up a few lines. Fix? Either create a .screenrc in the user’s account or globally change /etc/screenrc (which is what we do):

 

[root@server-replacement ~]# diff /etc/screenrc /root/oldserver-screenrc
216a217,218
>
> termcapinfo xterm* ti@:te@

Zoneminder, time zones, and php-fpm

I’m in the process of installing Zoneminder on our new server. It was a fairly straightforward process — stop Zoneminder on the old server, dump the SQL database, fix the DEFINER values since I’m using a central database server instead of a server on localhost, install Zoneminder, copy the config file, set up the database user, pull in the SQL file, and start it all up.

Visiting the website, I get “ZoneMinder is not installed properly: php’s date.timezone is not set to a valid timezone”. I’d forgotten to set the timezone in php.ini. Added ‘date.timezone = “America/New_York”‘, restarted httpd and Zoneminder. And got the same error.

<?php
error_reporting(E_ALL);
var_dump(ini_get('date.timezone'),date_default_timezone_get());
?>

It’s not set. This isn’t a funky Zoneminder thing — this is a PHP problem. I realized that PHP now runs as its own service. Restarting httpd is insufficient. Restarted php-ftm and the time zone I’d set in php.ini showed up. This is a case where a reboot would have sorted it … but good to remember that, when changing PHP settings, the php service needs to be restarted.