Month: October 2020

PrivateTmp Strangeness — apachectl v/s systemctl

This is a very strange problem — we had a web server upgraded recently. We use “sudo apachectl start” to bring up the server since the server is maintained by a dedicated Unix support team, and the site worked fine. Until … Sunday morning after the log rotation. Then Box Spout was unable to access the XML data for an Excel file to compress it. Lots of errors:

 

[Mon Sep 14 10:59:39.137728 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: ZipArchive::close(): Zlib error: stream error in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 199, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137785 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fopen(/tmp/xlsx5f5f934699dcd9.17695276.zip): failed to open stream: No such file or directory in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 213, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137803 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: stream_copy_to_stream() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 214, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.137814 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fclose() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 215, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.138360 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Fatal error: Uncaught exception ‘Box\\Spout\\Common\\Exception\\IOException’ with message ‘Cannot perform I/O operation outside of the base folder: /tmp’ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php:130\nStack trace:\n#0 /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php(82): Box\\Spout\\Common\\Helper\\FileSystemHelper->throwIfOperationNotInBaseFolder(‘/tmp/xlsx5f5f93…’)\n#1 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Helper/FileSystemHelper.php(369): Box\\Spout\\Common\\Helper\\FileSystemHelper->deleteFile(‘/tmp/xlsx5f5f93…’)\n#2 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Internal/Workbook.php(134): Box\\Spout\\Writer\\XLSX\\Helper\\FileSystemHelper->zipRootFolderAndCopyToStream(Resource id #26)\n#3 /path/to/site/ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php on line 130, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139468 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: ZipArchive::close(): Zlib error: stream error in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 199, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139504 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fopen(/tmp/xlsx5f5f934699dcd9.17695276.zip): failed to open stream: No such file or directory in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 213, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139515 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: stream_copy_to_stream() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 214, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139533 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Warning: fclose() expects parameter 1 to be resource, boolean given in /path/to/site/classes/vendor/box/spout/src/Spout/Writer/Common/Helper/ZipHelper.php on line 215, referer: https://hostname.example.com/path/to/code.php
[Mon Sep 14 10:59:39.139599 2020] [:error] [pid 57117] [client 10.1.2.3:49276] PHP Fatal error: Uncaught exception ‘Box\\Spout\\Common\\Exception\\IOException’ with message ‘Cannot perform I/O operation outside of the base folder: /tmp’ in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php:130\nStack trace:\n#0 /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php(82): Box\\Spout\\Common\\Helper\\FileSystemHelper->throwIfOperationNotInBaseFolder(‘/tmp/xlsx5f5f93…’)\n#1 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Helper/FileSystemHelper.php(369): Box\\Spout\\Common\\Helper\\FileSystemHelper->deleteFile(‘/tmp/xlsx5f5f93…’)\n#2 /path/to/site/classes/vendor/box/spout/src/Spout/Writer/XLSX/Internal/Workbook.php(134): Box\\Spout\\Writer\\XLSX\\Helper\\FileSystemHelper->zipRootFolderAndCopyToStream(Resource id #26)\n#3 /path/to/site in /path/to/site/classes/vendor/box/spout/src/Spout/Common/Helper/FileSystemHelper.php on line 130, referer: https://hostname.example.com/path/to/code.php

 

The postupdate script is “systemctl reload httpd.service” — so not exactly the same thing we used to launch the service originally. But I’ve never seen differing behavior between apachectl and systemctl started HTTPD instances. Quick/dirty solution is to disable PrivateTmp, but I’m hoping to be able to isolate why exactly the postupdate script appears to break the service’s access to the private tmp space.

30 November 2020 addendum — In discussing the issue with RedHat, they suggested using either

/sbin/killall -HUP httpd

or

/bin/systemctl restart httpd.service > /dev/null 2>/dev/null || true

Doing this has allowed continual access to the Private Tmp space after log rotation. Woohoo! Not sure why the default configuration that came from the Apache httpd package didn’t work (i.e. it’s not like we built some funky weird log rotation script). But success is good enough for me.

Python Time-Expiring Cache

I needed to post information into a SharePoint Online list. There’s an auth header required to post data, but the authentication expires every couple of minutes. Obviously, I could just get a new auth string each time … but that’s terribly inefficient. I could also use what I had and refresh it when it fails … inelegant but effective. I wanted, instead, to have a value cached for a duration slightly less than the server-side expiry on the auth string. This decorator allows me to use the cached auth header string for some period of time and actually run the function that grabs the auth header string before the string is invalidated. 

import functools
import time
from datetime import datetime

def timed_cache(iMaxAge, iMaxSize=128, boolTyped=False):
    #######################################
    # LRU cache decorator with expiry time
    #
    # Args:
    #    iMaxAge: Seconds to live for cached results. 
    #    iMaxSize: Maximum cache size (see functools.lru_cache).
    #    boolTyped: Cache on distinct input types (see functools.lru_cache).
    #######################################
    def _decorator(fn):
        @functools.lru_cache(maxsize=iMaxSize, typed=boolTyped)
        def _new(*args, __time_salt, **kwargs):
            return fn(*args, **kwargs)

        @functools.wraps(fn)
        def _wrapped(*args, **kwargs):
            return _new(*args, **kwargs, __time_salt=int(time.time() / iMaxAge))
        return _wrapped

    return _decorator

# Usage example -- 23 second cache expiry
@timed_cache(23)
def slow_function(iSleepTime: int):
    datetimeStart = datetime.now()
    time.sleep(iSleepTime)
    return f"test started at {datetimeStart} and ended at at {datetime.now()}"


print(f"Start at {datetime.now()}")

for i in range(1, 50):
    print(slow_function(5))
    
    if i % 5 is 0:
        print(f"sleeping 5 at {datetime.now()}")
        time.sleep(5)
        print(f"done sleeping at {datetime.now()}\n\n")

print(f"Ended at at {datetime.now()}")

Ohio Absentee Ballot — Confirming Rejected Ballot

I requested an absentee ballot this year — we’d used absentee ballots once back when we lived in Geauga county and delivered the ballots to the Board of Elections office, but we generally vote in person. I wanted Anya to experience the process (and they give out cute stickers!). But we’ll have more time to explore what the ballot actually is at home, and there won’t be a bunch of people about. But — and the one thing that makes me want to do the early in-person voting — they can reject your ballot if the signature doesn’t match. I didn’t have a problem with my absentee ballot request, but Scott’s got rejected for a signature mismatch. And that had be worried about my actual ballot.

But, amid all of Trump’s blathering last night? Biden managed to convey that a lot of states have a process for curing rejected absentee ballots. I had no idea. Per Ohio Rev Code ยง 3509.06:

“(b) If the election officials find that the identification envelope statement of voter is incomplete or that the information contained in that statement does not conform to the information contained in the statewide voter registration database concerning the voter, the election officials shall mail a written notice to the voter, informing the voter of the nature of the defect. The notice shall inform the voter that in order for the voter’s ballot to be counted, the voter must provide the necessary information to the board of elections in writing and on a form prescribed by the secretary of state not later than the seventh day after the day of the election. The voter may deliver the form to the office of the board in person or by mail. If the voter provides the necessary information to the board of elections not later than the seventh day after the day of the election and the ballot is not successfully challenged on another basis, the voter’s ballot shall be processed and counted in accordance with this section.”

Which means I’d be notified if my ballot is rejected, and I could go to the Board of Elections within seven days of the election to provide whatever sort of identification they need to be satisfied. A total of fifteen states provide a remediation path for signature challenges — details for those states at https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to-correct-signature-discrepancies.aspx