Tag: OUD

OUD 11g Failure

Ran out of disk space on the OUD partition (don’t let people turn on debug logging!!). Easily fixed — stop service, remove big log file, start again. Unfortunately that “start again” wasn’t as easy as it could have been. Server startup fails. It says the cryptomanager failed to publish the instance-key-pair in ADS …. which, Oracle says means you get to recreate the single instance. Honestly, this isn’t a horrible process since I’ve got a backup of the directory. But I’d rather not spend the next hour messing around with the server.

The underlying problem is that the admin partition didn’t dump out to its storage file (full disk), so admin-backend.ldif is a 0-byte file. Now, had the directory been running when I noticed this … I could have dumped the cn=admin data partition to another volume and copied the file in once I cleared up disk space. But I have a file-level backup … and it was easy enough to pull the file from last night back. Voila, one server online in a couple of minutes. And a good reminder for the future that a shutdown with a full disk … no good. I probably want to find something to free up a couple meg of space prior to shutting down the server. Extra-cautious option would be the MS Exchange approach of creating a few 5-meg files to ensure there’s space that can be freed up.

 

Added 06 Feb 2020: One of my colleagues encountered this problem again today, and it turns out you can copy the admin-backend.ldif from another server too. It may not be 100% — he cannot log into the server through the admin console. But he’s got a directory being served & no users complaining about an outage. It can get sorted properly later.

Gathering Info For Oracle – Oracle Unified Directory

I’ve been opening a lot of tickets for Oracle Unified Directory bugs recently. To save time gathering, I put together a quick script that gathers the data for an initial ticket. It depends on having a $PKGDIRECTORY environment variable set to the installation directory, and the data is stashed in a gzip’d tar file in the /tmp directory.

Script:

$PKGDIRECTORY/Middleware/asinst_1/OUD/bin/start-ds -s > /tmp/${HOSTNAME%%.*}-startds.txt
$PKGDIRECTORY/Middleware/asinst_1/OUD/bin/status -D “cn=directory manager” -j ~/pwd.txt > /tmp/${HOSTNAME%%.*}-status.txt

tar -cvzf /tmp/${HOSTNAME%%.*}.tgz /tmp/${HOSTNAME%%.*}-startds.txt /tmp/${HOSTNAME%%.*}-status.txt $PKGDIRECTORY/Middleware/asinst_1/OUD/logs/access $PKGDIRECTORY/Middleware/asinst_1/OUD/logs/admin $PKGDIRECTORY/Middleware/asinst_1/OUD/logs/errors $PKGDIRECTORY/Middleware/asinst_1/OUD/logs/replication $PKGDIRECTORY/Middleware/asinst_1/OUD/logs/server.out
chmod o+r /tmp/${HOSTNAME%%.*}.tgz

Updating Oracle Unified Directory (OUD) SSL Certificate

# PRE-CHANGE VERIFICATION
# There are two environment variables set to allow this to work:
# WLSTOREPASS=Wh@t3v3rY0uU53d # WLSTOREPASS is set to whatever is used for the keystore and truststore password
# OUDINST=/path/to/OUD/installation (root into which both java and OUD were installed — if you are using an OS package
# for java, your paths will be different)
# Log into OUD web management GUI (https://hostname.domain.gTLD:7002/odsm) and verify for each server:
# Configuration=>General Configuration=>Connection Handlers=>LDAPS Connection handler: “Secure access properties” section, Key Manager Provider & Trust Manager Provider are JKS. Certificate name is short hostname
# Configuration=>General Configuration=>Kery Managers=>JKS: Path is /$OUDINST/Oracle/Middleware/<short hostname>.jks

# During Change, server can be online
# Use the web GUI to issue certificates from WIN-WEB-CA. Export each cert as a PFX with keystore password $WLSTOREPASS
# On each server, place the approprate PFX file named with the hostname (i.e. the cert for LDAPFrontEndAlias.domain.gTLD will be stored to HOST1 as host1.pfx but stored on HOST2 as host2.pfx) in /tmp/ssl
# Alternatively, issue one certificate with each hostname and the front end alias as SAN values and use a static filename for the PFX file
# Put the root & web CA base-64 public key in /tmp/ssl/ as well (named Win-Root-CA.b64.cer and Win-Web-CA.b64.cer)

### Import the chain for the private key certificate
$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-ROOT -file /tmp/ssl/Win-Root-CA.b64.cer -keystore /tmp/ssl/${HOSTNAME%%.*}.jks -keypass $WLSTOREPASS -storepass $WLSTOREPASS
$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-WEB -file /tmp/ssl/Win-Web-CA.b64.cer -keystore /tmp/ssl/${HOSTNAME%%.*}.jks -keypass $WLSTOREPASS -storepass $WLSTOREPASS

# get GUID for the private key in the PFX file
HOSTCERTALIAS=”$($OUDINST/java/jdk/bin/keytool -v -list -storetype pkcs12 -keystore /tmp/ssl/${HOSTNAME%%.*}.pfx –storepass $WLSTOREPASS | grep Alias | cut -d: -f2-)”

# Change the cert alias to be the short hostname
$OUDINST/java/jdk/bin/keytool -importkeystore -srckeystore /tmp/ssl/${HOSTNAME%%.*}.pfx -destkeystore /tmp/ssl/${HOSTNAME%%.*}.jks -srcstoretype pkcs12 -deststoretype JKS -alias $HOSTCERTALIAS -storepass $WLSTOREPASS -srcstorepass $WLSTOREPASS
$OUDINST/java/jdk/bin/keytool -changealias -alias $HOSTCERTALIAS -destalias ${HOSTNAME%%.*} -keypass $WLSTOREPASS -keystore /tmp/ssl/${HOSTNAME%%.*}.jks -storepass $WLSTOREPASS

# Verify you have a WIN-ROOT, WIN-WEB, and hostname record
$OUDINST/java/jdk/bin/keytool -v -list -keystore /tmp/ssl/${HOSTNAME%%.*}.jks –storepass $WLSTOREPASS | grep Alias

# STOP THE LDAP SERVER AT THIS POINT
# Back up the current Java keystore file and move the new one into place
CURRENTDATE=”$(date +%Y%m%d)”
mv $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks $OUDINST/Oracle/Middleware/$CURRENTDATE.jks

cp $OUDINST/Oracle/Middleware/asinst_1/OUD/config/truststore $OUDINST/Oracle/Middleware/asinst_1/OUD/config/truststore-$CURRENTDATE
$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-ROOT -file /tmp/ssl/Win-Root-CA.b64.cer -keystore $OUDINST/Oracle/Middleware/asinst_1/OUD/config/truststore -keypass $WLSTOREPASS -storepass $WLSTOREPASS
$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-WEB -file /tmp/ssl/Win-Web-CA.b64.cer -keystore $OUDINST/Oracle/Middleware/asinst_1/OUD/config/truststore -keypass $WLSTOREPASS -storepass $WLSTOREPASS

mv /tmp/ssl/${HOSTNAME%%.*}.jks $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks

# START THE LDAP SERVER AND check for errors / test

# Backout:
# Stop the LDAP server
# mv $OUDINST/Oracle/Middleware/$CURRENTDATE.jks $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks
# mv $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks /tmp/ssl/${HOSTNAME%%.*}.jks
# Start the LDAP server

Updating Weblogic Certificate For OUD Management Utility

This is the process I use to update the WebLogic SSL certificate for our OUD management web interface. 


# PRE-CHANGE VERIFICATION
# There are two environment variables set to allow this to work:
# WLSTOREPASS=Wh@t3v3rY0uU53d # WLSTOREPASS is set to whatever is used for the keystore and truststore password
# OUDINST=/path/to/OUD/installation (root into which both java and OUD were installed — if you are using an OS package
# for java, your paths will be different)
#Log into https://hostname.domain.gTLD:7002/console (or whatever your WL console URL is)
# As my WebLogic instance auths users via LDAP, I log in with my UID & pwd … you may have a generic account like ‘admin’
#
#Navigate to Domain Structure => Environment => Servers
#Select “AdminServer”
#
#Keystores tab — will tell you the name of the keystore and trust store
#SSL tab — will tell you the friendly name of the certificate
# Verify the keystore and truststore are $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks,
# Verify the friendly name of the certificate is the short hostname
#
# Verify the keystore is using the normal keystore password
#[ldap@dell115 ~]$ $OUDINST/java/jdk/bin/keytool -v -list -keystore $OUDINST/Oracle/Middleware/dell115.jks –storepass $WLSTOREPASS| grep Alias
#Alias name: dell115
#Alias name: win-we
#Alias name: win-root
#Alias name: winca1-root
#Alias name: winca1-issuing
# *** If you do not get any output, remove the ” | grep Alias” part and check for errors. “keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect” means the password is different.
# *** either try to guess the password (company name or ‘a’ are good guesses, along with the java-typical default of changeit)
# *** to continue using the existing password or you’ll need to update the keystore and truststore passwords in the web GUI.
# *** Since the keystores are generated using the process below … 99% of the time, the password matches.
#
# Generate a cert with appropriate info, export public/private key as a PFX file named with the short hostname of the server (i.e. dell115.pfx here) and, as the keystore password, use whatever you’ve set in $WLSTOREPASS

 # DURING THE CHANGE, as the ldap service account on the server:

mkdir /tmp/ssl

# Put base 64 public keys for our root and web CA in /tmp/ssl as Win-Root-CA.b64.cer and Win-Web-CA.b64.cer
# Put public/private key export from above in /tmp/ssl 

# Import the keychain for your certificate
$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-ROOT -file /tmp/ssl/Win-Root-CA.b64.cer -keystore /tmp/ssl/${HOSTNAME%%.*}.jks -keypass $WLSTOREPASS -storepass $WLSTOREPASS

$OUDINST/java/jdk/bin/keytool -import -v -trustcacerts -alias WIN-WEB -file /tmp/ssl/Win-Web-CA.b64.cer -keystore /tmp/ssl/${HOSTNAME%%.*}.jks -keypass $WLSTOREPASS -storepass $WLSTOREPASS 

# get GUID for cert within PFX file
HOSTCERTALIAS=”$($OUDINST/java/jdk/bin/keytool -v -list -storetype pkcs12 -keystore /tmp/ssl/${HOSTNAME%%.*}.pfx –storepass $WLSTOREPASS | grep Alias | cut -d: -f2-)” 

# Import the private key
$OUDINST/java/jdk/bin/keytool -importkeystore -srckeystore /tmp/ssl/${HOSTNAME%%.*}.pfx -destkeystore /tmp/ssl/${HOSTNAME%%.*}.jks -srcstoretype pkcs12 -deststoretype JKS -alias $HOSTCERTALIAS -storepass $WLSTOREPASS -srcstorepass Ra1n1ng1

# Change the alias to match what is configured in the web GUI
$OUDINST/java/jdk/bin/keytool -changealias -alias $HOSTCERTALIAS -destalias ${HOSTNAME%%.*} -keypass $WLSTOREPASS-keystore /tmp/ssl/${HOSTNAME%%.*}.jks -storepass $WLSTOREPASS 

# Verify you have a WIN-ROOT, WIN-WEB, and hostname record

$OUDINST/java/jdk/bin/keytool -v -list -keystore /tmp/ssl/${HOSTNAME%%.*}.jks –storepass $WLSTOREPASS | grep Alias

# Stop the weblogic server

# Back up current keystore file and move new one into place
CURRENTDATE=”$(date +%Y%m%d)”
mv $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks $OUDINST/Oracle/Middleware/$CURRENTDATE.jks
cp /tmp/ssl/${HOSTNAME%%.*}.jks $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks

# Start the weblogic server in the screen session, then disconnect from the screen session

# Assuming success
rm -rf /tmp/ssl

# Backout is
# stop weblogic
mv $OUDINST/Oracle/Middleware/$CURRENTDATE.jks  $OUDINST/Oracle/Middleware/${HOSTNAME%%.*}.jks
# start weblogic

LDIF To Move User Accounts In Oracle Unified Directory

Since I keep wasting an hour to figure this out every time I need to move a user within OUD, I’m writing down the proper LDIF text to move a user from ou=disabled,o=orgName to ou=users,o=orgName.

dn: uid=TestUser123,ou=disabled,o=orgName
changetype: moddn
newrdn: uid=TestUser123
deleteoldrdn: 1
newSuperior: ou=users,o=orgName

For some reason, Oracle’s documentation omits the newrdn component and it all fails spectacularly.

OUD Returning Some DirectoryString Syntax Values As UTF-8 Encoded Bytes

We are still in the process of moving the last few applications from DSEE to OUD 11g so the DSEE 6.3 directory can be decommissioned. Just two to go! But the application, when pointed to the OUD servers, gets “Unable to cast object of type ‘System.Byte[]’ to type ‘System.String'” when retrieving values for a few of our DirectoryString syntax custom schema.

This code snippet works fine with DSEE 6.3.

string strUserGivenName = (String)searchResult.Properties["givenName"][0]; 
string strUserSurame = (String)searchResult.Properties["sn"][0]; 
string strSupervisorFirstName = (String)searchResult.Properties["positionmanagernamefirst"][0]; 
string strSupervisorLastName = (String)searchResult.Properties["positionmanagernamelast"][0];

Direct the connection to the OUD 11g servers, and an error is returned.

     

The attributes use the same syntax – DirectoryString, OID 1.3.6.1.4.1.1466.115.121.1.15.

00-core.ldif:attributeTypes: ( 2.5.4.41 NAME ‘name’ EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{32768} X-ORIGIN ‘RFC 4519’ ) 
00-core.ldif:attributeTypes: ( 2.5.4.4 NAME ( ‘sn’ ‘surname’ ) SUP name X-ORIGIN ‘RFC 4519’ ) 
00-core.ldif:attributeTypes: ( 2.5.4.42 NAME ‘givenName’ SUP name X-ORIGIN ‘RFC 4519’ ) 

99-user.ldif:attributeTypes: ( positionManagerNameMI-oid NAME ‘positionmanagernamemi’ DESC ‘User Defined Attribute’ SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN ‘user defined’ ) 
99-user.ldif:attributeTypes: ( positionManagerNameFirst-oid NAME ‘positionmanagernamefirst’ DESC ‘User Defined Attribute’ SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN ‘user defined’ ) 
99-user.ldif:attributeTypes: ( positionManagerNameLast-oid NAME ‘positionmanagernamelast’ DESC ‘User Defined Attribute’ SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE X-ORIGIN ‘user defined’ ) 

I’ve put together a quick check to see if the returned value is an array, and if it is then get a string from the decoded byte array.

string strUserGivenName = (String)searchResult.Properties["givenName"][0]; 
string strUserSurame = (String)searchResult.Properties["sn"][0]; 

string strSupervisorFirstName = "";
string strSupervisorLastName = "";
if (searchResult.Properties["positionmanagernamefirst"][0].GetType().IsArray){
    strSupervisorFirstName = System.Text.Encoding.UTF8.GetString((byte[])searchResult.Properties["positionmanagernamefirst"][0]);
}
else{
    strSupervisorFirstName = searchResult.Properties["positionmanagernamefirst"][0].ToString();
}

if (searchResult.Properties["positionmanagernamelast"][0].GetType().IsArray){
    strSupervisorLastName = System.Text.Encoding.UTF8.GetString((byte[])searchResult.Properties["positionmanagernamelast"][0]);
}
else{
    strSupervisorLastName = searchResult.Properties["positionmanagernamelast"][0].ToString();
}

Voila

The outstanding question is if we need to wrap *all* DirectoryString syntax attributes in this check to be safe or if there’s a reason core schema attributes like givenName and sn are being returned as strings whilst our add-on schema attributes have been encoded.

DSEE 6.3 To OUD 11g Transition

There’s no direct path to replicate data from DSEE6.3 to OUD11g. Not unreasonable since DSEE is the Sun product based on the Netscape Directory Server and OUD is the Oracle product based on OpenLDAP – they weren’t exactly designed to allow easy coexistence that would permit customers to switch from one to the other. Problem is, with Oracle’s acquisition of Sun & axing the DSEE product line … customers *need* to interoperate or do a flash cut.
Since our Identity Management (IDM) platform was not able to prep development work and implement their changes along with the directory replacement, a flash cut was right out. I’ve done flash cuts before — essentially ran two completely different directories in parallel with data fed from the Identity Management platform, tested against the new directory using quick modification to the OS hosts file, then reconfiguring the virtual IP on the load balancer to direct the existing VIP to the new service hosts. Quick/easy fail-back is to set the VIP to the old config and sort out whatever is wrong on the new hosts. A lot lower risk than a traditional ‘flash cut’ approach as long as you trust the IDM system to keep data in sync. But lacking an IDM system, flash cut is typically a non-starter anyway.
There is a migration path. Oracle put some development effort into the DSEE product line prior to discontinuing it. DSEE7 was the Sun distributed “next version”. It was not widely deployed prior to the Oracle acquisition. Oracle took over DSEE7 development but called it DSEE11 (to match the OUD version numbering, I guess?). Regardless of the rational, you’ll see the “next version” DSEE product referred to as both DSEE7 and DSEE11.
There’s not a direct replication between Oracle DSEE11 and Oracle OUD11. Oracle created a “replication gateway” that handles, among other things, schema name mapping (only Netscape would use attribute names like nsAccountLockout, and that nomenclature carried through to the Sun product). Oracle did a decent job of testing DSEE11<=>OUD11 Replication Gateway interoperability. I don’t know if they just assumed DSEE6 would work because DSEE11 did or if they assumed the installation base for DSEE6 was negligible (i.e. didn’t bother to test older revisions) but we found massive bugs in the replication gateway working with DSEE6. “You cannot import the data to initialize the OUD11 directory” type of bugs which I was willing to work around by manually editing the export file, but subsequent “updates do not get from point ‘A’ to point ‘B’ bugs too. The answer from Oracle was essentially “upgrade to DSEE11” … which, if i could flash-cut upgrade DSEE6 to DSEE11 (see: IDM platform couldn’t do that), I could just cut it to OUD11 and be done. Any non-trivial change was a non-starter, but Oracle wasn’t going to dump a bunch of development time into fixing replication for a dead product to their shiny new thing.
I worked out a path that used tested and working components — DSEE6 replicated just fine with DSEE11. DSEE11 replicated just fine with the OUD11g replication gateway, and the OUD11g replication gateway replicated fine with OUD11g. Instead of introducing additional expense and time setting up dedicated replication translation servers, I installed multiple components on the new servers. There is a DSEE11 directory on one of the new OUD servers, the replication gateway on another one of the new OUD servers, and (of course) the OUD11g directory that we actually intended to run on the new servers is on those new OUD servers.
This creates additional monitoring overhead – watching replication between three different directories and ensuring all of the services are running – but allows the IDM platform to continue writing changes to the DSEE6.3 directory until they are able to develop and test changes that allow them to use OUD11g directly.