Category: Technology

Java Heap Stats with JStat

While there are plenty of third-party utilities for looking at the java heap space, I just use jstat (in OpenJDK, this means installing java-<Version>-openjdk-devel

JStat will display the following columns:

--------------------------------------------------------------------------------
S0C: Survivor space 0 size in K
S1C: Survivor space 1 size in K

S0U: Survivor space 0 usage in K
S1U: Survivor space 1 usage in K

--------------------------------------------------------------------------------

EC: Eden space size in K
EU: Eden space usage in K

--------------------------------------------------------------------------------

OC: Old space size in K
OU: Old space usage in K

--------------------------------------------------------------------------------

MC: Meta space size in K
MU: Meta space usage in K

--------------------------------------------------------------------------------

CCSC: CodeCache size in K
CCSU: CodeCache usage in K

--------------------------------------------------------------------------------

YGC: Young generation garbage collection count
YGCT: Young generation garbage collection total time in seconds

FGC: Full garbage collection count
FGCT: Full garbage collection total time in seconds

CGC: Concurrent garbage collection count
CGCT: Concurrent garbage collection time in seconds
GCT: Total garbage collection time in seconds

--------------------------------------------------------------------------------

https://stackoverflow.com/questions/13660871/jvm-garbage-collection-in-young-generation/13661014#13661014 does a good job of explaining the nomenclature & how stuff gets moved around in the heap space

Sample output — this command is for java PID 19356 and will list 100 lines 2 seconds apart (2000 ms)

server01:bin # jstat -gc 19356 2000 100
S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT
68096.0 68096.0 0.0 64207.5 545344.0 319007.2 30775744.0 19221750.2 137452.0 124322.4 18860.0 15380.6 324697 14589.985 228 45.830 14635.815
68096.0 68096.0 0.0 64207.5 545344.0 386674.5 30775744.0 19221750.2 137452.0 124322.4 18860.0 15380.6 324697 14589.985 228 45.830 14635.815
68096.0 68096.0 0.0 64207.5 545344.0 457055.4 30775744.0 19221750.2 137452.0 124322.4 18860.0 15380.6 324697 14589.985 228 45.830 14635.815
68096.0 68096.0 0.0 64207.5 545344.0 485538.8 30775744.0 19221750.2 137452.0 124322.4 18860.0 15380.6 324697 14589.985 228 45.830 14635.815
68096.0 68096.0 0.0 64207.5 545344.0 505893.4 30775744.0 19221750.2 137452.0 124322.4 18860.0 15380.6 324697 14589.985 228 45.830 14635.815

And this is a time where a third-party tool would be helpful but I never really ‘get’ what is and what is not OK to install on servers, so try not to install things — because the *useful* bit of information for any of this is really the usage / size percent utilization value.

That last grouping of stuff — I look at those v/s how long the pid has been running. If you’ve gotten a billion GC’s and the PID has only been running for eight seconds, that is a crazy amount of I/O. If I’ve only had 3 GCs and the pid has been running for seven years, it hasn’t been doing anything. In between? I don’t really find the numbers useful unless I’ve got a baseline from normal operation.

NEO4J: Union Sets

There are a lot of places where cypher queries don’t seem to have an “OR” type of functionality … but you can union two returned sets provided they have the same properties list
MATCH z=(:TOMATO {name: ‘Black Krim’})
RETURN z
UNION
MATCH z=(:TOMATO {name: ‘Cherokee Purple’})
RETURN z;

NEO4J: Debugging a Cypher Query

It is somewhat ironic that I continue to use print statements as my debugging tool of choice when programming but spent a decent bit of time trying to find a cypher query debugger. Just use a print statement — or, in this case, return.

When my query returned an error indicating that the variable isn’t defined even though I copy/pasted the variable name from whence I defined it:

I could just omit the component of the query with the error and try returning this variable

And performing operations on null values may not get me anywhere. Adding a replacement command to drop the commas produces integer values:

NEO4J: Installing Plugins

Since I have the /plugins directory persisted, I can use wget <URL> to grab a JAR file and place it into the plugins folder. Restart the container and the plugin is active.

Example installing APOC extended plugin:

cd /docker/neo4j/plugins;wget https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/5.11.0/apoc-5.11.0-extended.jar;docker stop neo4j;docker start neo4j

NEO4J: APOC Plugin in Docker Container

I wanted to test some of the Awesome Procedures on Cypher (APOC) functions, but first I needed to install the plugins. I needed to modify my docker run statement slightly to persist the /plugins directory and install APOC core:

 

docker run -dit -p 7474:7474 -p 7687:7687 --volume=/docker/neo4j/data:/data --volume=/docker/neo4j/conf:/var/lib/neo4j/conf --volume=/docker/neo4j/plugins:/plugins -e NEO4J_AUTH=none -e NEO4J_apoc_export_file_enabled=true -e NEO4J_apoc_import_file_enabled=true -e NEO4J_apoc_import_file_use__neo4j__config=true -e NEO4J_PLUGINS=\[\"apoc\"\] --name neo4j neo4j

Maintaining an /etc/hosts record

I encountered an oddity at work — there’s a server on an internally located public IP space. Because it’s public space, it is not allowed to communicate with the internal interface of some of our security group’s servers. It has to use their public interface (not technically, just a policy on which they will not budge). I cannot just use a DNS server that resolves the public copy of our zone because then we’d lose access to everything else, so we are stuck making an /etc/hosts entry. Except this thing changes IPs fairly regularly (hey, we’re moving from AWS to Azure; hey, let’s try CloudFlare; nope, that is expensive so change it back) and the service it provides is application authentication so not something you want randomly falling over every couple of months.

So I’ve come up with a quick script to maintain the /etc/hosts record for the endpoint. 

# requires: dnspython, subprocess

import dns.resolver
import subprocess

strHostToCheck = 'hostname.example.com' # PingID endpoint for authentication
strDNSServer = "8.8.8.8"         # Google's public DNS server
listStrIPs = []

# Get current assignement from hosts file
listCurrentAssignment = [ line for line in open('/etc/hosts') if strHostToCheck in line]

if len(listCurrentAssignment) >= 1:
        strCurrentAssignment = listCurrentAssignment[0].split("\t")[0]

        # Get actual assignment from DNS
        objResolver = dns.resolver.Resolver()
        objResolver.nameservers = [strDNSServer]
        objHostResolution = objResolver.query(strHostToCheck)

        for objARecord in objHostResolution:
                listStrIPs.append(objARecord.to_text())

        if len(listStrIPs) >= 1:
                # Fix /etc/hosts if the assignment there doesn't match DNS
                if strCurrentAssignment in listStrIPs:
                        print(f"Nothing to do -- hosts file record {strCurrentAssignment} is in {listStrIPs}")
                else:
                        print(f"I do not find {strCurrentAssignment} here, so now fix it!")
                        subprocess.call([f"sed -i -e 's/{strCurrentAssignment}\t{strHostToCheck}/{listStrIPs[0]}\t{strHostToCheck}/g' /etc/hosts"], shell=True)
        else:
                print("No resolution from DNS ... that's not great")
else:
        print("No assignment found in /etc/hosts ... that's not great either")

NEO4J: Adding Indices

Like SQL-based databases, you can create indices in Neo4j to optimize frequently performed searches. In my particular case, I am usually searching by the plant name. Adding an index on the plant’s name, therefore, makes sense:

CREATE INDEX index_tomato_name IF NOT EXISTS
FOR (n:TOMATO)
ON (n.name)

I am also interested in mapping the genetic lineage, so I’ve added indices for the male and female parent plants:

CREATE INDEX index_tomato_parentm IF NOT EXISTS
for ()-[r:PARENT_M]-()
on (r.name)

CREATE INDEX index_tomato_parentf IF NOT EXISTS
for ()-[r:PARENT_F]-()
on (r.name)

 

Hypothetically, you can use the procedure CALL db.indexes(); to view all of the indices in a database, but SHOW PROCEDURES; shows me that procedure isn’t registered in the community edition.

╒═════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════╤═══════╤═════════════╕
│name                                                     │description                                                           │mode   │worksOnSystem│
╞═════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════╪═══════╪═════════════╡
│"db.awaitIndex"                                          │"Wait for an index to come online (for example: CALL db.awaitIndex(\"M│"READ" │true         │
│                                                         │yIndex\", 300))."                                                     │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.awaitIndexes"                                        │"Wait for all indexes to come online (for example: CALL db.awaitIndexe│"READ" │true         │
│                                                         │s(300))."                                                             │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.clearQueryCaches"                                    │"Clears all query caches."                                            │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.createLabel"                                         │"Create a label"                                                      │"WRITE"│false        │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.createProperty"                                      │"Create a Property"                                                   │"WRITE"│false        │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.createRelationshipType"                              │"Create a RelationshipType"                                           │"WRITE"│false        │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.index.fulltext.awaitEventuallyConsistentIndexRefresh"│"Wait for the updates from recently committed transactions to be appli│"READ" │true         │
│                                                         │ed to any eventually-consistent full-text indexes."                   │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.index.fulltext.listAvailableAnalyzers"               │"List the available analyzers that the full-text indexes can be config│"READ" │true         │
│                                                         │ured with."                                                           │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.index.fulltext.queryNodes"                           │"Query the given full-text index. Returns the matching nodes, and thei│"READ" │true         │
│                                                         │r Lucene query score, ordered by score. Valid keys for the options map│       │             │
│                                                         │ are: 'skip' to skip the top N results; 'limit' to limit the number of│       │             │
│                                                         │ results returned; 'analyzer' to use the specified analyzer as search │       │             │
│                                                         │analyzer for this query."                                             │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.index.fulltext.queryRelationships"                   │"Query the given full-text index. Returns the matching relationships, │"READ" │true         │
│                                                         │and their Lucene query score, ordered by score. Valid keys for the opt│       │             │
│                                                         │ions map are: 'skip' to skip the top N results; 'limit' to limit the n│       │             │
│                                                         │umber of results returned; 'analyzer' to use the specified analyzer as│       │             │
│                                                         │ search analyzer for this query."                                     │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.info"                                                │"Provides information regarding the database."                        │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.labels"                                              │"List all available labels in the database."                          │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.ping"                                                │"This procedure can be used by client side tooling to test whether the│"READ" │true         │
│                                                         │y are correctly connected to a database. The procedure is available in│       │             │
│                                                         │ all databases and always returns true. A faulty connection can be det│       │             │
│                                                         │ected by not being able to call this procedure."                      │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.prepareForReplanning"                                │"Triggers an index resample and waits for it to complete, and after th│"READ" │true         │
│                                                         │at clears query caches. After this procedure has finished queries will│       │             │
│                                                         │ be planned using the latest database statistics."                    │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.propertyKeys"                                        │"List all property keys in the database."                             │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.relationshipTypes"                                   │"List all available relationship types in the database."              │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.resampleIndex"                                       │"Schedule resampling of an index (for example: CALL db.resampleIndex(\│"READ" │true         │
│                                                         │"MyIndex\"))."                                                        │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.resampleOutdatedIndexes"                             │"Schedule resampling of all outdated indexes."                        │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.schema.nodeTypeProperties"                           │"Show the derived property schema of the nodes in tabular form."      │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.schema.relTypeProperties"                            │"Show the derived property schema of the relationships in tabular form│"READ" │true         │
│                                                         │."                                                                    │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.schema.visualization"                                │"Visualizes the schema of the data based on available statistics. A ne│"READ" │true         │
│                                                         │w node is returned for each label. The properties represented on the n│       │             │
│                                                         │ode include: `name` (label name), `indexes` (list of indexes), and `co│       │             │
│                                                         │nstraints` (list of constraints). A relationship of a given type is re│       │             │
│                                                         │turned for all possible combinations of start and end nodes. The prope│       │             │
│                                                         │rties represented on the relationship include: `name` (type name). Not│       │             │
│                                                         │e that this may include additional relationships that do not exist in │       │             │
│                                                         │the data due to the information available in the count store. "       │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.clear"                                         │"Clear collected data of a given data section. Valid sections are 'QUE│"READ" │true         │
│                                                         │RIES'"                                                                │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.collect"                                       │"Start data collection of a given data section. Valid sections are 'QU│"READ" │true         │
│                                                         │ERIES'"                                                               │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.retrieve"                                      │"Retrieve statistical data about the current database. Valid sections │"READ" │true         │
│                                                         │are 'GRAPH COUNTS', 'TOKENS', 'QUERIES', 'META'"                      │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.retrieveAllAnonymized"                         │"Retrieve all available statistical data about the current database, i│"READ" │true         │
│                                                         │n an anonymized form."                                                │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.status"                                        │"Retrieve the status of all available collector daemons, for this data│"READ" │true         │
│                                                         │base."                                                                │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"db.stats.stop"                                          │"Stop data collection of a given data section. Valid sections are 'QUE│"READ" │true         │
│                                                         │RIES'"                                                                │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.cluster.routing.getRoutingTable"                   │"Returns the advertised bolt capable endpoints for a given database, d│"DBMS" │true         │
│                                                         │ivided by each endpoint's capabilities. For example an endpoint may se│       │             │
│                                                         │rve read queries, write queries and/or future getRoutingTable requests│       │             │
│                                                         │."                                                                    │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.components"                                        │"List DBMS components and their versions."                            │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.info"                                              │"Provides information regarding the DBMS."                            │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.killConnection"                                    │"Kill network connection with the given connection id."               │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.killConnections"                                   │"Kill all network connections with the given connection ids."         │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.listCapabilities"                                  │"List capabilities"                                                   │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.listConfig"                                        │"List the currently active config of Neo4j."                          │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.listConnections"                                   │"List all accepted network connections at this instance that are visib│"DBMS" │true         │
│                                                         │le to the user."                                                      │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.queryJmx"                                          │"Query JMX management data by domain and name. For instance, \"*:*\"" │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.routing.getRoutingTable"                           │"Returns the advertised bolt capable endpoints for a given database, d│"DBMS" │true         │
│                                                         │ivided by each endpoint's capabilities. For example an endpoint may se│       │             │
│                                                         │rve read queries, write queries and/or future getRoutingTable requests│       │             │
│                                                         │."                                                                    │       │             │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.upgrade"                                           │"Upgrade the system database schema if it is not the current schema." │"WRITE"│true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"dbms.upgradeStatus"                                     │"Report the current status of the system database sub-graph schema."  │"READ" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"tx.getMetaData"                                         │"Provides attached transaction metadata."                             │"DBMS" │true         │
├─────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼───────┼─────────────┤
│"tx.setMetaData"                                         │"Attaches a map of data to the transaction. The data will be printed w│"DBMS" │false        │
│                                                         │hen listing queries, and inserted into the query log."                │       │             │
└─────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┴

NEO4J: Searching

Returning data from NEO4J is matching — we can match a node using MATCH p=(a:TOMATO {name: 'Black Krim'}) return p;

where the object “p” becomes a set of nodes labeled with ‘TOMATO’ where the value of ‘name’ is ‘Black Krim’

More advanced matches set the return object to a set of nodes and relationships — here we set p to the set of items starting at the node labeled TOMATO with name ‘PLANT0’ with relationships and nodes until you get to a node labeled TOMATO with name ‘Tomato0000007’

MATCH p=(a:TOMATO {name: 'PLANT0'})-[*]->(b:TOMATO {name: 'Tomato0000007'}) RETURN p

NEO4J: Relational Databases

I always found it odd that relational databases didn’t really have relationships as part of the stored data. Technically, they could if you had a view or stored procedure with a bunch of JOIN’s in there. But relational databases stored data about which you could build relationships. And people using the data may not even know about relationships that other people saw within that data. Some of our databases at work have amazing documentation — a hundred meg of PDF files detailing the relationships that the database creator wanted people to use. Other databases? No such luck!

To me, that’s the big advantage of a Graph database — the relationships are stored within the data, so anyone viewing it for the first time doesn’t need to poke around, see where values could be correlated across tables, and create their own JOIN statements to build out the relationship. Recording data in the database means defining those relationships. It would still be good to document a graph model (we have ‘people’ nodes who ‘act in’, ‘cast’, ‘produce’, or ‘direct’ ‘movie’ nodes), but you could figure all of those things out by perusing the database content.