Data Quality as Meta?

dataflow No Comments »

When presenting data, try to include some sense of quality or accuracy, even if it’s just a flag “I derived this” or “I got this from a very accurate source” or “this is a space-filler”.

I wanted to highlight something I saw quite interesting in Axeda Corporation‘s Gateway and Connector technologies: Quality of metrics. Axeda uses an enumeration of simple qualities (Good, Bad, or Unknown), and this could theoretically be used when choosing which of two conflicting data types to show.

The simple act of collecting and summarizing metrics is not necessarily made easier when the precision meta is tracked, but it can help the end-user make better decisions based on this data: if you see an aberrant data point, do you know it’s seriously out-of-norm and needs to be acted upon, or is it based perhaps on a ratio with a questionable denominator, and should be taken with a bit of skepticism?

Consider precision, or at least define why it’s out-of-scope for your work.

Bundling in Java

Uncategorized No Comments »

In providing a simple java-stack copy of an internal tool, I found that a few resources helped me define a basic layout, which I’ve hand-generated in the makefile since then.

Bart’s work on bundling an OSX App from a Java Jar helped to get a start, but more helped to find the other bits. helped to remind to use Spotlight to find the tools that move around occasionally; unfortunately, that’s a ANT-focused article, and I’m not really interested in adding build-level dependencies yet. Eventually, I’m probably going to have to look at where a resource is inside the jar, but that’s really unrelated to Apple’s bundler.

In general, I used Bart’s work in Nakaya’s spotlight-found Bundler to make the original Info.Plist, built a diff-target to show that I was generating an exact copy, then expanded the built/generated copy to include more jar files as needed. If it gets more complex, or I need to combine JARs, I’ll probably look at ANT at the time, but for now, just text-printing XML, not a big requirement.

Yeah, this is another not to remid myself later 🙂

Use UDCs to Group ISL Trunks or PortChannels

How to, samplecode, UDC No Comments »

I recently worked with a VirtualWisdom Administrator who wanted to group his ISL port utilization to match his ISL trunks, so we worked out a method of doing this, and I wanted to share it.  As a Field Application Engineer at Virtual Instruments, I tend to focus on these lower-level “how to” issues, working with users to achieve the data representation they need to make informed decisions in lieu of guesses and rule-of-thumb.

Initially, this administrator and I spoke of “trunks”, but between Brocade and Cisco terminology, these mean different things. The aggregations of ISLs into single logical units are “Trunking” in Brocade, but “Port Channels” in Cisco. Trunking E ports in Cisco is a different thing. I’ll use “Aggregate” as much as possible to refer to this in terminology as vendor-neutral as VirtualWisdom is.

We discussed why this Admin wanted to see more than just “the top 20″ values on a list of ISLs.  Diving deeper, this was because the top 24 entries are all the same aggregate: essentially, the first entire page is taken up by channels 1 and 2 of a a single aggregate ISL.  He wanted to skip beyond this to see the next 20 or 40 ISLs so he could see which ISLs were getting near 90% utilization.  So… why not combine these into a single filter or expression that matches the aggregate, and make sure that each aggregate uses only one row of this resulting table?

Additionally, one switch vender implements this aggregation of ISLs as balanced utilization across a collection of links between the same endpoints; conversely, another vendor implements this by overflowing one container, then moving to the next. In essence, an abstract aggregation that is has 60% Utilization may look like a collection of ports or links each with utilization 60%, or might look like 6 out of 10 links with Utilization 100%, and 4 of 10 with 0%.  That’s very difficult to separate in the data, and can obfuscate which ISL aggregations are approaching maximum desired load.

VirtualWisdom’s focus is on ports: what type, attached to what, etc.  Using Aliases or Nicknames, we can describe the endpoint, and in VirtualWisdom 3.0.0 and later, those ISL Nicknames are determined for us.  Unfortunately, these all have switch/blade/port, they’re too detailed. We cannot use that combination for a “group-by”expression to separate out the ISL aggregate.

VirtualWisdom is “too detailed” in this case: it wants to show all the ports individually.

A User-Defined Context, or UDC, is a metric with constant values applied using filter expressions. We often use these to automatically apply a logical grouping that better represents the real world implementation. One ISL aggregate between two switches A1 and A2 tends to encompass all E or TE ports on A1 connected to A2, and conversely, all A2 E or TE ports attached to switch A1.  That tends to make this one ISL unique from others.  We create a UDC in the SNMP/Link scope with values based on the “name of the switch” in an ISL: for example, in “SW12A44:3:1 ISL” as a link name, “SW12A44″ is the switch name.  ISLs between two switches share the same switch names, but are distinct by this same manner from ISLs to other switches. All we need is a UDC with values such as “SW12A44″ where “Attached Port Name MATCHES ^SW12A44:*”, and “SW12B44″ where “Attached Port Name MATCHES ^SW12B44:*”.

An example UDC would look like (Using terminology that’s a bit Brocade-leaning for this UDC because the Administrator favoured Brocade terminology) :

UDC to Group ISL Trunks

As you can see, grouping these ISL connects by “Probe Name”, “Channel”, and “TrunkChannel”, and filtering by “Attached ISL” would summarize traffic on all ISLs by the switches each connects, but aggregating bandwidth of all trunk members between each switch. Grouping by Channel continues to help us keep the directions separate so that a trunk loaded with 95% in one direction and 5% in the other shows “95%” and “5%” rather than “50%”.

You’ll notice, too, that we’ve added short-circuit to mark any non-ISL as a “NoTrunk”, the same as the default value. This avoids running the heavier “MATCHES” expression to evaluate ports that aren’t even ISLs. Your Portal server will thank you.

This logic assumes that all ISLs between two switches are in the same Aggregate; if you have any two switches with more than one distinct aggregation of ISLs, our logic no longer applies. One of our Analysts has seen ISLs grouped into multiple distinct aggregates even though they’re between the same switches, but it wasn’t the case in the discussion sponsoring the work I wanted to share.

Some customers have smaller SANs with a few dozen switches; others exceed 280 switches. This number of switches, and the various ISL possibilities between these, makes writing and maintaining a UDC with over 200 values very difficult and labour-intensive.  Because the user is effectively transferring config information from one format to another, accuracy risks can enter where users are transposing digits, or delayed in echoing the updated config information, or (more often) is simply not informed that any change needs to be echoed or copied. These risks are significant detriments to using this method.

To de-risk this implementation and help you try it out, we’ve created a script to convert a basic list of ISLs with nicknames into a UDC: while the blogging engine doesn’t let me upload this file, your VI Support and Services teams can help you get “ISL2TrunkChannelUDC.awk”, and a version of “awk” to run it. If your report with a Data View of “Table” is saved as a CSV with the AttachedPortName in the 4th column, you would run this script as:

awk -v COL=4 -f ISL2TrunkChannelUDC.awk 'Table-(summary).csv' > ISLAggregation.udc

(resulting UDC tested on version 3.0.2 and version 3.1.0 pre-release, incompatible before version 3.0.0)

I hope this helps you keep watch on your ISL utilization, and show the correct justification for adding ISLs to an aggregation, or balancing traffic to another less-used edge switch.

Keep those nicknames updated, and have a great holiday.

How to Grab Brocade and Cisco WWN Aliases by CommandLine

howto No Comments »

In my work, I find that customers need to continually grab some updated labels and data, and re-import. This is tedious.

Worse, it’s in the Windows world, so by comparison, scripting is in a toddler world (small, doesn’t understand, and has tantrums)

I end up using something like the following, understanding that pre-sharing a public SSH key is safer.

@echo off

plink.exe -l ciscouser1 -pw Secr3tP@ssw0rd "show device-alias database" > cisco1.csv
plink.exe -l ciscouser2 -pw Secr3tP@ssw0rd "show fcalias" > cisco2.csv
plink.exe -l brocadeuser1 -pw Secr3tP@ssw0rd "zonecfg" > brocade1.csv
plink.exe -l brocadeuser2 -pw Secr3tP@ssw0rd "alishow" > brocade2.csv

gawk.exe -f brocade-alishow2wwncsv.awk cisco1.csv cisco2.csv brocade1.csv brocade2.csv > nicknames-by-WWN.csv
gawk.exe -f unique-nicknames.awk nicknames-by-WWN.csv > E:VirtualWisdomDataDeviceNicknamenicknames.csv

We’ve edited “brocade-alishow2wwncsv.awk” to accommodate broader formats, but I haven’t been able to check it on a wide range of platforms.

NTP Diagnostic

Uncategorized No Comments »

I’ve been working quite a bit recently with NTP, so I wanted to record some notes.

I’ve found that when the NTP server is fewer than 4x reached, then it will not choose a source. When it chooses a source, a local source might cause it to flag itself as “unreliable” (leap flag with a bad result). That or a poor stratum (less than client’s fudged is causing a client to ignore a source.

In our case, we saw an SNTP reject on status == 3 (see sntp/main.c::read_packet : “if (failed || data->status == 3”); this “status” is actually the Leap Indicator being 0%11 ( == 3), which was re-instated in RFC-2030 (superseding RFC-958) as an alarm condition (when the last second plus the next second are leap-seconds).

The client is flagging 0x2001 and 0x6001 quite frequently; this is clearly PLL/FLL changeovers, implying that it often sees a “weak” source, perhaps one that jitters too often, and swaps mode.

I’ll add to this over time if I see additional factors. As always — ALWAYS — “ntpq -c opeers” (in whatever form is permitted by your OS) is the best first-step, although “readvar” is also a good first step.

WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in