Blog

MIB Smithy SDK 4.0 Released

MIB Smithy SDK 4.0 is released at long last! This is a major release composed of significant architecture changes, new features, and assorted bug fixes. High level changes include IPv6 support, a new license key format that supports licensing based on user name (as an alternative to host-based licensing), many new validation rules and error level adjustments, and better integration with Tcl’s own support for binary data, and platform changes.

As of 4.0, some older platform versions are no longer officially supported, but newer versions are:

This is not to say definitively that the SDK will not work with the older platforms, however, as it’s likely just a matter of having the appropriate dependencies installed (e.g. libstdc++). Rather, these are the platforms the releases are built on, and the older platforms have been (or will soon be) retired, so we won’t be able to guarantee support that far back anymore.

Some changes still need to be made to the web site to support Linux x86_64 and user-based licensing, and I’ll get to work on those right away. In the mean time, if you are interested in either of these, please contact support and we’ll get you squared away.

If you haven’t already, and you’re currently using an older version of the SDK, be sure to read my other post about potential compatibility issues posed by the binary data changes.

Changes in MIB Smithy SDK 4.0:

As always, your feedback is important in determining changes and fixes that will be made in coming releases. If you have questions or comments, please don’t hesitate to contact us. The following changes have been made in this release:

1359: New options for formatting OCTET STRINGs

A new -strformat option was added to [smilib format] to enhance support for formatting OCTET STRINGs (now treated as byte arrays via APIs). The default, configurable for the database, is “1x:” for colon-delimited hex format.

[smilib format] also now allows specifying a type in lieu of record name or OID, e.g.:

% smilib format “OCTET STRING” foo
66:6f:6f

1658: Return indexparts for strings as byte arrays

[smilib get -indexparts] will no longer return a custom hex format for OCTET STRING and BITS index data, but will instead return raw binary (via Tcl_NewByteArrayObj) to make it easier to work with data at the script level. Use [smilib format] to format data for display, which will use return a hex format in lieu of DISPLAY-HINT.

362: Add Linux x86_64 platform support

The Linux platform with x86_64 architecture is now supported.

1836: Validation for COPS-PR-SPPI’s OBJECT-TYPE STATUS

Validation rules were added for COPS-PR-SPPI’s OBJECT-TYPE STATUS field. Previously, values that are not allowed by COPS-PR-SPPI were not reported as errors the way values disallowed by other versions were.

2160: Support multiple Host IDs in license key

Certain users in the past have needed to swap license key files depending on which host they’re using. With the new license key format, we can provide these users with a single unified key if switching to user-based licensing is not appropriate for their needs.

1707: Check for circular indexing dependencies

Validation rules were added to recursively check through INDEX, AUGMENTS, EXTENDS, PIB-INDEX, and UNIQUENESS relationships to detect loops and improper table normalization (such as A EXTENDS B, B EXTENDS A; or A AUGMENTS B, but B’s INDEX lists columns from A).

2161: Add support for username-based licensing

A new license key format has been implemented that allows for licensing based on username, rather than on Host ID. Where host-based licenses allow any user on a specific host, username-based licenses allow a specific named user on any host. Existing users who purchased prior to this feature being available can request their license to be permanently converted to a username-based license at no charge provided their support contract is current.

776: Relaxed rule for column syntax vs. row definition

It is now a warning, rather than an error, for a column’s type in the SEQUENCE definition and the column’s OBJECT-TYPE to be different, provided the SEQUENCE uses the proper base type.

699: Normalized filename properties

The -filename property for SMI databases and file records is now normalized automatically when set to make its value invariant if the current directory is changed in the Tcl shell (or script) after assignment.

1849: Message clarifications

Many warning and error messages from the parser and validator were reworded (some slightly, some more) for clarification.

1850: Auto-correct swapped subrange endpoints and doublets

When validating size/range specs, swapped and duplicate endpoints in subranges are now automatically corrected after reporting the error. e.g., (3..2|5..5) becomes (2..3|5).

304: Add IPv6 Support

Communicating with SNMP agents over IPv6 is now supported. (Note: nmtrapd protocol doesn’t currently have a way to indicate an IPv6 source address for traps that aren’t received directly by the SDK, but there’s a plan in place to address this limitation.)

1840: Clarify SMI version in error/warning messages

Improved module SMI version detection for application of version-specific rules when imports are missing and changed most validation-stage messages to include the proper version string. (For example, COPS-PR-SPPI inherits most of its rules from SMIv2. Most of these messages previously mentioned SMIv2 even for PIB modules, but will now say COPS-PR-SPPI.)

1837: Disallow non-attributes in COPS-PR-SPPI’s OBJECT-GROUP

A validation rule was added to generate an error if an OBJECT-TYPE listed in a COPS-PR-SPPI OBJECT-GROUP is not an attribute OBJECT-TYPE.

1895: Setting SMI database options at creation time

The [smilib new] command, when used to create a new SMI database, can now accept options for setting the database properties the same way [snmplib new] can. Previously this required a separate [smilib set] call to set database properties, except when creating records in the database.

1115: Allow named ports, in addition to numeric

TCP ports for sending or receiving SNMP messages can now be specified by service name, rather than just numeric (e.g. “snmp” for port 161, if such a named entry is present in /etc/services).

2185: INDEX/related lookup enhancements

Added -extendsdecl, -pibindexdecl, -uniquedecl options to smilib get/set commands for OBJECT-TYPEs to function like -indexdecl and -augmentsdecl in returning the value directly defined in the record, while the old -extends and -pibindex options will return the value from the associated row.

The -index, -pibindex, -pibreferences, etc. options now trace AUGMENTS and EXTENDS to find the values inherited from augmented tables, rather than only looking one node up/down to find the row.

Also added -row and -table properties to return the record for the row or table OBJECT-TYPE associated with the specified OBJECT-TYPE (which could itself be a row, table, or column).

1153: XMLSMI: relax parsing of unrecognized elements

The XML parser now only generates a warning and treats as a successful parse (rather than an error and failed parse) when encountering tags that aren’t recognized. This should allow new tags to be added down the road, but still allow the file to load into versions back to 4.0.

2184: Correct multiple INDEX/AUGMENTS/etc. at parse time

The SMI parser previously recovered from having too many or out-of-order INDEX, AUGMENTS, etc. clauses by preserving them all and erroring at validation time, despite only having one legal value, which complicated validation and usage. Instead of preserving them all, the parser now reports the extras as an error only at parse time and discards them, but still supports (and reports) recovery from ordering issues.

1234: Provide socket errors as strings rather than numbers

In a few cases, socket-related errors were provided just as an error number. They should all now give a more meaningful error message. Some of the existing messages have been further clarified.

1871: Add checks for STATUS consistency

Validation rules were added for checking consistency between related records and dependencies. An error or warning may be issued depending on the type of relation and difference (for example, it’s an error for a row and table to be different, but a warning for a “current” OBJECT-TYPE to use a “deprecated” TEXTUAL-CONVENTION).

1830: VARIATIONs with DEFVAL and ACCESS not-implemented, accessible-for-notify

A validation rule was added to issue a warning for a VARIATION having both a DEFVAL and ACCESS ‘not-implemented’ or ‘accessible-for-notify’ (the DEFVAL is superfluous).

1702: Check for scalar objects with MAX-ACCESS not-accessible

A validation rule was added to produce an error for a scalar SMIv2 OBJECT-TYPE having MAX-ACCESS ‘not-accessible’. If the object’s value is only available via notification, it should have MAX-ACCESS ‘accessible-for-notify’.

1835: OBJECT-TYPEs not in OBJECT-GROUPs in SMIv2, COPS-PR-SPPI

It is now an error, rather than a warning, for OBJECT-TYPEs and NOTIFICATION-TYPEs to not be a member of any OBJECT-GROUP or NOTIFICATION-GROUP when they otherwise must be.

1834: Auto-correct missing ‘Z’ suffix in ExtUTCTime values

The validator now automatically corrects a missing ‘Z’ suffix in LAST-UPDATED and REVISION timestamps after reporting the issue.

1831: Use of type names (rather than objects) as INDEXes

It is now an error, rather than a warning, for SMIv2 and COPS-PR-SPPI OBJECT-TYPE to use INDEX values pointing to type names rather than object names. For RFC-1212, it’s now a forward-compatibility warning.

1742: Additional validation for read-create

A validation rule was added to disallow any column from having access “read-write” if any column in the same table has access “read-create”, per RFC 2578 section 7.3.

1833: Restrictions on MANDATORY-GROUPS and GROUPs

It’s now an error for MODULE-COMPLIANCE’s MANDATORY-GROUPS or GROUP clauses to reference anything other than an OBJECT-GROUP or NOTIFICATION-GROUP. Previously they allowed any OID value, like AGENT-CAPABILITIES’ INCLUDES allows specifying SMIv1 branches, but the text of RFC 2576 that allows this (in section 2.3) applies only to AGENT-CAPABILITIES.

1841: Disallow empty binary/hex strings as enum values

Hex/binary strings as enumeration values are not explicitly legal (or illegal) in SMI, but the SDK supports them (with warning) just in case. However, it makes no sense for something that should be an integer to have no bits, so a validation rule was added to produce a warning specifically for empty hex/binary enumerations and integral DEFVALs.

1842: Do not error for ASN.1 types in plain ASN.1 modules

Towards support for plain ASN.1 modules, error levels for validation rules regarding the use of ASN.1 types not supported by SMI or COPS-PR-SPPI were changed. If the module is clearly intended to be a MIB or PIB module, an error is produced. If nothing is imported from SMI or COPS-PR-SPPI base modules, it’s assumed to be plain ASN.1 and no error or warning about the type is produced.

1838: Improved parser recovery for enumerated types

Enumerated types are now parseable with empty braces or no braces/enumerations, including BITS (where the legal syntax requires at least one bit to be defined and braced). This is mainly to allow incomplete MIB definitions to be saved and still reloaded in the MIB Smithy editor.

1660: Misleading error when SDK falls back to direct trap port binding

The failure messages from attempts to bind to port 162 before connecting to nmtrapd via Tcl_OpenCommandChannel were left in the interpreter result despite connection succeeding a TCL_OK result being returned. The message could mislead users (at least in the interactive shell) into thinking the [snmplib bind] command failed when it didn’t.

375: Return OCTET STRINGs as binary

SNMP session APIs no longer automatically encode/decode binary OCTET STRING data to/from hex format, but instead return and accept Tcl’s native binary format and conventions (e.g., byte array objects, [binary] command, and “\x” to specify hex-encoded data in scripts).

This change is intended to simplify scripted use of and prevent APIs from interpreting values as hex encoded when they are intended to be taken as literal.
It affects variable bindings, session properties (such as community strings and passwords), and callback parameters alike.

Use the [smilib format] command to format values for display to a user. You can specify “OCTET STRING” to format values that aren’t associated with a varbind, such as the SNMPv3 Engine ID passed to request callbacks.

1718: ExtUTCTime validation wrong after 2038

ExtUTCTime validation (e.g. LAST-UPDATED and REVISION timestamps) is now Y2K38 compliant, no longer based on Unix epoch.

2209: Wrong keywords for usm privKeyChange -privproto option

The [snmplib usm privKeyChange] command’s -privproto option was expecting auth protocol keywords rather than priv protocol keywords. Consequently, one could not override the privacy protocol for the session without error.

1832: Warning for duplicate INDEXes

It is now a warning, rather than an error, for an OBJECT-TYPE’s INDEX to list the same object multiple times. SMIv2 discourages it, but it is not forbidden.

1726: False “recursive loop” error with conformance groups

A GROUP clause in MODULE-COMPLIANCE pointing to a non-group with missing dependencies could previously generate a “OBJECT IDENTIFER value assigned…is defined as a recursive loop” error due to the way OIDs were indexed in the old architecture. In the new architecture, along with forbidding non-groups, this is no longer an issue.

1705: Using EXPORTS in SMIv2 modules should be an error

It is now an error for an SMIv2 or COPS-PR-SPPI module to use the ASN.1 module’s EXPORTS clause. Previously it was a warning because the MIB Smithy editor automatically excluded it upon saving, but since it’s forbidden it should be an error for standalone validation.

1428: [smilib new] doesn’t allow some set options

The set of record properties supported by [smilib new] and [smilib set] had gotten out of sync, so some properties of some record types could not be set at creation time. They should be back in sync now, allowing all properties to be set at record creation time.

Addendum

The following additional cases were also fixed in 4.0 but didn’t make it into the release notes (they’d been dual-committed for 4.0 and a possible 3.4.9, but there was no 3.4.9 release):

1815: Additional validation for first two subidentifiers

ASN.1 places limits on the range of the first two subidentifiers of an OID. Validation previously did not check both values in some cases, depending on the form used to define the OID. Out-of-range values should now be reported as an error regardless of form.

1782: Disallow hyphens in COPS-PR-SPPI enum/bit labels

A warning will now be produced for enumeration and bit labels containing hyphens in COPS-PR-SPPI, as with 3.4.8, which added such warnings to record names.

1779: DEFVAL identifier list not fully validated versus named bits

When using the identifier list form for DEFVAL (used for named bits), the identifiers were not properly checked against the defined bit names to see if they exist.

1802: Validation for single-valued size/range refinements

Validation for size and range refinements was failing to detect and report a value as being out of the base size or range if it was only a single value (rather than a sub-range) in the refinement.

1786: Hex/Binary enumeration order checking incorrect

When using hex or binary values for enumeration and bit values, a false error was reported regarding the order due to treating the value as a hex or binary representation of a string rather than an integer.

1809: Warning to start enums at 1 should not trigger when 1 is refined away

A warning to start enumerations at 1 will no longer be generated for a record that refines the value away (such as a TEXTUAL-CONVENTION that starts at 1, but the OBJECT-TYPE using it refining it away).

1808: Overlapping sizes/ranges should be an error

There was a bug in the rule for detecting overlapping size and range lists that prevented the overlap from being detected in some cases. These will now be reported as an error as they should be.

1810: Duplicate enum values in hex/binary not detected

Duplicate enumeration values were not previously reported as an error if one or both of the values were in hex or binary string format (which is not explicitly legal or illegal in SMI).