[netmrg-logo-medium.png] The NetMRG Reference Brady Alleman Douglas E. Warner Copyright © 2003-2004 Brady Alleman and Douglas E. Warner This project is licensed under the terms of the MIT License. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. All product names are property of their respective owners. Such names are used for identification purposes only and are not indicative of endorsement by or of any company, organization, product, or platform. _________________________________________________________ Table of Contents I. Overview 1. Introduction 2. Revision History II. Concepts 3. System Structure 4. Monitoring and Reporting Concepts 5. Graphing Concepts III. Installation 6. Software Requirements 7. Satisfying Requirements 8. Installation Procedure 9. Upgrade Procedure 10. Basic Tutorial 11. Graph Tutorial 12. Interface and Disk Cache Tutorial IV. Feature Guide 13. External Authentication 14. Workday Highlighting V. Reference I. Manual Pages netmrg-gatherer -- Gathers data from devices. II. Web Interface Descriptions Monitoring -- Items associated with setting up gathering of information Groups -- A container for a collection of devices Devices -- A container for a collection of sub-devices Sub Devices -- A container for a collection of monitors Monitors -- A container for a collection of tracked values Events -- A container for a collection of actions to take based on monitor values Conditions -- A collection of paramenters to trigger an Event Reporting -- Items whose purpose is to display and report information to the user. Graphing -- Where graphs (compositions of one or more monitors) are created. Tests -- Tests represent actions taken to gather data. Admin -- Administrative functions not related to the reporting or gathering of information. Help -- How to get assistance with NetMRG. 15. Contact Information I. Overview Table of Contents 1. Introduction 2. Revision History _________________________________________________________ Chapter 1. Introduction Welcome to NetMRG! NetMRG is a database-driven network monitoring and graphing tool for use on Linux systems. It offers an intuitive web-based configuration, customized monitoring capabilities for unusual situations, a fast multi-threaded polling application, and brilliant graphs rendered by RRDTOOL. This document aims to guide you swiftly through the installation and configuration of NetMRG, as well as act as a reference for those with existing installations. _________________________________________________________ Chapter 2. Revision History * 0.17 2004.09.13 Minor Feature Enhancements; Added colorized gatherer output, checkbox support on multiple pages, the ability to disable users, fixed a potential SQL-injection bug, updated navigation on group pages, updated tutorials. * 0.16 2004.06.10 Minor Bugfixes and Feature Enhancements; fixed a user management bug and several bugs related to 'single view only' users not being able to see graphs, added the ability to ignore SNMP uptime checks for devices (permitting the monitoring of broken devices), added the ability to do an SNMP walk of an OID (useful for the Windows SNMP agent), enhanced group/device management, cleaned up autoconf (now supports FC2), and added experimental syslog and daemon support in the gatherer. * 0.15 2004.05.20 Minor Bugfixes and Feature Enhancements; fixes for various db errors (including the infamous one on snmp_cache_view.php), better cross-platform compilation, select all/none/invert checkbox support, support for choosing what graph to apply to an interface, better support for php < 4.2.0, and some security fixes for single-view-only users. * 0.14 2004.03.10 Minor Bugfixes and Feature Enhancements; escaping issues in template dereferencing, broken links on various pages, security issues regarding manual SNMP recaching, broken duplication on non-cutting-edge MySQL, more parameter substitution in SQL tests, better date parsing for advanced graph view, nav-bar and group navigation enhancements, advanced view capability for monitor-only graphs. * 0.13 2003.12.19 Minor Bugfixes and Feature Enhancements; slide show ordering, previous slide, resume slide show, better formatting of 'sums', errors on view editing, fixes to permissions checking, and an occassional segfault in the gatherer was fixed. * 0.12 2003.12.15 Major Feature Enhancents and Bugfixes; added SNMPv2 support, fixed a bug that would cause a device to be deleted from all groups, fixed session timeout when using external auth, added checkbox support to interface/disk cache pages, slide show enhancements, fixed permission issue where unauthorized users could view/edit test_* pages * 0.10 2003.10.05 Major bug fix; fixes a db error when trying to view monitors, a bug that would always set the width and height to zero when creating graphs, and makes conditions with 'rate of change' work. Many documentation upgrades. * 0.10pre2 2003.09.15 This release focuses on adding new features and sqaushing bugs. In addition, there is now an 'updater.php' that is required to be run between versions. A minor change needs to be made to everyone's 'netmrg.xml' to add an 'externalauth' field inside the 'website' section. * 0.10pre1 2003.07.28 Initial publication II. Concepts Table of Contents 3. System Structure 4. Monitoring and Reporting Concepts 5. Graphing Concepts _________________________________________________________ Chapter 3. System Structure The components of NetMRG. * Gatherer The Gatherer is a multi-threaded C++ program that interacts with the Database and RRDTOOL to gather and store data. The program is run by cron, or another scheduling system of your choice, on five minute intervals. * Web Interface The Web Interface consists of several PHP scripts, static HTML files and images, as well as CSS for formatting. It interacts with the Database for the storage of configuration data and with RRDTOOL for the rendering of graphs for presentation. * Database The Database is a relational database running on MySQL. It is utilized by the Web Interface and the Gatherer. * RRDTOOL RRDTOOL is a graphing system based upon round-robin databases (RRDs). It is used for both storing data into the databases and creating graphs using the databases. * RRDs RRDs are the round-robin databases used by RRDTOOL. They are individual files that are writable by the Gatherer and readable by the Web Interface. _________________________________________________________ Chapter 4. Monitoring and Reporting Concepts * Groups Groups are organizational containers used in NetMRG. They are capable of containing Devices, as well as having a View associated with them. As they are strictly organizational, the Web Interface is the only component to utilize them. They are ignored by the Gatherer. * Devices Devices can be thought of as any physical device on your network that will be monitored. They may also be used organizationally to group Monitors that are not associated directly to any physcial host. Devices may have, but are not required to have, IP addresses, SNMP community strings, and so forth. Data gathering for Devices may be explicitly disabled. Devices contain Sub-Devices. * Sub-Devices Sub-Devices are discrete entities on a Device. Examples of these are network interfaces, disk drives, and so forth. Sub-Devices may have parameters or variables associated with them for identification purposes. For example, a web server has a network interface designated "eth0." In NetMRG, the server would have a network interface Sub-Device which would have a variable of "ifDescr" set to "eth0." Sub-Devices contain Monitors. * Monitors Monitors represent discrete monitorable quantities. The Gatherer performs the Monitor's Test upon the Monitor's Sub-Device. The result is then stored in a RRD. Monitors are also associated with Graph Items. If NetMRG is being used to track events, a Monitor may also contain Events. * Tests Tests are data gathering operations associated with Monitors. There are four types of tests: Scripts, SNMP Queries, SQL Queries, and Internal tests. Script tests call a script and use either its return code or its output as data. SNMP Tests perform a SNMP GET operation against a device using a supplied OID. SQL Tests perform a MySQL query and use a value from a returned row as data. Internal tests are frequently used tests built into the Gatherer. They typically start as Script Tests but are integrated for the sake of efficiency. * Events Events contain a group of Conditions and Responses. If the Conditions of an Event are met, the appropriate response is performed. _________________________________________________________ Chapter 5. Graphing Concepts * Graphs Graphs are the abstraction of a single chart generated by NetMRG using RRDTOOL. Graphs have properties such as sizes, a title, and vertical labels. They also contain a set of Graph Items which represent the separate pieces of data graphed. + Custom Graphs Custom graphs have "absolute" referencing of data to be displayed. They are typically used for graphs that are applicable in only one instance, such as a stack graph of all Internet links on your network. + Templated Graphs Template Graphs have "relative" references to data. They are used for graphs such as CPU load on a server or traffic on a network interface. They are applicable in many instances. The use of templates allows you to create one graph for a set of variables on a Sub-Device and then apply that same graph to other Sub-Devices for graphs formatted identically. This allows considerable savings of time as there is no need to create a Custom Graph for each instance of a graph. * Graph Items Graph Items represent data on a graph. They can represent areas, lines, and stacks on Graphs. Each Graph Item is linked to a Monitor for dynamic data or given a static value. Options for Graph Items include type (such as line, area, stack), color, value (either static or dynamic based on a Monitor), and legend options. * RRDs and Variables Graphs and variables (subdevice parameters) won't become available until after the gatherer runs. The gatherer creates RRD files that the graphs use to display and inserts the variables into the database so they can be used to be displayed in the graphs. This can occasionally bite you when you add a new parameter and expect it to be displayed immediately in your graph. * Variables and Parameters Subdevice parameters become variables that can be used in various places inside NetMRG. Variables can be used inside Tests, Graph Legends and Graph item multipliers. Inside tests, it allows you to use use the same test for multiple subdevices by defining a parameter that will be substituted in. Inside graph items, it will allow you to use a graph for multiple subdevices. Here's a partial list of some variables that will be defined by NetMRG for a device/subdevice + %dev_name%: your device's name + %ip%: the IP entered for your device + %test_name%: the name of the test executed + %test_result%: the results of the test + %snmp_read_community%: the SNMP community for the device + %subdev_name%: name of the subdevice There are usually several "Dynamic Parameters" for a subdevice; if you go to your device, then click parameters next to your subdevice, you will get a list of parameters you entered an a list of dynamic parameters that NetMRG has created. III. Installation Table of Contents 6. Software Requirements 7. Satisfying Requirements 8. Installation Procedure 9. Upgrade Procedure 10. Basic Tutorial 11. Graph Tutorial 12. Interface and Disk Cache Tutorial _________________________________________________________ Chapter 6. Software Requirements NetMRG Requires the following packages for compilation. * PHP, Version >= 4.1 * MySQL, Version >= 3.23 * RRDTOOL, Version >= 1.0.28 * Libxml2, Version >= 2.30 * Libstdc++, Version >= 3.2 Versions earlier than this may compile just fine, but may experience runtime issues. Specifically, we experienced software crashes on an SMP machine running version 2.96 that were resolved by upgrading to 3.2. * NET/UCD-SNMP, Version >= 4.2.2 _________________________________________________________ Chapter 7. Satisfying Requirements Specific instructions to satisfy dependencies for certain platforms are presented below. They should be used more as suggestions, not necessarily step-by-step instructions. You may need to do more than what is listed, or much less, depending on the options you selected when installing your operating system. * Red Hat Linux + In order to satisfy the libstdc++ requirement, Redhat 8.0 or above is required. NetMRG may not be stable on older versions of libstdc++. + Install the following RPMs from your Red Hat installation media: o httpd o php o php-mysql o mysql o mysql-server o mysql-devel o libxml2 o libxml2-devel o net-snmp o net-snmp-devel o net-snmp-utils + Installing RRDTOOL o Download an RRDTOOL RPM from http://www.freshrpms.net/ o OR Download the RRDTOOL source from http://www.rrdtool.org/ and compile it. + Installing precompiled NetMRG RPM Download the latest NetMRG RPM and install it as follows: rpm -Uvh netmrg-0.10.rpm You will still have to perform the following tasks listed in the Install Instructions: o Setup the Database o Configure Apache o Configure the Installation o Setup Crontab * FreeBSD (4.8) + Install the following ports: o www/apache13 o rrdtool o net-snmp o libxml2 o mod_php4 (specify MySQL support) o lang/gcc32 + Run configure, specifying gcc32: ./configure 'LDFLAGS=-L/usr/local/gcc3/lib' 'CPPFLAGS=-I/usr/local/gcc3 /include' 'CXX=g++32' _________________________________________________________ Chapter 8. Installation Procedure 1. Untar to some directory like /tmp/ (it will untar into a 'netmrg' directory) cd /tmp wget http://www.netmrg.net/download/release/netmrg.tar.gz tar xzvf netmrg.tar.gz 2. Now we need to compile and install the source. cd into the directory made by your tar command and type the following: ./configure make make install Note If configure fails to locate the needed libraries they can be manually specified. Run ./configure --help for details on the necessary options. 3. Setup the database Now we need to setup the database. Please change at least the password (netmrgpass below). mysqladmin create netmrg mysql -u root -p netmrg < share/netmrg.mysql mysql -u root -p > grant all on netmrg.* to netmrguser@localhost identified by 'netmrgpa ss'; 4. You need to modify your apache config in order to make your NetMRG installation live (commonly in /etc/httpd/conf/httpd.conf). The first is to alias /netmrg to the installed web directory (commonly /usr/local/var/www/netmrg/webfiles). Alias /netmrg "/usr/local/var/www/netmrg/webfiles" The easier way (if you're running Apache >= 1.3) is to just Include the conf file we've created anywhere in your file (near the bottom is fine). Include /usr/local/etc/netmrg.conf You will need to restart your Apache after making config changes. Telling it to reload its config should be good enough killall -HUP httpd 5. Configure the Installation All of NetMRG's common configuration options are controlled by an XML file, normally located at /usr/local/etc/netmrg.xml. You should edit this file to make any necessary changes before continuing with installation. For most installations, you will at least need to edit netmrg.xml to reflect your database username and password. 6. Setup Permissions NetMRG does not need superuser permissions to run, therefore we will setup a user for it. # useradd netmrg # chown netmrg:netmrg /usr/local/var/log/netmrg # chown netmrg:netmrg /usr/local/var/lib/netmrg/rrd 7. Setup crontab You can setup the crontab entries in two places: either edit /etc/crontab, or run crontab -e as the netmrg user. For 'crontab -e', use an entry similar to this: su netmrg */5 * * * * /usr/local/bin/netmrg_cron.sh exit To edit the master crontab, edit /etc/crontab and add a line similar to this: */5 * * * * netmrg /usr/local/bin/netmrg_cron.sh 8. Point your web browser at your install, and start graphing things! The default login is admin and the default password is nimda. Point your web browser at the host you built this on (something similar to http://yourhost.com/netmrg/) and please change your password! _________________________________________________________ Chapter 9. Upgrade Procedure After doing a 'make install' as described in the chapter on Installing NetMRG, the next step is to run the web-based upgrader. This is accomplished by logging into NetMRG with administrator privileges, and following the prompts. NetMRG's web interface will not operate normally until the web-based upgrade is performed, and non-administrator users will only receive a message directing them to contact their NetMRG administrator. _________________________________________________________ Chapter 10. Basic Tutorial This brief tutorial will walk you through the major features of NetMRG. The tutorial assumes that you have installed the Net-SNMP daemon on a system which you want to monitor. 1. Create a new group. a. Click on Groups in the Monitoring menu. b. Click on Add. c. In the Name textbox, enter a name for your group, such as "Home." d. In the Comment textbox, enter a description for your group, such as "Computers in my house." e. In the Parent drop down box, select -Root- as we want this device in the root of the device tree. You can create nested groups. f. Click the Save Changes button to create your new group. 2. Create a new device. a. Click on the name of your new group. You are now looking at an (empty) list of devices in your new group. b. Click on Add in the Monitored Devices section. c. Click on Create a new device. d. In the Name textbox, enter a name for your device, such as "My Server." e. In the IP or Host Name textbox, enter a domain name or IP address of your device. Domain names must be resolvable from your monitoring host. f. In the SNMP Support dropdown, select the appropriate version. If unsure, select SNMPv1. g. In the SNMP Read Community textbox, enter the SNMP community string for your device. h. Click the Save Changes button to create your new device. 3. Create a new sub-device. a. Click on the name of your new device. You are now looking at an (empty) list of sub-devices in your new device. b. Click on Add. c. In the Name textbox, enter "System." This will be a pseudo-sub-device used for monitoring the device as a whole. d. In the Type drop down box, ensure that Group is selected. e. Click the Save Changes button to create your new "System" sub-device. 4. Add graph templates. Since devices running Net-SNMP are common, NetMRG comes with several graph templates for commonly used variables. a. Click on Template Graphs in the Graphing menu. b. Click on the Apply Template To... next to the Linux Box - CPU Utilization template graph. c. In the Subdevice drop down box, select My Server - System. d. Click on the Save Changes button to apply the template. e. Repeat this step for any other graph templates you find interesting. 5. View your new graphs. a. Click on Device Tree in the Reporting menu. b. Click on the name of your group to expand it. c. Click on the graph icon beside your device (orange graphs mean there are graphs associated with this device, black means there are no graphs attached to this device. d. You should now see graphs! If you don't, you may need to wait for the gatherer to perform another cycle. _________________________________________________________ Chapter 11. Graph Tutorial This tutorial aims to show you the basics of how to create a Custom or Template Graph. It will be useful to look at existing graphs that are supplied with NetMRG for examples. Custom and Template graphs are very similar, but have slightly different applications. A Custom Graph is designed to be used only once, while a Template Graph is designed to be applied to multiple devices and subdevices Let's try a basic graph using no subdevice parameters. 1. Creating a Template/Custom Graph a. Goto the Template Graph item under the Graphing menu. b. Click on Add. c. Click on Add. This leads you to a screen where you can combine multiple monitors (usually of the same subdevice) into one graph using various styles; areas, stacked areas, and thicknesses of lines. d. Click on add This page allows you to edit various aspects of this monitor's appearance. Name: Name that will be displayed in the legend. If omitted, this item will not be displayed in the legend. Item Type: can be various thickness of lines (1 being thinnest, 3 being thickest), area (which will always start at '0' from the graph - positive or negative), and stack (which will 'stack' on top of other areas and stacks). Item Color: color of the item on the graph and in the legend. It can be omitted to have a transparent item. Fixed Value or Multiplier: change the value displayed by your monitor. The rest of the checkboxes should be self explanatory e. By repeating these steps you can add multiple monitors to a single graph. _________________________________________________________ Chapter 12. Interface and Disk Cache Tutorial The Interface and Disk Cache (that is available next to your device names under the Groups menu) are quick tools for setting up interfaces and disk graphs on your devices. These caches provide useful mechanisms for indexing disks and interfaces. For example, commonly you want to graph your interfaces by their ifName (interface name) which is commonly something like 'Se2/1'. Doing a lookup each time you gather data would be very time-consuming, so the cache allows you monitor an interface by using it's name, and NetMRG will do the mapping to the correct ifIndex. NetMRG will do the same thing for disks * The options from next to your device are as follows 1. View Interface Cache: This is only available after you have cached your interfaces. Most of the management is done through here 2. Recache Interfaces: If you add or remove interfaces to your device, this can force a recache. You can also setup NetMRG to automatically recache your interfaces when the device reboots or the number of interfaces changes by editing your device. 3. View Disk Cache: Like the interface cache; the list of disks that NetMRG currently knows about. 4. Recache Disks: You'll probably only do this once when you create your device (unless you add/remove disks a lot). * Inside the Interface Cache 1. Checkboxes: You can select multiple interfaces to monitor by using the '*' (all), '0' (none), and '-' (inverse) items, or by clicking each checkbox individually. 2. View: View the graph that has been created by clicking 'monitor/graph'. 3. Monitors: View the monitors under the subdevice that was created by clicking 'monitor/graph'. 4. Monitor/Graph: Apply the graph template selected in the dropdown box by clicking this item 5. Dropdown box: The list of graph templates that are available in your NetMRG installation; it's very useful when you have different types of interfaces with different monitors you want applied to an interface (maybe you have one graph with a maximum line, or different multiplers to show bits instead of bytes). * Inside the Disk Cache 1. The options inside the Disk Cache are very similar to the Interface Cache. Refer there for their descriptions. IV. Feature Guide Table of Contents 13. External Authentication 14. Workday Highlighting _________________________________________________________ Chapter 13. External Authentication NetMRG handles authentication internally by default, checking passwords against its own database. It is possible for NetMRG to use other systems for username/password authentication. This is accomplished by handing off the authentication task to the HTTP server. Note You must still create accounts in NetMRG for each user when using external authentication in order to assign them permissions and rights. Details on methods of authentication using the Apache web server are available here. There are add-on modules for Apache that allow for various authentication mechanisms, including LDAP and custom database authentication systems. Using Apache-based authentication, it is possible for an organization to maintain users and passwords in one location for a variety of web-based services, only needing to set up application-specific permissions in each application. To enable external authentication, edit netmrg.xml and set externalAuth to true. _________________________________________________________ Chapter 14. Workday Highlighting Shading specific time periods on a graph is possible using NetMRG. Typically this is used for highlighting the workday or SLA distinctions based on time of day. The following steps will walk you through the setup of workday highlighting on a graph. 1. Add a new Graph Item. 2. Select Show Advanced. 3. Leave the title of the item blank if you want it to be omitted on the legend, otherwise enter a label such as "Business Hours." 4. Set the Item Type to Area. 5. Select a color for the highlighting. 6. Select Fixed Value in the Monitor drop-down. 7. Type in "INF" as the fixed value. This allows the graph item to take on an infinite height, which is rendered by filling the background. 8. Unselect all checkboxes under Legend. 9. Enter start and stop times in their respective text boxes. Times should be prefixed with a plus sign ('+') and be consistent with GNU Date Input Formats. An example would be +9am for a start time and +5pm for an end time, to highlight the "typical" business day. 10. Click Save Changes and observe the colored rectangle cover a lot of the data on your graph. 11. Click Move Up beside your new graph item until it reaches the top of the list. You now have a graph with workday highlighting! V. Reference Table of Contents I. Manual Pages II. Web Interface Descriptions I. Manual Pages Table of Contents netmrg-gatherer -- Gathers data from devices. netmrg-gatherer Name netmrg-gatherer -- Gathers data from devices. Synopsis /usr/local/bin/netmrg-gatherer [OPTIONS] Description netmrg-gatherer performs tests on all enabled hosts in NetMRG. It will store results in RRD files using RRDTOOL(1) and perform actions in response to events. Global Options -h Displays usage information and exits. -v Displays software version and exits. Operating Options -t num threads Run no more than the specified number of threads simultaneously. -X Become a daemon. -M mode Mode of operation. This option controls the manner in which the gatherer operates. The modes currently supported are "once" (the default, best used manually or via cron) which runs all tests once then exits, and "wait" (best used when daemonized) which will run all tests, wait for the gatherer interval to expire, and then repeat. Note To run netmrg-gatherer daemonized, the recommended set of options at this point is "-X -S -M wait" which will daemonize, repeat the gather cycle on each interval, and write output to syslog. Logging, Debugging and Verbosity Options -S Syslog. Logs messages to syslog instead of stdout. -b Bland. Disables color output. By default, netmrg-gatherer will display color output when connected to a terminal, but will use bland output when its output is written to a file. This option should only be needed when the detection mechanism has failed, or the user doesn't want to see colors when running the program by hand. -a All. Displays all log messages. -m Most. Displays more messages than default. -q Quiet. Silences all output. -l level mask Only outputs messages that match the specified level mask. -c component mask Only outputs messages that match the specified component mask. -s Safe. Tries to keep sensitive information out of the displayed messages. Configuration File Options -C config file Uses the specified XML configuration file. -K config file Parses the specified XML configuration file and exits. Database Options -H host Uses the specified host name for connecting to the database. -D database Uses the specified database name. -u user name Uses the specified user name for connecting to the database. -p [password] Uses the specified password for connecting to the database. If no password is specified, the user is prompted to enter one. Recaching Options -i device id Performs an interface recache on the specified device. -d device id Performs a disk recache on the specified device. Note Recaching options are intended for use by the web interface. These options should not be generally used by end users. II. Web Interface Descriptions Table of Contents Monitoring -- Items associated with setting up gathering of information Groups -- A container for a collection of devices Devices -- A container for a collection of sub-devices Sub Devices -- A container for a collection of monitors Monitors -- A container for a collection of tracked values Events -- A container for a collection of actions to take based on monitor values Conditions -- A collection of paramenters to trigger an Event Reporting -- Items whose purpose is to display and report information to the user. Graphing -- Where graphs (compositions of one or more monitors) are created. Tests -- Tests represent actions taken to gather data. Admin -- Administrative functions not related to the reporting or gathering of information. Help -- How to get assistance with NetMRG. Monitoring Name Monitoring -- Items associated with setting up gathering of information Description Groups Management of Groups, Devices, Sub-devices, Monitors, Events, and the Notifications used by the events. Device Types Types of devices; unused at this point. Notifications Types of notifications that can be tied to events. * Name - Descriptive name of the notification * Command - Command to execute for the notification. Notifications that start with a '/' will use the ful path the the specified command, whereas commands without a starting '/' will start using NetMRG's libexec directory. Parameter substitution is fully functional here, as well. Groups Name Groups -- A container for a collection of devices Description List List of all groups. This is the default view when you first click on Groups in the nav bar. * Name - click on the name of the group to view the devices contained in it * Comments - description of a group * Add - add another group * View - views graphs assocated with this group and allows you to edit what graphs are associated with the group * Edit - edits parameters of a group * Delete - deletes the group and all devices, sub-devices, etc underneath it Edit Edit information about a group * Name - Name of the group * Comment - Description of the group * Parent - Parent group for this group; sets up subgroups Devices Name Devices -- A container for a collection of sub-devices Description List List of devices in a group; this is the default view when you click on a group name. * Name - click on the name of the device to view the sub devices contained in it * SNMP Options + View Interface Cache - views a list of cached interfaces (interfaces and values that NetMRG knows about) + Recache Interfaces - recache the interface list (to get new ifAliases or new interfaces, for example) + View Disk Cache - views a list of cached disks (disks and their values that NetMRG knows about) + Recache Disks - recache the list of disks (to get new disks or paths, for example) * Add - add a new device or an existing device to this group * View - views graphs assocated with this device and allows you to edit what graphs are associated with the device * Edit - edits parameters of a device * Delete - deletes the device all sub-devices, monitors, etc underneath it Add * Create a new device - creates a new device in the current group * Add an existing device to this group - puts a link to an existing device in this group (does not copy it, just refers to it; they are the same device + Device - Name of the device to add to this group Edit Edit info about a device * General + Name - Name of a device; commonly the hostname + IP or Hostname - IP address of the device, or hostname (must be resolvable by the monitoring box) + Device Type - Type of device (currently not used) + Disabled - Don't gather data for this device * SNMP + SNMP Support - The version of SNMP to use when querying this device, if any. + SNMP Read Community - The SNMP community string to use when querying this device. + Recaching Method - Methods used by the gatherer for determining if its SNMP cache is stale. o Never refresh cache - The gatherer will never automatically refresh the cache. The cache must be refreshed manually when changes occur. o Refresh on SNMP agent restart - The gatherer will refresh the cache if the SNMP agent uptime goes backwards, or if the device has become reachable again after a period of unreachability. This option is useful for devices that change interface counts too often, such as a router serving DSL sessions. o Refresh on interface count change - The gatherer will refresh the cache if the "ifNumber" value has changed since the last gathering. This typically happens when cards are added to or removed from a router, or when PPP sessions go up or down. (Also includes the above.) o Refresh on interface count mismatch - The gatherer will refresh the cache when the number of items in the interface cache do not match the "ifNumber" on a device. This can prevent problems when devices are in the process of booting. This is the preferred option for most devices. (Also includes the above.) o Always refresh cache - Always refresh the cache on each gather. This is useful for devices that are always changing, when you care about the changes. * Advanced SNMP Options + Disable SNMP Uptime Check - Allows NetMRG to work with devices that are broken or abnormal with respect to system.sysUpTime.0. NetMRG usually uses this value to determine if a host is up and if the SNMP agent has restarted since the last polling cycle. It should only be used when absolutely necessary as it reduces the robustness of the SNMP avoidance and recaching mechanisms which can be crutial to performance. + SNMP UDP Port - UDP port to direct SNMP queries to (default is 161). + SNMP Timeout (microseconds) - Number of microseconds to wait for a response to a query before retrying (default is 1000000 or 1 second). An exponential backoff is used after each failed query; this is the initial delay. + SNMP Retries - Number of times to query the device before giving up (default is 3). SNMP Cache View List of cached values for a device Interface Cache List of Interfaces NetMRG knows about * Index - SNMP ifIndex * Status - SNMP ifAdminStatus/ifOperStatus * Name - SNMP ifName * Description - SNMP ifDescr * Alias - SNMP ifAlias (commonly the 'description' field on routers and switches) * IP Address - IP address of this interface * MAC Address - MAC address of this interface * View - View the graph for this interface (if monitored) * Monitors - View the monitors for this interface (if monitored) * Monitor/Graph - Choose to graph this interface (adds an interface subdevice based on ifDescr, adds monitors for ifInOctets, ifOutOctets, and ifInErrors for the subdevice, and assocates a graph with the device) Disk Cache List of Disks NetMRG knows about * Index - SNMP dskIndex * Device - SNMP dskDevice * Path - SNMP dskPath * View - View the graph for this disk (if monitored) * Monitors - View the monitors for this disk (if monitored) * Monitor/Graph - Choose to graph this disk (adds a disk subdevice based on dskPath, adds monitors for KB Used and KB Available for the subdevice, and assocates a graph with the device) Sub Devices Name Sub Devices -- A container for a collection of monitors Description List list of sub devices in a device * Sub Devices - Name of the sub devices for this device; click on a name to view its monitors * Type - Type of sub device (group, interface, or disk) * Add - add a new sub device to this device * Parameters - View/edit available parameters for this sub device (such as what ifName to use to graph this interface) * View - views graphs assocated with this sub device and allows you to edit what graphs are associated with the sub device * Edit - edits options of a sub device * Delete - deletes the sub device all monitors, events, etc underneath it Edit edit info about a sub device * Name - Name of this sub device * Sub-Device Type - Type of sub device (group, interface, or disk) Parameters items that alter what interface or disk a sub device monitors Configured Parameters * Name - Name of paramter to match this sub device to (commonly 'dskPath' for disks or 'ifDescr' for interfaces) * Value - Value of the Name to match (commonly '/' for disks or 'eth0' for interfaces) * Add - Adds a Name/Value pair for this sub device * Edit - Edits the values above for this sub device * Delete - Deletes this Name/Value pair for this sub device Dynamic Parameters * Name - Name of paramter that is available in template graphs as a wildcard (put between two '%' in the title) * Value - Value of the Name that will be displayed Monitors Name Monitors -- A container for a collection of tracked values Description List list of monitors for a sub-device * Test - Name of what is being monitored; click to view/edit events for this monitor * Data - Information about the last gather of this monitor + Value - value gathered in the last run + Rate of Change - Difference between the current value and the last value, divided by the time between samples + Time Stamp - Date/Time of last gather for this monitor * Graph - A small representation of the current data gathered for this monitor; click on the graph for more options * Add - add a new monitor to this sub device * Edit - edits options of a monitor * Delete - deletes the monitor and everything underneath it Edit edit info about a monitor * Monitoring Type - specifies the type of test used for this monitor (Script, SNMP, SQL, or Internal) * Test Options - this section varies depending on the setting of Monitoring Type + Script Options o Script Test - specifies which script to run o Parameters - specifies a string which will be appended to the script's parameters. Parameter expansions are honored in this field. + SNMP Options o SNMP Test - specifies which SNMP test to run o Parameters - specifies a string which will replace the expansion %parameters% in SNMP OIDs. + SQL Options o SQL Test - specifies which SQL test to run o Parameters - specifies a string which will replace the expansion %parameters% in SQL queries. + Internal Options o Internal Test - specifies which internal test to run o Parameters - specifies a string used by the internal test * Graphing Options + Data Type - the RRDTOOL data type for this monitor o Absolute - a counter reset when read o Counter - an incrementing value interpreted as a rate (such as calculating your average speed with odometer readings) o Derive - a counter without overflow protection, permits graphing of negative rates o Gauge - the value read is the value used, such as temperatures For a more complete discussion on when to use which data type, see the RRD Create manual page. + Minimum Value - the minimum value accepted by RRDTOOL as valid. + Maximum Value - the maximum value accepted by RRDTOOL as valid. Events Name Events -- A container for a collection of actions to take based on monitor values Description List list of events for a monitor Note Events displayed in bold are currently triggered. Edit edit info about an event * Name - a name for the event, unique within a device. This name is used when displaying entries in the Event Log. * Trigger Type - when to trigger an event. When events are triggered, they are logged in the Event Log and any Responses associated with the event are executed. + On Change - The event is triggered when its conditions are met, and the conditions had not been met on the last run. Example: If you have an event to warn you when CPU Usage climbs above 80%, it will trigger this condition if the previous value was 72 and the current value 85, but not if the previous value was 82 and the current value 85. This prevents duplicate triggerings to a situation spanning multiple polling intervals. + Never (disabled) - This event is ignored and responses for it are not processed. * Situation - the severity of the situation the event indicates. + Disabled - the tested service is in a non-operational, yet normal, condition. + Normal - the tested service is operating normally. + Warning - the tested service is operating in a degraded condition. + Critical - the tested service is not operating acceptably. Note Situations for an event propagate up the Device Tree. Each level takes on the worst situation of its descendents. For example, if you have a Critical situation, the Event as well as the Monitor, Sub-Device, Device, and Groups above that Event will take on the situation Critical as well. Conditions Name Conditions -- A collection of paramenters to trigger an Event Description List list of conditions for an event Edit edit info about an condition * Logical Operation - the logical operator (AND or OR) to apply between this condition and the previous one. This parameter is not valid, nor is displayed, for the first condition in a set of conditions. * Value Type - the type of data about the Monitor to be used for comparision. + Current Value - the current value of the Monitor + Delta Value - the difference between the current value and the previous value. + Rate of Change - the delta value divided by the difference in timestamps between the current and previous values. When using counters, this gives you the "per second" value. * Condition - the arithmetic operator to use when comparing the Value Type of the Monitor and the Value of the Condition. * Value - the number used for comparision. Reporting Name Reporting -- Items whose purpose is to display and report information to the user. Description Device Tree Displays graphs attached to Groups, Devices, etc, down to the monitor level. Graph icons next to each level represent where views are attached to items. Orange icons mean one or more graphs are linked here; grey icons mean there are non currently. Groups will have a Slide Show icon next to them which will show all views in subgroups and devices underneath this group. Most items will also have a pencil icon (when a user has Read/Write permissions) in order to quickly edit this item. Event Log When events are triggered, the record of their status changes can be viewed here. Slide Show Starts an in-order or random slide show of all views. A 'Resume Slide Show' link will also appear to go back to the place where you left off if you leave the slide show. If you have auto-scroll enabled, clicking anywhere on the page will pause the slideshow, and clicking again will resume scrolling. Graphing Name Graphing -- Where graphs (compositions of one or more monitors) are created. Description Custom Graphs Custom Graphs are graphs meant to only be used once. If you have an application where you are going to apply the same type of monitors to more than one device (even if the parameters are different), use a template graph. Template Graphs Graphs intended to be used more than once. Template graphs are applied to a sub-device at the view level. It will look for the same type of monitors as it was created with and use the matching monitor types from whichever sub-device was selected. Template graphs also have the added benefit of being applied to a sub-device, allowing you to automatically create the same monitor types inside a new sub-device, quickly adding more monitors to your device. Tests Name Tests -- Tests represent actions taken to gather data. Overview A monitor can be thought of as a specific instantiation of a test. There are currently four types of tests supported by NetMRG: Scripts, SNMP, SQL, and Internal. Scripts Arbitrary scripts can be executed and their output or return values used as values to be monitored. Edit * Name - An description used to identify the test. * Command - The program to execute for this test. If the value of this field does not start with a slash, it is assumed that the command is located in the default NetMRG tests directory. Commands are passed to a shell for execution by the gatherer. * Data Type - The type of data gathered from the program. + Standard Out - Data from the program's standard output is returned. The program should output data on one line which should be a numeric value or "U" to specify an unknown value. + Error Code - The integer return code from the executed program is returned. SNMP SNMP (Simple Network Management Protocol) is widely used for accessing statistics on network infrastructure and hosts. Edit * Name - An description used to identify the test. * SNMP OID - The OID to be obtained. The value may be numeric or of any form your Net-SNMP library can understand. Because the installed MIBs can vary from machine to machine, use of numeric (or at least common) values is preferred. * Type - specifies whether to GET or WALK the OID + Direct (Get) - The specific OID specified will be returned. + Nth Item (Walk) - The OID specified will be walked until the Nth item is encountered. This item will be returned. * Item # - Specifies which item to return when doing an SNMP walk. This option has no effect on SNMP GETs. SQL SQL (more specifically, MySQL) tests are useful when you have data that is otherwise in-accessable, or takes a long time to gather. Rather than making the netmrg-gatherer run a script test that will take a long time, consider writing the script to run periodically (out of Cron, for example) and dumping its output into a MySQL database for NetMRG to gather later. This greatly reduces NetMRG's runtime. Edit * Name - An description used to identify the test. * Host - The host running the MySQL server you wish to connect to. * User - The MySQL user to connect as. * Password - The password for the MySQL user. * Query - The SQL query to perform. Note No default database is specified, so be sure to include the database name in the table names that you use in your query. * Column Number - The column to return. NetMRG will collect the value from this column in the first row of the result set. Internal Internal tests are compiled into the gatherer. These tests typically start as script tests, and are then migrated to the gatherer for efficiency. At the moment, the gatherer source code is the only documentation for such tests, however some are obvious. The intention is to expand this category to include DSO-based test packages. Admin Name Admin -- Administrative functions not related to the reporting or gathering of information. Description Users The area where you manage users that are allowed to view or modify graphs and parameters in NetMRG. If you're using External Auth, you will also need to come here to create users to setup permissions for them to access your NetMRG installation. Prefs Preferences for your users (such as auto-scroll in the Slide Show, or changing your password). If you're an admin, you will be able to change your users' prefs here, or you will only be able to change your own. Logout Logout of NetMRG. NetMRG will keep its session active for 24 hours; this may allow other users to come and use your computer to view or modify your graphs if you don't close your browser. Logging out of NetMRG will prevent accidental (or malevolent) activities in your NetMRG installation. Help Name Help -- How to get assistance with NetMRG. Description About Some basic information about NetMRG, its authors, the license, and credits to noteworthy contributors. Manual A link to the NetMRG Manual. Forum A link to the best place to go for useability questions. Bugs If you think you have found a bug, submit it here. _________________________________________________________ Chapter 15. Contact Information We have great aspirations for what NetMRG could become, but as with all open-source software, such aspriations can only be achieved with the active participation of the open-source community. Please do your part and support software that you use by helping to answer support questions on forums and report bugs to developers. Thanks! * Main Website - http://www.netmrg.net/ * Support Forum - http://lists.netmrg.net/ * Bug Reporting and Tracking - http://bugs.netmrg.net/