2

I have a case where I have collected a lot of SNMP data and stored it via rrdtool. (Using OpenNMS)

My goal is to determine, among hundreds of servers, which ones haven't had memory usage exceed a certain amount within the past six months. (say, 64 gigs)

My plan was to write a bash script to extract and process the data from rrdtool, but unsure how to start. This seems like a common enough task that I felt I should ask on here if anyone has any ideas.

Thanks!

BMW
  • 42,880
  • 12
  • 99
  • 116
cat pants
  • 1,485
  • 4
  • 13
  • 22
  • Interesting topic. can you first check in OpenNMS forum if there are any exit sulotions. – BMW Mar 05 '14 at 22:01

1 Answers1

2

RRDTool terminology:

  • RRD : The RRDTool database file
  • DS : Data Source. One of the variables being measured
  • RRA : Roundrobin Archive. Consolodation archive definedin the RRD file
  • CF : Consolodation Factor. MAX, MIN, AVERAGE, LAST. How the RRA consolodates the data
  • DP : Data point. A sample of data as stored into the RRD before consolodation
  • CDP : Consolodated data point. A data point in an RRA, which corresponds to one or more DP merged using the CF of that RRA.

I would suggest doing this in two parts.

Firstly, extract the Maximum of the values over the time period for each DS. This step is simplified considerably if you create an RRA that has a MAXIMUM CF and an appropriate granularity, such as 1 day. How you do the extract will depend on if you have a single RRD with many DS, or many RRDs with one DS in each; however you will need to use the rrdtool xport and NOT rrdtool fetch to retrieve the data so that you get a single data value for each DS. The xport function of rrdtool will allow you to further consolodate your 1CDP==1day RRA to get a single CDP; do this by setting 'step' to be 6 months, and force your DEF to use a MAX CF. The reason we use a 1day RRA rather than a 6month one is so that we can run the calculation on any date, not just once every 6 months.

Assuming your file is data1.rrd containing a single DS dsname for host host1 :

rrdtool xport --end now --start "end - 6 months" --step 15552000 --maxrows 1
    DEF:x=data1.rrd:dsname:MAX 
    XPORT:x:host1

Next, you will need to threshold and filter these to get the list of DS that have a MAX value below your threshold. This would be a simple process in bash that shouldn't tax you!

Steve Shipway
  • 3,754
  • 3
  • 22
  • 39