Welcome, Guest. Please login or register.

Author Topic: Performance collector for NEDI  (Read 5208 times)

AJobs

  • Guest
Performance collector for NEDI
« on: February 10, 2009, 05:02:59 PM »
Hi *,

first of all: Great work :-)

I would like to contribute my "perfi.pl" to the community.

So what does perfi do?
perfi collects performance data from the devices discoverd by nedi. It actually
gathers and writes the rrd files. The rrd-files now have a STEP of 300, which
gives you quite nice and useful graphs.

Why another process? Because discovery takes around 4 hours :-) Our "little"
network contains around 1750 switches and 14 routers; roundabout 42000
resulting rrd files.

There are two problems. Get this data and write it to disk. perfi is able to
gather all the data needed for the nedi-graphs in less than 5 Minutes (but you
need a ram-disk to store the rrd-files, because writing 40000 rrdfiles takes
longer than 5 minutes :-)

Here is the timing of the 2 runs:

Starting perfi run Tue Feb 10 16:28:01 CET 2009
Finished perfi run Tue Feb 10 16:29:02 CET 2009
Done. Tue Feb 10 16:32:46 CET 2009
Starting perfi run Tue Feb 10 16:33:01 CET 2009
Finished perfi run Tue Feb 10 16:34:02 CET 2009
Done. Tue Feb 10 16:34:52 CET 2009

Between "Finished" and "Done" a tar archive of the rrd directory is created
(and restored during reboot). This tar is 4GByte uncompressed.

To use it you have to patch nedi, because perfi need the SysObjID, which is not
stored the the database, and you should set $rrdcmd to "" to prevent nedi to
collect the data aswell.

perfi: http://noc.rub.de/~jobsanzl/nedi/perfi.pl (still contains some debuging code)
so-field-patch: http://noc.rub.de/~jobsanzl/nedi/nedi-so-field.patch

--Andreas

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2691
    • View Profile
    • NeDi
Re: Performance collector for NEDI
« Reply #1 on: February 10, 2009, 08:02:46 PM »
Wow  8) Great job! Hope everyone is aware, that I'm quite busy these days. But I'll spend more time to follow up on all your great work as soon as I have some air...I need another Saturday a week!
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

oxo

  • Guest
Re: Performance collector for NEDI
« Reply #2 on: February 11, 2009, 12:03:12 AM »
Quote
To use it you have to patch nedi, because perfi need the SysObjID, which is not
stored the the database

I would also hope sysObjID gets into the next release DB ... :)

(Rushing of to try Performance Collector ...)

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2691
    • View Profile
    • NeDi
Re: Performance collector for NEDI
« Reply #3 on: February 14, 2009, 01:10:45 PM »
The reason why sysobj is not in there (as explained to oxo before), is that the intention is to keep NeDi as simple as possible (for a network discovery tool), thus hiding the gory details. Over time NeDi has gone beyond my imagination and I see it'll benefit, of leaving such things in there...next release will have a SysObjID column  8)

Note: This will allow for simpler access to editing via Defgen (just copy the .def you wish to edit to html/log and allow write access for the websrever). Now that the sysobj link is available for recognized devices as well, you simply click on it in Devices-List...
« Last Edit: February 14, 2009, 01:36:25 PM by rickli »
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

shadowcaster

  • Guest
Re: Performance collector for NEDI
« Reply #4 on: February 16, 2009, 12:18:19 PM »
Hi *,

first of all: Great work :-)

I would like to contribute my "perfi.pl" to the community.

So what does perfi do?
perfi collects performance data from the devices discoverd by nedi. It actually
gathers and writes the rrd files. The rrd-files now have a STEP of 300, which
gives you quite nice and useful graphs.


Great job!
Just one thing to fix - you can't poll device for ifHC or ifOctets only. If device supports HC (64 bit), it means that ONLY high-speed interfaces get 64 bit counters, other will get 32 bit ones. This is per RFC 2233. At least Cisco seems to follow that RFC, and that was a "problem" I hit. This is about to be fixed in next nedi version.

RFC 2233:
Quote
   Instead, this memo adopts expanded, 64 bit, counters.  These
   counters are provided in new "high capacity" groups.  The old,
   32-bit, counters have not been deprecated.  The 64-bit counters
   are to be used only when the 32-bit counters do not provide enough
   capacity; that is, when the 32 bit counters could wrap too fast.

   For interfaces that operate at 20,000,000 (20 million) bits per
   second or less, 32-bit byte and packet counters MUST be used.  For
   interfaces that operate faster than 20,000,000 bits/second, and
   slower than 650,000,000 bits/second, 32-bit packet counters MUST
   be used and 64-bit octet counters MUST be used.  For interfaces
   that operate at 650,000,000 bits/second or faster, 64-bit packet
   counters AND 64-bit octet counters MUST be used.

See full RFC for all possible caveats with interface counters.

folke

  • Guest
Re: Performance collector for NEDI
« Reply #5 on: March 06, 2009, 04:22:52 PM »
There are two problems. Get this data and write it to disk. perfi is able to
gather all the data needed for the nedi-graphs in less than 5 Minutes (but you
need a ram-disk to store the rrd-files, because writing 40000 rrdfiles takes
longer than 5 minutes :-)

--Andreas
Hi,

This is wonderful news!
We use Cacti today like alot of other folks I think.
We have ~250 switches and a few routers that are being crawled by Nedi.

Do you think that we can use this patch without ram-disk?
Our cacti installation handles the load today...

--
Regards Folke
Sweden

folke

  • Guest
Re: Performance collector for NEDI
« Reply #6 on: October 20, 2009, 10:00:47 PM »
Well, it took a while for me to try this out :)

So one year after I checked it out.

The thing is that we installed 1.04, does perfi.pl work with that?

From what I can see, the rrd files get created and so.
But I get no graphs.

btw, I've changed the rrdstep to 300 in nedi.conf

--
Regards Folke