NeDi 1.0.7 Progress

[rickli]

Coming along nicely! I'm finishing multi-tenancy (meaning a user can be given access to just some deivces, based on location, contact, type, community or vtp domain for example). While testing notification in a Ski resort (hey, I'm Swiss go figure) some SMS notification issues surfaced, which need to be addressed.

I just found a bug, where the wrong vlan is assigned to the nodes. Amongst many other minor fixes and improvements I expect to get everything done within a month or so!

[rickli]

Life is great when you get to have a drink on a glacier (via helicopter) on a Sunday morning and find out afterwards, that whatever you hacked together Saturday night actually works!

Honestly a chopper ride in the Alps beats everything by a far shot, but having mails and SMS aggregated and only sent to the users having access to the involved devices is pretty cool too...


---------- Forwarded message ----------
From: <nedi@smtp.nedi.ch>
Date: 2011/9/11
Subject: NeDi Discovery Alert
To: rickli@some-glacier.ch


Hello admin

2520G8+P: Link  LLDP:C2940-8TT,Gi0/1 on 5 changed type from CDP to LLDP
2520G8+P: Link  LLDP:1800-8G,Port#1 LLDP:Sesxi,vmnic0 on 4 changed type from CDP to LLDP
2520G8+P: Link  LLDP:Aastra IP Phone,00085d219bc2 on 3 changed type from CDP to LLDP
2520G8+P: Link  LLDP:SIP000E08D217E9,Port1 on 10 changed type from CDP to LLDP
1800-8G: Link  LLDP:MSR20-15,Et0/2 on Port#5 changed type from CDP to LLDP
1800-8G: Link  LLDP:2520G8+P,4 on Port#1 changed type from CDP to LLDP
MSR20-15: Link  LLDP:1800-8G,Port#5 on Et0/2 changed type from CDP to LLDP
Node 000c29555c17 reappeared as plutus.nedi.ch/10.10.10.11 on Port#3 of 1800-8G
Node 00c0a8bf9549 reappeared as cerberus.nedi.ch/10.10.10.188 on Do0 of C-AP1231

Visit http://www.nedi.ch NeDi for more information

Kind regards & lots of fun
-Remo


This brings 1oh7 a whole lot closer. I need to do some more testing and will release a snapshot soon. Please don't forget this, if you have ideas for better usability: http://forum.nedi.ch/index.php?topic=1032.0

[raider82]

I am glad, 1.0.6 is out and it works really well. Thanks for that.

However, I still got issues with my nodes and have put some efforts into troubleshooting.
The thing is that I see the nodes on uplinks of the wrong switches. One example from the actual output of NeDi:
Situation: Node (a printer) 0000ff4b6fdf  is connected to switch access02 Fa0/42; all access switches are connected to the core
However, NeDi claims it is connected to access61 Gi0/49 -> that's the uplink of a completely different access switch to the core

Here is the ouptput, stripped down to the relevant information for this node:
core1
ARPS:0000ff4b6fdf=172.26.245.190 on Vl1 vl1
FWDC:0000ff4b6fdf on Gi2/2 vl1

core2
FWDC:0000ff4b6fdf vl1, no IF Port-channel1

access71
FWDC:0000ff4b6fdf on Gi0/49 vl1

access01
FWDC:0000ff4b6fdf vl1, no IF Gi1/3

access02
FWDC:0000ff4b6fdf vl1, no IF Fa0/42

access20
FWDC:0000ff4b6fdf on Gi1/1 vl1

access32
FWDC:0000ff4b6fdf vl1, no IF Gi0/1

access31
FWDC:0000ff4b6fdf vl1, no IF Gi0/1

access33
FWDC:0000ff4b6fdf vl1, no IF Gi0/1

access61
FWDC:0000ff4b6fdf on Gi0/49 vl1

access43
FWDC:0000ff4b6fdf vl1, no IF Gi0/1

access42
FWDC:0000ff4b6fdf vl1, no IF Gi0/2

access41
FWDC:0000ff4b6fdf on Gi0/1 vl1

access51
FWDC:0000ff4b6fdf on Gi0/51 vl1

access52
FWDC:0000ff4b6fdf on Gi0/51 vl1

access53
FWDC:0000ff4b6fdf on Gi0/49 vl1

NODE:0000ff4b6fdf pn01001. M4352 access53 Gi0/49 M4352 access52 Gi0/51 M4352 access20 Gi1/1 M4352 core1 Gi2/2 M4352 access41 Gi0/1 M4352 access51 Gi0/51 M4352 access71 Gi0/49 M4352 access61 Gi0/49 M4352 = access61 Gi0/49

M4352 access61 Gi0/49
I added a line for debugging in the source and it says:
Metric: 4096, Newmet: 4352
ifmetric however is 4352 in the database -> where does 4096 come from?
Funny thing: why does access02 not appear in this list, where it is check if an update is required?
Conclusion: access61 is by accident the last entry in this row and therefore is chosen. access02 is missing, although it would be the correct device.

Configuration:
getfwd          sec

Anyone who can help?

Btw. with the new version, I get this error message all the time:
Use of uninitialized value $ifnamO in concatenation (.) or string at ./inc/libsnmp.pm line 945.

[rickli]

If nodes are not assigned correctly it's usually because the MAC is not found on the proper switch, like you noticed already. Do you see the printer listed in the CLI? is access02 discovered at all?

4096 is the new linkmetric (Hover over the calculator in Reports-Nodes Distribution for more info). Wired ports receive an additional 256. Wlan interfaces use the client's SNR as metric (implemented on Cisco APs so far).

The error you see, should be from missing IFname in the .def. I added this option since some switches use ifdesc, some only offer ifidx (in which case IFname should be emtpy).

[raider82]

Thanks for the quick reply.

Setting the interface name in the def file solved this issue. The device is now shown on the correct port.

I have copied over some definitions from RC2, so I guess I have to check for more ports with missing interface names.

[rickli]

Have a look at todays news on www.nedi.ch

Discover new report and map features, many usability fixes and the beautiful "Neo" theme...even if you don't care about multi-tenancy

[pc_sg]

Hi Rickli, I've just update my "test" NeDiO 1.0.6 to 1.0.7 to experience your improvement.
When I'll have time for, I'll try to understand how to configure and use "multi-tenancy", but my current test environment is "one site" limited by design, so I'm the only one "tennent" 

Anyway seems a bit more responsive (but 1.0.6 is more responsive then 1.0.5 by itself!).

One of the most interesting thing to have in future releases, for a multi site WAN environment, is multi threading, to parallelize discovery and queries, but I know that is not as easy as say it... Don't worry, the current release seems speedy enough (for me...) !!!

A little "glitch": looking at "Monitoring Health" page, the bottom part shows fields from SNMP Location trunked at 9th character, while NeDi is able to handle longer ones, even if containing spaces.

If I see something other, I'll keep you informed!

As always, thanks a lot !

P.C.

[rickli]

Thanks for your quick feedback!

Yes, parallel discovery is the next quantum leap and I've been postponing this successfully for years Let me just say that multi-tenancy is just a side-product of the overhaul...

I know documentation is way behind, but I'm catching up (see website). I also plan some useful tutorials soon!

- Multi-tenancy: Create a new user and don't add him to the admin group (won't make sense, thus filtering is disabled). Now you can create a filter based on those fields available. Click on the eyes to verify the user's "view"...

- Multi-threading on the other hand can be done manually. Just add your WAN router as border and start a thread for each remote location. This is actually better than the "real" parallel discovery mentioned above, because you will exactly have 1 thread per WAN link this way.

[pc_sg]

You're welcome!

• I'll do some try in the future. First I've to do a new "master" (all sites managing) NeDi VM (I prefer to start from scratch, to have installed only what is needed to run NeDi.
  I think I'll again choose Debian. My "production" VM was made on Debian, following Ubuntu guide, but I preferred to put NeDi under "var" intead of "opt" (never understand why this change under Ubuntu...)
  And I like to wait the stable 1.0.7 (even if current beta seems stable enough!)
  
• I suppose saying "thread" you mean start different cron task using different seeds and conf.
  I've already done this, after a grouping based on "discovery time" to guarantee that all sites will be checked in less than a quarter of hour, so I can repeat a "all sites" discovery every 15 minutes, but not knowing if running all task in parallel may be too heavy for the serve, or if this may corrupt database consistency, I start every task at different (interleaved) time.

If you have some advice, of course are wellcome!

P.C.

[gbine]

First of all, nice enhancements Remo, Thank You.

I'm experiencing an issue with interface counters on all my switches (HP and Cisco).  They seem to be increasing with each discovery.  More precisely, the numbers I'm seeing seem to be the raw counters from the switch, NOT the deltas for the timespan since last scan (120 minutes in my case).
The RRDs appear to be correct.

I'm sure it's something I've done or setup something incorrectly, but any help would be appreciated.

Regards,

Gordon

[rickli]

I did optimize some things there for link failure detection and the likes. However I can't reproduce this problem. Anybody else?

I did find a bug where interfaces are not written, if you don't use -v (due to unexpected perl behavior). Quick fix is to just add -v (verbose output won't harm anyway).

[gbine]

Thank you, that fixed it.  I have corrected my crontab jobs accordingly.

FYI - my version of perl is v5.8.8

[rickli]

If you don't get graphs have a look at html/inc/libgraph.php and change lines 36,37 accordingly:

   #$agrtyp = 'MAX'; #TODO check whether this makes sense with aggregation or remove from RRDs to preserve space!
   $agrtyp = 'AVERAGE';

I removed the MAX values from the rrds, thus they won't show up anymore. This reduces the rrdsize, but also averages out peaks over time. I was playing around with using MAX values instead, but the results where not convincing either...

[raider82]

Have a look at todays news on www.nedi.ch

Checked 1.0.7 now and it's working great. Thanks a lot for your efforts.
I just got a few questions (that are not directly related to this specific version):
1. Is it possible to switch off the usage of Google Maps? We mainly use location specific information that is not recognized.
2. What is influences by "rrdstep"? The description says:
"This value needs to be set to the discovery interval in seconds. Make sure it  corresponds to the NeDi crontab entry (e.g. hourly 1 * * *  => 3600)."
However I do not use NeDi with cron, since I set up a logic to run one discovery after the other with certain pauses in between. Runtimes differ and some locations are only discovered once. I would assume that e.g. transferred data per hour is displayed incorrectly then. Any other impact (I could live with incorrect counters)?
3. Have you already thought about IPv6? This is going to come soon in our organization and I would love NeDi to be compatible ;-)

[rickli]

Glad you like it. A release candidate will follow soon!

1. Each user can enable google-maps in his profile. I'm not quite sure what the next steps will be in that respect. Apparently a proper license would cost a lot of money for the user, but it obviously would be more interesting for me to make money with NeDi myself I'll look at alternatives and how it's going to be used (e.g. in monitoring etc.)

2. Upon creating a new rrd, a value needs to be set with that it can calculate valid graph data. As long as you stay within a certain time frame, rrd will calculate the value for a given slot. I also use that value to calculate link usage, errors and aggregation of events. In your case it should still work. You might consider skipping RRDs altogether by commenting rrdcmd in nedi.conf. BTW you can also skip IF counters if you don't care (-Stebd) to speed up discoveries.

3. Yepp, but I don't know how to store such an address efficiently in the DB. It seems I'm not the only one if you google a bit...