NeDi Community

NeDi General => News => Topic started by: rickli on February 12, 2012, 10:41:48 AM

Title: NeDi 1.0.8 Progress
Post by: rickli on February 12, 2012, 10:41:48 AM
With all the groundwork laid in the previous version, the new development is geared towards enterprise networks. Amongst many other ideas I want to pursue the following:

Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on February 13, 2012, 10:29:52 AM
+1 Parallel monitoring

Maybe, more custom oids that can be graphed instead of only 1.
Title: Re: NeDi 1.0.8 Progress
Post by: vtur on February 15, 2012, 10:52:31 AM
HI
What about BGP  view and monitor ? :-)

Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 15, 2012, 08:57:01 PM
Can you elaborate a bit? At least Topo-Routes lets you see the routing table, but this is only intended for campus routing...
Title: Re: NeDi 1.0.8 Progress
Post by: vtur on February 16, 2012, 12:58:24 PM
Hi
I mean the view state of the BGP or OSPF neighbors on routers in Devices-Status page.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 16, 2012, 10:37:40 PM
I got an interesting contribution, which suggests inline Device-Writer functions. This could be an application for it as well. Unless there are some OIDs to use?
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on February 17, 2012, 09:51:07 AM
Hi Remo, I don't want to stress you...

Have you evaluated a way to "move" a device under Stock when it become substituted or voluntarily removed, keeping all information, without the need to delete it from "online" database?
I mean: currently, if I remove a switch from LAN and put it storehouse, even if "copy" it in Stock, it remains in other pages as a "failing" device.
I think that, if I "move" it into Stock, is a good think if it will be no more shown ad "failing" but remains in database, so, if I reintroduce it somewhere, even with different hostname, NeDi can warn me of a kind ok "duplicated" device, and I can remove it (manually ? automatically ?) from Stock.

Another thing: there is a way to put an "out of order" device in a "not failed" status? May be nice that, if it become again working (after repair), it  will be restored as monitored (not only by moni, but even by discovery).

I don't know if I asked too much  :-[ , but even more, if I made myself clear, english is not my native language...  :(  ???

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 17, 2012, 09:11:21 PM
Now if a device gets discovered its status and comment will be updated, if it's in stock. What would you like to retain, if you delete it in Devices-Status? I could think of generating a different event, if a serial was kept in stock prior discovery.

If you wish to temporarily remove a device from monitoring, just select (None) as test in Monitoring-Setup..
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 18, 2012, 10:34:52 PM
Another topic, I want to add some properties to the .defs. E.g. height in RU, whether it's stackable (preferrably an OID providing stack info). What else can you think of?

I realize there will be multiple CPUs and MEM statistics to be gathered. Also an option to graph multiple custom values would be nice, but I hope you agree parallelizing and IPv6 deserve more attention for now...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on February 20, 2012, 10:19:16 AM
Now if a device gets discovered its status and comment will be updated, if it's in stock. What would you like to retain, if you delete it in Devices-Status? I could think of generating a different event, if a serial was kept in stock prior discovery.

If you wish to temporarily remove a device from monitoring, just select (None) as test in Monitoring-Setup..

I try to explain my ideas: If I have a particular device layout, and I upgrade it for example substituting a couple of 24 port with a (brand new or stocked) 48 port switch, if I don't divest them, I like to keep relevant asset information, so "stocking" the old ones seems a good way. Currently I need to remove them from discovered, else at everey diiscovery I see  faulty devices. Lately, if I reuse them, they remain in stock or NeDi keep care of it?

Another (i think) uselful think is a possibility to put temporarily "on line" my stock devices (attached to net, not operational), do a discovery (of something similar) to capture HW and SW configuration (and serial number, of course), and "archive" it (in stock, again?)

My other question (instead of asking for a new feature, that may be also avaliable in other ways) concerns broken devices. You suggest that I  should keep them in discovered database (even if shown as failing, that is not always nice), but in case exclude it from monitored by "moni"?
I've been thinking that a kind of manual "on hold" until discovered again may be kind. Am'I a "sophist"  ???  :-[

Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on February 20, 2012, 01:14:52 PM
I have been experimenting with SNMP traps as some of the equipment I have doesn't support syslogs.

I installed snmptt (on ubuntu this is done by using sudo apt-get install snmptt).  Attached are the files used for a working configuration.  (The snmptt.conf file is too large to post!)

Currently I use a rudimentary viewer but what I want to do is create a Monitoring-Traps.php based on Monitoring-Events.php  After that I'll try and do a count per hour of a trap per device instead of storing each trap.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 21, 2012, 09:54:45 PM
Good stuff. I put it in contrib. You can try posting the big file as zip or mail it to me and I'll include it as well...
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on February 22, 2012, 08:17:13 PM
How about adding image links to the def files, so when the topo maps are drawn they come up with the correct image of the switch/router?
This can be used in the device list also instead of w2b3 etc images?
Maybe keep the current codes for ones without images?

(And also a way to upload them via GUI to the nedi server.)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 22, 2012, 09:14:32 PM
Basically you want to remoe my handcrafted device icons with something like visio stencils?  :o ??? ::)

Not a bad idea...they are about 10 years old after all! But this might be a tedious process unless it can be scripted. Another question would be the copyright...

I'm willing to pursue the idea, if the community helps out...
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on February 22, 2012, 09:20:07 PM
I think your handcrafted icons have a place where they are but if there is an image available for the device then it uses that instead.
Can probably screen cap the ones from visio....?

Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 22, 2012, 10:16:52 PM
No really, I actually like the idea and don't need to maintain them in parallel. But I'm worried about copyright issues (a quick look at the general MS visio stencil license states that redistribution is not allowed).

As I plan to add a rackview, this would really top it off. Ok putting modules into the right places of chassis would be the icing of the cake...
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on February 22, 2012, 10:22:34 PM
Hmmm.... they probably don't want it to compete with ciscoworks. :(
It was a good idea while it lasted.
Title: Re: NeDi 1.0.8 Progress
Post by: Al on February 22, 2012, 11:09:13 PM
Wlan controller support (mainly HP-MSM, Cisco and Aruba) - Yay, that'll be excellent!
Title: Re: NeDi 1.0.8 Progress
Post by: rskjels on February 24, 2012, 02:38:29 PM
If i may be so bold as to come with a suggestion, since Nedi 1.0.8 is geared towards a enterprise networks.
I would like to se the use of the SNMP location string format to be more flexible. Right now there is :

Region;City;Building;Floor;[Room;][Place within room;][Whatever additional info you want]

I would like to group them first as Customer (or organization entity), then country, region, city etc.

Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 24, 2012, 07:24:31 PM
Unfortunately my location format hits the limit on some devices already. I actually need to look at shortening the string and have the full location entered in Loced...

But how about the contact field? IMHO it doesn't make sense to put the same info on all devices. The network guys know they're responsible for their switches  ;) I'm also thinking of the user view (in User-Accounts) which can easily link to that.

Alternatively, a Region could be (ab)used to include customer info (e.g. a number?). With that all those not using this field, won't have to reorganize all their location strings...
Title: Re: NeDi 1.0.8 Progress
Post by: harry on February 28, 2012, 03:29:24 AM
Hello All,
Is there any possibility of Monitoring status of ISDN PRI/BRI modules on 28xx and 38xx Cisco routers...?  with  graph of number of calls in a day from that router.

Cisco switches (all models)should show the power on device status. (Not all the switches report the power.)

Just checking if anyone would be interested in this...
I would also like to see the Nodes (Aruba Wireless) associate with the AP which it is connected to.
Number of users per controller and if possible per AP.
People are talking about Enterprise version. That is all about naming.
We are using this wonderful master piece work since last three years. We have stopped renewing our support contract for "Whats up".  Nedi does Monitoring and management for our network, what else can you ask for?  i.e. it is an enterprise  product.
All other improvement will definitely make it more useful  and can be used with different product range in the market and in a heterogeneous network.

I love NEDI.
Title: Re: NeDi 1.0.8 Progress
Post by: eviltrooper on February 28, 2012, 09:51:46 AM
it would be nice when nedi can deploy new Firwmware to an Switch ^^ :D

 ;)
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on February 28, 2012, 01:02:05 PM
it would be nice when nedi can deploy new Firwmware to an Switch ^^ :D

 ;)

Buy CiscoWorks, is sooooooooooooooooo cheap!!!!  :P
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 28, 2012, 05:14:58 PM
Darn, I should start making money then? Hope with 1.0.8 nedi will actually replace other tools more officially and not kept hidden as a "secret weapon"  ;)
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on February 28, 2012, 10:18:48 PM
It's the best free "secret weapon" I've found in my 10ish years networking.
Keep it up!

185 devices monitored.
Mostly cisco shop, with some MDS switches, and a couple of foundrys.
I wish I had more time to include firewalls and packet shapers into it. (not sure if the OIDs are supported though)

Love it.
Mark
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 29, 2012, 06:26:43 PM
Cool  8)

BTW eviltrooper, I've thought about FW update, but doing this safely on different vendors and models is  ??? :( :'(

I resort to writing the necessary commands into an editor and paste them into several terminals at once. I did several 100 switches this way and was sure each device went well...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on March 01, 2012, 09:43:43 AM
Remo, I totally agree with you!

CiscoWorks (now called LMS by Cisco, LAN Management System) do this job, but is painfull even with a so integrated product!
Some times (most of them) the best way is do it is "by hand" and "one by one". For example, if a 3570 stack update fails on one switch, you may have an uncontrolled situation! And even LMS is not always able to distiguish this situation, or fail update!

Anyway, a way to keep a local repository of installed firmwares, grabbed directly from devices (TFTP ???), and organized version by version, may be useful. So, you need only a TFTP server (o daemon) to distribute firmware updates (downloaded from manufacturer support site) or downgrade if you have any problem with new firmware version.

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on March 01, 2012, 10:41:09 PM
Tx for the feedback  8) NeDi can fill this gap with System-Files. Uploading images should be fairly easy through the GUI as well. Maybe an addition in Devices-Write could further assist the process (e.g. perform the download), but leave the rest to the admin...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on March 05, 2012, 12:03:01 AM
Ooops, guess I'll need to fix this, if staying with OpenBSD:

n7dev:~#/var/nedi/moni2.pl -v
This Perl not built to support threads
Title: Re: NeDi 1.0.8 Progress
Post by: fblank on March 09, 2012, 09:47:41 AM
I would love to see a way to check and report on compliant configurations up against a policy. The policy would consist of regexes indicating a valid configuration by device type ex. switch / ap / router / etc.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on March 09, 2012, 08:29:41 PM
Indeed I've been thinking about this as well (I know, I've come up with this line a lot, but then again lots of thoughts cross my mind). It shouldn't be to hard to add a OS based list, with strings to match and mails being sent. Any things in particular, the community would be looking for?
Title: Re: NeDi 1.0.8 Progress
Post by: swepart on April 01, 2012, 02:09:53 AM
Suggestions

- Identify 802.1x ports in "Device Status View"
  More and more networks is using 802.1x for security but still there is need for some ports to be open,  it would be very helpful to easily  view and monitor what ports are "open state"
 
- Check ports as "monitored" or "not monitored"
Sometimes you don't want to monitor a certain port, the reasons could be several.

- Support for external pollers.
When monitoring large networks and

This is a biggy :) and probably not doable

- From the "device list"  placing objects on a map :)..  system will then automatically make/draw  the connections
- In a map,  change the positions of a object and make it more viewable (I know the map is dynamic and updated each time u use it,, so I can see the challenge in this)


Right now,,  monitoring ~2000 devices with ~3700 modules,  ~140K interfaces
Rg
Håkan Sjöberg

Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on April 02, 2012, 08:39:32 AM
First of all - thanks for 1.0.7; I am making great progress with this version.
Code: [Select]
mysql> select count(*) from devices;
+----------+
| count(*) |
+----------+
|    10302 |
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from nodes;
+----------+
| count(*) |
+----------+
|   195398 |
+----------+
1 row in set (0.00 sec)
This is only switches, controllers and Access Points, no phones, voice gateways, etc.

With those numbers, I tend to say: mass changes would be very helpful for me. Maybe some stuff is already in 1.0.7 and I just did not know:
A question that I guess is solved in 1.0.7 - does NeDi store the nodes DNS names?
Currently I only discover once a day, but I would like to resolve node DNS names only at night and therefore speed up discovery within working hours.
How many threads can I run in parallel? Currently, I am working with 3, but it already takes too long with DNS lookups to finish them in office hours.
Title: Re: NeDi 1.0.8 Progress
Post by: acoto on April 03, 2012, 12:56:52 AM
Hello Håkan,

About your map request, I´m not sure if I clearly understand your objective, but there are some interesting things that can be done in yEd (once you have your schematic in GraphML)

I found out that Tools --> Select Elements very powerful.

For instance, you can select an interesting node, then Tools -->Select Elements, then:

  General Tab
    Check only "Select Inverse" option

  Nodes Tab
    Check "Use This Criteria"
    Criteria: "Neighbors of Selected Nodes"
    Maximal Path Distance: 1 (or 2.. or 3...)
 
  Edges Tab
    Uncheck "Use this Criteria"

  Bends Tab
    Uncheck "Use this Criteria"

Click OK.

At this point you should have selected: the interesting node, and all nodes which are 2 or more hops  away.

Now, press Shift and click over the interesting node to unselect it, then press Delete.

Result:

A simplified schematic containing only the selected node (or nodes) and 1 hop (or 2.. or 3... if you changed Maximal Path Distance) neighbors

Regards,

Adrian


Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on April 12, 2012, 09:27:51 AM
Another thing that might be interesting for 1.0.8.
Currently, the hostname is used to identify a device. Nevertheless, this is not always correct - e.g. with primary and secondary firewalls.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 12, 2012, 09:06:15 PM
Yes, the SNMP sysname is used as primary key. What would you change? BTW, I wouldn't worry about DNS too much. It takes very long on the first discovery, when all nodes are being resolved. Subsequent discoveries only resolve, if a MAC-IP change is detected or the last ipupdate was longer ago than retire days (which means 30 days at default). Therefore be aware that 30 days after the first discovery, it might take a little longer too....
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on April 13, 2012, 09:55:08 AM
Yes, the SNMP sysname is used as primary key. What would you change?
Maybe the prompt or the serial? As mentioned: all secondary firewalls are currently missing in my installation. And renaming leads to duplicate entries (saw this today with leightweight APs).

BTW, I wouldn't worry about DNS too much. It takes very long on the first discovery, when all nodes are being resolved. Subsequent discoveries only resolve, if a MAC-IP change is detected or the last ipupdate was longer ago than retire days (which means 30 days at default). Therefore be aware that 30 days after the first discovery, it might take a little longer too....
Ok, I have set it down to 14 while testing with 1.0.6. Maybe I should increase it again. Another question: can this be the source for the graph issue? Every 2nd week, all data comes in later?

Another idea: Any option to include functionality which:
- compares seedlists to the devices table (can't report on failed discoveries at the moment) with a report that shows missing devices?
- deletes devices that were removed from the seedlists (maybe with a threshold)?

And one more idea ;-)
- what about an error log in the database, e.g. for any command that did not run? Just found a switch that needs CISCO-OLD although the IOS version is > 12.1 (it requires show mac-address-table instead of show mac address-table); I found this by accident since I was searching for a specific node that appeared on the uplink
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on May 01, 2012, 07:51:03 AM
I would like to suggest a dedicated database poller.  The scheme is:
Description - time period (configurable)
-------------------------------------------------
High      - 5 minutes
Medium - 20 minutes
Normal  - 1 hour
Low      - 1 day
The poller looks at the database and polls the device using the above scheme.  Let us assume the devices file has a new column named polling, then every hour the following would be done as an example: ./nedi.pl -A 'polling regexp "normal"'
Let us assume the devices file also has another column named disco.  The default for the field is 0 and it is marked as 1 if the device is going to be used as a discovery node.  Once a day the following would then be done as an example: ./nedi.pl -p -A 'disco !regexp 0'
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 10, 2012, 11:30:44 PM
raider:
- did you check events for "not discoverable" info? You can click on the radar icon, to filter on discovery related events as well...
- device-list sort by last discove. Then you'd need to manually delete devices via device-status. Do you need to do this a lot?
- Errors on getting configs are caugt...will check with forwarding (and ARP for ASAs)

redpineapple:
This sounds like a major redesign. Variable RRDs and highly scalable discovery. Maybe some day, but not on my priority list right now...
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on May 16, 2012, 02:30:52 PM
raider:
- did you check events for "not discoverable" info? You can click on the radar icon, to filter on discovery related events as well...
Looks good. I will check this.

- device-list sort by last discove. Then you'd need to manually delete devices via device-status. Do you need to do this a lot?
Yes. With a few hundred locations ... I would prefer using existing interfaces to other tools. With an existing seedlist, automated deletion would be perfect.
[/quote]
Title: Re: NeDi 1.0.8 Progress
Post by: swepart on May 23, 2012, 12:01:24 AM
As you mentioned at the webb

"Threaded discovery is more  complex than expected, especially with perl on OpenBSD
not supporting threads out of the box!"

Maybe its time to look into some other distro,, 

To me,, and I am a rookie in this field,  it looks like Centos is highly respected ServerOS.
Idont know if Centos is better supporting threads,  but as it is a "cousin" to RH it might be more
matured than Ubuntu ( u had a video how to install on Ubuntu,  thats why I mention it)

just a thought,,

right now having around ~2500 devices,  ~3500 modules,  ~165 000 interfaces,  113 450 Vlans   :)

The system feels a little sluggish with this volumes :)

/Swepart
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 23, 2012, 08:27:14 AM
Consider also Debian, that is very stable (more then Ubuntu), very well supported, and have a small Server distribution.
In this case Ubuntu is a mercy, because all Ubuntu programs runs well on Debian (from which Ubuntu is a derivative)

Only my two cents...

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: eviltrooper on May 23, 2012, 02:36:16 PM
is it possible to change the Alias and or the speed duplex setting with SNMP WRITE Community string?
Title: Re: NeDi 1.0.8 Progress
Post by: acoto on May 23, 2012, 06:39:12 PM
I think Arch Linux makes a good fit too. The base install is lightweight, and very easy to customize.

It also may be a good time to switch to MariaDB.

Adrian


Title: Re: NeDi 1.0.8 Progress
Post by: pluto01 on May 24, 2012, 05:40:44 PM
Yes, the SNMP sysname is used as primary key. What would you change?

I have noticed previously that if we have a switch, SWITCH1, and we replace that with a new switch (possibly with a new management IP) which we also call SWITCH1, then nedi will not be able to discover the new switch until I go in and delete the old SWITCH1 from devices. It will simply keep the old entry in there and the 'last discovered' will tick further and further back forever.

Does using sysname as the primary key cause this?

I would think something like serial number might make more sense as that should be guaranteed to be unique. If I replace a switch, it would be nice for the nedi entry to update automatically with the new device/IP information.

B
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on June 01, 2012, 11:30:58 PM
Yes, I'm considering a debian based alternative, also since OBSD keeps growing in Size. Not sure if  this is going to happen with 1oh8...

@Eviltrooper: Only IF status changes via SNMP for now, but I think at least alias would be nice to have...will look into it!

@Swepart: Do you mean the GUI by sluggish? Only a better machine would fix this. You can still manually start parallel threads, but make sure they don't "cross-discover".

@Pluto01: Yes name is key, I had SN at the very beginning, but can become very ugly in mixed networks etc. Thus, NeDi requires unique names. There have been requests to NOT overwrite existing devices with same name, but for now you can rely on "SN has changed" and related events to detect such incidents...
Title: Re: NeDi 1.0.8 Progress
Post by: steffen1 on June 10, 2012, 09:52:34 AM
I want to ask, if GBic discovering will also be an issue in v1.0.8 e.g. serial numbers, power etc.?

Steffen
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on June 10, 2012, 06:57:06 PM
Transceivers are discoverable besides modules already, if a serial# is available. Power levels could be added, if the switch supports DOM and info is available in the module OIDs (mostly entity MIB). If it's tied to the interfaces, additional fields would be necessary. Of course this would enable great new features, like alarms on low signal levels as the SFP ages over time...
Title: Re: NeDi 1.0.8 Progress
Post by: steffen1 on June 10, 2012, 09:44:40 PM
Are the GBic's organized in the module MIB (Entity-MIB or properitary one) with each vendor always so that its coverd by the def-file as you described or could it some times happen, that GBic's are organized at seperate MIBs or need to be queried via CLI with the consequence to need additional code in nedi to retrieve the information?
Its a pity that GBic's does not contain the relation to the interface it belongs to in the MIB, maybe its possible by quering via CLI - Wow, you knows a lot.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on June 10, 2012, 11:22:28 PM
Well, I collected some knowledge within those 11 years of working on NeDi :)

Yes, transceivers are classified as 10 (modules are 9) in the entity MIB and this behaviour can be matched with the Classvalue regexp (in Defgen). They usually show the interface in the description or model OID...

I suspect you're still dealing with Juniper switches?
Title: Re: NeDi 1.0.8 Progress
Post by: steffen1 on June 11, 2012, 10:39:39 AM
yes mainly Juniper, but also with Cisco, HP, Foundry/Brocade.

They usually show the interface in the description or model OID...
means its already now possible to create a PHP-report based on a MySQL-query with transeivers tied together with the interfaces belongs to?
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on June 12, 2012, 08:07:48 AM
I think Arch Linux makes a good fit too. The base install is lightweight, and very easy to customize.

It also may be a good time to switch to MariaDB.

Adrian
Arch is a really good distro, especially, if you are free to choose.
For companies, I suggest to keep testing with the latest versions of SLES and Red Hat also. Most companies do not allow consumer versions.
Postgre and Oracle support would also be nice.
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on June 19, 2012, 09:52:27 AM
Other suggestions for useful enhancements:
- Add name of seedlist for events with "not discoverable" information -> this makes it a lot easier to find the correct contacts (my seedlists contain the location name)
 ==> "No response from seed706 in seedlist xy, not discoverable"
 -> alternative: DNS lookup
- If SSH Keys are incorrect, NeDi resets the Cliport to 1; nevertheless, deleting old values out of ~/.ssh/known_hosts would be better. It is hard to manually maintain these keys in big networks
 -> alternative: cron job to delete known_hosts?

Hope these are considered reasonable for other users, too.
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on June 19, 2012, 01:13:37 PM
- Add name of seedlist for events with "not discoverable" information -> this makes it a lot easier to find the correct contacts (my seedlists contain the location name)
 ==> "No response from seed706 in seedlist xy, not discoverable"
I implemented this one. Maybe someone is interested:
inc/libmisc.pm
line 689:
Code: [Select]
$dv  = &snmp::Identify($id, $seedlist);

inc/libsnmp.pm
line 92:
Code: [Select]
sub Identify {

        my ($id) = @_;
        my ($seedlist) = $_[1];
line 108:
Code: [Select]
&db::Insert('events','level,time,source,info,class',"\"100\",\"$main::now\",\"$ip\",\"IP=$ip in $seedlist, not discoverable\",\"nedi\"");
}

                &misc::Prt("IDNT:$ip in $seedlist is not discoverable\n","Not discoverable\t\t");
line 161:
Code: [Select]
&misc::Prt("ERR :$err\n","No response from $id in $seedlist, not discoverable\t\t");
                if($misc::notify =~ /d/i){
                        &db::Insert('events','level,time,source,info,class',"\"100\",\"$main::now\",\"$ip\",\"No response from $id in $seedlist, not discoverable\",\"nedi\"");

Have fun! ;-)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on June 19, 2012, 08:57:14 PM
Thanks for your valued feedback. I'll look into it...btw downloaded Arch, just to check it out again (did look at it years ago, but got stuck on Ubuntu)
Title: Re: NeDi 1.0.8 Progress
Post by: foxtrom on June 21, 2012, 04:18:25 PM
Does Alcatel Switch will be part in the next release ?  :-*

Regards

Good job :)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on June 21, 2012, 06:48:53 PM
What exactly do you mean? The current version monitors and backs up Alcatel switches in a very nice ski resort here in Switzerland for example ;)
Title: Re: NeDi 1.0.8 Progress
Post by: fblank on July 01, 2012, 08:31:26 PM
Not necessarily an absolute must but certainly worth to think about would be a possibility to suppress certain monitoring messages on a device (type) level.
The event list can quickly grow substantially when running several Nedi threads say on a half-hourly base with more than 1000 devices.
Another goodie would be the ability to define monitoring thresholds based on device type: An UPS device doesn’t need much free memory whereas a switch/router definitely does  :)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 02, 2012, 12:14:17 AM
Which events would you like to supress? In Monitoring-Setup you can add regexps for Events-Delete (next to the bell icon) in order to discard syslog and traps. Would that cover your needs?

Good point about more specific thresholds. I'm actually thinking about making this more flexible...
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on July 05, 2012, 01:37:44 PM
  • Wlan controller support (mainly HP-MSM, Cisco and Aruba)
Just out of curiosity? Is this topic still on the list? This would be one of the best features, especially with mixed networks.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 05, 2012, 06:22:37 PM
Yes it is and you actually provided a part to the solution with Aurba APs! AFAIK they don't even support LLDP, right? I have access to MSM and might get connected to a Cisco controller. Maybe Harry can help me with Aruba again. The art would be to map the wireless nodes to the APs (assuming they're in devices)...

I plan to get IPv6 ready and do some cleanup for a alpha release. Then I'd use the time to get the above in, while fixing the reported bugs...
Title: Re: NeDi 1.0.8 Progress
Post by: harry on July 06, 2012, 01:43:07 AM
Sure I can,
I am on a week leave from Monday.
Let me know ASAP if you want to do it in next week so taht I can set up in my Home netwrok and give access to you.
otherwaire any time is good for me, just let me know.
-Harry
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on July 06, 2012, 08:29:10 AM
I have access to MSM and might get connected to a Cisco controller. Maybe Harry can help me with Aruba again.
I might also be able to support testing with our labs. Just need to check how to get clients in ...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 06, 2012, 07:40:43 PM
Great! But let me get IPv6 out of the way first and I'll pick that topic up next...
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on July 11, 2012, 04:11:03 PM
Great! But let me get IPv6 out of the way first and I'll pick that topic up next...
Both features are more than welcome ;-)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 11, 2012, 07:11:26 PM
Ok, making progress on at least one of them. For your eyes only  ;)
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on July 11, 2012, 10:50:49 PM
Question: How are you storing the IPv6 address' ?
I've started to learn php myself and the best way I've found so far was using inet_pton(), but it can be hard to use when using a framework like kohana.

Great work, looking forward to the next version!

Mark.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 11, 2012, 10:55:49 PM
varbinary(16) but I'm not sure if it'll work out. Apparently MySQL 5.6 (http://dev.mysql.com/doc/refman/5.6/en/miscellaneous-functions.html) has new functions (similar to PostgreSQL). I'm not sure how to search those fields now (e.g. partial address? I have no idea how to use CIDR notation etc.). This will be so much better with proper DB support  :(
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on July 11, 2012, 11:02:59 PM
Yeah thats how I've been using it also.
varbinary(16) in mysql, keeps both ipv4/v6 addresses with php inet_pton() func.

I agree about the DB support. (It's such a nightmare otherwise).

Cheers
Mark.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 12, 2012, 06:38:18 PM
So how do you search for a partial IPv6 address (e.g. like or regexp)? inet_pton() only seems to work on complete addresses :(

EDIT: Hmm, actually no, regexp even seems to work :D
EDIT2: Not always, will need some further testing...
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on July 13, 2012, 04:42:51 PM
Great to see, IPv6 will be coming. I also like the design updates.

Maybe one additional idea: would it be ok to store the ssh timeouts in nedi.conf? Some devices take extremely long to build the config (had to set the timeout to +40). Don't know if this is necessary, since other discoveries should not be slower when the device answers faster, right? Maybe 40 as default does not make my runtimes longer ... I'll check that next week.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 13, 2012, 09:10:22 PM
I've added OS specific timeouts in libcli. Hope that will do the trick...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 22, 2012, 09:35:13 PM
 How about a nifty little "tool", accessible by everyone (just like Mobile-Health). This would allow for a quick network check on any client in your network...could even be done by the regular office user. It'll probably make a nice web-widget as well...
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on July 22, 2012, 11:30:30 PM
Pretty cool.
Could it display switch and port it is conencted to? Or is that (Home E) and port 5 on vlan 1 in the pic?

Handy tool possibly for troubleshooting.
Title: Re: NeDi 1.0.8 Progress
Post by: harry on July 23, 2012, 03:19:31 AM
Good Idea.
Can be very helpful for Support officer.
Eagerly waiting for bita ver.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 23, 2012, 07:51:56 PM
Yepp, switch 2520 (it's in my Home on the E floor) and the notebook ikarus is connected to port 5 (I was thinking about a more graphical representation). You get IF speed & duplex as well as the traffic stats below...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 28, 2012, 09:56:28 PM
Not really intended, but NeDi supports DD-WRT routers now. I still want to properly support enterprise Wlan solutions, but I just couldn't resist (for peronsal use...)


Title: Re: NeDi 1.0.8 Progress
Post by: harry on July 29, 2012, 08:53:06 AM
This is very good.
How about HP and MSM and.....Aruba...? I know you are too busy.....just a gentle reminder that I can provide the Aruba setup whenever you want to test.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on July 29, 2012, 12:16:44 PM
I'll let you know. The tricky part is to integrate those different solutions with a coherent look and feel. A lot changed in the MSM software and once I got this reflected, I'll look at the others...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 02, 2012, 01:55:01 PM
Making good progress. The latest change is a more efficient way to write the nodes tables. This should have a big impact on large installation (e.g. 100k nodes) and especially when running several discovery threads. Then not all nodes have to be rewritten with every thread, but really only those which are actually updated...

I also include the vlanid in the node MAC address field now. This only matters for IVL enabled vlans (see useivl in nedi.conf). It's hard to foresee problems, since I cannot test this thoroughly. So let me know, if you find any issues...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on September 03, 2012, 07:53:26 AM
Hi Remo, only a litte question: if you write "So let me know, if you find any issues..." you mean that a 1.0.8 (beta) is available to test.

So, where I can download it???  :)

Or only to beta testers?  :(

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on September 03, 2012, 10:59:39 PM
haha, I wondered the same thing. :)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 04, 2012, 12:09:40 AM
Not quite, but beta is not far now! Going to San Francisco this Friday, but I consider making a test version available before I leave. It won't contain proper wifi-controller support yet, but lets you play with all the other stuff...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 04, 2012, 08:13:33 PM
An important step for supporting WiFi controllers and apparently loadbalancers, which create lots of virtual interfaces. I've added an IF index range to the .def. The SNMP library will only retrieve indexes from start to end (or all if not set)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 06, 2012, 07:27:51 PM
Ok, as promised: http://www.nedi.ch/pub/nedi-250.tgz

Easiest way for testing. Get a NeDiO7 VM up and running and update with System-Files (the Readme is displayed afterwards!). Then use System-NeDi to initialise the DB (use root/dbpa55).

Have a look at Defgen if you want to add IPv6 support. There are new options for ARP/ND (I recommend only setting this on proper L3 devices) and IFIPs. Both can be tested with the walk icons next to it. You might notice some requiring a combination of two MIBs to gather IPv4 and IPv6 info, which can be selected with the appropriate option...

Try http://nedio7/me.php for a quick port check

As mentioned before, Wlan controllers are not supported yet. Next thing on my todo-list...

System-Files shows news structure for maps, fotos and tftp files. The latter can also be generated now by clicking on the star, or by selecting a device config (if stored as file), which will be great for auto-provisioning supported devices. If you don't know what I mean, don't worry tutorial will follow...

All lists feature limits and the last map option now. In addition they support XLS export (works with Excel, but probably not all other spreadsheet apps as it's HTML based).

Have fun, I'll be back in 10 days...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on September 07, 2012, 08:23:56 AM
I'll wait until you come back....

 :(

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 07, 2012, 10:18:33 AM
Ok, last post before I leave :) Check max upload file size and max post size being at least 8Mb in php.ini. Also /var/nedi needs to have write permissions set for the www user. You can still cd to /var/nedi and extract with tar zxvf <archive>
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on September 07, 2012, 11:08:56 AM
Still not go, but don't worry, I'll try untar files...

Else I can wait, 1.0.7 is running good enough!

Good journey (work? vacancy? :) )

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on September 07, 2012, 02:13:00 PM
Ok, as promised: http://www.nedi.ch/pub/nedi-250.tgz
Thanks Remo. It's great to see NeDi is developing in such a good direction.
Just installed it on a test box and it looks good at first glance.

Attached is a backport of the new node functionality for 1.0.7. It might be useful for people with large networks and several threads for discovery. Just replace sub WriteNod with the attachment. Note: don't take WriteNod from 1.0.8, since the database structure is different (e.g. IPv6). This is the same functionality - only the new database fields of 1.0.8 were removed.
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on September 14, 2012, 10:10:23 AM
Realtime traffic:

Please put in a dropbox for the poller interface: 5secs, 20 secs and 30 secs.

Tx
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on September 24, 2012, 03:58:42 PM
Resported really in late, but I've successfully installed 1.0.8 250 on my production server (scaring!!!)
To avoid problems, from scratch, except configuration parameters (well refined by me...)

Good! Still to deep to explore any new feature!

I've seen some alternative script to launch in multithreaded mode, but I absolutely don't understand the right way!
Also I've seen something about an "agent" and collector environment, but I think that we should wait instructions from Remo ;)

Ah, Remo, do you need any report or test I'll do to refine your masterpiece?

Paolo.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on September 25, 2012, 06:18:31 PM
@red, done...expecting your donation after final release  ;)

@pc, Readme should tell you about all changes. moni2.pl was an attempt for threaded monitoring, but this is actually alpha code!
The master agent part ist not thoroughly tested either, so I'll document this, if it turns out to be usable...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on September 26, 2012, 09:54:00 AM
@pc, Readme should tell you about all changes. moni2.pl was an attempt for threaded monitoring, but this is actually alpha code!
The master agent part ist not thoroughly tested either, so I'll document this, if it turns out to be usable...

Do you need some beta (alpha) test?
If you give me enough instruction, I can try, if advisable in a parallel instance.

Anyway,  8)

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on October 04, 2012, 06:46:29 AM
@red, done...expecting your donation after final release  ;)

 ;D

Can ship you wine, caps, tshirts, polars and scarfs!!!

BTW: Do you have my mods for:
- Ciena www.ciena.com
- Mikrotik www.routerboard.com
- Radwin www.radwin.com

?
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on October 04, 2012, 08:16:24 AM
Hi Remo,
do you think you can add an "Alarm" test button, for example in User Profile page, to check both e-mail and SMS functionality (one for each)?
Else we need to wait any real alarm condition.

TIA!

P.S. in libmon.pm I've seen you commented code for SMTP SMS sending. I've tried to adjust it, copying some code from smsd part, but I'm not sure, because I absolutely don't know PERL language and programming. If I find some spare time, I'll try to look around for some other examples, and if I'll obtain a running code, I'll send you.

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 04, 2012, 11:16:10 PM
@red, Southafrican wine, mmmh that'll work! Going to Seychelles soon, will pick it up on the way back ;)

@pc, you can test it with moni.pl -t200, but I suppose a GUI feature could be added if time permits...

And here's why one shouldn't be coding close to midnight, after a full day of work (presenting a network design and working on a new cloud architecture):

my $mc = sprintf "%02x%02x%02x%02x%02x%02x",$i[12].$i[13].$i[14].$i[15].$i[16].$i[17];

Who gets it? :)
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on October 05, 2012, 08:24:16 AM
@pc, you can test it with moni.pl -t200, but I suppose a GUI feature could be added if time permits...
Code: [Select]
nedi:/var/nedi# ./moni.pl -v -t200
RMON:238 entries (dev) read from nedi.monitoring
RMON:0 entries (node) read from nedi.monitoring
RUSR:1 entries (groups & 8 AND (phone != "" OR email != "")) read from nedi.users
ALRT:0 mails from 0 events and 0 SMS sent

Anyway, don't worry, I'll wait until a real event... :)
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on October 05, 2012, 09:02:45 AM
@red, Southafrican wine, mmmh that'll work! Going to Seychelles soon, will pick it up on the way back ;)

Drop in when you are in Joburg and we'll fatten you up on wors & biltong!
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 05, 2012, 09:57:36 PM
Sounds good :)

@pc, good point, that first device needs to have sending mails enabled. I'll look at improving this...
Title: Re: NeDi 1.0.8 Progress
Post by: lukas on October 06, 2012, 09:02:17 PM
my $mc = sprintf "%02x%02x%02x%02x%02x%02x",$i[12].$i[13].$i[14].$i[15].$i[16].$i[17];

Who gets it? :)

s/[.]/,/gsmx;
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 07, 2012, 02:54:48 PM
Exactly, I was concatenating the array to a string and wondered about a really strange MAC address  :-[
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 11, 2012, 10:12:24 PM
I know you want it....I also know what you're going to ask next: Can we add servers? It's on the todo-list ;)
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on October 12, 2012, 08:16:02 AM
Slurp!!!

If you like graphical mapping, a somethig useful thing is a printable switch front panel map of ports, with some (even only one a time) info about port configuration, like assigned VLAN, description fields, etc.

Until now a colleague of me has made an Excel file who parse a CSV export of devices database, but is a rough approach, and absolutely not usable...

Anyway, very nice, Remo!!!
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 12, 2012, 09:43:21 PM
Yeh, I was thinking about adding the ports (running out of space on 48p 1RU switches), but how do you handle chassis and stacks? It's ok as long as the entity mib is used, but still needs more walks, which will slow down the discovery. On the other hand, what do you really gain out of it? Pretty pictures, ok but you know what the switch looks like for real, right? So I went for a more basic look, with quick links to the details instead...
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on October 15, 2012, 11:35:56 AM
Hi Remo,

I managed to do a test installation of the 250 beta last week.
Again you have added some very nice features! Thanks for that.

Especially I like the link to the neighbor map over the link list of a device.
Another enhancement for this feature would be that one can select a deeper level of neighbors
in the topology map. For example to show the device, the neighbors and the neighbors of the neighbors in level 2.

Do you think this is possible? (without spending too much time)

Christoph

PS: I can't wait to see the WLAN controller feature in the final 1.0.8  8)
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on October 15, 2012, 02:24:42 PM
Yes, Remo, is impossible to include too many infos in a "printable" switch front panel map.

Our idea was to document VLAN port mapping, to help operator to choose right switch port to interconnect to wall network sockets...

But is not so important! Don't worry at all!

Thanks anyway!!!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on October 16, 2012, 11:44:10 PM
I haven't completely given up on the panel, but it's just not a big priority right now...

cmesser, yes that would be nice, but is not done that easily with the current logic. Will keep it in mind too :)

Thanks to Jürgen for isolating a bug, which caused monitoring graphs to have gaps. Funny I never really saw it...and I'm running NeDi on various locations myself!

Still wrapping up a few things for beta2, but it won't be long anymore!
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on November 05, 2012, 09:28:19 PM
Just noticed Remo has posted a youtube on how to use the defgen features in nedi.
Not sure if it's been mentioned elsewhere in forums, but it's quite informative.
Take 30mins out of your day to get some handy tips on using it.
Defgen-Tutorial (http://youtu.be/bunFHB-RoUQ)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 07, 2012, 10:10:54 PM
Tx, I thought putting it on the webiste and in the newsflash here (that fading headline on the top) would be enough. Guess the hardcore users only look at new posts ;)
Title: Re: NeDi 1.0.8 Progress
Post by: ntmark on November 08, 2012, 04:44:37 AM
haha, yeah I'm a new post only guy.
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 08, 2012, 12:31:51 PM
Hi Remo, a little urgent question:

update from 1.0.8-250 to 1.0.8-309 needs database reinitialization?

If so, how can I save and then restore current database information?

I've done an update trough System-Files, but i had some trouble, old config files was not really backupped (so I'd remodified them by hand, based on my remembrance...)
and now I've a lot of alarms.

A manual device add give this errors:
Code: [Select]
DBD::mysql::st execute failed: Unknown column 'devgroup' in 'field list' at ./inc/libdb-msq.pm line 509.
DBD::mysql::st execute failed: Unknown column 'devgroup' in 'field list' at ./inc/libdb-msq.pm line 509.

Please, help!

Paolo

P.S. I know, a good rule is to ALWAYS to save at least config files.... my mistake!!!
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 08, 2012, 04:06:49 PM
Addendum: look at attachment.

May I done something wrong?

in the meanwhile I'm losting devices one by one...

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 08, 2012, 05:44:01 PM
Well, it looks like you're using the old DB scheme. I know it's cumbersome, but you need to initialize the DB. However you could have exported just the configs table and import it afterwards, as this one hasn't changed from 1.0.7.

You can check any changes in the Readme.txt file now...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 09, 2012, 08:57:46 AM
Thank Remo.

Erased all. I'm not able to do export/import...

Could you write somewhere the right procedure to export and then reimport recoverable database tables?
Particularly if a future upgrade will need again a database reinitialization.

I'm not skilled in database administration, but are you sure that tables structure is always not modifiable if not reinitialising at all?

Thanks a lot anyway!!!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 09, 2012, 09:52:45 AM
Hi Remo.
Reinitialized database and reacquired all devices...

But System-Export page now is completely messed up.

Look to attached snapshot (only a part of the screen, because of attachment size limit)

Any patch to correct it?

TIA!

paolo
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 09, 2012, 02:03:33 PM
Hi Remo, sorry to bother you again... another problem relevanto to Cacti integration. The same was in previous 250 release.
Consider that now I've correctly set all Cacti parameters...
When in Device-Graph page, I select a 3560 switch, select all ports, and then click on Add button under Cacti (traffic selected, but don't change if other choosed) it shows below ERROR: Invalid snmp version (10).
Adding "&debug=1" at the end of the URL it shows

/usr/bin/php /usr/share/cacti/cli/add_device.php --description="Switch_Name" --ip="numeric_ip_address" --template=1 --version="10" --community="Read_SNMP_Community"

(of course up here I've cleaned sensible information...)

and next

/usr/bin/php /usr/share/cacti/cli/add_graphs.php --graph-type=ds --graph-template-id=2 --host-id=ERROR: Invalid snmp version (10) --snmp-query-id=1 --snmp-query-type-id=14 --snmp-field=ifName --snmp-value="cpu"

/usr/bin/php /usr/share/cacti/cli/add_graphs.php --graph-type=ds --graph-template-id=2 --host-id=ERROR: Invalid snmp version (10) --snmp-query-id=1 --snmp-query-type-id=14 --snmp-field=ifName --snmp-value="Fa0/1"


The error seems that "10" is the binary form of 2 (that is the SNMP version supported by switches)

Instead, on 3750 switches, when clicking on Cacti Add button, nothing happens at all, appears only the usual graphs,  CPU Load, Temperature and MemIO.

P.S. but is needed or not to choose which interface or parameter I like to graph, or "all" is the default?

Thanks again in advance!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 09, 2012, 06:01:32 PM
It's actually read + writeversion * 4. I wrote Devices-Graph way before I've added any snmpwrite features and never checked back! So, good catch many tx and try this for line 59:

         $sp = ($d[2] & 3);
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on November 11, 2012, 07:29:20 AM
What is the defgen file for dd-wrt?  I want to see if I can mod it to work with ubnt.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 11, 2012, 10:43:04 AM
Unfortunately they're just using UC-Davis (1.3.6.1.4.1.8072.3.2.10)...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 11, 2012, 12:57:24 PM
Hi Remo, thanks, now is OK for Cisco 3560 (and also for 3500XL) but still trouble with 3750. I don't know why.
As previsously wrote, with Cisco 3750 nothing happens, only default NeDi graphs appears in the window.

To avoid troubles, could you add an automatism to select all switch parameters graphable by Cacti, instead of needs of select them one by one?
(I don't know if I've explained in a intelligible way...)

Paolo

Addendum: adding "&debug=1" doesn't show usual debug information... Is this useful to identify the problem? Any other tricks to use to help troubleshooting?

P.S. I was working on Sunday because our main DataCenter remained without power supply... all devices and servers to be restartede in the right sequence, and some dead switches... No Comment!

Addendum 2: about 3750: seems that I must select only device ports, and one device a time in a stack, else Cacti add fails... On 3560 and 3500XL I can select all items in the second list (with CTRL-A) and click Add, and NeDi do the job, at most with an error if already added or unmanageable item.
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on November 15, 2012, 11:32:23 AM
Hi,

I installed 1.0.8 RC (309) and ran into a problem. Don't know if it's a bug or due to a miss-configuration by me.
In the new version, the nodes are not associated to the interfaces on the switches but  to the VLAN interfaces on the core switch/router.

Seen this before (look attached pictures) ?
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 15, 2012, 04:17:11 PM
Same to me.

Noted today, looking for a Printer, and finding them on core switch and relevat VLAN.
Found real switch/port with the usual MAC table finding, switch by switch following the chain...

Remo, help!!!  ;)

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 15, 2012, 06:03:49 PM
Try following the MAC in the -v output. It's all about the ifmetric which will decide at the (BNOD: lines), where it's going to. You can also send me a -v output via email...
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on November 16, 2012, 09:01:14 AM
Remo,
a question. Will there be two link types for APs in the future? If not, will the link be between the controller and the AP or between the actual switch and the AP?
Sorry, can't test myself at the moment ...
Thanks,
raider82
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 16, 2012, 09:30:56 AM
Hi Remo, very strange situation:
I've checked where Nodes table shows the printer MAC was connected, and this moring was still on core (layer 3) device and relevat VLAN.
Then I've done a discovery on the real switch this printer is connected to (using NeDi web interface, so the commenad was "/var/nedi/nedi.pl -l100 -v -a <switch address>, and then NeDi Nodes shows the right port on the right switch.

I run every 15 minutes a "segmented" discovery, one for any site, in this case for any Group (you mean VTP Domain), statring in different minutes trying to avoid concurrent discoveries, using this type of command line:
Code: [Select]
/var/nedi/nedi.pl -vprNA "location regexp '^NE;' OR device regexp '^NE' ORDER BY devip" > /tmp/nedi.ne.lastrun 2>&1 
Is this command line (the same I use as from 1.0.7 after an advice from Steffen) the reason for missing node updates?

Another question: I'm easily able to send snmp traps of the kind "snmp trap mac-notification added - snmp trap mac-notification removed" to NeDi server, but I don't understand if NeDi is able to handle them.

Thanks in advance!
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on November 16, 2012, 01:45:58 PM
Hi,

maybe I found out. I checked the -v output.

In 1.0.8 the CLI:
Code: [Select]
show mac address-table | e CPU|Switch|Router|/.* is issued. That leads to nothing.

In 1.0.7:
Code: [Select]
show mac address-table dynamic is used, and that shows up a lot of entries.

Is there a problem in parsing the .def files?

@Remo: I always thought NeDi retrieve the MAC tables via SNMP ?!?
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on November 16, 2012, 02:07:56 PM
Hi again  ;)

I modified libcli-iopty.pm line 195 to:
Code: [Select]
$cmd{'IOS'}{'dfwd'} = 'show mac address-table | e CPU|Switch|Router'; #|/.*';                           # tx colejv
Just commented out that strange asterisk.
That seems to work at first sight, but I don't know if there are any side effects.

Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 16, 2012, 02:10:27 PM
Ah!!!

This morning I've modified thie line in libcli-iopty.pm, as noted in another thread:

You missed the trailing comma (in 1.08 309), which causes it not to work at all

Code: [Select]
$cmd{'IOS'}{'dfwd'} = 'show mac address-table | e CPU|Switch|Router|/.*';                              # tx colejv


Should be

Code: [Select]
$cmd{'IOS'}{'dfwd'} = 'show mac address-table | e CPU|Switch|Router|/.*,';                              # tx colejv


and this seems to solve the problem in 1.0.8 b309.

Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on November 16, 2012, 03:39:58 PM
OK, also fixed it.

Thanks!
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on November 16, 2012, 09:40:11 PM
@raider, you mean Aruba (the others should have links)? LLDP support is on their roadmap AFAIK. However I plan to bring back MAC based links to a certain extent. That could address this...

@cmesser, watch the brand-new CLI tutorial :) Glad, that friggin "," fixed it anyway ;)

@pc_sg, yes traps can be received. NeDiO handles them by default, but does not really decode them. Have a look at trap.pl (should be configured as trap-handler)
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on November 19, 2012, 09:05:52 AM
@raider, you mean Aruba (the others should have links)? LLDP support is on their roadmap AFAIK. However I plan to bring back MAC based links to a certain extent. That could address this...
Both. Cisco APs have Links and appear as a Node (at least in the discovery; NeDi then ignores them if set via nosnmpdev). The question is:
Will there be 2 entries in the link table per AP? 1 to the Controller, 1 to the Switch?
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on November 19, 2012, 11:09:27 AM
@pc_sg, yes traps can be received. NeDiO handles them by default, but does not really decode them. Have a look at trap.pl (should be configured as trap-handler)

Thanks Remo, but I'm not skilled enough to add needed code to trap.pl to make it able to decode "trap notification add" (and remove, if really needed) and put this info in NeDi nodes tables.
Really, this will keep nodes database realtime updated!

Thanks anyway!!!
Title: Re: NeDi 1.0.8 Progress
Post by: redpineapple on November 27, 2012, 05:52:33 AM
How about an autodelete process for monitoring?

e.g. if the device is down > 10 days then delete?
Title: Re: NeDi 1.0.8 Progress
Post by: raider82 on November 27, 2012, 03:15:32 PM
How about an autodelete process for monitoring?

e.g. if the device is down > 10 days then delete?
Just make a cron job with an SQL.
mysql <database>--batch --user=<user> --password=<password> < <sql_file>

in the sql file:
delete from monitoring where <condition>;
Title: Re: NeDi 1.0.8 Progress
Post by: Peter@Lichtenberg on November 30, 2012, 05:13:35 AM
Hello,

at my installation "Locate me IP" for NeDi 1.08-309 shows wrong traffic, errros, discards and broadcasts counters.

My patch for "Locate me IP" for NeDi 1.08-309 (<nedipath>/html/me.php) with little modifications for traffic, errros, discards, broadcasts and a little bit of debugging.

Code: [Select]
--- html/me.php 2012-11-04 13:03:07.000000000 +0100
+++ /var/nedi-1.08/html/me.php 2012-11-30 04:30:23.000000000 +0100
@@ -38,6 +38,7 @@
 <?PHP
 $link  = @DbConnect($dbhost,$dbuser,$dbpass,$dbname);
 $query = GenQuery('nodes','s','nodes.*,location,speed,duplex,pvid,dinoct,doutoct,dinerr,douterr,dindis,doutdis,dinbrc','lastseen','1',array('nodip'),array('='),array( ip2long($_SERVER[REMOTE_ADDR]) ),'','LEFT JOIN devices USING (device) LEFT JOIN interfaces USING (device,ifname)');
+if(isset($_GET['debug'])) echo "<h4>$query</h4>";
 $res   = @DbQuery($query,$link);
 if($res){
  $n   = @DbFetchRow($res);
@@ -49,7 +50,7 @@
  echo "<tr class=\"txtb\"><th class=\"imgb\" width=\"20\"><img src=\"img/16/net.png\" title=\"Network\"></th><td class=\"blu code\">".long2ip($n[1])."</td><td class=\"prp code\">".(($n[16])?inet_ntop($n[16]):'')."</td></tr>\n";
  echo "<tr class=\"txta\"><th class=\"imga\" width=\"20\"><img src=\"img/16/dev.png\" title=\"Device\"></th><td>$n[6]</td><td>$l[2] $l[3]</td></tr>\n";
  echo "<tr class=\"txtb\"><th class=\"imgb\" width=\"20\"><img src=\"img/16/port.png\" title=\"Interface\"></th><td>$n[7]</td><td>".DecFix($n[24])."-$n[25] vl$n[8]</td></tr>\n";
- echo "<tr class=\"txta\"><th class=\"imga\" width=\"20\"><img src=\"img/16/grph.png\" title=\"In/Out\"></th><td colspan=\"2\">Traffic: <b class=\"blu code\">".DecFix($n[26])."/".DecFix($n[27])."</b> Errors:<b class=\"drd code\"> ".DecFix($n[28])."/".DecFix($n[29])."</b> Discards:<b class=\"prp code\">".DecFix($n[30])."/".DecFix($n[31])." </b> Bcast:<b class=\"dgy code\"> ".DecFix($n[32])."</b></td></tr>\n";
+ echo "<tr class=\"txta\"><th class=\"imga\" width=\"20\"><img src=\"img/16/grph.png\" title=\"In/Out\"></th><td colspan=\"2\">Traffic: <b class=\"blu code\">".DecFix($n[27])."/".DecFix($n[28])."</b> Errors:<b class=\"drd code\"> ".DecFix($n[29])."/".DecFix($n[30])."</b> Discards:<b class=\"prp code\">".DecFix($n[31])."/".DecFix($n[32])." </b> Bcast:<b class=\"dgy code\"> ".DecFix($n[33])."</b></td></tr>\n";
  echo "</table>";
  }else{
  echo "<h4>$_SERVER[REMOTE_ADDR] was not found</h4>";

Regards

Peter
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on December 15, 2012, 11:07:02 AM
Forgot to adjust that one indeed, tx! I'd use field names next time, rather than indexes :)

BTW, I'm back from the Seychelles and working on finalizing 1.0.8 now...
Title: Re: NeDi 1.0.8 Progress
Post by: mpamio on January 18, 2013, 02:23:54 PM
Found a strange behavior on a Cisco WS-C3560X-48
The Arp table is not readed via SNMP.

Note that I have another WS-C3560X-24 that works fine.
It seems that the first one is slow.
snmpwalk -v2c -c #### ##.##.##.## ipNetToMedia responds in different speed.


The log is:
Arp (SNMP)   ------------------------------------------------------------------
SNMP:Connect ip.ad.dr.ess #community# v2 Tout:2s MaxMS:1472
ERR :No response from remote host "ip.ad.dr.ess"
ARPS:0 ARP entries found

Is possibile to tune the response time?

 :( :( :( This is our core L3 switch, with a lot of subnets, in such subnets the nodes are discovered as 0.0.0.0, because of "no arp entries found"  :( :( :(

So I switched back to 1.0.7 . . .. .. .
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on January 19, 2013, 01:59:04 AM
1. Do you have the same timeout values?
2. Did you look at the .def? oh8 supports IPv6, but needs your help :) Watch the Defgen tutorial on youtube, if you don't know what I mean...
Title: Re: NeDi 1.0.8 Progress
Post by: OMFan on January 28, 2013, 12:23:39 PM
Hello,

I have just tried the new Nedi Version (nedi1.0.8-027).

I have installed it from scratch.

I have modified nedi.conf for Mysql database parameters and i have run : ./nedi.pl -i

When I go to the interface, and i want to show, for example, the devices list, i have this error "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'y G:i LIMIT 200' at line 1".

I don't understand why.

Do you have any idea ?

Thanks by advance.

Bye.
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on January 28, 2013, 01:56:46 PM
Hi again Remo, I'm just updated to last "027", but still have the messed "System Export" page problem, both with IE9 and Chrome.
Have a look in attached snapshot...

Can you help me?

TIA!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on January 28, 2013, 07:15:55 PM
Can you both log out and back in again? If you haven't updated User-Profile before it should fix it...

@omfan do you seen any of the GUI? Can you click on the lock to reveal the query, if so? Looks like the dateformat got into it somehow...
Check out User-Profile and update it or add another user with User-Accounts and try with that one...

@pc-sg what's wrong with that, it's quite pretty ;) Set # of columsn in User-Profle...

In addtion I noticed a few minor thingies, but they'll cleaned up for the relase of course!
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on January 29, 2013, 08:09:21 AM

@pc-sg what's wrong with that, it's quite pretty ;) Set # of columsn in User-Profle...


Arggggghhhhhhhhhhhhh!!!!

My mistake, not correctly interpreted columns # meaning... not yet, indeed...

In the meanwhile, could you enlarge Alias field in  Device Status window? Sometimes is not large enough.
And Top VLANs is always too short even when 31 is the limit. But we use extensively VLANs, and have much more the 31 active... :) But is not important, there are other place to list them.

TIA!!!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on January 31, 2013, 07:32:00 PM
I've coupled the input fields with the labelsize now, hoping this works for everybody...

I'm limiting vlan and module lists, as devices can have a lot of both and clutter the GUI. But I've added the icons to quickly see the whole lists...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 04, 2013, 12:36:05 AM
Bummer, I was hoping not to add DB fields for a while, but those are too cool to leave out! Adding "maxpoe SMALLINT unsigned and totpoe SMALLINT unsigned" to devices, with that everyone stays on top of PoE budgets!

Thanks to all for testing the latest RC :D
Title: Re: NeDi 1.0.8 Progress
Post by: ascii on February 05, 2013, 08:46:41 AM
i just updated to new 027.
i also had to alter ther table monitoring and add a collum eventlvl after eventfwd
Code: [Select]
ALTER TABLE `monitoring` ADD `eventlvl ` TINYINT NULL DEFAULT '0' AFTER `eventfwd`
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 05, 2013, 06:18:15 PM
Yepp, that's why I suggested -I, but you seem to know your way around SQL, so that's much easier of course. :) This is for the new event and monitor handling. Works quite nicely already and will allow for granular filtering and forwarding of events (which can be traps, syslog messages or discoveries)...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on February 06, 2013, 09:33:55 AM
Hi Remo, this database modifications are needed in current in current 027 installations, or will be applied (automatically) in future builds?

A little hint: for the ones not skilled in SQL (like me), could you show how to do thing like "you’ll only need to recreate the monitoring table" as you wrote in 027 update instructions? I was forcet to reinitializa all tables   :'(

Thanks a lot!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 06, 2013, 06:19:30 PM
I know it's annoying and I'm sorry to keep doing this. I really hope those changes will be a lot less and better documented (even automated) in the future!

So far, you'll need those lines after upgrading from 027 to the final release:

alter table devices add column maxpoe SMALLINT unsigned;
alter table devices add column totpoe SMALLINT unsigned;

There's a nice alias (db) in NeDiO8 to call mysql...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 15, 2013, 12:47:35 AM
Quick update. The final release draws closer! While I'm waiting for more feedback, I managed to implement some POWER-ETHERNET mib based features, like detecting failed power supplies (at least on PoE switches) and overloaded ones...

In addition the new elevation logic, lets you send email alerts on some and completely suppress events on other devices!

Another bonus is the new Device-Status bar showing total PoE load (not related to interface PoE OIDs). You'll be able to turn PoE on and off on some (mostly HP) devices as well...

With the increasing VoIP and Wireless demands I think those additions will be well perceived ;)
Title: Re: NeDi 1.0.8 Progress
Post by: Al on February 15, 2013, 03:00:57 AM
Great work, however, one "small" request - could you (or someone skilled in SQL) please provide a SQL script to upgrade the database from 1.0.7 to 1.0.8 so we can keep our now quite historic data? I tried to do it manually from comparing the database creation in both versions but mangled it quite badly (yay for server snapshots) and after a few hours of trying in frustration thought I'd ask here.

Cheers Al
Title: Re: NeDi 1.0.8 Progress
Post by: harry on February 15, 2013, 05:46:43 AM
Hi Remo,
Can we have Aruba device port links on device status. linking back to Cisco Switches Controller and  if possible AP's.

The me.php is giving me wrong result. I could not found the problem. can you plase have a look?

Regards and Good luck
Harry
Title: Re: NeDi 1.0.8 Progress
Post by: ascii on February 15, 2013, 08:29:09 AM
i found 2 smaller bugs.

Snap1:
when i go to Devices->Moduel and select inventory as Report.
the Software and Firmware output for the modules/Supervisor is wrong. It needs to be switched

Snap2:
when i take a closer look at a node the output for device is a little bit of.
the icons indicate Location and Contact. but it shows twice the location.
i just deleted the node and rediscoverd it to see if it's just off somewhere in the database from an old scan.


EDIT
i would have 1 more request/suggestion.
when i take a look at a device i see all the port with description, vlan, etc.
i would love to see if these port is a trunk and what vlans are on it.
on cisco devices i get vlan 0 in the vlanid field.
maybe it would be possible to have an extra field with the tagged vlans
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on February 15, 2013, 09:25:43 AM
Not really important: I've found that some .def point to same "Type"

1.3.6.1.4.1.11.2.3.7.11.34.def
1.3.6.1.4.1.11.2.3.7.11.36.def
1.3.6.1.4.1.11.2.3.7.11.44.def
1.3.6.1.4.1.11.2.3.7.11.45.def
1.3.6.1.4.1.11.2.3.7.11.82.def

all shows HP-2626.

I've a temporary switch replacement of a failed one, is an HP, model E2650B, and looking to HP MIBs expressly downloaded from HP support site, I've found that his OID is 1.3.6.1.4.1.11.2.3.7.11.44.

File: hpicfOid.mib

the lines that shows this:

Code: [Select]
    hpSwitchJ4899B OBJECT-IDENTITY
        STATUS      current
        DESCRIPTION "The authoritative identifier for the
                    J4899B HP E2650B Switch."
        ::= { hpEtherSwitch 44 }

This file may be useful to adjust .def files for other HP switches.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 18, 2013, 07:21:09 PM
@Al, I really do want to get better at this and provide a script to have the DB updated automagically. Unfortunately the changes from oh7 are too many for me to backtrack. On the upside, most of the info will be discovered anyway. Stock for example hash't changed at all and could be carried over via export and import functions...

@Harry, I have an idea, how to fix the wrong port, thanks for bringing this up. But for the links to the APs we'll have to wait until Aruba adds LLDP support...

@Ascii, you're correct on both findings, tx! Unfortunately the FW/SW issue is a result of wrong .defs! I'll try to fix them!
All Vlans per port is still kind of on the todo-list, but I need to investigate how to cover this for all supported devices.

@pc_sg, I remember vaguely not being able to identify them properly. I'll try to match them accordingly...

To all, I know I stated that I'd release before my trip to Houston, but there's a problem :) I really want proper Nexus support, but this requires some more testing and I simply don't have the time right now!
Title: Re: NeDi 1.0.8 Progress
Post by: Al on February 20, 2013, 09:45:39 PM
Thanks Rick, I find the last-used info per port extremely useful so if that can't be migrated, I might build a new box and keep the old one running for a month or two until the new one has built up a history. Cheers Al
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 21, 2013, 07:59:20 PM
Ok, good you bring this up....it's noted for a showcase in youtube :)

How exactly do you check it? There are several options and with 1.0.8 you can even track the date when the last interface status change was detected. So you'll even get more ways to check!

And on the 2nd picture you'll see the reason why you'd want to delete all if graphs...yeah now you can look back for if changes (as long as it happened within a discovery interval)
Title: Re: NeDi 1.0.8 Progress
Post by: Al on February 24, 2013, 10:36:16 PM
How do I check for unused ports? Pretty basic really, someone from the Helpdesk comes and asks for a spare port in a building / floor. I search for switches in that floor, click on each one until I see one with ~100 days plus of unused and they can use it. The "days since last used" is probably more helpful than a "date last used" (is that a field I can choose still?) in the display as it's pretty easy to eyeball (based on a number) the oldest unused port to give out, knowing that I'm probably not going to piss some lecturer who's come back from sabbatical to find no connection for their machine.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on February 27, 2013, 04:09:27 AM
Depends where you look. In Devices-Status it's actually fetched realtime from the switch. So it's not even stored in the DB. If the traffic field has a blue background, it means there was traffic on this port, even if they're 0 now and the link is down. Now you loose this "view to the past" when the device is rebooted. That's why I've added the lastchg field to the interface table. This lets you check the used/patched interfaces, if the device is down...
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on March 25, 2013, 03:01:25 PM
Hi Remo,

in another thread you wrote:
Yes leave it off, the master daemon collects alerts from nedi agents, but it's not officially documented yet. Will need to do some testing...

How is the road-map here? Will the master service only collect alerts or also device and node data from "NeDi-probes"
in other networks?
I'm very interested in a distributed NeDi setup, because we think about implementing it globally.
So I wold be very pleased about a short docu and your thoughts about pushing NeDi to "enterprise-grade".

Regards Christoph
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on March 25, 2013, 07:25:30 PM
Only events level > 150 and incidents for now. On the master NeDi adjust agentlist, only enable monitorin-master, events and incidents in nedi.conf  and run the master.pl (e.g. from System-Services). No need to change anything on the agent NeDies. The master will poll through the webinterface...

Update: I have made some improvements and will provide a tutorial with oh8 final...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on April 09, 2013, 02:32:03 PM
Hi Remo,
two little questions:

(Always) Thanks in advance!

Paolo
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 09, 2013, 05:36:01 PM
Emails where broken in 027 (that's why I still haven't released) and yes stack changes can be detected with "M" in notify, because it's effectively a module change...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on April 10, 2013, 04:09:04 PM
Emails where broken in 027 (that's why I still haven't released) and yes stack changes can be detected with "M" in notify, because it's effectively a module change...
Any way to restore emails, even with a patch/old version?
Or should I wait until 1.0.8 release?

:)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 13, 2013, 09:25:24 PM
Right! After countless hours of testing, fixing and testing again to find what I broke this time, I'd like you to look at this :)
http://www.nedi.ch/pub/nedi-103.tgz

Let me know, how it works...
Title: Re: NeDi 1.0.8 Progress
Post by: cmesser on April 15, 2013, 05:24:37 PM
Hi,

tried to install 103!

I noticed that there are two new fields in the devices table called "maxpoe" and "totpoe" of type SMALLINT.
Are there any other changes that I have to add in order to go on with the existing database?

Christoph
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 15, 2013, 06:21:39 PM
It's described in the Readme.txt. You should be able to update the DB without init via System-Export...
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 25, 2013, 09:49:45 PM
Ok faithful forum visitors. 1.0.8 is on the website for your early access pleasure ;) I'll announce it on the official website on the weekend (unless someone reports a major foppa). I've also added addstatus2ifrrd.pl in the contrib directory, which modifies 1.0.7's interface RRDs to include the status. Note that nedi.pl -i will keep RRDs now (-I won't)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on April 26, 2013, 06:14:23 PM
 >:( I realized today that OUI/IAB files changed the format. As a result nodes' NIC vendors are not recognised properly. This will be fixed in nedi-1.0.8-116 (archive will still be nedi-1.0.8.tgz)...

BTW, thanks for "Prometheus", whoever that was  :D
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 02, 2013, 10:35:33 AM
Good news!

Upgraded from  build 103 to build 116.
So no need of database init (at least not indicated in readme).

But something is not ok, at least in monitoring. See attachment.

Should I do something more?

Thanks!

P.S. instructions in addstatus2ifrrd.pl are not clear. I use Debian, so apt-get are ok and modules were installed smoothly, but no Makefile.pl exist at all, in every directory on my server... so I'm not able to "make"...


Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 02, 2013, 06:27:39 PM
Looks like toprrd is not getting executed with every discovery or rrdstep is not matching your crontab schedule. grep for TRRD in the output files...

Admittetly the instructions for building the RRD:Simple dependencies are rather short:

1. apt-get install libtest-deep-perl libtest-pod-perl libtest-pod-coverage-perl
2. Download http://search.cpan.org/~nicolaw/RRD-Simple-1.44/lib/RRD/Simple.pm and cd into the extracted dir
3. perl Makefile.pl
4. make
5. make install
6. cd /var/nedi
7. contrib/addstatus2ifrrd.pl

But you've been following the RCs, so don't have have the status in the RRDs alread? :)
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 03, 2013, 11:19:19 AM
Looks like toprrd is not getting executed with every discovery or rrdstep is not matching your crontab schedule. grep for TRRD in the output files...

Admittetly the instructions for building the RRD:Simple dependencies are rather short:

1. apt-get install libtest-deep-perl libtest-pod-perl libtest-pod-coverage-perl
2. Download http://search.cpan.org/~nicolaw/RRD-Simple-1.44/lib/RRD/Simple.pm and cd into the extracted dir
3. perl Makefile.pl
4. make
5. make install
6. cd /var/nedi
7. contrib/addstatus2ifrrd.pl

But you've been following the RCs, so don't have have the status in the RRDs alread? :)

Thanks!

I suppose line 2. should be
Download http://www.cpan.org/authors/id/N/NI/NICOLAW/RRD-Simple-1.44.tar.gz somewhere, extract with "tar xzpf RRD-Simple-1.44.tar.gz" ,cd into the extracted dir

Anyway, all now is Make and done well...
...except that script retuns an error (I put "omissis" is in place of the real name of the first switch in my NeDi list):

nedi:/var/nedi# contrib/addstatus2ifrrd.pl
rrd/omissis -----------
Failed to add new data source 'status' to RRD file 'rrd/omissis/Gi1%2F0%2F1.rrd': new RRD file '/tmp/pBPYPpA8n3' does not contain expected data source names at contrib/addstatus2ifrrd.pl line 34
Processing rrd/omissis/Gi1%2F0%2F1.rrd...

and it stops.

About toprrd, is executed on some discoveries. I see some lines without errors on some logs. My scheduling is one every 15 minutes, but is partially parallelized, I have launch one discovery for each site (9, of different size), spaced by one minute each other. The bigger sites keeps more time to finish (up to 10 minutes), the smaller less (even less than a minute for some). So some have the TRRD lies at the and of logs, some not.
I don't know if there is another way to parallelize discoveries in the right and fastest way.
Up to 1.0.7, a single discovery of all devices in all sites was too slow (up to more than an hour). But indeed, WAN bandwidths were much less than current ones...

Any advice? :)

Thanks again.
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 03, 2013, 08:08:55 PM
 :o  :-[

line 32 should read:
         unless( $rrdfl eq 'system.rrd' ){

Sorry, my bad...

But the error might indicate you have that status DS already. You can check with rrdtool info rrd/omissis/Gi1%2F0%2F1.rrd and if you see lines starting with ds[status]. you'll be ok.

Running them every 15 minutes, when some take 10 can be tight. I'd go for a 20 or even 30 minute interval (rrdstep 1200 or 1800 respectively) to be safe. You can still try 15, but then you need to make sure, that you start the longest running ones first. TopRRDs should be drawn by the last one to finish.

In your current setup, do you ever see nedi not running in System-Services (and 0 threads)?
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 07, 2013, 04:10:06 PM
:o  :-[

line 32 should read:
         unless( $rrdfl eq 'system.rrd' ){

Sorry, my bad...

But the error might indicate you have that status DS already. You can check with rrdtool info rrd/omissis/Gi1%2F0%2F1.rrd and if you see lines starting with ds[status]. you'll be ok.

Running them every 15 minutes, when some take 10 can be tight. I'd go for a 20 or even 30 minute interval (rrdstep 1200 or 1800 respectively) to be safe. You can still try 15, but then you need to make sure, that you start the longest running ones first. TopRRDs should be drawn by the last one to finish.

In your current setup, do you ever see nedi not running in System-Services (and 0 threads)?

Modded, same error. May be not important... :)

I've checked some logs (found a way to have 24 hours of logs, a lot but rotating), and the slowest discovery takes 7 minutes (once I've seen 10, I don't know why).
Anyway, 3 minute before first scheduled discovery the icon is grey, and no nedi thread is running.

Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 07, 2013, 09:23:42 PM
Yeh, posibly you've got the status already in there...

So, if Threads finish within 15m you're good! Is rrdstep 900 in nedi.conf? Additionally you can grep for TRRD in the output, to look for missing ones (should really occur in the same thread everytime)...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 08, 2013, 09:53:56 AM
Yes, rrdstep is of course 900, and currently the situation seem more stable even not perfect). Yesterday I've removed a site (migrated inside another site not under my management), and this seed up discoveries (no fake failing discovery).

As you can see in attachment, except near midnight, RRDTOPs are generated correctly (hope from now on).

The difference ad midnight is that at midnight I've scheduled a minimized (but still threaded) discovery used to backup device configs.
The command line is similar to
Code: [Select]
/var/nedi/nedi.pl -vpN -B0 -SarfsgmitedbpowmAO -A "location regexp '^XX;' OR device regexp '^XX' ORDER BY devip" > /tmp/nedi.xx.bck.`date +%H-%M` 2>&1(XX is a two character Site identifier)
Then at 15, 30 and 45 minutes is schedule a standard discovery
Code: [Select]
/var/nedi/nedi.pl -vpNA "location regexp '^XX;' OR device regexp '^XX' ORDER BY devip" > /tmp/nedi.xx.`date +%H-%M` 2>&1and then every hour, except 0:00,at 0, 15, 30 and 45 minutes, the same command line as above.

I don't know if there be a better mode to do all this tasks in a more complete and fastest way...
Any advice is welcome!

P.S. sometimes I receive further fake module modification alerts.

Thanks!
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 08, 2013, 05:05:02 PM
Wow, good job! I guess you could remove o from skip or let the config backup thread run entirely parallel and create another one for the usual discovery stuff...

As to the modules issue. Do you see any failed SNMPwalks? I really can't explain what else would be going on there...
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on May 09, 2013, 08:48:33 AM
Wow, good job! I guess you could remove o from skip or let the config backup thread run entirely parallel and create another one for the usual discovery stuff...
Done, now I'll wait... :)

As to the modules issue. Do you see any failed SNMPwalks? I really can't explain what else would be going on there...
I suppose this lines:
Code: [Select]
Modules      ------------------------------------------------------------------
SNMP:Connect ---cut--- v2 Tout:5s MaxMS:1472
MOD :Walking module description
MOD :Walking module class
MOD :Walking module HW
MOD :Walking module SW
MOD :Walking module FW
MOD :Walking module SN
MOD :Walking module model
MOD+:26-GigabitEthernet0/1 - unknown Gigabit Ethernet
MOD :24-FastEthernet0/23 class 10 !~ /9/  100BaseTX/FX Fast Ethernet
MOD :13-FastEthernet0/12 class 10 !~ /9/  100BaseTX/FX Fast Ethernet
MOD :14-FastEthernet0/13 class 10 !~ /9/  100BaseTX/FX Fast Ethernet
MOD :11-FastEthernet0/10 class 10 !~ /9/  100BaseTX/FX Fast Ethernet

... and so forth ...

Not so light... what text should I espect? Any trick to grep it easily ?

Anyway, I think NeDi should ignore failed SNMP returns. Or is it possible that a SNMP query returns garbage?
Any idea on how to save somewhere any debug info useful to debug this issue?
Else, I like very much more a way to generate a distingushed alert when a switch stack member fails. In my mind, is a higher emergency that a single module fail, but for other may be different, so don't care :) ;) :)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 09, 2013, 11:32:52 AM
I'd create the -v output files with names matching their start time. Then you can easily compare them, when such an event occurs or mail them to me...
Title: Re: NeDi 1.0.8 Progress
Post by: eviltrooper on May 15, 2013, 02:22:10 PM
hi,

i just init new Database.. and i want to reimport my users database

i do it like this.

export:
Code: [Select]
mysqldump nedi monitoring user[b]s[/b] stock -u nedi -p > nedi-mon-usr.sql
works good! no error so long.

import

Code: [Select]
cat nedi-mon-usr.sql |mysql -u nedi -p nedi.
no error..

but i cant login it always says
Quote
Incorrect login!

what can i do?


EDIT:

see it in another Thread.. i dont import my sql so i make new users^^
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 15, 2013, 11:42:23 PM
Unfortunately monitoring and user tables changed quite a bit. The reason you cannot login is the passwords. However you could reset them by clicking on the key before loging out...
Title: Re: NeDi 1.0.8 Progress
Post by: eviltrooper on May 22, 2013, 02:16:27 PM
Hello,

what does this mean?

TRRD:No successful check at all, is moni running?


Code: [Select]
root@wetinfo:/tmp# ps -efa
UID        PID  PPID  C STIME TTY          TIME CMD   
root     16430     1  0 May15 ?        00:00:04 /usr/bin/perl ./moni.pl -D

i started moni.pl from the CLI
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 24, 2013, 06:46:49 PM
Are any targets actually checked (e.g. uptime or ping)?
Title: Re: NeDi 1.0.8 Progress
Post by: eviltrooper on May 27, 2013, 08:55:09 AM
i checked uptime with snmp.. but no ping
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on May 28, 2013, 03:52:45 PM
Ok, but is anything actually up? :) Not sure, why else you'd get this message...
Title: Re: NeDi 1.0.8 Progress
Post by: dobst on June 12, 2013, 03:21:22 PM
Finally I updated from 1.0.7 to 1.0.8, an imported the nodes table. Everything works fine and the db is running faster. During the installation I noticed these things, maybe they can be fixed in further versions:

1. nedi.conf changed in 1.0.8 but the version is commented with '# NeDi 1.0.7 configuration file' - a bit confusing :-/
2. file structure: all files (binaries, scripts and data) are stored in /var per default. Especially rrd can get very big (30 gig in my set up). I recommend to put
on different partitions/volume groups. At the moment I solved the problem with links.
Title: Re: NeDi 1.0.8 Progress
Post by: pc_sg on June 12, 2013, 03:40:16 PM
Finally I updated from 1.0.7 to 1.0.8, an imported the nodes table. Everything works fine and the db is running faster. During the installation I noticed these things, maybe they can be fixed in further versions:

1. nedi.conf changed in 1.0.8 but the version is commented with '# NeDi 1.0.7 configuration file' - a bit confusing :-/
2. file structure: all files (binaries, scripts and data) are stored in /var per default. Especially rrd can get very big (30 gig in my set up). I recommend to put
  • binaries to /opt/nedi and
  • data to /var/nedi
on different partitions/volume groups. At the moment I solved the problem with links.

1. +1 :)

2. depends on your server partition structure. In mine there is only one partition (all are virtual machines, not created nor HW managed by me).
Anyway a bit skilled *nix server manager can add a new partition/volume using an approriate mount point, like using nedi folder under /var, so all NeDi files (except databases) will have a dedicated disk space.
My two cents... ;)

Paolo

Title: Re: NeDi 1.0.8 Progress
Post by: Hannu Liljemark on August 08, 2013, 09:31:02 AM
Note that nedi.pl -i will keep RRDs now (-I won't)

1.0.8-116 warns "DB, RRDs and configs files will be cleared" if you run it with -i... maybe for next version that message could be made a bit more clear? :)
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on August 08, 2013, 08:40:18 PM
True, missed that one :) tx
Title: Re: NeDi 1.0.8 Progress
Post by: Hannu Liljemark on August 09, 2013, 11:30:09 AM
What's the deal with top.rrd and system.rrd after 1.0.7->1.0.8 upgrade? I've updated now four NeDi environments and did the addstatus2ifrrd.pl magic to the interface rrd files. However system.rrd and top.rrd files stop updating so I've deleted them to get them recreated (no big deal). There's still few 1.0.7 sites that I need to update. Removal of old top.rrd and system.rrd files is necessary or am I doing something wrong?

Thanks.

Br,
Hannu
Title: Re: NeDi 1.0.8 Progress
Post by: rickli on August 12, 2013, 10:10:11 PM
I think it was global discards. For the IF RRD there's addstatus2ifrrd.pl in contrib, but I dunno if it works in every case...