NeDi Community

NeDi General => Other => Topic started by: acoto on February 25, 2012, 02:36:50 PM

Title: NeDi2GrapML
Post by: acoto on February 25, 2012, 02:36:50 PM
Hello NeDi Community,


Some time ago Dave Wapstra, a friend from Cisco created a perl script to convert a text file containing CDP details into a network Schematic in GraphML format.

On last week I rewrote part of the script and created NeDi2GraphML.pl

NeDi2GraphML.pl reads some tables from MySQL NeDi database, and optionaly some some control files and then generates a detailed network schematic in GraphML format

yEd from yWorks is an excellent free application to view/edit GraphML.

I placed the script, some helping files and screenshots in:

http://sourceforge.net/projects/nedi2graphml/ (http://sourceforge.net/projects/nedi2graphml/)

Attached a couple of schematics from my lab.


Please take a look.

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: pc_sg on February 27, 2012, 08:53:26 AM
Very interesting!

Could you give us (at least me) some more hints (or a procedure) on how to "install and configure" NeDi2GraphML in a running NeDi environment?

TIA!

Paolo
Title: Re: NeDi2GrapML
Post by: acoto on February 27, 2012, 02:56:42 PM
Hello Paolo,

Well, letīs see.

Unzip the file on a working directory on the same machine running NeDi.

The folder:  shall contains
  - NeDi2GraphML.pl: the script file.
  - icons.csv: a sample icon reference file
  - groups.csv: a sample groups reference file
  - geo.csv: a sample (x,y) position reference file.
  - icosns: a folder containing some icons in SVG format


You have to modify next lines at the beginning of the script:
  my $mysql_db="nedi";                   # mysql database
  my $mysql_user="nedi";                 # mysql username
  my $mysql_pass="dbpa55";               # mysql password
  my $mysql_host="localhost";            # mysql host


and place the correct credentials.

Now, we are ready to go. A simple run:

  perl NeDi2GraphML.pl -o File.GraphML

should create a simple schematic. using circles for routers, squares for switchs and triangles for endpoints. Open it in yEd.

At this point all devices are overlapped at position (0,0), not fancy at all. Experiment a bit with yEd Automatic Layout tools, I bet you will be happy soon... Try the Layout Organic :)

Regards,

Adrian


Title: Re: NeDi2GrapML
Post by: acoto on February 27, 2012, 03:26:27 PM
Now,

For a better looking schematic, we can add SVG icons!

The option "--icn icons.csv" instructs NeDi2GrapML to use the references in "icons.csv" and add icons to schematic.

"icons.csv" is a comma separated values file, with the next format per line:

#   icon,<type>,<regex>,<link>,<width>,<weight>,<boxcolor>
#
#      <type>:  "cdpsvc" - CDP announced services: "Router" "Switch" "Bridge"...
#               "devtype" - Device Type "Cisco-3640" "Cisco-3845"...
#               "devname" - Specific device Name "RCore001" "RAccess001" "SWDCT..."
#     <regex>:  matching regular Expression
#      <link>:  file location
#     <width>:  icon width
#    <height>:  icon height     
#  <boxcolor>:  color used when no icon, or zoom out


NeDi2GraphML will follow next type priority: devname > devtype > cdpsvc

An Example File:

icon,cdpsvc,Router,./icons/cisco/blue/router.svg,32,24,#0065A2
icon,cdpsvc,Bridge,./icons/cisco/blue/switch.svg,36,24,#0065A2
icon,devtype,^Node,./icons/cisco/grey/generic-pc.svg,20,20,#707070
icon,devtype,^Hub,./icons/cisco/cyan/hub-small.svg,32,24,#00B38A
icon,devtype,^Cisco-364,./icons/cisco/blue/router.svg,32,24,#0065A2
icon,devtype,^Cisco-720,./icons/cisco/blue/router-mpls.svg,32,24,#0065A2
icon,devname,^RCore,./icons/cisco/red/router-mpls-p.svg,48,48,#CC092F
icon,devname,^RDist,./icons/cisco/orange2/router-mpls-pe.svg,48,40,#FF5A00


The first two lines will place generic router and switch icons to devices that match "Router" or "Bridge" capabilities on CDP.

The next four lines will add icons, to specific device types "Node" "Hub" "Cisco-364x" and "Cisco-720x".

The final two lines, will add icons to specific device names.

The last column is a bit cryptic. When zooming out in yEd, at some point it will replace the SVG icon for an squared box. That column tells yEd to use an specific color for the box.

Now, after we modified icons.csv file to our specific needs, we can proceed:

perl NeDi2GraphML.pl -o NiceSchematic.graphml --icn icons.csv

Get the file. Have fun!

A note:

  NeDi2Graph.pl may add some "Hubs" when it thinks the real topology has them.

A couple extra options:

"-n" will add NeDi nodes (aka endpoints) (be careful, may be slow and memory consuming)
"--edgelabels full" will add labels containing interface names

perl NeDi2GraphML.pl -o NSEdgeLabels.graphml -n --edgelabels full --icn icons.csv

Title: Re: NeDi2GrapML
Post by: pc_sg on February 27, 2012, 03:53:06 PM
Thanks!
 I've just done a first rough test and (with minimal command line) result is interesting!
Without your note about "all icon collapsed" the shown images was unusable...
This confirm that a short "how to" is useful!
Yes, I agree, Organic is one of the best arrangement.

I've seen the strange (and fake) HUB connection. I supposed that is related to an irregular behaviour of old 3500XL switches (that CiscoWorks show connected by a backbone, instead that cascading as really are).
But this doesn't happen in every 3500XL stack. And I see some 3500XL switches in another stack completely disconnected.

May depend on another reason...

Anyway, I'll play some more on this nite addon!

Thanks again!

Paolo
Title: Re: NeDi2GrapML
Post by: pc_sg on February 27, 2012, 04:08:36 PM
May be first bug?

Using --icn and both your original example icons.csv, or a new one with the exact line from you previous message returns this (second case):

Code: [Select]
Useless use of sort in void context at ./NeDi2GraphML.pl line 1011.
Useless use of sort in void context at ./NeDi2GraphML.pl line 1155.
--------------------------------------------------
Enabled Options:
  + Output file: NiceSchematic.graphml
  + Icon file: ./icone.csv
--------------------------------------------------
Loading Icons
Connecting Database
Processing Devices Table
Processing Links Table
Hunting Hubs
Use of uninitialized value $i in array element at ./NeDi2GraphML.pl line 777.
Use of uninitialized value $i in array element at ./NeDi2GraphML.pl line 777.
Use of uninitialized value $i in array element at ./NeDi2GraphML.pl line 777.
Use of uninitialized value $i in array element at ./NeDi2GraphML.pl line 777.
Use of uninitialized value $i in array element at ./NeDi2GraphML.pl line 777.
Attempt to free unreferenced scalar: SV 0xa0e5590, Perl interpreter: 0x9e4e008 at ./NeDi2GraphML.pl line 786.
Attempt to free unreferenced scalar: SV 0xa0e5590, Perl interpreter: 0x9e4e008 at ./NeDi2GraphML.pl line 788.
Segmentation fault

debug switch doesn't add any more significant (for me)

May I missed something?

TIA!

Paolo
Title: Re: NeDi2GrapML
Post by: acoto on February 27, 2012, 04:25:45 PM
Hello Paolo,

Yep, looks like a beginner mistake on that lines... (nested for with the same variable name... doh!...)

Will post a correction asap.

I wonder why I never get that on my lab...


AC

Title: Re: NeDi2GrapML
Post by: pc_sg on February 27, 2012, 04:28:27 PM
Fine!

I'll wait!

Paolo

P.S. seen your PM.
Title: Re: NeDi2GrapML
Post by: acoto on February 27, 2012, 04:38:12 PM
NeDi2GraphML v0.7 uploaded!

Please try again with this new version.

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: acoto on February 27, 2012, 05:01:36 PM
Second Update Today NeDi2GraphML v0.8
  Corrected another bug on node description links list

AC
Title: Re: NeDi2GrapML
Post by: pc_sg on February 28, 2012, 09:23:49 AM
Hi Adrian, v0.8 runs fine. No more errors.

Is very strange the behaviour on 3500XL switches stacks. Look at attached snapshot. Is a stack of 6 switches, but is shown as 4 connected (through a fake hub) and another 2 connected each other (ever through a fake hub), but separed from other 4.

This is the "show cdp neighbors" output of one of the "alone switches"
Code: [Select]
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
NENS0051.sas.itn Gig 0/2            157         T S       WS-C3524-XGig 0/2
NENS0052.sas.itn Gig 0/2            151         T S       WS-C3524-XGig 0/2
NENS0055.sas.itn Gig 0/2            149         T S       WS-C3524-XGig 0/2
NENS0054.sas.itn Gig 0/2            122         T S       WS-C3524-XGig 0/2
NENS0056.sas.itn Gig 0/2            172         T S       WS-C3524-XGig 0/2

Nothing strange in this configuration...

Anyway, I remind you that CiscoWorks LMS shows 3500XL switches stacked trough a sort of bus (or a LAN backbone). Look at the second attachment.

Bye!

Paolo

Title: Re: NeDi2GrapML
Post by: acoto on February 28, 2012, 02:45:06 PM
Hello Paolo,

The logic besides that fake hub is, if we have a local interface with two or more CDP neighbors, we can draw several times the same link, or assume there is a layer 1 or layer 2 device in the middle, working like a Hub. I prefer that second approach so NeDi2GraphML inserts a virtual node, and call it "Hub-XX". At some point you may change the Hub Icon for something that represents better the Stack...

Now, what I find strange in this specific case, is that we are getting two Hubs, one joining switches 51, 52, 54 and 56, and the other joining switches 53 and 55.

May you please check the links table for that 6 switches, to verify which CDP links NeDi has on Database.

Regards,

Adrian



 
Title: Re: NeDi2GrapML
Post by: pc_sg on February 29, 2012, 11:53:53 AM
AHi Adrian!

Here some dump:

Code: [Select]
NENS0051
Gi0/1 NENS0002 on Gi2/0/5 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0056 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0055 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0054 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0053 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0052 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Code: [Select]
NENS0052
Gi0/2 NENS0056 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0053 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0051 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0055 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0054 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Code: [Select]
NENS0053
Gi0/2 NENS0056 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0055 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0054 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0052 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0051 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Code: [Select]
NENS0054
Gi0/2 NENS0056 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0055 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0053 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0052 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0051 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Code: [Select]
NENS0055
Gi0/2 NENS0053 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0056 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0054 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0052 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0051 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Code: [Select]
NENS0056
Gi0/1 NENS0002 on Gi3/0/5 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0055 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0054 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0052 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0053 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012
Gi0/2 NENS0051 on Gi0/2 (Vlan0 FD) 1G CDP Discovered Wed Feb 29 11:45:01 2012

Is enough for your debuging?

There is another way to dump needed information?

Paolo
Title: Re: NeDi2GrapML
Post by: acoto on February 29, 2012, 03:24:06 PM
Hello Paolo,

Your database seems to be correct, you have a full CDP mesh between NENS005x devices.

At this point I think my initial algorithm to discover possible hubs is not bulletproof. I have to re-think it.

If anyone knows a good book about node-edge algorithms it will help  :)

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: acoto on March 01, 2012, 02:49:56 AM
Hello

I just uploaded NeDi2GraphML v0.9.

A little change, added a protection to -n option, to avoid adding links for orphan nodes.

(pending the correction for hubs detection... still need more time for that.)

Regards

Adrian
Title: Re: NeDi2GrapML
Post by: pc_sg on March 01, 2012, 08:53:53 AM
Tested v0.9.
Done a full map,in cluding nodes (-n) with both Icons e Groups handling with no error. Seems good!

But yEd crashes opening the (big, 6.5 MB) file.

Look error in attached snapshot.

If yEd is unusable, also this maps become unusable...  :'(

Any idea?

Paolo
Title: Re: NeDi2GrapML
Post by: acoto on March 01, 2012, 03:03:13 PM
Hello Paolo,

We need to narrow down where does the error comes from.

May you please try each option independently (-n, --icn, --grp) and check if you get the same error.

If not, then start with combinations: -n --icn, or -n --grp

My guess is that a name somewhere includes invalid XML characters, but got to figure out where and which character.

Regards,

Adrian
Title: Re: NeDi2GrapML
Post by: pc_sg on March 02, 2012, 08:02:01 AM

Sorry... Done a run with only -n switch and again yEd crashes, with the same error.

I don't know how to analyze if output file is really consistent and compliant with GraphLM format constraints.
Or if there is any utility to purge (or convert, or identify) invalid characters.

Paolo
Title: Re: NeDi2GrapML
Post by: acoto on March 02, 2012, 05:33:09 PM
Hello Paolo,

I think the problem may be an "&" character on OID field, for some of your nodes.

I had protection for non valid XML characters on nodename, but not for other fields on NeDi DB.

Now I added protection for a bunch of fields, including group name, device type, device location, device contact, link type, node OID...

Uploading NeDi2GraphML v0.10 to sourgeforge.

Regards

Adrian

Title: Re: NeDi2GrapML
Post by: acoto on March 05, 2012, 08:28:48 PM
Hello NeDi Community,

I just updated NeDi2GraphML to version 0.11

On this release, i added an option to avoid creating fake hubs (NeDi2GraphML -nhh ...). I do not like much that idea, but may work in some cases, the final schematic will present all CDP links, repeating in some case interface numbers.

Also added some extra code to sort the links array, prior to hunting hubs. In theory this may increase accuracy on hub detection, without a total rewrite of hub detection code.

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: tristanbob on March 06, 2012, 07:26:56 PM
I just tested version 0.11 with 985 nodes and 1185 edges.  This is a fun tool!  I like both the organic and circular layouts. 

I still use the diagram tool that Remo includes with Nedi, but this tool adds more options (although it takes many more steps).

Is there anyway you can make this automatically generate pictures within a web-interface (maybe a section of Nedi)?

Thanks for your contribution!

Tristan
Title: Re: NeDi2GrapML
Post by: acoto on March 06, 2012, 10:50:49 PM
Hello Tristan,

I'm glad you like the tool. I hope it becomes useful for more NeDi users.

About the integration, I think it is possible up to some point.

It does not seems too complicated to integrate the graphml diagram generation in the web server. User may select some options and run the script.

A second integration level, may include icons.csv, groups.csv and geo.csv file editing... things like that.

We need Remo's opinion here :)

Now, trying to include web visualization or editing is another story. Even that graphml is an open standard I do not know a good open source plugin that supports graphml. May be some day we will see something, but by now we are tied to yEd.

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: rickli on March 06, 2012, 10:55:14 PM
I definitely like the idea to have more alternatives to my map.  8)  Calling a perl script should be fairly easy, but having it generated by PHP would make the integration seamless (e.g. another output format in Topo-Map). I plan inline maps anyway, thus I'll keep graphml in the back of my head, while remodeling...
Title: Re: NeDi2GrapML
Post by: pc_sg on March 07, 2012, 08:28:25 AM
To Adrian:
Well, 0.11 runs without errors in all my tries. The -nhh switch doesn't do a bad work. The "star" interconnection of 3500XL family of switches stacks shown is not an error, rather is the real situation! Have a look to attached snapshot.
Indeed, the Cisco choice is to show it as a bus, as you have seen in a my previous message attached.

I'll play on it in the future.

Thanks so much!

To Remo:
Yes! I like very much if it or something like it become part of NeDi, with a minimal map editing capability (at least icon movement/arrangement).


Paolo
Title: Re: NeDi2GrapML
Post by: acoto on March 12, 2012, 02:28:54 AM
Hello,

I just uploaded NeDi2GraphML v 0.12 to sourceforge.

In this release I added some small optimization to code and  support for groups based on SNMP/NeDi location field.

(Note: still do not support hierarchical groups... some day )

#
# Sample Groups CSVFile
# Format:
#   group, <type>, <group>,<regex>
#
#      <type>:  "cdpsvc"   - CDP announced services: "Router" "Switch" "Bridge"...
#               "devtype"  - Device Type "Cisco-3640" "Cisco-3845"...
#               "location" - SNMP/NeDi Location Field "San Jose;Costa Rica"
#               "devname"  - Specific device Name "RCore001" "RAccess001" "SWDCT..."
#     <regex>:  matching regex
#     <group>:  Group Name
#
#
##########################################################

group,cdpsvc,Router,Routers
group,devtype,^Cisco-36,Cisco3600s
group,devtype,^cisco26,Cisco2600s
group,location,^San Jose;Costa Rica,SJO
group,devname,^RCore,Core
group,devname,^RDist1,Pop1
group,devname,^RAccs1,Pop1
group,devname,^RDist2,Pop2
group,devname,^RAccs2,Pop2
group,devname,^Raccs2,Pop2
group,devname,^RR0,Pop2


Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: acoto on March 12, 2012, 11:30:35 PM
Hello,

Second update today. NeDi2GraphML v0.13 is on sourceforge now.

I added another command line option "-ipg".

When using groups and -ipg option, NeDi2GraphML will search for IP Phones, and place them in the same group as the parent switch.

Regards,

Adrian


Title: Re: NeDi2GrapML
Post by: acoto on April 03, 2012, 12:26:57 AM
Hello,

I just Uploaded NeDi2GraphML v0.14 to sourceforge.

On this version I added "--url [telnet|ssh|http|https]" option. This will populate the URL field for each node.

On yEd right click on node, then select "Go to URL"

Regards,

Adrian



Title: Re: NeDi2GrapML
Post by: OMFan on April 17, 2012, 03:10:10 PM
Hello acoto,

Firtaval, Thanks to your great work about graphML generation which is absolutely fantastic !!!!!

All the people I have made a presentation of generated maps had the same reaction : WOUAH !!!

I have one question : in your last release (v0.14), you have added telnet,ssh,http or https links where you click on Node.

But when I have clicked on one node with Telnet link, anything happened. Apparently, Yed have not yet implemented this function.

Do you find any solution ?

Thanks a lot by advance.

Bye.
Title: Re: NeDi2GrapML
Post by: acoto on April 17, 2012, 04:13:53 PM
Hello OFMan,

Thank you for your comments. Iīm glad NeDi2GraphML works for you.

About the URL issue, it works for http and https on my Mac, but not for telnet or ssh. I started a topic on yEd support board:

http://yed.yworks.com/support/qa/874/url-telnet-ssh-not-working-in-os-x

This is the last answer I got:

"It seems that current versions of Java come with support for opening any protocol that is registered with the operating system and, thus, it shouldn't be too hard to support additional protocols in yEd as well. However, I cannot make any promises whether or when this will be done.
commented 3 days ago by michael [yWorks]"

So, the solution is now on yWorks hands.

Now, if we can get some NeDi users registering and voting +1 on this question on yEd site, the solution may come faster :)

Regards,

Adrian





Title: Re: NeDi2GrapML
Post by: OMFan on April 18, 2012, 03:38:12 PM
Thanks a lot for your answer.

I will go to Yed Website many many times in order to find a solution !!!

NB : Soory for my bad english (I am French).

Bye.
Title: Re: NeDi2GrapML
Post by: OMFan on April 19, 2012, 01:39:05 PM
Hello acoto,

I have just tried your last version and i have several questions about it :


Thanks a lot for your answer.

Title: Re: NeDi2GrapML
Post by: acoto on April 19, 2012, 04:00:43 PM
Hello OMFan,

The [-a] option was my first idea for IP Phone Autogroup, then I changed it to [-ipg], i did not noticed I left it on the help message. Got to correct it.

About individual GraphML files by building, not currently implemented in NeDi2GraphML. The logic to work it out seems complex at this point. Got to think about it.

Besides that, have you tried the grouping option?

You can group nodes by building, then in yEd select the group, Grouping-->View Group Contents. Yed Will Only show the elements inside the group.

Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: OMFan on April 19, 2012, 04:52:34 PM
Hello,

Thanks for your answer. I make a little modification on your perl script.

I have added one option and modify your SQL request.

I Will post it tomorrow if you are interested by this feature.

Bye.
Title: Re: NeDi2GrapML
Post by: acoto on April 19, 2012, 05:10:09 PM
Cool. Please share your ideas.

Adrian
Title: Re: NeDi2GrapML
Post by: OMFan on April 26, 2012, 11:05:29 AM
Hello,

I share my modifications about your initial script.

I have just added --buil "string" option.

If you use it, only devices which contains "string" in their name will be on the GraphML.

For example, if you have 30 switches, and you have 10 switches with string like 'Miami' in their name, then only 10 switches will be appear :

Code: [Select]
perl NeDi2GraphML.pl -o File.GraphML --url http --icn icons.csv --buil Miami --edgelabels full
I know it's not very useful for everyone, but in my case, all our switches (approximatively 200) have one information about localization in their name.

So we generate as much maps as we have buildings.

Tell me if you have any ideas to optimize it.

Bye.
Title: Re: NeDi2GrapML
Post by: acoto on May 03, 2012, 10:26:11 PM
Hello OMFan,

That is cool. Now, I wonder which will be more helpful, having the build option on the command line, or using an inclusion (filter) list in an external file, similar to the group list, and only add nodes if they match a statement on the list.

for instance...

perl NeDi2GraphML.pl -o File.GraphML --url http --icn icons.csv --nodefilter include.csv --edgelabels full

include.csv
incnode   cdpsvc   Router
incnode   devtype   ^Cisco-36
incnode   devtype   ^cisco26
incnode   location ^San Jose;Costa Rica
incnode   devname   ^RDist
incnode   devname   ^RAccs

That will permit to include nodes in several conditions.

What do you think?

Adrian
Title: Re: NeDi2GrapML
Post by: OMFan on May 04, 2012, 09:59:27 AM
Hello,

I think it is a very good idea which offers a lot of capabilities to build maps with only few devices !!!!!!!!!!!!!

If you have time for develop this functionnality, i will be your first personal tester  ;) ;)
Title: Re: NeDi2GrapML
Post by: acoto on May 05, 2012, 10:02:59 PM
Hello,

NeDi2GraphML v0.15 is on Sourceforge now.

New option Device filters list. Thank You OMFan!!

Use devices filter to include/exclude devices based on specific characteristics.

perl NeDi2GraphML.pl -o Schematic.graphml --dfr devicefilter.csv

# Sample Devices Filter CSVFile
# Format:
#   devfilter, <type>, <regex>, <condition>
#
#      <type>:  "cdpsvc"   - CDP announced services: "Router" "Switch" "Bridge"...
#               "devtype"  - Device Type "Cisco-3640" "Cisco-3845"...
#               "location" - SNMP/NeDi Location Field "San Jose;Costa Rica"
#               "devname"  - Specific device Name "RCore001" "RAccess001" "SWDCT..."
#     <regex>:  matching regex
# <condition>:  "inc" Include
#               "exc" Exclude
#

devfilter,cdpsvc,Router,exc
devfilter,devtype,^Cisco-36,exc
devfilter,location,^San Jose;Costa Rica,inc
devfilter,devname,^RCore,inc



Note: There is a change on -n option. Now we only include nodes if parent device exists ( or it is not filtered out :) )

Adrian
Title: Re: NeDi2GrapML
Post by: ablkshrt1 on May 08, 2012, 07:57:03 PM
First off, great tool!

I am having a problem with how it links my CatOS devices.  CatOS reports device IDs a little different via CDP....it shows the serial number if you do a show CDP neighbor detail and just the name if your just do a show cdp neighbor.

My problem is that when your tool is building the links, it sees the serial number and thinks it is a missing device, and therefore never creates the link.  I am attaching a screenshot of my debug.  Can you help me out.
Title: Re: NeDi2GrapML
Post by: rickli on May 08, 2012, 10:38:19 PM
This might actually be related to the discovery. I changed it around in order to support Nexus. Can you check around line 1460 of libsnmp.pm, whether the WS-C regexp matches or not?
Title: Re: NeDi2GrapML
Post by: acoto on May 08, 2012, 10:44:57 PM
Hello Ablkshrt,

NeDi2GraphML takes the link information directly from NeDi Database. I think it may be better to clean the database input process than trying to correct it on the output side.

Can you share a "show version" and a "show cdp neighbor details" of the affected switch and some neighbors.

Remo, can you give us your opinion here

Regards

Adrian

Title: Re: NeDi2GrapML
Post by: rickli on May 09, 2012, 09:39:05 PM
Actually I forgot about a typo of mine:

-                                       if($lneb{"1.3.6.1.4.1.9.9.23.1.2.1.1.8.$k[14].$k[15]" =~ /^WS-C/}){
+                                       if($lneb{"1.3.6.1.4.1.9.9.23.1.2.1.1.8.$k[14].$k[15]"} =~ /^WS-C/){

(the line with a + replaces the line with a -. The curly bracket at the end needs to be before =
This should fix it...
Title: Re: NeDi2GrapML
Post by: acoto on May 10, 2012, 12:47:23 AM
Great catch Remo!

Thank You!

Adrian
Title: Re: NeDi2GrapML
Post by: ablkshrt1 on May 10, 2012, 05:59:18 PM
Fixing the typo fixed my issue. Thanks to this tool and Nedi, I finally have an updated network map (5 years in the waiting)

I do have a feature request.  It is great to be able to filter the devices, it would be equally great to be able to filter nodes.  If I could filter nodes by IP, Name, MAC and whatever other methods you can think of, it would be a huge help.  Also, Nedi already does a good job of using png icons for nodes.  Is there a way to get the nodes to show up in yEd with the same icons that are found in Nedi. That would make identification of nodes very easy.

Also, when I add nodes to my graph and do a Tree layout, is there a way to get it to order the nodes by port number....right now it is very random.
Title: Re: NeDi2GrapML
Post by: acoto on May 11, 2012, 07:15:10 AM
Hello ablkshrt1,

Node filters... good idea, almost done. uploading now.

PNG icons...yuck!.. I do not like bitmaps on my network schematics...
  one of my main goals when I started on this project was to get nice schematics. Vector graphics are a must if you want to print...
  may we can work on the other way around, instead of using png, we can try to get/make all requiered icons in SVG format,
  my current icon gallery is just a start :)

About the ordering of nodes/ports... yEd do not work on that way.
  My best shot to get ordered, consistent schematics is to use auto-layouts, then finetune by hand,
  then, export coordinates in a CSV File (geo file: devicename,coordX,coordY), and use it as geo file on NeDi2GraphML


Regards,

Adrian
Title: Re: NeDi2GrapML
Post by: acoto on May 11, 2012, 07:28:46 AM
Hello NeDi Community

NeDi2GraphML v0.16 is on server now :)

New option: Node Filter list

perl NeDi2GraphML.pl -o Schematic.graphml --nfr nodefilter.csv

#
# Sample Node Filter CSVFile
# Format:
#   nodfilter, <type>, <regex>, <condition>
#
#      <type>:  "name"     - Node Name
#               "nodip"    - Node IP Address
#               "mac"      - Node MAC Address
#               "oui"      - Node OUI
#               "devname"  - Parent Device Name
#     <regex>:  matching regex
# <condition>:  "inc" Include
#               "exc" Exclude
#
#
###############################################################################

nodfilter,devname,^RAccs1,inc
nodfilter,nodip,10.1.112.102,exc
nodfilter,devname,^RAccs2,exc
nodfilter,mac,^000002210102,inc
nodfilter,oui,^Cisco,inc
nodfilter,oui,^D-Li,exc
nodfilter,name,^NeDi,inc



Notes:

  When node filters are enabled, by default all nodes are excluded

  Filter priority is name > nodip > mac > oui > devname

  It is possible to create hierarchical filters. for instance include nodes connected to device, but exclude some specific mac address.

  It is possible to use device filters and node filters on the same NeDi2graphML run.


Regards,

Adrian
Title: Re: NeDi2GrapML
Post by: OMFan on May 11, 2012, 03:18:33 PM
Hello Adrian,

I have just tested your modifications and It works so great !!!

Thanks a lot !!

It can suggest you a little bit modification.

Since you have several criteria for nodfilter (nodeip, mac, oui), could you have the same thing for icons.csv ?

Example :

I have used nodfilter in order to have only IP Phones on map (Thomson ST2030) :

Code: [Select]
nodfilter,oui,Thomson Telecom Belgium,inc
It works great, but i would like to have a special icon for those materials.

But only filters in icons.csv are cdpcsv,devtype,devname so i have one rectangle for one thomson IP phone.

I hope you understand my description.

Thanks.

Bye.

Title: Re: NeDi2GrapML
Post by: acoto on May 11, 2012, 04:53:58 PM
Hello OMFan

Done! uploading NeDi2GraphML v0.17 to sourceforge.

Now we can match parent device, oui, mac, node ip, and name on icon assignment for nodes.

There is an small syntax change on icons and node filter files. To match for parent device we need to use "pdevname"


Regards,

Adrian

Title: Re: NeDi2GrapML
Post by: OMFan on May 15, 2012, 02:11:28 PM
Hello Adrian,

You job is really fantastic. I have just tested and it works great !!

I have one question about GraphML generation file.

Do you know if it is possible to generate automatically a PNG of JPEG file instead of GraphML file , in using yEd CLI for example ?

Thanks a lot.

Bye
Title: Re: NeDi2GrapML
Post by: acoto on May 15, 2012, 02:43:04 PM
Hello OMFan

yEd licence forbidens the use of yEd in a script.

http://yed.yworks.com/support/qa/6/can-i-invoke-yed-layouts-from-the-command-line

yFiles is probably your best option.

Regards,

Adrian
Title: Re: NeDi2GrapML
Post by: dobst on February 15, 2013, 04:16:45 PM
Hi there,

it's a very magic tool do visualize your network. But i've one problem: The connecting lines between the devices overlap some devices and their description so i can't see the description of our main switch because a hundred links leave them. Any idea how to bring the labels to the forground?

Yours, Daniel
Title: Re: NeDi2GrapML
Post by: acoto on February 15, 2013, 05:20:03 PM
Hello Daniel,

This may help...

http://yed.yworks.com/support/qa/1630/can-i-lower-the-edge (http://yed.yworks.com/support/qa/1630/can-i-lower-the-edge)

     See "File" -> "Preferences", tab "Display" option "Paint Nodes over Edges".

Regards,

Adrian
Title: Re: NeDi2GrapML
Post by: dobst on February 15, 2013, 05:22:40 PM
Thx! That was the button i looked for :-)
Title: Re: NeDi2GrapML
Post by: rickli on July 05, 2014, 11:25:06 AM
I had a hard time finding this great contribution again, so I made it sticky  :)
Title: Re: NeDi2GrapML
Post by: OMFan on March 19, 2015, 01:38:17 PM
Hi Guys,

Since NeDi 1.4 version, NeDi2GraphML don't work properly because database scheme has changed.

If you want to use it with the last version, i have found how to modify NeDi2GraphML.pl by changing line 862 :

Old line

Code: [Select]
my $query="SELECT name, nodip, mac, oui, device, ifname FROM nodes";

New Line

Code: [Select]
my $query="SELECT nodes.nodesc,nodip,nodes.mac,oui,device,ifname FROM nodarp,nodes WHERE nodes.mac = nodarp.mac";
Bye.
Title: Re: NeDi2GrapML
Post by: acoto on May 08, 2015, 07:46:22 AM
Excellent OMFan, thank you!

I added your change and uploaded v0.18

I do not have a recent NeDi installed around here, please check if it works fine.

Regards,

AC