Welcome, Guest. Please login or register.

Author Topic: speeding up discovery  (Read 3411 times)

big

  • Newbie
  • *
  • Posts: 26
    • View Profile
speeding up discovery
« on: June 08, 2015, 10:22:35 PM »
Hi Folks

another question if I may. Is it possible to speed up the discovery interval? right now I need about 2hrs do
discover everything on the network. I see a reference to "borders" but I don't see a configuration example
or much documentation on configuring this nedi.pl switch. Any one done this? if so can you share some specifics?

thanks

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2786
    • View Profile
    • NeDi
Re: speeding up discovery
« Reply #1 on: June 14, 2015, 03:40:57 PM »
No one dares to answer?  :)

What I do is divide a large network into smaller parts and have staggered cronjobs in parallel. See here:
https://plus.google.com/106414135314831644755/posts/32a5bQ7f4P7

You can also skip certain things like reading modules and vlans during the day (-Smv) and have one discovery at night skipping nodes instead (-SAF)

If you use other tools for monitoring traffic, errors, discards or broadcasts, you can skip NeDi's interface statistics altogether (-Stedb)
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

dohco

  • Newbie
  • *
  • Posts: 33
    • View Profile
Re: speeding up discovery
« Reply #2 on: June 17, 2015, 04:20:33 PM »
Hmm, our network takes about 1-2 hours to complete a scan with -pob
I did a test run with some new settings and run 6 parallel discovery processes and it took ~25 minutes!

This is what I put in a script called hourly.sh:
/path/to/nedi.pl -Smvj -A"device regexp '^(a|b|c|d|e|f)'" &
/path/to/nedi.pl -Smvj -A"device regexp '^(g|h|i|j|k|l)'" &
/path/to/nedi.pl -Smvj -A"device regexp '^(m|n|o|p|q|r)'" &
/path/to/nedi.pl -Smvj -A"device regexp '^(s|t|u|v)'" &
/path/to/nedi.pl -Smvj -A"device regexp '^(w|x|y|z)'" &
/path/to/nedi.pl -Smvj -A"device regexp '^[0-9]'" &

I'm thinking af running this script every hour and probably need to run with other options once per day (midnight).
Please give advice about a script to run at midnight and any modifications or thoughts to the script above.

/Conny
« Last Edit: June 17, 2015, 08:03:39 PM by dohco »

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2786
    • View Profile
    • NeDi
Re: speeding up discovery
« Reply #3 on: June 19, 2015, 10:10:53 AM »
Good stuff :-)

You could add "sleep 10" in between the nedi calls to stagger initial (heavy) DB access a bit...

The regexps could be written as '^[a-f]'

The midgnight run could be -SAFao (skip ARP, Bridge-Forward, admin and oper status of interfaces)
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

dohco

  • Newbie
  • *
  • Posts: 33
    • View Profile
Re: speeding up discovery
« Reply #4 on: June 24, 2015, 11:30:46 AM »
OK, after some testing, this is what I came up with.
Instead of making a script I did it all in crontab (edit via System - Files).
The reason I choose to use device names instead of IP is that I find it more flexible.
I added a line to cover for any new entries in the seedlist-file.

I'm I missing anything, or is it all covered?!

0 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[a-f]'" > /tmp/nedi-`date +\%H`_a-f.run 2>&1
1 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[g-l]'" > /tmp/nedi-`date +\%H`_g-l.run 2>&1
2 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[m-r]'" > /tmp/nedi-`date +\%H`_m-r.run 2>&1
3 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[s-v]'" > /tmp/nedi-`date +\%H`_s-v.run 2>&1
4 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[w-z]'" > /tmp/nedi-`date +\%H`_w-z.run 2>&1
5 * * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[0-9]'" > /tmp/nedi-`date +\%H`_0-9.run 2>&1
10 0 * * *   /var/nedi/nedi.pl -SAFao > /tmp/nedi-`date +\%H`.run 2>&1
11 0 * * *   /var/nedi/nedi.pl -u /var/nedi/seedlist > /tmp/nedi-`date +\%H`_seedlist.run 2>&1

/Conny

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2786
    • View Profile
    • NeDi
Re: speeding up discovery
« Reply #5 on: June 28, 2015, 02:18:36 PM »
I'd try this:

0 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[a-f]'" > /tmp/nedi-`date +\%H`_a-f.run 2>&1
1 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[g-l]'" > /tmp/nedi-`date +\%H`_g-l.run 2>&1
2 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[m-r]'" > /tmp/nedi-`date +\%H`_m-r.run 2>&1
3 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[s-v]'" > /tmp/nedi-`date +\%H`_s-v.run 2>&1
4 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[w-z]'" > /tmp/nedi-`date +\%H`_w-z.run 2>&1
5 1-23 * * *   /var/nedi/nedi.pl -Smvj -A"device regexp '^[0-9]'" > /tmp/nedi-`date +\%H`_0-9.run 2>&1

0 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[a-f]'" > /tmp/nedi-`date +\%H`_a-f.run 2>&1
1 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[g-l]'" > /tmp/nedi-`date +\%H`_g-l.run 2>&1
2 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[m-r]'" > /tmp/nedi-`date +\%H`_m-r.run 2>&1
3 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[s-v]'" > /tmp/nedi-`date +\%H`_s-v.run 2>&1
4 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[w-z]'" > /tmp/nedi-`date +\%H`_w-z.run 2>&1
5 0 * * *   /var/nedi/nedi.pl -SAFao -A"device regexp '^[0-9]'" > /tmp/nedi-`date +\%H`_0-9.run 2>&1

30 0 * * *   /var/nedi/nedi.pl -u /var/nedi/seedlist > /tmp/nedi-`date +\%H`_seedlist.run 2>&1
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo