Welcome, Guest. Please login or register.

Author Topic: Too much duplication in Nodes table? (table growing excessively large)  (Read 2054 times)

tristanbob

  • Full Member
  • ***
  • Posts: 152
    • View Profile
I noticed my Nodes feature has been getting slower and slower, so I checked on the quantity of nodes:

New    2,171
Active    4,525,308
Wlan    4,796,687
IP Change    1,063,297
IF Change    15,805,663
No Name    18,231
No IP    16,408
Discover Once    167,450

Total    24,563,767

There is no way there are that many nodes on our network! And these numbers keep growing.  My theory is that Nedi is creating a new Node entry any time there is a unique "DNS name + IP + MAC" combo.  On our wireless, we have 15 min DHCP leases so users could be seen on hundreds of different IPs.

I searched for one node with DNS name of "amelia-pond", and it came up with the 527 results.  The DNS name name was always the same, but the MAC address (including vendors) and IPs varied a lot. 

I am running Nedi version 1.5.225.  We didn't have this explosive growth problem with previous versions.

Cheers!

Tristan


Please visit "Other"->"Invoices" on your NeDi installation to make an annual contribution and support Nedi!

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2697
    • View Profile
    • NeDi
Indeed that's an issue I'm working on. I've made some improvements in 1.6, but I haven't identified every possibility, causing this yet...

To clean out your multiples, you can use those queries:

Delete duplicate DNS entries (instead aname you can also use nodip):
DELETE t1 FROM dns t1, dns t2 WHERE t1.dnsupdate < t2.dnsupdate AND t1.aname = t2.aname;
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

tristanbob

  • Full Member
  • ***
  • Posts: 152
    • View Profile
Thanks Remo!  I ran your SQL command using your nifty "System>Database" tool and it reduced my total nodes by ~90%.   

Before: 24.0 million nodes
After: 2.2 million nodes

Needless to say, Node reports are much faster.  (although I still wonder why I have 2 million nodes)

Thanks for your assistance!

Tristan
Please visit "Other"->"Invoices" on your NeDi installation to make an annual contribution and support Nedi!

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2697
    • View Profile
    • NeDi
I'll have a patch for 1.6 soon, which uses the 2nd retire value (used for retiring links) defaulting to 7 days. I plan a 3rd value for 1.7 for retiring ARP values (defaulting to 1 day). This should help mitigate the mysterious node multiplication...
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo