Welcome, Guest. Please login or register.

Author Topic: Distributed discovery  (Read 3966 times)

raider82

  • Jr. Member
  • **
  • Posts: 91
    • View Profile
Distributed discovery
« on: July 19, 2012, 10:42:35 AM »
Just tried to merge the data of two systems into one central NeDi installation. And it seemed to work. Just a small description of the steps, in case someone wants to try this:
  • Export data from any local agent: mysqldump nedi --compact --no-create-info --user=xxx --password=xxx > sqlfile_x
  • Transfer data to central system via scp
  • Transfer all seedlists and conf files to central system via scp
  • Clean NeDi database on central system: nedi.pl -i
  • Import all local databases: mysql nedi --batch --force --user=xxx --password < sqlfile_x
  • Enjoy all data in one system

Things that will not work:
  • Graphs (rrds missing)

pc_sg

  • Guest
Re: Distributed discovery
« Reply #1 on: July 19, 2012, 01:56:46 PM »
Sorry, Raider82, your is not a Distributed discovery, but something similar to a systems consolidation.
Anyway interesting.

Are you meaning something like: I have two (or more) NeDi instances (on different servers ?), I export all data separately, and import all of that (merge data) in a brand new NeDi server?

Paolo

P.S. I'm greatly interested ii a real distribuited discovery, with a remote NeDi server in each remote site, and a central NeDi server that collects all data. If any remote istance is able to run concurrently as a collector and as a autonomous NeDi this should be the best!

raider82

  • Jr. Member
  • **
  • Posts: 91
    • View Profile
Re: Distributed discovery
« Reply #2 on: July 20, 2012, 11:26:59 AM »
Are you meaning something like: I have two (or more) NeDi instances (on different servers ?), I export all data separately, and import all of that (merge data) in a brand new NeDi server?
Yes. This is what this example describes. You could also make one of the agents the central server ...

P.S. I'm greatly interested ii a real distribuited discovery, with a remote NeDi server in each remote site, and a central NeDi server that collects all data. If any remote istance is able to run concurrently as a collector and as a autonomous NeDi this should be the best!
The only difference is, that you will not get data immediately per device, but for the whole local agent.

You can also replace "localhost" with a central server in nedi.conf. This collects data like you want, but you will have the same problems like a single NeDi with locked tables for node updates and so on. In addition, you lose the local database, so the agent is no full NeDi any more.

If one would disable constraint checking in the central database (much faster imports) and import the agents' data every few hours (maybe 1 or 2), I guess this would be the solution with the least problems for now. Since we are rebuilding from local databases which check the constraints, there should be no issue at all.

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2785
    • View Profile
    • NeDi
Re: Distributed discovery
« Reply #3 on: July 20, 2012, 06:17:49 PM »
What would a hierarchical NeDi be worth to you? If I ever was to make a living off of NeDi, this might be the road...

From the purely technical side, I got a proof of concept working, where a Master talks to other NeDies via HTTP(S) and gathers incidents and events (level > 200). Quick links let you access the Remote Agent for more details. Have a look at the screenshot of the master console (note that the targets are actually NeDi Agents)...
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

pc_sg

  • Guest
Re: Distributed discovery
« Reply #4 on: July 23, 2012, 10:57:33 AM »
Hi Remo, I'm not sure I've understand what you mean.

My idea is based on the fact that every discovery use some network bandwidth, and polling a remote site occupy WAN bandwith, with problem if timeout occours.

I think that a local NeDi server (with a future multithread capability) has less problem,  and shoul be able to poll device more often that a single central one. At the same time, each local NeDi could be very useful for local ICT colleagues, they don't usually need to watch other sites networks.

The central NeDi server can collect other sites data in a smart way (compressed? compacted? consolidated?).
And this central server may use and show only global infos, not necessarily all.
If the master network administrator need some local site network information, can access each local NeDi server...

So, in my mind, for example, one company, 10 sites, 10 local NeDi servers, 1 Central neDi server.

But is only a  wish  ;)

Paolo

rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2785
    • View Profile
    • NeDi
Re: Distributed discovery
« Reply #5 on: July 23, 2012, 07:55:43 PM »
This is pretty much it, except the central master only syncs incidents and polls the alerts of the agents. As soon as you (or the local tech) acknowledges an incident, it'll dissapear from the master automatically. If this takes off, top-ifstats and other important specs could be added...
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo

ntmark

  • Full Member
  • ***
  • Posts: 136
    • View Profile
    • tvnz.co.nz
Re: Distributed discovery
« Reply #6 on: July 23, 2012, 11:16:24 PM »
I think of a distributed system a little differently.
Where the edge Nedis do the grunt work, polling and all the other stuff the current one does, but they send it all to a central one. (and maybe keep a copy on thier local disk[just in case])
The central Nedi server holds the DB and configs, and receives the traps(either, sent directly or via edge nedi), and is where the operator(s) do all their work on.

Sending the data compressed/consolidated and/or in chunks is a great idea.
Maybe even scheduled updates at whatever time you configure, excluding monitor alerts or something like that.


NB. I'm currently not using the monitor part of nedi as we already have another system for this.

Mark

pc_sg

  • Guest
Re: Distributed discovery
« Reply #7 on: May 23, 2013, 11:41:05 AM »
Hi Remo,

I'm starting to develope a distributed NeDi envoronment.
I've already build two running VM in my site.
One as agent (but a full NeDi instance, it polls all network device in tis site)
The second one should be the master.
After a lot of try, a missing PERL module install, and some config adjustment, "master.pl" contact the agent, ad as you said, polls some info, but retuns some error:
Code: [Select]
RDEV:1 devices read from nedi.devices
SEED:Using ./agentlist
SEED:AGENT agent0 added for discovery
RMON:1 entries (dev) read from nedi.monitoring
RMON:0 entries (node) read from nedi.monitoring
RUSR:0 entries (groups & 8 AND (phone != "" OR email != "")) read from nedi.users

QERY:admin@AGENT + SELECT * FROM events WHERE level > 150 ORDER BY id DESC LIMIT 1
ERR :Events - 501 Protocol scheme 'ping' is not supported
INS :1 ROWS INTO events (level,time,source,info,class) VALUES ("150","1369300925","AGENT","Events - 501 Protocol scheme 'ping' is not supported","mast")
QERY:admin@AGENT + SELECT * FROM incidents WHERE time = 0
ERR :Incidents - 501 Protocol scheme 'ping' is not supported
INS :1 ROWS INTO events (level,time,source,info,class) VALUES ("150","1369300925","AGENT","Incidents - 501 Protocol scheme 'ping' is not supported","mast")
UPDT:1 ROWS FROM monitoring SET status="6",lost="6" WHERE name ="AGENT"
WDEV:AGENT written to nedi.devices

Took 0s, sleeping 60s

Seemed necessary to add AGENT (in this example a fake hostname or address) as a monitored device...

You already said that is not documented at all, ad is only a firts try, not deeply tested.

Anyway, exactly as ntmark uphold, I think that the right way to create a distributed NeDi archiecture is to mantain a local NeDi instance in any site (minimizing time and WAN traffic), ad to collect centrally all possible information, having there all remote device datum (like a unique NeDi instance that collects/polls all sites device).

Anyway, I'm at your disposal for "beta" testing!

Thank in advance!

Paolo


rickli

  • Administrator
  • Hero Member
  • *****
  • Posts: 2785
    • View Profile
    • NeDi
Re: Distributed discovery
« Reply #8 on: May 24, 2013, 07:02:33 PM »
You need to change the test to either http or https (depending on your agent setup). I actually updated this in Monitoring-Master's help.

I'd agree that it would be a lot more powerful the way Mark and you suggest. But it's also more challenging to code :) In its current state, this is more or less a proof of concept and can certainly be extended if it works as intended...
Please consider Other-Invoices on your NeDi installation for an annual contribution, tx!
-Remo