more robust ip->mac translation in reportEvent
Vis a vis event 2244:
- reported on 22.214.171.124 at 2011-11-06 00:07:08
- at that time, there was no record in dhcplease table (the record was written LATER after the event detection at 2011-11-06 00:11:57)
- also at that time no record in arp_table_intervals yet
- hence the HwFactory._resolve method went to MISCOMP with an ip address, NOT a mac, and found system r-s-edge-2 (which was NOT the system using this ip address at the time).
Problem is exasperated by the fact that a node that is statically registered in miscomp was actually being used by a different mac (and had been for at least a week or two).
After much discussion we think the following changes will help:
1) For detected_timestamp in the "recent" past (@3hrs or so) FIRST query InfoBlox for ip->mac translation as master and most up-to-date source of information; if not found there, THEN look in nimi (which is not as up-to-date). 3 hours is chosen because Krysia thinks that InfoBlox has about 6 hours of history (configurable, evidently); and nimi seems to be lagging somewhere between 15 and 50 minutes behind "reality", so 3 hours should fall comfortably between these limits.
2) If no ip->mac translation can be made in the HwFactory.getSystem(ip, timestamp) method, then raise some "NotFoundInNimi" exception (not quite the same as "NotOnNetwork", because clearly a realtime detector DID find this system on the network; but we cannot deduce the mac address from the specified ip address). In other words, do NOT ever go directly from ip to system, ALWAYS go from ip to MAC to system.