Fusion is explanation It is from the various intelligence, surveillance, and reconnaissance (ISR) assets, from automated sensor platforms to verbal reports from personnel, that we achieve the information dominance that enables comprehensive situation awareness. The common operating picture (COP) is a hypothesis that we have accepted as the best explanation for the ISR data. The COP is a complex, composite cause that leads to detectable effects on our ISR assets.

Abduction enables explanation The reasoning component of our approach is based on abductive inference, which is inference to the best explanation. It uses commonsense logic to reason from the evidence to an explanation, from effects to causes, leveraging corroboration and contradiction, and making and testing hypotheses. By operating in such a natural manner, an abductive approach eases the important tasks of transferring reusable knowledge from the analyst to the software. In addition, the system collaborates with the analyst so that they can understand how the system is justifying its conclusions based on the evidence. The analyst can even assist and control the inference processes.

Abduction is efficient A common problem with generalized data fusion is that there are too many hypotheses to consider as an explanation for each time-segment of incoming data. Our inference process finds the most confident conclusions first, and works outward from those. It keeps track of its beliefs, testing them against reality, and works to amend its beliefs when reality presents new anomalies. So, the more easily reached conclusions are available earlier, and changes in conclusions are prompted by corresponding changes in the observations. So, the system is not continuously rediscovering things already known, nor is it allowing the difficult, ambiguous data to halt its work completely.

Multi-level fusion The general literature makes it clear that the data fusion community has made significant progress with techniques for lower-level fusion, and rather limited progress with higher levels. However, many of the solutions that exist are limited in scope. Abductive inference provides a framework where true multi-level fusion can be realized. It achieves this level of integration so processing can be parallelized, expectations and questions from higher levels affect the interpretation of data at lower levels, and the conclusions of lower levels become data to be explained at higher levels.

Model-based inference Computational models exist for many aspects of battlesphere behavior. Abduction can make use of these during its inference process: a hypothesis becomes a model that we simulate, then the simulation results become predictions of the hypothesis that can be tested against actual observations.

Complex scenes A remarkable aspect of abduction is its ability to assemble complex scenes from simpler parts, in assembling consistent hypotheses from fragments. Various parts of the observed situation each evoke hypotheses that could explain them. Abductive inference works to discover which of these scene fragments are the most plausible and brings them into its composite view of accepted reality. An important aspect of a scene fragment's merit is the degree to which it appears to be consistent with what we already believe about the scene. A sufficiently compelling fragment that does not match the system's beliefs can cause it to fragment its beliefs again, retaining some while reexamining others. This capability is far beyond many other traditional reasoning approaches, but is required for a system that is a cornerstone of battlesphere situation awareness.

Examples Projects that Aetion has worked on include:

Level 1 fusion, with multiple targets, and multiple sensor locations, modalities and resolutions. Each sighting of an object explains its appearance in sensor data. Generic models were used of how a sighting on a particular sensor display constrains where, in spacetime, the object must have been. An acceptable explanation must involve a target that is within the intersection of the regions of spacetime that correspond to the sensor readings that the target purports to explain. The sensors with greater resolution, or with a more constraining modality (e.g., including range information), require acceptable explanations to be within a smaller region of spacetime. Corroboration or contradiction from other sensors also surveilling the same region is leveraged. Anomaly-driven processing allows the discrimination of objects that are separately visible to some sensors, but that appear to mass together to other sensors. Object tracks are inferred from sequences of sightings, and can then be used to generate expectations that aid the processing of further sightings. Furthermore, sightings of reference objects could be used to infer a sensor's location and alignment.

Level 2+ fusion, where the analyst encodes, prior to the engagement, their hypotheses about enemy intent, the indicators of that intent, and how observations can confirm or disconfirm those indicators. During the engagement, our system is provided with the incoming reports about types of military activity, vehicle types, etc. at various locations and times. Abductive inference transforms the message stream into beliefs about enemy intent, based on the explanatory hypotheses encoded by the analyst. The analyst can explore these beliefs, overriding them if necessary, by reviewing the evidence for and against them.