Existing software techniques are adequate for many problems. However, as the complexity and/or uncertainty of the input increases, traditional computational methods become increasingly inadequate. For some of these problem spaces, various soft computing methods such as neural networks, fuzzy logic, Decision Trees, Bayesian processing, etc. have been quite successful. But each of these technologies has various strengths and weaknesses. Though existing techniques can sufficiently address small parts of an overall problem space, substantial value can be provided by a cohesive system that can effectively reason about the entire problem space while applying the best technology for each part of the problem. The complex, real world problems we need to address require a mix of traditional and soft computing technologies in a cohesive, multi-paradigm hybrid framework.
One of many guiding factors in determining what technology to apply is the nature of the information we have available on which to act. Sometimes we have data that contains a buried wealth of information, other times we have knowledge (rules). Each of these characteristics leads us towards a different solution based on the technology that is best suited to acting on that kind of information.
The concept we have been investigating is an advanced application framework built on the IBM InfoSphere Streams platform that incorporates numerous leading edge aspects. Three independent, but interacting, challenges needed to be overcome.