Many researchers argue that logics and connectionist systems complement each other nicely. Logics are an expressive formalism for describing knowledge, they expose the common form across a class of content, they often come with pleasant meta-properties (e.g. soundness and completeness), and logic-based learning makes excellent use of knowledge. Connectionist systems are good for data driven learning and they’re fault tolerant, also some would argue that they’re a good candidate for tip-toe-towards-the-brain cognitive models. I thought I’d give d’Avila Garcez and Lamb (2006) a go [A Connectionist Computational Model for Epistemic and Temporal Reasoning, Neural Computation 18:7, 1711-1738].
- Ω is a set of possible worlds.
- R is a binary relation on Ω, which can be thought of as describing connectivity between possible worlds, so if R(ω,ω’) then world ω’ is reachable from ω. Viewed temporally, the interpretation could be that ω’ comes after ω.
- v is a lookup table, so v(p), for an atom p, returns the set of worlds where p is true.
- How can these nets be embedded in a static population coded network. Is there any advantage to doing so?
- Where is the learning? In a sense it’s the bit that does the computation, but it doesn’t correspond to the usual notion of “learning”.
- How can the construction of a network be related to what’s going on in the brain? Really I want a more concrete answer to how this could model the psychology. The authors don’t appear to care, in this paper anyway.
- How can networks shrink again?
- How can we infer new sentences from the networks?
I received the following helpful comments from one of the authors, Artur d’Avila Garcez (9 Aug 2006):
I am interested in the localist v distributed discussion and in the issue of biological plausibility; it’s not that we don’t care, but I guess you’re right to say that we don’t “in this paper anyway”. In this paper – and in our previous work – what we do is to say: take standard ANNs (typically the ones you can apply Backpropagation to). What logics can you represent in such ANNs? In this way, learning is a bonus as representation should preceed learning.
The above answers you question re. learning. Learning is not the computation, that’s the reasoning part! Learning is the process of changing the connections (initially set by the logic) progressively, according to some set of examples (cases). For this you can apply Backprop to each network in the ensemble. The result is a different set of weights and therefore a different set of rules – after learning if you go back to the computation you should get different results.
I wonder to what extent ideas developed by the likes of Martin Ward (there are many others – for some reason I just remember him) on refinement and its inverse be used in the field of connectionist-symbolic integration? More generally help us to understand the different forms a representation can take and to relate different flavours of representation – even the anti-representation if you should choose to take such a vulgar viewpoint.
The first paper I read on this which I vaguely understood was by Pinkas (1995). There he showed how Hopfield nets can be thought of as finding preferred models of a non-monotonic sentential logic he named penality logic. Another way of thinking of this could be that the network configuration is a refinement of the logical theory? Or the logical theory is a reverse engineering of the network?
Pinkas, Gadi (1995), Reasoning, Nonmonotonicity and Learning in Connectionist Networks that Capture Propositional Knowledge, Artificial Intelligence, 77(2), 203-247