I wonder to what extent ideas developed by the likes of Martin Ward (there are many others – for some reason I just remember him) on refinement and its inverse be used in the field of connectionist-symbolic integration? More generally help us to understand the different forms a representation can take and to relate different flavours of representation – even the anti-representation if you should choose to take such a vulgar viewpoint.
The first paper I read on this which I vaguely understood was by Pinkas (1995). There he showed how Hopfield nets can be thought of as finding preferred models of a non-monotonic sentential logic he named penality logic. Another way of thinking of this could be that the network configuration is a refinement of the logical theory? Or the logical theory is a reverse engineering of the network?
Pinkas, Gadi (1995), Reasoning, Nonmonotonicity and Learning in Connectionist Networks that Capture Propositional Knowledge, Artificial Intelligence, 77(2), 203-247