Strömfelt, H; Dickens, L; d'Avila Garcez, A; Russo, A; (2022) Formalizing Consistency and Coherence of Representation Learning. In: Advances in Neural Information Processing Systems. NIPS

Preview

PDF 11840_formalizing_consistency_and_co.pdf - Published Version Download (3MB) | Preview

Abstract

In the study of reasoning in neural networks, recent efforts have sought to improve consistency and coherence of sequence models, leading to important developments in the area of neuro-symbolic AI. In symbolic AI, the concepts of consistency and coherence can be defined and verified formally, but for neural networks these definitions are lacking. The provision of such formal definitions is crucial to offer a common basis for the quantitative evaluation and systematic comparison of connectionist, neuro-symbolic and transfer learning approaches. In this paper, we introduce formal definitions of consistency and coherence for neural systems. To illustrate the usefulness of our definitions, we propose a new dynamic relation-decoder model built around the principles of consistency and coherence. We compare our results with several existing relation-decoders using a partial transfer learning task based on a novel data set introduced in this paper. Our experiments show that relation-decoders that maintain consistency over unobserved regions of representation space retain coherence across domains, whilst achieving better transfer learning performance.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item