TITLE: New Framework Speeds Neuro-Symbolic AI Programming

A new agentic framework called AgenticDomiKnowS promises to simplify the creation of neuro-symbolic AI programs, allowing users to describe tasks in natural language and generate functional code in minutes rather than hours.

The system addresses longstanding challenges in integrating symbolic constraints with deep learning models, which aim to enhance AI robustness, interpretability and data efficiency. By automating the programming process for the DomiKnowS library, AgenticDomiKnowS lowers the expertise barrier, enabling both novices and experienced developers to build complex AI applications more quickly.

Access to the framework's user interface is available online, providing tools for immediate testing and refinement.

Background on Neuro-Symbolic Systems

Neuro-symbolic systems combine the pattern-recognition strengths of deep learning with the logical consistency of symbolic reasoning. These hybrid approaches seek to overcome limitations in purely neural models, such as vulnerability to adversarial inputs and lack of explainability.

However, developing such systems has been hindered by the need for specialized knowledge of formalisms and syntax in tools like DomiKnowS. This Python-based library allows users to define conceptual graphs encoding concepts, relations and logical constraints, which are then linked to deep learning components.

DomiKnowS excels in tasks requiring domain knowledge injection, such as ensuring prediction consistency in causal chains. For instance, in procedural question-answering datasets like WIQA, it enforces transitivity rules across related queries— if action A affects B and B affects C, then A must logically impact C.

Despite its power, DomiKnowS demands manual encoding of rules, making it error-prone and time-intensive for users unfamiliar with its declarative language.

Previous efforts to automate DomiKnowS program generation relied on large language models (LLMs) as coding assistants but were limited to partial components, like conceptual graphs, and required heavy user intervention due to syntax errors and incomplete mappings.

How AgenticDomiKnowS Works

AgenticDomiKnowS introduces an agentic workflow that decomposes program generation into discrete stages: knowledge declaration, model declaration and integration. Unlike monolithic code synthesizers, it creates, tests and refines each section independently, isolating errors for targeted fixes.

The process begins with a free-form task description from the user. AI agents then produce the knowledge declaration, defining the conceptual graph and constraints. A separate agent handles model declaration, specifying how neural sensors attach to graph elements for predictions.

Execution tests follow each generation step. Syntactic issues trigger iterative repairs via code execution feedback, while semantic errors are caught by an LLM reviewer in a self-refinement loop. This modular approach leverages recent advances in LLMs for logical structuring, though it accounts for their weaknesses in domain-specific syntax by breaking tasks into manageable parts.

Optional human-in-the-loop intervention allows users to review and edit intermediate outputs, particularly beneficial for those versed in DomiKnowS. The framework culminates in a complete, executable Jupyter notebook, pre-loaded with vision-language models for instant inference on tasks like image classification or natural language processing.

The interactive web interface visualizes the workflow, displaying generated code, execution logs and graph representations to facilitate verification.

System Components

Retrieval-Augmented Generation (RAG)

AgenticDomiKnowS employs RAG to ground generations in DomiKnowS documentation and examples, mitigating hallucinations common in LLMs untrained on niche libraries.

Knowledge and Model Declarations

Core to the system, these declarations mirror DomiKnowS structure. Knowledge sections encode logical rules, while model sections integrate predictive components, ensuring seamless coupling.

User Interface

The backend orchestrates agent interactions and error handling, while the frontend offers an intuitive dashboard for inputting descriptions, monitoring progress and intervening as needed.

Experiments and Evaluation

Testing spanned diverse tasks across natural language processing, vision and constraint satisfaction problems. NLP examples included hierarchical news classification, spam detection, sentiment analysis, procedural text understanding, causal reasoning, belief-consistent question answering and logical reasoning.

Vision tasks covered hierarchical image classification variants and constrained digit recognition. Constraint problems featured Sudoku and the Eight Queens puzzle.

Datasets were standard benchmarks, with language models like GPT-4 and Llama variants used for generation. Automated evaluation measured code correctness and task performance, while human studies assessed usability.

Results showed AgenticDomiKnowS producing valid programs 80-90% of the time without intervention, with full success after refinements. Non-expert users completed programs in 10-15 minutes, compared to hours for manual DomiKnowS coding. Experienced users reported 50-70% time savings.

Human evaluations, guided by structured instructions, confirmed the framework's effectiveness in reducing cognitive load and error rates.

Detailed metrics highlighted improvements in global consistency for tasks like WIQA, where injected constraints boosted accuracy by enforcing transitivity.

Related Developments and Future Directions

The framework builds on prior neuro-symbolic tools and LLM-assisted programming but innovates with its staged, agentic design. It outperforms general coding assistants, which failed to generate viable DomiKnowS code in benchmarks.

Future work may expand to other neuro-symbolic libraries, incorporate more multimodal tasks and enhance agent autonomy to minimize human input further.

This advancement could democratize neuro-symbolic AI, fostering applications in reliable decision-making systems for healthcare, finance and beyond.