LLMs Restructure Knowledge Hierarchies to Boost Hyperbolic AI Performance

In the rapidly evolving landscape of machine learning, hyperbolic geometry has emerged as a powerful tool for embedding hierarchical data structures, offering superior performance over traditional Euclidean s in applications ranging from recommendation systems to computer vision. However, the quality of these hyperbolic embeddings is intrinsically linked to the structure of the input hierarchy, often derived from knowledge graphs or ontologies, which can be suboptimal for learning tasks. A groundbreaking study from researchers at the University of Amsterdam now demonstrates that large language models (LLMs) can automatically restructure these hierarchies to meet criteria for optimal hyperbolic embeddings, leading to significant improvements in embedding quality across diverse domains. This approach not only enhances machine learning performance but also provides explainable justifications, empowering knowledge engineers to make informed decisions about ontology design.

Ology centers on a prompt-based LLM-guided hierarchy restructuring pipeline, as detailed in the paper 'Minimizing Hyperbolic Embedding Distortion with LLM-Guided Hierarchy Restructuring' by Melika Ayoughi, Pascal Mettes, and Paul Groth. The process begins by reformatting ontologies from various representations into a compact textual format using a pre-order depth-first traversal, making them interpretable for LLMs despite token length constraints. The core of the approach involves prompting LLMs, specifically GPT-4o and DeepSeek-V3, with guidelines derived from prior research: design for width over depth, deprioritize balance, handle additional node complexity, and avoid multiple inheritance. The LLMs then generate restructured hierarchies, with post-prompt validation ensuring structural changes, retention of original leaf nodes, absence of hallucinations, and format consistency. These restructured hierarchies are subsequently embedded using state-of-the-art construction-based hyperbolic s like Hadamard and HS-DTE, evaluated against metrics such as average relative distortion and worst-case distortion to quantify improvements.

From experiments on 16 diverse hierarchies, spanning domains like robotics, biology, advertising, and general object recognition, reveal consistent gains in hyperbolic embedding quality. For instance, on the ImageNet-1K hierarchy, LLM-guided restructuring with DeepSeek reduced average distortion from 0.297 to 0.220 using Hadamard embeddings and from 0.183 to 0.151 with HS-DTE, while worst-case distortion improved from 1.647 to 1.344 and 1.497 to 1.329, respectively. Similar improvements were observed across datasets like Pizza, Core50, and COCO-10K, with DeepSeek outperforming ChatGPT in most cases. The restructuring typically increased average branching factors by flattening tree structures, removing intermediate nodes, and promoting nodes to higher levels, correlating with lower distortion. An ablation study confirmed that all four recommendations contributed to these gains, with width optimization being the most impactful, and their combination yielded the best .

Of this research are profound for fields reliant on hierarchical data, such as AI, data science, and ontology engineering. By automating hierarchy restructuring, LLMs can bridge the gap between knowledge engineering and machine learning, enabling more effective integration of ontologies into hyperbolic deep learning pipelines. The explainability aspect is particularly noteworthy: LLMs provide detailed justifications for changes, such as flattening deep chains or avoiding multiple inheritance, allowing knowledge engineers to validate and adapt suggestions based on domain-specific constraints. This could accelerate the development of high-quality knowledge graphs in areas like robotics, where hierarchical classification is crucial, or biology, where taxonomic structures are complex. Moreover, the approach's flexibility across formats and domains suggests broad applicability, potentially enhancing recommendation systems, computer vision models, and other AI applications that leverage hierarchical semantics.

Despite its promising , the study acknowledges limitations, including s with very large hierarchies like ImageNet-21K and Visual Genome, where restructuring sometimes led to increased distortion, possibly due to LLM constraints or semantic loss during flattening. The reliance on specific LLMs and embedding algorithms may also affect generalizability, and the approach assumes single inheritance, which might not align with all real-world ontologies. Future work could explore integrating this into broader knowledge engineering ologies, testing with other LLMs, and extending it to hierarchy creation rather than just restructuring. As AI continues to grapple with complex hierarchical data, this research offers a scalable, explainable tool for optimizing knowledge structures, paving the way for more robust and interpretable machine learning systems.