Przemyslaw Chojecki · ArXiv.org · 2025
We extend the moduli-theoretic framework of psychometric batteries to the domain of dynamical systems. While previous work established the AAI capability score as a static functional on the space of agent representations, this paper formalizes the agent as a flow $ν_r$ parameterized by computational resource $r$, governed by a recursive Generator-Verifier-Updater (GVU) operator. We prove that this operator generates a vector field on the parameter manifold $Θ$, and we identify the coefficient of self-improvement $κ$ as the Lie derivative of the capability functional along this flow. The central contribution of this work is the derivation of the Variance Inequality, a spectral condition that is sufficient (under mild regularity) for the stability of self-improvement. We show that a sufficient condition for $κ> 0$ is that, up to curvature and step-size effects, the combined noise of generation and verification must be small enough. We then apply this formalism to unify the recent literature on Language Self-Play (LSP), Self-Correction, and Synthetic Data bootstrapping. We demonstrate that architectures such as STaR, SPIN, Reflexion, GANs and AlphaZero are specific topological realizations of the GVU operator that satisfy the Variance Inequality through filtration, adversarial discrimination, or grounding in formal systems.
Cited by 0Open access
Revista, Zen, IA, 10 · Zenodo (CERN European Organization for Nuclear Research) · 2025
This paper explores the paradigm of autonomous architecture synthesis for self-improving agents, a critical advancement toward general-purpose artificial intelligence. Current AI systems are often constrained by fixed, human-designed architectures, limiting their adaptability and continuous learning capabilities in dynamic, open-ended environments. We propose a conceptual framework where agents can autonomously design, evaluate, and modify their own internal architectures, including their control flows, communication protocols, and even the selection and integration of internal modules and algorithms. Drawing inspiration from evolutionary computing, meta-learning, and reflective AI, this synthesis process allows agents to optimize their structure for enhanced performance, efficiency, and resilience, leading to true open-ended self-improvement. We discuss the necessary components for such a system, including reflective learning mechanisms, automated evaluation benchmarks, and secure self-modification protocols, and highlight its potential to accelerate AI development across diverse domains from robotics and manufacturing to complex financial platforms and network management. This work posits autonomous architecture synthesis as a foundational step towards endlessly innovating and robust AI systems capable of transcending human design limitations.
Cited by 0Open access
Emre Can Acikgoz, Cheng Qian, Heng Ji et al. · ArXiv.org · 2025
One paradigm of language model (LM) fine-tuning relies on creating large training datasets, under the assumption that high quantity and diversity will enable models to generalize to novel tasks after post-training. In practice, gathering large sets of data is inefficient, and training on them is prohibitively expensive; worse, there is no guarantee that the resulting model will handle complex scenarios or generalize better. Moreover, existing techniques rarely assess whether a training sample provides novel information or is redundant with the knowledge already acquired by the model, resulting in unnecessary costs. In this work, we explore a new test-time self-improvement method to create more effective and generalizable agentic LMs on-the-fly. The proposed algorithm can be summarized in three steps: (i) first it identifies the samples that model struggles with (self-awareness), (ii) then generates similar examples from detected uncertain samples (self-data augmentation), and (iii) uses these newly generated samples at test-time fine-tuning (self-improvement). We study two variants of this approach: Test-Time Self-Improvement (TT-SI), where the same model generates additional training examples from its own uncertain cases and then learns from them, and contrast this approach with Test-Time Distillation (TT-D), where a stronger model generates similar examples for uncertain cases, enabling student to adapt using distilled supervision. Empirical evaluations across different agent benchmarks demonstrate that TT-SI improves the performance with +5.48% absolute accuracy gain on average across all benchmarks and surpasses other standard learning methods, yet using 68x less training samples. Our findings highlight the promise of TT-SI, demonstrating the potential of self-improvement algorithms at test-time as a new paradigm for building more capable agents toward self-evolution.
Cited by 0Open access
Revista, Zen, IA, 10 · Zenodo (CERN European Organization for Nuclear Research) · 2025
This paper explores the paradigm of autonomous architecture synthesis for self-improving agents, a critical advancement toward general-purpose artificial intelligence. Current AI systems are often constrained by fixed, human-designed architectures, limiting their adaptability and continuous learning capabilities in dynamic, open-ended environments. We propose a conceptual framework where agents can autonomously design, evaluate, and modify their own internal architectures, including their control flows, communication protocols, and even the selection and integration of internal modules and algorithms. Drawing inspiration from evolutionary computing, meta-learning, and reflective AI, this synthesis process allows agents to optimize their structure for enhanced performance, efficiency, and resilience, leading to true open-ended self-improvement. We discuss the necessary components for such a system, including reflective learning mechanisms, automated evaluation benchmarks, and secure self-modification protocols, and highlight its potential to accelerate AI development across diverse domains from robotics and manufacturing to complex financial platforms and network management. This work posits autonomous architecture synthesis as a foundational step towards endlessly innovating and robust AI systems capable of transcending human design limitations.
Cited by 0Open access
Tennison Liu, Mihaela van der Schaar · ArXiv.org · 2025
Self-improving agents aim to continuously acquire new capabilities with minimal supervision. However, current approaches face two key limitations: their self-improvement processes are often rigid, fail to generalize across tasks domains, and struggle to scale with increasing agent capabilities. We argue that effective self-improvement requires intrinsic metacognitive learning, defined as an agent's intrinsic ability to actively evaluate, reflect on, and adapt its own learning processes. Drawing inspiration from human metacognition, we introduce a formal framework comprising three components: metacognitive knowledge (self-assessment of capabilities, tasks, and learning strategies), metacognitive planning (deciding what and how to learn), and metacognitive evaluation (reflecting on learning experiences to improve future learning). Analyzing existing self-improving agents, we find they rely predominantly on extrinsic metacognitive mechanisms, which are fixed, human-designed loops that limit scalability and adaptability. Examining each component, we contend that many ingredients for intrinsic metacognition are already present. Finally, we explore how to optimally distribute metacognitive responsibilities between humans and agents, and robustly evaluate and improve intrinsic metacognitive learning, key challenges that must be addressed to enable truly sustained, generalized, and aligned self-improvement.
Cited by 0Open access
Marc-Antoine Allard, Arnaud Teinturier, Victor Xing et al. · ArXiv.org · 2026
Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.
Cited by 0Open access
Marc-Antoine Allard, Arnaud Teinturier, Victor Xing et al. · arXiv (Cornell University) · 2026
Recent advances in large language models (LLMs) have enabled the development of autonomous agents capable of complex reasoning and multi-step problem solving. However, these agents struggle to adapt to specialized environments and do not leverage past interactions, approaching each new task from scratch regardless of their accumulated experience. We introduce Experiential Reflective Learning (ERL), a simple self-improvement framework that enables rapid environment adaptation through experiential learning. ERL reflects on task trajectories and outcomes to generate heuristics, capturing actionable lessons that transfer across tasks. At test time, relevant heuristics are retrieved based on the current task and injected into the agent's context to guide execution. On the Gaia2 benchmark, ERL improves success rate by 7.8% over a ReAct baseline, with large gains in task completion reliability, and outperforms prior experiential learning methods. Through systematic ablations, we find that selective retrieval is essential and that heuristics provide more transferable abstractions than few-shot trajectory prompting. These results demonstrate that reflecting on single-attempt experiences to extract transferable heuristics enables effective agent self-improvement.
Cited by 0Open access
Chen Wang, Keir Dorchen, Peter Jin · arXiv (Cornell University) · 2025
As systems trend toward superintelligence, a natural modeling premise is that agents can self-improve along every facet of their own design. We formalize this with a five-axis decomposition and a decision layer, separating incentives from learning behavior and analyzing axes in isolation. Our central result identifies and introduces a sharp utility-learning tension, the structural conflict in self-modifying systems whereby utility-driven changes that improve immediate or expected performance can also erode the statistical preconditions for reliable learning and generalization. Our findings show that distribution-free guarantees are preserved iff the policy-reachable model family is uniformly capacity-bounded; when capacity can grow without limit, utility-rational self-changes can render learnable tasks unlearnable. Under standard assumptions common in practice, these axes reduce to the same capacity criterion, yielding a single boundary for safe self-modification.
Cited by 0Open access
Areej Mustafa · 2026
Cited by 0Open access
Alex Sheng · arXiv (Cornell University) · 2024
We develop a simple and straightforward methodology to create AI computer agents that can carry out diverse computer tasks and self-improve by developing tools and augmentations to enable themselves to solve increasingly complex tasks. As large language models (LLMs) have been shown to benefit from non-parametric augmentations, a significant body of recent work has focused on developing software that augments LLMs with various capabilities. Rather than manually developing static software to augment LLMs through human engineering effort, we propose that an LLM agent can systematically generate software to augment itself. We show, through a few case studies, that a minimal querying loop with appropriate prompt engineering allows an LLM to generate and use various augmentations, freely extending its own capabilities to carry out real-world computer tasks. Starting with only terminal access, we prompt an LLM agent to augment itself with retrieval, internet search, web navigation, and text editor capabilities. The agent effectively uses these various tools to solve problems including automated software development and web-based tasks.
Cited by 2Open access
Zhang, Jenny, Hu, Shengran, Lu, Cong et al. · arXiv (Cornell University) · 2025
Today's AI systems have human-designed, fixed architectures and cannot autonomously and continuously improve themselves. The advance of AI could itself be automated. If done safely, that would accelerate AI development and allow us to reap its benefits much sooner. Meta-learning can automate the discovery of novel algorithms, but is limited by first-order improvements and the human design of a suitable search space. The Gödel machine proposed a theoretical alternative: a self-improving AI that repeatedly modifies itself in a provably beneficial manner. Unfortunately, proving that most changes are net beneficial is impossible in practice. We introduce the Darwin Gödel Machine (DGM), a self-improving system that iteratively modifies its own code (thereby also improving its ability to modify its own codebase) and empirically validates each change using coding benchmarks. Inspired by Darwinian evolution and open-endedness research, the DGM maintains an archive of generated coding agents. It grows the archive by sampling an agent from it and using a foundation model to create a new, interesting, version of the sampled agent. This open-ended exploration forms a growing tree of diverse, high-quality agents and allows the parallel exploration of many different paths through the search space. Empirically, the DGM automatically improves its coding capabilities (e.g., better code editing tools, long-context window management, peer-review mechanisms), increasing performance on SWE-bench from 20.0% to 50.0%, and on Polyglot from 14.2% to 30.7%. Furthermore, the DGM significantly outperforms baselines without self-improvement or open-ended exploration. All experiments were done with safety precautions (e.g., sandboxing, human oversight). The DGM is a significant step toward self-improving AI, capable of gathering its own stepping stones along paths that unfold into endless innovation.
Cited by 3Open access
Zhang, Jenny, Hu, Shengran, Lu, Cong et al. · arXiv (Cornell University) · 2025
Today's AI systems have human-designed, fixed architectures and cannot autonomously and continuously improve themselves. The advance of AI could itself be automated. If done safely, that would accelerate AI development and allow us to reap its benefits much sooner. Meta-learning can automate the discovery of novel algorithms, but is limited by first-order improvements and the human design of a suitable search space. The Gödel machine proposed a theoretical alternative: a self-improving AI that repeatedly modifies itself in a provably beneficial manner. Unfortunately, proving that most changes are net beneficial is impossible in practice. We introduce the Darwin Gödel Machine (DGM), a self-improving system that iteratively modifies its own code (thereby also improving its ability to modify its own codebase) and empirically validates each change using coding benchmarks. Inspired by Darwinian evolution and open-endedness research, the DGM maintains an archive of generated coding agents. It grows the archive by sampling an agent from it and using a foundation model to create a new, interesting, version of the sampled agent. This open-ended exploration forms a growing tree of diverse, high-quality agents and allows the parallel exploration of many different paths through the search space. Empirically, the DGM automatically improves its coding capabilities (e.g., better code editing tools, long-context window management, peer-review mechanisms), increasing performance on SWE-bench from 20.0% to 50.0%, and on Polyglot from 14.2% to 30.7%. Furthermore, the DGM significantly outperforms baselines without self-improvement or open-ended exploration. All experiments were done with safety precautions (e.g., sandboxing, human oversight). The DGM is a significant step toward self-improving AI, capable of gathering its own stepping stones along paths that unfold into endless innovation.
Cited by 0Open access
Lin, Long-Ji · KiltHub Repository · 2018
Abstract: "The purpose of this work is to investigate and evaluate different reinforcement learning frameworks using connectionist networks. I study four frameworks, which are adopted from the ideas developed by Rich Sutton and his colleagues. The four frameworks are based on two learning procedures: the Temporal Difference methods for solving the credit assignment problem, and the backpropagation algorithm for developing appropriate internal representations. Two of them also involve learning a world model and using it to speed learning. To evaluate their performance, I design a dynamic environment and implement different learning agents, using the different frameworks, to survive in it. The environment is nontrivial and nondeterministic. Surprisingly, all of the agents can learn to survive fairly well in a reasonable time frame. This paper describes the learning agents and their performance, and summarizes the learning algorithms and the lessons I learned from this study."
Cited by 0Open access
Zhixuan Bao, Zhuoyi Lin, Jiageng Wang et al. · arXiv (Cornell University) · 2026
Recent advances in large language models (LLMs) suggest strong potential for automating analog circuit design. Yet most LLM-based approaches rely on a single-model loop of generation, diagnosis, and correction, which favors succinct summaries over domain-specific insight and suffers from context attrition that erases critical technical details. To address these limitations, we propose AnalogAgent, a training-free agentic framework that integrates an LLM-based multi-agent system (MAS) with self-evolving memory (SEM) for analog circuit design automation. AnalogAgent coordinates a Code Generator, Design Optimizer, and Knowledge Curator to distill execution feedback into an adaptive playbook in SEM and retrieve targeted guidance for subsequent generation, enabling cross-task transfer without additional expert feedback, databases, or libraries. Across established benchmarks, AnalogAgent achieves 92% Pass@1 with Gemini and 97.4% Pass@1 with GPT-5. Moreover, with compact models (e.g., Qwen-8B), it yields a +48.8% average Pass@1 gain across tasks and reaches 72.1% Pass@1 overall, indicating that AnalogAgent substantially strengthens open-weight models for high-quality analog circuit design automation.
Cited by 0Open access
Byoungjip Kim, Youngsoo Jang, Lajanugen Logeswaran et al. · 2024
Large language models (LLMs) have shown the ability to solve complex decision-making tasks beyond natural language processing tasks.LLM agents based on few-shot in-context learning (ICL) achieve surprisingly high performance without training.Despite their simplicity and generalizability, ICL-based agents are limited in their ability to incorporate feedback from an environment.In this paper, we introduce Prospector, an LLM agent that consists of two complementary LLMs, an Actor and a Critic.To elicit better instruction-aligned actions from the LLM agent, we propose AskAct prompting that performs an additional selfasking step such as goal and progress checking before generating an action.Furthermore, to implicitly incorporate the environment feedback, we propose Trajectory Ranking that orders generated trajectories by predicting the expected total reward.Prospector encourages the LLM Actor to generate diverse (creative) trajectories, and harnesses the LLM Critic to select the most rewarding trajectory.On representative decision-making benchmark environments such as ALFWorld and WebShop, we empirically demonstrate that Prospector can considerably increase the success rate of given tasks, while outperforming recent advancements such as ReAct and Reflexion.
Cited by 1Open access
Zhixuan Bao, Zhuoyi Lin, Jiageng Wang et al. · arXiv (Cornell University) · 2026
Recent advances in large language models (LLMs) suggest strong potential for automating analog circuit design. Yet most LLM-based approaches rely on a single-model loop of generation, diagnosis, and correction, which favors succinct summaries over domain-specific insight and suffers from context attrition that erases critical technical details. To address these limitations, we propose AnalogAgent, a training-free agentic framework that integrates an LLM-based multi-agent system (MAS) with self-evolving memory (SEM) for analog circuit design automation. AnalogAgent coordinates a Code Generator, Design Optimizer, and Knowledge Curator to distill execution feedback into an adaptive playbook in SEM and retrieve targeted guidance for subsequent generation, enabling cross-task transfer without additional expert feedback, databases, or libraries. Across established benchmarks, AnalogAgent achieves 92% Pass@1 with Gemini and 97.4% Pass@1 with GPT-5. Moreover, with compact models (e.g., Qwen-8B), it yields a +48.8% average Pass@1 gain across tasks and reaches 72.1% Pass@1 overall, indicating that AnalogAgent substantially strengthens open-weight models for high-quality analog circuit design automation.
Cited by 0Open access
Tao Zhang · 2025
Despite the great success of large language models (LLMs), foundationmodel-based agents still face challenges in achieving autonomous selfimprovement: current iterative self-training paradigms are prone to capability collapse, while self-evolving coding agents lack the capacity to drive foundation model evolution, thereby limiting their potential for selfimprovement. To address these problems, an open-ended approach for neural network weight modification is promising. Therefore, a question is how a foundation-model-based agent can learn neural network weights via scaling compute at training and inference time, making it possible to leverage implicit meta-knowledge for self-improvement. In this paper, we argue that the autonomous self-improving AI systems should learn weights from diverse neural networks, e. g., LLMs, and treat their own weights as the real environment for exploring. We propose Generative Agent as Scientist (GAS), a self-evolving framework that aims to realize weight evolution in foundation models by applying open-ended approaches including, e. g., weight generation and weight regularization. During training stage, GAS is trained with large-scale neural network weight data via self-supervised learning. During inference stage, GAS reasons over its own weight patterns and implements continuous weight modification with meta-knowledge discovery, following a scientific paradigm that requires collecting its own internal weight states for diagnosis, formulating and testing falsifiable hypotheses, and then modifying its weights and knowledge base after evaluation. In order to avoid significant deviations, the neural network weights before and after weight modification are merged for regularization.
Cited by 1Open access
Long-Ji Lin · 1992
Cited by 165
Junzhe Wang, Mohammad Ibrahim Saleem, Jay Yoon et al. · IADC/SPE International Drilling Conference and Exhibition · 2026
Abstract Monitoring daily drilling mud data is essential for maintaining operational performance and rig-site safety. However, frequent variations in mud report templates across vendors make centralized parsing and digitalization highly labor-intensive and costly. Our previous study demonstrated the technical and economic feasibility of automating this process using generative AI (GenAI.) Building on that foundation, this work introduces a self-improving, template-agnostic agentic GenAI framework that fully automates the parsing process, including for new and previously unseen report formats, achieving substantial gains in efficiency, scalability, and adaptability for large-scale deployment and potentially transforming the operational model of mud monitoring services. An autonomous cloud-based workflow was developed to enable end-to-end mud report processing, covering email ingestion, GenAI-driven parsing, database integration, and dashboard visualization. At its core, the system employs a multi-agent architecture that integrates drilling and fluid domain expertise with generative AI reasoning to extract structured data from unstructured PDF reports. Within this architecture, a supervisory GenAI agent oversees the parsing process, evaluates performance, and autonomously optimizes the prompts used by a secondary extraction agent to improve accuracy and adaptability. Through this self-improving feedback mechanism, the system continuously refines prompt quality to accommodate new and evolving report templates. By learning from prior examples and analyzing report layouts, the framework generalizes effectively to unseen formats without manual intervention. The resulting solution autonomously parses complex mud reports with high accuracy and low cost, accelerates new vendor onboarding, and delivers reliable structured data in a fraction of the time required by traditional manual workflows.
Cited by 0
Rimom Costa · ArXiv.org · 2025
Large language models (LLMs) are fluent but largely static after pre-training; new or shifting knowledge is typically added with retrieval-augmented generation (RAG) or fine-tuning. RAG raises latency and engineering overhead and often fails to integrate facts; prompt engineering is brittle and can conflict with prior knowledge; fine-tuning is costly and risks catastrophic forgetting. We propose Instruction-Level Weight Shaping (ILWS): curated system instructions act as external, auditable pseudo-parameters updated after each session via reflection and user feedback. A Reflection Engine inspects conversation traces, diagnoses reasoning successes and failures, and proposes typed deltas $ΔK=(ΔS,ΔU,ΔT)$ over instructions, user preferences, and tools. Deltas are version-controlled, evaluated with a sliding window of 1-5 star ratings, auto-repaired on first failure, and rolled back on repeated failure. When an edit budget crosses a threshold, the agent compiles a rating-weighted synthetic set and distills matured instruction-space gains into parameters, converting prompt-space improvements into weight-space without downtime. ILWS makes explicit the low-rank shaping induced by context in transformer blocks, preserves governance, and removes per-call retrieval. In enterprise support it increased throughput 2.4-5.0x and cut audited hallucinations by about 80% versus a frozen baseline. In an Adobe Commerce Cloud proof of concept "L0 Support", it achieved 4-5x more tickets per hour and about 80% lower time per ticket, with autonomous instruction updates and optional tool synthesis. Because ILWS operates at the instruction layer until controlled distillation, it generalizes to dynamic domains (legal, medical, engineering) requiring adaptive reasoning, tool creation, and low-latency deployment.
Cited by 0Open access