LiteGUI 审稿意见与 Rebuttal 作战面板

This paper proposes LiteGUI, an SFT-free framework combining Guided On-policy Distillation with Multi-solution Dual-level GRPO for compact GUI agents. It achieves strong results on ScreenSpot-Pro, OS-World, and Lite-Bench

Strengths Addresses multi-solution ambiguity and long-horizon planning Simple, well-motivated training design with clear ablations Strong performance and data efficiency at the 2B and 3B scale Major Weaknesses The combination of hard nearest-GT selection and reverse KL may create a self-reinforcing mode-selection loop: the student’s current preference determines the target that subsequently reinforces that same preference. The paper claims improved multi-solution exploration but reports no diversity, solution-coverage, or policy-entropy analysis. The claimed on-device advantage is not empirically substantiated: the paper reports neither inference latency, memory nor the one-off distillation and training data collection cost, leaving the accuracy–efficiency trade-off relative to directly deploying the 32B teacher unclear. This is particularly relevant for LiteGUI-30B-A3B, whose total parameter footprint remains 30B despite activating only 3B parameters and whose performance in Table 1 is comparable to that of the teacher. The methodological novelty of MD-GRPO appears limited, as it mainly linearly scalarizes action-matching and plan-quality scores before applying standard GRPO. Despite being described as “dual-level,” both signals are collapsed into a single response-level advantage and uniformly credit all generated tokens, which may reinforce an incorrect action when accompanied by a high-quality plan, or vice versa. Prior work [1,2] on fine-grained credit assignment suggests that structured intermediate feedback can be more effectively incorporated through step-specific advantages. The paper lacks comparisons against such alternatives, sensitivity analysis of the reward weight, and the missing multi-solution=no, dual-level=yes ablation needed to isolate the planning reward. The main text and Appendix describe inconsistent GRPO objectives: Eqs. (18–19) use token-level importance ratios, whereas Appendix A.6 states that sequence-level importance sampling is used. Since these choices yield different clipping behavior and gradients, the actual training objective remains unclear. Minor Weaknesses Fig. 1 “taks list” is typo Tab. 1 only bolds the Avg best results [1] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models . 2024 [2] VinePPO: Refining Credit Assignment in RL Training of LLMs . ICML 2025

Given that recent long-horizon coding-agent systems such as GLM-5.2 have moved from group-wise optimization to critic-based PPO to accommodate highly variable, compacted trajectories, could the authors justify why GRPO is more appropriate for long-horizon GUI agents? In particular, please discuss the trade-off between GRPO’s lower training overhead and PPO’s finer-grained credit assignment under GUI-specific challenges such as costly environment interaction, stochastic visual states, and delayed rewards. A compute-matched empirical comparison would be valuable.

No. The appendix discusses annotation coverage and cost, training-time judge bias, generalization, and societal risks, but should also acknowledge the unvalidated GPT-5.4 evaluation judge, the lack of repeated OS-World or Lite-Bench runs, and the absence of on-device latency, memory, and energy measurements

This paper studies post-training methods for compact vision-language GUI agents. It proposes a two-stage pipeline that avoids conventional supervised fine-tuning. In the first stage, Guided On-policy Distillation provides the teacher model with privileged information derived from a human-verified set of valid actions. Among several variants, the proposed Most-Matched-GT method selects the valid action most similar to the student's sampled action and conditions the teacher on that reference. In the second stage, Multi-solution Dual-level GRPO combines two reward components: a local action reward that takes the maximum matching score over multiple annotated valid actions, and a planning reward assigned by a vision-language judge to the model's generated subtask list. The paper also presents a semi-automated trajectory-generation pipeline, a training collection containing 30K trajectories and 11K multi-solution annotations, and Lite-Bench, a 160-task evaluation suite covering web, terminal, and file-system interactions. Experiments with Qwen3-VL-2B and Qwen3-VL-30B-A3B report improvements on ScreenSpot-Pro, OS-World, and Lite-Bench, with ablations comparing standard SFT, standard on-policy distillation, different guided-distillation variants, and different GRPO reward components.

The paper targets an important practical problem: improving compact GUI agents without relying on increasingly large inference models. Its focus on multi-solution ambiguity is well motivated, since many GUI states admit several semantically equivalent actions and single-reference supervision can incorrectly penalize valid behavior. The use of a valid-action set both as privileged teacher context and as a max-over-solutions reward is a coherent design choice. However, the following technical issues currently limit the strength of the conclusions. The optimization procedure is described inconsistently between the main text and the appendix, making it unclear which formulation produced the reported results. For Guided-OPD, Equation (12) in the main text defines a sampled-sequence log-ratio objective: L_OPD=E_(y ˆ∼π_θ (⋅∣x) ) [∑ (j=1)^|y ˆ |▒(〖log⁡π〗 θ (y ˆ_j∣x,y ˆ (<j) )-〖log⁡π〗 T (y ˆ_j∣x ˜,y ˆ (<j) )) ]. In contrast, Appendix A.5 states that the implemented GKD objective directly minimizes the token-averaged divergence between the complete student and teacher next-token distributions: L_GKD=1/|T| ∑ (j∈T)▒D_KL (p_θ (⋅∣x,y ˆ_(<j) ) ⓜ‖ p_T (⋅∣x ˜,y ˆ_(<j) ) ). Although these formulations are related conceptually, they are not interchangeable as implementation specifications. The former is written as a log-ratio evaluated only at sampled tokens and requires clarification about how gradients through the stochastic sample are handled, whereas the latter requires full-vocabulary student and teacher logits and provides a directly differentiable per-prefix KL loss. This distinction is particularly relevant for Most-Matched-GT, because the teacher context g=h(A^*,y ˆ ). depends on the complete sampled output through a discrete matching operation. The paper should therefore clarify whether (g) is selected once and treated as fixed for the entire optimization step, where stop-gradient operations are applied, and whether Equation (12) is intended as an abstract objective or the actual estimator used in training. Similar inconsistencies appear in the GRPO description. Equation (18) uses token-level importance ratios, ρ_j^((i) )=(π_θ (y ˆ_j^((i) )∣x,y ˆ_(<j)^((i) ) ))/(π_(θ_old ) (y ˆ_j^((i) )∣x,y ˆ_(<j)^((i) ) ) ). whereas Appendix A.6 states that sequence-level importance sampling is used: ρ^((i) )=(π_θ (y ˆ^((i) )∣x))/(π_(θ_old ) (y ˆ^((i) )∣x) ). Moreover, Equation (14) defines the reward using only the multi-solution action reward and the subtask-planning reward, R^((i) )=R_ms^((i) )+λ_sub R_sub^((i) ). while Appendix A.6 reports that the actual training reward also includes a format/length component with fixed weights: R=0.60R_GUI+0.30R_subtask+0.10R_(format/length). These differences affect the gradient estimator, clipping granularity, and effective reward being optimized. The authors should reconcile the main-text equations with the appendix, provide one authoritative formulation or pseudocode for each training stage, and explicitly confirm which configuration was used to obtain the reported experimental results. The proposed GRPO stage is described as long-horizon reinforcement learning, but the formal objective is primarily per-state optimization. Equations (13)-(19) sample several candidate outputs for a fixed input state (x_t), score their immediate actions against (A_t^\ast), score their textual subtask plans with a judge, and normalize these scores within the group. The formulation contains no executed environment transition, no sampled future state, no trajectory-level return, and no terminal task-success reward. In particular, it does not optimize a return such as G_t=∑_(k=t)^T▒γ^(k-t) r_k. The planning reward evaluates whether the generated subtask text appears consistent with the current history, but it does not directly establish that following the plan leads to better future states, recovery behavior, or final task completion. The method may therefore be better characterized as contextual per-state policy optimization with structured reward shaping, rather than exploration over long-horizon trajectories. Since improved long-horizon planning and recovery are major motivations, the paper needs experiments stratified by trajectory length, explicit recovery or perturbation tests, and clarification of whether any training candidates are actually executed in the GUI environment. Otherwise, the current results do not isolate whether the gains arise from multi-solution action supervision, judge-model imitation, history conditioning, or genuine multi-step optimization.

Please reconcile the optimization formulations in the main text and appendix and clarify which formulations produced the reported results. Please provide pseudocode for one optimization step and explicitly state whether the student minimizes a full-vocabulary per-prefix divergence, ∑ j▒D_KL (π_θ (⋅∣x_t,y ˆ (t,<j) ) ⓜ‖ π_T (⋅∣x ˜ t,y ˆ (t,<j) ) ). a sampled-token surrogate, or a score-function estimator. Please also specify all stop-gradient operations and whether the teacher is queried once per complete sequence or once per token prefix. A precise and mathematically consistent clarification, together with confirmation that the reported experiments use that objective, would materially improve my assessment. If the implementation follows Equation (12) literally with ordinary detached sampling, this would be a major technical problem. Does MD-GRPO execute multi-step environment rollouts during training? For each candidate (\hat{y}_t^{(i)}), is the predicted action executed to obtain a subsequent screenshot, or are all candidates scored against an offline annotation at the same state? If training is per-state, please revise the long-horizon exploration characterization and provide targeted evidence for the planning component, such as performance by episode length, recovery from injected errors, loop/stuck-state subsets, and an ablation that removes the generated subtask memory while keeping the same action data. Evidence that the reward improves subsequent executed progress rather than only judge-assessed plan text would substantially strengthen the paper. How are benchmark separation, result discrepancies, and statistical uncertainty handled? Please describe de-duplication between Lite-Dataset and Lite-Bench at the instruction, task-template, application, screenshot/state, and trajectory levels. An unseen-template or unseen-application split would be particularly useful. Please also reconcile the ScreenSpot-Pro values 46.86 and 43.13 for Lite-GUI-2B, and the Lite-Bench baseline values 32.35 and 33.35. My score could increase if the authors demonstrate strict benchmark independence, resolve the table inconsistencies, and provide uncertainty estimates; it would decrease if substantial train-test overlap is present.

yes

This paper proposes LiteGUI, a reinforcement learning pipeline with distillation capacity for better agent performance. SFT has many problems, like overfitting on the dataset and limited performance. Thus, LiteGUI proposes to finetune the agent by RL with the assistance of an expert model. LiteGUI uses a stronger teacher model (i.e., Qwen3-32B in the paper) ot genrate or retrieve reference trajectories and guide the student model during on-policy training, which can better compact agent learning from its own action distribution rather than only offline imitation demonstrations. The final results demonstrates the superior performance of the proposed method.

Strengths: The paper is generally well written and structured. The reviewer can easily follow and understand the story. The motivation is clear and it is an important problem for real-world applications. The methodology looks good and sufficient for a clear demonstration. The results also look good, i.e., LiteGUI achieves the SOTA performance. However, there are several concerns: The claim "SFT for small-scale models often leads to overfitting...." (line 4) is not accurate. SFT fails because of a low amount of high-quality data, which is not the problem of SFT (or say, imitation learning). In fact, if the data scale is sufficient, SFT can show impressive results. For many real-world applications like autonomous driving and embodied AI, RL cannot be used due to the sim-to-real gap. SFT can already achieve satisfying results for these tasks. Thus, the authors should be careful with their claims, especially strong claims in the abstract and intro. The reason for small model distillation research is for on-device applications (line 1). However, in the whole paper, the authors do not mention or prove that your method can be applied to on-device applications. Does it mean that 2B or 3B is for on-device? 7B cannot? What about 0.5 or 0.6B models? The RL framework looks like the combination of many existing methods, e.g., distillation, GRPO, which is not new and impressive at all. For imitation learning, there is DAgger (very early work, in 2011), which uses an online expert for better supervision, which looks very similar to your proposed algorithms. It looks like your "new" framework is just for the very specific application with the combination of existing methods, i.e., small agent distillation. The authors might want to further claim their main contributions and new stuff. "To implement guided OPD and multi-solution dual-level GRPO, we leaverage our data generation pipeline to expand the training dataset" (lien 276). It looks like the authors use additional training data, which should be explained. Does the superior performance of your model come from the additional data? It is hard to fairly compare LiteGUI with past methods due to different training data and setting gaps, instead, the authors should pay more attention to the proposed method, i.e., the new RL distillation pipeline. Thus, the authors might want to do more analysis about "WHY the proposed method is better, why it is new, why it achieves better results than past RL/IL methods". Currently, the experiments are not convincing.

Please refer to the weakness part.

Weak novelty, combination of existing methods. Unconvincing experiments.

This paper proposes LiteGUI, a two-stage training framework for lightweight vision-based GUI agents. It first uses Guided On-policy Distillation, where the teacher receives privileged reference actions from a multi-solution action set, and then applies Multi-solution Dual-level GRPO with both action-level and planning-level rewards. The goal is to improve small on-device GUI models without relying on standard SFT.

strengths: good paper，see sum weakness: the evidence for Guided-OPD is relatively weak. Its improvement over standard OPD is modest, and the paper does not analyze how privileged reference actions affect teacher confidence, entropy, or token-level supervision. Since the teacher sees information unavailable to the student at inference time, there may be a train-test mismatch. (Rethinking On-Policy Self-Distillation for Thinking Models) The planning reward is useful in principle, and the rebuttal ablations partially support it, but the paper still needs stronger analysis of judge reliability and correlation with actual task success. More broadly, the final gains may come from curated data and strong teacher/judge supervision rather than the proposed algorithmic design alone.

How does privileged guidance change the teacher distribution compared with standard OPD? Can the authors analyze whether Guided-OPD improves real decision making or mainly increases answer-conditioned teacher confidence? How much of the gain comes from the training objective versus the curated multi-solution annotations?

see weakness

This paper presents LiteGUI, a novel post-training framework designed to develop small VL GUI agents (in the 2B–3B parameter range) without relying on SFT. The authors argue that SFT causes overfitting, catastrophic forgetting, and policy rigidity in lightweight models. To address this, they introduce a two-stage training paradigm: 1. Guided On-policy Distillation (Guided-OPD): Integrates generalized knowledge distillation into the GUI domain. It leverages human-verified valid action sets as privileged training-time information to dynamically align the teacher model's context with the student's execution intent, reducing teacher hallucinations and cognitive misalignment. 2. Multi-solution Dual-level GRPO (MD-GRPO): An RL framework using Group Relative Policy Optimization. It incorporates a macro-level subtask planning reward (evaluated by a strong VLM judge) alongside a micro-level multi-solution action matching reward to prevent "false negative" penalties from rigid single-action references. Additionally, the authors design a semi-automated data generation pipeline, releasing Lite-Dataset (30K trajectories with 11K multi-solution annotations) and Lite-Bench (a 160-task benchmark for Ubuntu system interaction). Experimental results across ScreenSpot-Pro, OS-World, and Lite-Bench show that LiteGUI sets a new state-of-the-art for small models and competes with much larger models.

Strengths: The paper uses Most-Matched-GT Guided OPD as privileged information to anchor on-policy distillation in multi-solution environments in the GUI domain. The evaluation is thorough, leveraging established benchmarks (OS-World, ScreenSpot-Pro) alongside a newly curated live environment benchmark (Lite-Bench). The inclusion of extensive ablation studies (Table 4) isolates the compounding value of Guided OPD, multi-solution verification, and dual-level rewards. Developing high-performing 2B–3B scale models that can execute complex, long-horizon computer tasks is impactful for practical on-device deployment where latency, privacy, and compute constraints are severe. The release of Lite-Dataset and Lite-Bench adds high-quality, multi-solution visual interaction data to the open-source community, addressing a notable data scarcity bottleneck. Weaknesses: While combining on-policy distillation (GKD) and GRPO is highly effective, both core optimization algorithms are known in the literature. The originality lies primarily in their domain-specific adaptation, reward structures, and framework integration The dual-level RL setup relies heavily on a 32B VLM judge (Qwen3-VL-32B) during training. The paper lacks a deep analysis or validation of the judge's own accuracy, bias, or potential agreement rate with human intent, which leaves questions about reward hacking open.

VLM Judge Reliability: Can you provide an evaluation or proxy analysis of the Qwen3-VL-32B judge’s reliability for the Long-horizon Planning Reward? For example, what is the agreement rate between the VLM judge scores and human evaluations on a subset of trajectories? Stating this would heavily increase confidence in the Quality dimension. Compute Overhead of Training: Bypassing SFT and moving straight to an on-policy rollout paradigm (GKD + GRPO with group size G = 16 ) typically incurs massive training-time sample complexity. Could you quantify the total GPU hours required for training Lite-GUI-2B compared to a standard SFT baseline?

The authors have done a commendable job mapping out limitations in Appendix A.10, particularly highlighting: • The multi-solution set being a finite approximation of a potentially infinite correct action space. • The human-in-the-loop verification cost for scaling up multi-solution data. • The training compute overhead generated by invoking an online VLM judge. • Potential safety risks (privacy leakage, unauthorized operations) from autonomous GUI control.