PORTool: A reward-tree and importance-aware policy optimization method for multi-tool integrated reasoning
PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning
AI BriefingThe research team proposes the PORTool algorithm to address ambiguous credit assignment caused by relying solely on outcome rewards in multi-tool integrated reasoning. Through importance-aware policy optimization, the method strengthens agents’ tool-use capabilities under outcome-level supervision while enabling step-level reward assignment. PORTool generates reward trees to identify critical decision steps, guiding models more precisely toward effective tool-call sequences and improving the efficiency and reliability of complex task solving.
Source: Apple Machine Learning Research · machinelearning.apple.com