Select and replace text via Accessibility API in TextEdit
Session from pi-computer-use-sessions on Hugging Face by thomasmustier · MIT · condensed by Coders Talk
Click a segment to jump to that moment
11ssession
0interventions
GOAL
Use accessibility-based selection and text replacement to select the word 'middle' in a TextEdit document and replace it with 'REPLACED', then confirm the final text.
- +0Prompt · +0Screenshot TextEdit, select range covering 'middle', replace it with 'REPLACED', then quote the final AX value.
- +0mAgent did · +0mTook a screenshot of TextEdit and captured the semantic window state, identifying the AXTextArea element @e1 containing 'before middle after'.
- +0mAgent did · +0mSelected the text range at start=8, length=6 on @e1, targeting the word 'middle'.
- +0mAgent did · +0mTyped 'REPLACED' into the selected range, updating the AXTextArea value to 'before REPLACED after'.
- +0mOutcome · +0mFinal AX value confirmed as 'before REPLACED after\n', matching the intended replacement.
VERDICT
I'd start with the same clear step-by-step instructions since the AX-based select and replace worked correctly on the first try. Using select_range with explicit start/length against the AXTextArea target avoided any coordinate-click guesswork, so no corrections were needed here.