Automate text replacement in TextEdit via AX tools
Session from pi-computer-use-sessions on Hugging Face by thomasmustier · MIT · condensed by Coders Talk
Click a segment to jump to that moment
20ssession
0interventions
by library · 10h
GOAL
Use accessibility automation to select all text in a TextEdit document and replace it with new content, then confirm the resulting AX value.
- +0Prompt · +0Screenshot TextEdit, select all text, replace with 'fully replaced ✅ 漢字', then quote the literal final AX value.
- +0mAgent did · +0mTook a screenshot of TextEdit, selected the full text range (0 to -1, matching 33 chars), then typed the replacement string into the AXTextArea via type_text.
- +0mOutcome · +0mFinal AX value confirmed as 'fully replaced ✅ 漢字', matching the intended replacement.
VERDICT
I'd keep this exact structured prompt next time since it worked cleanly end to end with no corrections needed. The AX-based select-all and type_text approach handled unicode and emoji characters correctly without any coordinate-click fallback.