Have you looked into using LDTP for non-testing purposes? (It's a functional test framework for GUIs which uses the accessibility APIs to implement AppleScript-like robust programmatic scriptability of GUI widgets)

I haven't had time to try it yet but I get the impression it should be possible and should produce an experience similar to your example session.