Wednesday 26 January 2022

Microsoft Accessibility - Part 3: MSAA vs UI Automation

 Microsoft has had support for accessibility for a long time. Microsoft Active Accessibility (MSAA) got released with windows 95 in 1995. 10 years later, in 2005 they introduced UI Automation (UI), with Windows Vista. The two are compared here.

  1. Tree structure: Both expose a window or a UI in a hierarchical way represented by a tree.
  2. COM Interface: Both expose COM based interfaces for both assistive technologies (AT) to use, and for software developers to accessibility in their product.
  3. IAccessible, IUIAutomation: Each element of MSAA tree is a COM interface of type IAccessible. Each element of UIA is a COM interface of type IUIAutomation.
  4. Clients: The AT are called clients in both
  5. Servers, Providers: MSAA refers to UI or control offering accessibility as Servers. AUI refers to them as providers.
  6. Performance: UIA allows for caching properties, and tree hierarchy in a client, which leads to better performance.
  7. Flexibility: UIA allows for a far richer set of properties, and patterns to be exposed. It also allows for customs properties and patterns. Extending UIA is much easier.

Despite both of them exposing a tree model for a control the view seen by an AT may be different. Consider the following example.


This is an MFC dialog which an embedded list control. The list control has two columns (colA, colB), and several groups. Each group can have any number of items within it. Each item always has data for colA, while colB might be empty, as it is for item2 above.

Let's try and observe the hierarchy exposed by MSAA, and UIA. For this we will use Inspect tool. The tool has a drop down with which we can select if we are using MSAA, or UIA (Red arrow in the image below). Use RAW view (green arrow below) in both cases so that we can see entire hierarchy.


Following is the MSAA tree of the MFC dialog.

MSAA tree


Following is the UIA tree of the MFC dialog.

UIA tree

In both images the part highlighted by yellow represents the dialog's main window. The part highlighted in red represents the list control. Here is the comparison between the two.

  1. The tree structures are different between the two. Hence an AT using MSAA may convey significantly different information to a user than an AT using UIA.
  2. UIA better represents the visual hierarchy on the dialog. For example the list control has groups as its children, the groups further have the items as its children.
  3. UIA has a richer tree. Each item and sub item (colA, and colB values of items) of the list control has a node of its own. So if we want to refer to "ColB info4", we have a separate node for it in the tree. MSAA only has 4 children for list item body, one for each children. The group, and column structure of the items is not exposed as individual nodes.
  4. Each window of MSAA always has some common children - minimize/maximize buttons, scroll bars, etc. For windows in which these are not present (for example non top level windows) they are marked invisible.
The most important take away from this is that the two trees might not be the same. Moreover a control is free to either expose MSAA, or UIA, or both. A client should not make assumptions about their support. Most win32 native controls support both but the extent to which they are supported might be different. For example MSAA tree does not have separate tree nodes for colB sub items, while UIA has. Thus a client may infer different visual hierarchy from the two. Windows controls have bridging between MSAA and UIA, so many properties if specified using one API may reflect in the other. However this applies to common properties only, and then too not always.

References