- Usability Standards
- Implementing Usability Standards in Code
- GUAT Framework Overview
- Implementation Iterations for Validation Tasks
- Language-based Guidelines
- Font-based Guidelines
- Spatial-based Guidelines
- Guidelines over all of an Application
- Reporting Results
- Types of Results
- Unimplemented Guidelines
- Guideline Classifications
- Gnome HIG 2.0 Coverage
- Section 508 Coverage
- A real-world test case: MySQL Tools
- MySQL Administrator
- MySQL Query Browser
- MySQL Workbench
- MySQL Migration Toolkit
- Results for interfaces shared by multiple tools
- Future Work
- Summary of unimplemented guidelines for Gnome HIG 2.0
- Summary of standards not implemented for Section 508
Usability guidelines for software interfaces can help improve many aspects of interface design including learnability, consistency and intuitiveness . This clearly can bring benefits to both developers and end-users. But there are few interface designers that know the existence of guidelines . Even if designers are aware of guidelines, it can be overwhelming for them to decide which set of guidelines to choose. Guidelines can be lengthy or unclear, which can make it difficult for developers to verify that their interfaces adhere to all of the guidelines. Another concern of manually verifying guidelines is that it leaves room for human error: designers might have incorrect interpretations of what some of the guidelines specify , or if guidelines are not carefully read and understood, subtle but crucial elements could be overlooked. This paper reports on the development of a tool called GUAT (GNOME Usability Analysis Tool), which automates the process for validating user interfaces against a subset of the GNOME Human Interface Guidelines 2.0 (GNOME HIG 2.0) and Section 508 standards.Codifying a full set of interface guidelines for automating checks is a complex problem: some guidelines may rely on human judgement (for example determining whether label concisely describes its associated control). Guidelines can also be partially implemented, where clauses are implemented if they do not depend on clauses that cannot be automated. Implementations can sometimes yield incorrect results (excluding buggy implementations) because the validation of a guideline based on static interface specifications can sometimes at best be inferred. Very few validation implementations for guidelines can be deduced.
An estimated 40% of GNOME HIG 2.0 was implemented, and only a single standard in section 508 could be codified.
A test-driven development process was employed for the implementation process of the selected guidelines to ensure that guidelines could be failed and passed correctly. In chapter , GUAT is used against a suite of open source tools developed by MySQLfor providing database-related tasks for MySQL databases. Limitations of GUAT arise from evaluating the results, which are of concern to automated guideline validation tools in general.
There are many different published interface guidelines. They generally fall into two classes: technology centric and general . Technology centric guidelines are usually platform specific, for example guidelines that are based on conventions used for all interfaces in a specific operating system or virtual machine.Two different guidelines were chosen for this investigation: A Technology-centric set of guidelines called GNOME HIG 2.0 and a general set of guidelines called Section 508.
GNOME HIG 2.0 was a logical choice because it is specific to the GNOME platform – which is Glades target platform as outlined in section . The GNOME HIG 2.0 guidelines cover a range of general usability principles and accessibility related principles1.2. They contain conventional guidelines that are specific for the GNOME desktop environment.
Section 508 is a law that requires Federal agencies’ electronic and information technology is accessible by people who have disabilities1.3. Although Section 508 law is enforced in the U.S. it has been made available to the public through a website to encourage accessibility in IT.
In a nutshell, GUAT reads Glade files and outputs results based on the given Glade file contents. Figure shows a high level diagram representing actual classes and relationships as well as other key entities. The GUAT framework takes one or more Glade files as its input. These files are loaded as GladeFile instances. The Validator has a set of ValidationTask‘s which perform a range of usability checks on the GladeFile‘s. The Validator performs all the ValidationTask‘s per GladeFile. When a ValidationTask is finished is assessment it returns a set of Result‘s. A Result is either a pass, warning or error. They contain information about why the result failed (if applicable), as well as any documented references (represented as DocReference‘s) give reason to why the particular content had failed. Line numbers as well as the actual XML content were added to the Result‘s for verbose feedback. Several observers (such as the TextResultWriter listens for results from the Validator and output the results in a specific format.
Document references are specific sections of a specific published usability/accessibility guidelines/standards. It is important that these are provided with error messages so that the developer can become educated in usability/accessibility aspects as these references usually provide reasoning’s for why the guidelines are constituted. To supply sufficient guideline information for each DocReference, not only the document name and section is provided but also a description of the actual guideline as well as an URL that refers to the original guideline documents.
To help make the code cleaner and easier to follow, an embedded database what written so that all document references for GNOME HIG 2.0 and Section 508 were centralised. In figure shows how the database is an embedded XML file and is queried via a utility class called DocReferenceRepository. Every time a document reference needs to be created the DocReferenceRepository supplies the appropriate document reference given a guideline name (for example “Section 508”) and a section number. This was helpful for keeping document reference data consistent (since specific guidelines can be requested more than once through the ValidationTask implementations) and also for keeping track of which references were implemented.
An abstract class called AbstractValidationTask developed cumulatively over time while implementing the standards to help with common parsing tasks, for example extracting access keys from a label.
Implementation Iterations for Validation Tasks
A test-driven development process was employed to ensure that the validation tasks were failing and passing Glade interfaces correctly. Testing validation tasks was of high importance to this study because it has given stronger grounds for results and conclusions conveyed in this study. Unit testing was used for checking utility code and validation tasks. A fail model was created in Glade Interface Designer, which breached all the guidelines that were codified one or more times. Unit tests were written which ran the validation tasks against a fail model, the results were checked that the exact amount of expected failures for each guideline were raised. Coding the guidelines was performed in iterations; where each iteration went through the following steps:
- Select a section and fully read it – noting codable guidelines.
- Add each guideline into the document reference XML database.
- Using the Glade Interface Designer breach the selected constraints for the fail model at least once.
- Write unit tests to assert expected number of failures in fail model for selected guidelines.
- Discover the Glade XML elements needed to be checked in order to validate the selected guidelines.
- Write code to validate the selected guidelines.
- Run unit tests: enter test->debug cycles until all tests pass.
- Back to step 1.
The final iteration checked that all guidelines are passable. A pass model was created in Glade Interface Designer once all guidelines were implemented. The pass model started as a copy of the fail model and was continuously amended until all errors and warnings were overcome. There were some instances where tasks were found to be impossible to satisfy because of bugs. The code was amended accordingly so that eventually all coded guidelines passed in the pass model and failed in the fail model.
A unit test was written to check that all document references in the document reference database were considered in the tests so that no guideline was missed. This helped keep track of that guidelines that were implemented.
Glade’s XML format is not publicly documented anywhere. This made it more difficult to codifying standards because relevant XML elements had to be discovered by inspecting existing Glade files. The problem became more sever because Glade XML omits all elements with default values. The Glade Interface Designer really helped with this because it provided all the possible Glade options and widgets. By enumerating over all the possible values and inspecting the Glade XML all the XML elements, attributes and values were discovered as well as the defaults. This highlights the importance of providing documentation for file formats – especially for open source software where developers may want to write software concerning file contents. It is possible that some Glade XML elements were missed in step 5 during the task implementation iterations (see above). Thus it would have been not only been easier to develop an automated guideline check for Glade if it were well documented but more robust.
There were many guidelines through out the Gnome HIG 2.0 reference which required text to follow sentence or header capitalization conventions. For example, a guideline section 11.1.3 requires all menu items and titles to use header capitalizationREFERENCE NEEDED. Section 8.3.2 outlines the two capitalization as follows:
- Header capitalization
- Capitalize all words in the element, with the following exceptions:
- Articles: a, an, the.
- Conjunctions: and, but, for, not, so, yet …
- Prepositions of three or fewer letters: at, for, by, in, to …
- Sentence capitalization
- Capitalize the first letter of the first word, and any other words normally capitalized in sentences, such as application names.
Checking for capitals after each white-space or word-breaking punctuation would not be sufficient for validating the capitalisation guidelines because types of words must be considered. So to validate whether or not widget adheres to these types of guidelines a language utility was developed. Figure shows a class diagram of linguistic services added to GUAT. The GnomeLanguageManager is a singleton which provides the interface for all linguistic services outside the package. All validation tasks query for sentence and header capitalization, as well as spell checks, via the GnomeLanguageManager. Before such services can be invoked a language must be set. At this stage only an English implementation has been implemented (GnomeEnglishLanguageUtil in Figure ). Other languages can be supported by implementing the GnomeLanguageUtil interface and not having to alter validating task implementations. Not all languages will share the same rules as English when it comes to capitalization (as noted in section 8.3.2), for example, Swedish has no concept of Header capitalization.The GnomeEnglishLanguageUtil supports multiple dictionaries: for example US, Canada or UK dictionaries. These are selected via the GnomeLanguageManager. In order to classify word types three types of dictionaries must be given; the dictionary of all words, all prepositions and all conjunctions. This gives the means to validate header capitalization. Only one dictionary was gathered for this tool but would be very simple to add support of other dictionaries.
A guideline in section 8.2.3 in Gnome HIG 2.0 states: “Be consistent. Use the same spacing, alignment, and component sizes in all dialogs appearing in your application…”. Glade files can contain multiple windows, but an application can specify the entire applications interface over many Glade files. For example many (or possibly all) of the standard GNOME applications use one Glade file per window. GUAT was therefore extended to support validation of guidelines over all glade files provided as input. Such tasks are refereed to as global tasks.After all Glade files are processed, PostValidationTask‘s are run. As indicated in Figure , a PostValidationTask is a ValidationTask. Initially they are treated like an ordinary ValidationTask by the Validator: where they inspect Glade files one by one. The Validator then invokes a post-validation method. All information extracted from the first (core) validation phase is then used. This therefore requires that all files passed into a GUAT session should be from the same application (or many applications in the same software suite for keeping layouts consistent).
shows some sample results written in HTML format. It was important to note to the users that the guidelines were only a subset of the full guidelines. This was achieve be adding a note in the footer of the HTML document as shown in Figure . The layout was refined while amending the pass model as discussed in Section . The code was useful for testing cycles, but may not be useful for target users as they would probably not directly edit the actual Glade XML.
There were five different classifications realized while implementing HIG and Section 508 guidelines for Glade:
- Human judgement
- There were many guidelines that could not be automated because they required human judgement. Such guidelines are generally opinionated, where an interface could be argued whether they pass or fail. Three different classes of human judgement were realized: visual, textual and conditional:
- Based on the visual content of an interface element, where even when supplied with visual data (such as pixels) a compelling result can be inferred. For example, a guideline that states the design of icon for a button must be suggestive of what the action of clicking that button would do. Even if it were possible to use state of the art image classification techniques, such guidelines are opinionated; therefore cannot be reduced to a pass or fail result.
- Based on the meaning of textual content. For example, a guideline that states that tip tool texts for button widgets must clearly describe its function.
- Content Relationships
- Based on whether content is related by the shared purpose of the widget. For example, a guideline that states that views with a lot of widgets should be positioned in groups, where a group of widgets share a similar purpose.
- Guidelines that are only possible to validate by accessing the behaviour of interface. For example, whether the click of a button responds with visual feedback, and responds within a specific time threshold.
- Impossible to violate
- Guidelines which cannot possibly be violated with Glade due to Gtk and the Glade interfaces both enforce such guidelines. For example, guidelines stating that windows should have close boxes as the right-most button on the top of a window. Such guidelines are conventional, that is, specific to a platform or language or general conventions that have been used by many software interfaces in the past. This classification can be abstracted for any format of interface markup, for example Microsoft User Interface guidelines might be impossible to violate if using .NET for the interface specification.
- Automatable but out of scope
- These are guidelines were it is conceivable to automate them (that is, they do not require human judgement or behaviour assessment), but depends on information not supplied within Glade files. For example, a guideline stating that the application should provide a desktop item (shortcut) within the applications menu cannot be validated by looking at the glade files, however it could be validated given the (automated) installer (by checking the application menu on the Gnome desktop after installation). This classification can be abstracted for any format of interface markup.
- Statically Automatable
- Guidelines that can be codified given only the interface markup, not the source or binaries. Such guidelines can be implemented and are implemented in GUAT.
Figure illustrates the crossovers with the different classifications. It highlights that guidelines can fall into more than one classification, for example a guideline could be classified as being both dynamic and requiring human judgement. The diagram also shows which classifications are mutually exclusive, for example a guideline cannot be classified as both dynamic and statically automatable; as their very definitions are converse. Also note how Figure indicates that guidelines for all classifications can be impossible to violate depending on the interface specification and guideline.
A total of 143 of the Gnome HIG 2.0 guidelines were codified. Many of which can be violated in multiple ways. For example, section X can be violated if there is no label to the left of or above of a slider, and if there is a label left of or above of the slider, the guideline can be still violated if the label does not have an access key, or if the label does not follow sentence capitalization rules.
A approximated method was used for quantifying how much of Gnome HIG 2.0 was codified. Gnome HIG 2.0 contain 13 major sections. The following sections were discounted from being regarded as specifying one or more guideline:
- Section 1: Usability Principles
- Provides a background to the guidelines. Although some guidelines are mentioned within this section they are only used as concrete examples and are covered in later sections (that are already considered).
- Section 12: Checklists
- Essentially a guide for designing interfaces accordingly to the Gnome HIG 2.0 reference.
- Section 13: Credits
- The Contains credits of the authors, reviewers and contributors.
The total amount of guidelines were estimated by counting all listed items under the “guidelines” headings in the guideline documents. For sections that did not have guideline heading, they were counted as only having one outline. It is estimated that are total of 354 guidelines in the Gnome HIG 2.0, giving an estimation of 40% of the Gnome HIG 2.0 being covered.
The method of counting guidelines is not accurate because some guidelines can be listed in other forms such as tables or paragraphs. Thus the estimation of coverage for Gnome HIG 2.0 is an over-estimation of what was actually covered. However there were many guidelines that could not be breached (as outlined below) because they were handled by Gtk, such guidelines are thus implicitly verified as passes, thus by excluding these guidelines as implemented when quantifying coverage the estimation is an underestimate. Although overall it still would probably be an over estimation.
Only one standard in section 508 was able to be codified in GUAT: standard G under section 1194.21, which states: “Sufficient information about a user interface element including the identity, operation and state of the element shall be available to assistive technology. When an image represents a program element, the information conveyed by the image must also be available in text.” This standard code be statically violated where icon elements did not provide a textual alternative (such as a tool-tip or an alternative label). It was more difficult to make a connection between the Section 508 standards and the Glade markup than it was for Gnome HIG 2.0 because of Section 508 being more abstract.Section 508 is broken for into four sub parts Below outlines the sub parts and what sub parts were considered as being able to be validated in code:
- Sub-part A
- General: Essentially outlines the scope and coverage of the standards. Does not specify standards that can be validated in any form.
- Subpart B
- Technical standards: Contains several sub sections of standards.
- Subpart C
- Functional performance criteria: Contains several sub sections of standards.
- Subpart D
- Information, documentation, and support: These are more service based standards rather than the actual interface itself – hence not material.
Sub part B and C contains statards which can be conceivably validated automatically. However the majority of these are not relevent to application interfaces, only subsection 1194.21: Software applications and operating systems, were relevent for subpart C. Sub section 1194.22 for example was not relevent as it was specific to web applications. All except for one of the standards outline in sub section 1194.21 and subpart C were unable to be codified because only one of the stardards was classified as being statically automatable. See Appendix for summary of Section 508 standards that were not implemented.
A real-world test case: MySQL Tools
GUAT was implemented with a test-driven style of development, however only two models were used for testing and those models were somewhat biased in that they were developed specifically for tests. This chapter evaluates GUAT by validating a popular suite of GUI tools released by MySQL for database development. Limitations of GUAT are discovered, which are limitations that are of concern for all types of automated tools in general.There are four open source tools available to the general public which use Glade on Linux ports. Each of the tools source code includes the Glade files for all of the interfaces. The four tools are MySQL Administrator, MySQL Query Browser, MySQL Workbench and MySQL Migration Toolkit.
A total of 2,025 errors and 27 warnings resulted from validating the 32 glade files that specifics all of the MySQL tools. These errors and warnings collectively referenced to a total of 2,692 guidelines in both the GNOME HIG 2.0 and Section 508 (note that a single result can have multiple document references). A total of 59 unique guidelines were violated.
Figure plots all of the guidelines violated by (top most) section name. Note that Section 508 refers to the one and only violatable section outlined in section TODO: The ref for that section, all other sections refer to the Gnome HIG 2.0 reference. As shown in Figure the majority of violations are attributed to visual design guidelines. These failures are mostly spatial-based guidelines (under section 8.2: Window Layout) which are validated with an element of uncertainly as described in section TODO: Spatial guidelines ref. The controls section is also responsible for a relatively significant amount of violations. This would be because the controls section contains most of the guidelines as there are 19 sub-sections under controls (for example section 6.14 covers all guidelines with Listbox controls).
Table contains the five guidelines that were violate the most cumulatively over all of the Glade files – which happen to be all GNOME HIG 2.0 guidelines. Section 8.2.3.a is a window layout guideline, for which the GUAT implementation has a higher level certainty of producing genuine results than other spatial based guidelines. The overwhelming amount of window layout based errors might suggest that many false negatives are being produced by incorrect implementation, however these guidelines apply to every widget in a windows and therefore are validated the most. Furthermore this error can cascade: thus one adjustment to the error-some user interface could fix multiple violations of these guidelines.
The MySQL Administrator tool allows users to administer and monitor their MySQL environments. Figure shows an annotated screen-shot of MySQL Administrator’s main window in the Glade Interface Designer. It highlights some of the violations detected by GUAT.
The MySQL Administrator tool contains 15 windows within 14 glade files, which excludes common interfaces shared between tools (see section TODO: Sec ref for commons). Over all of the Glade interfaces 751 errors and 19 warnings, violating guidelines 996 times4.1. The guideline that was violated the most was GNOME HIG 2.0 section 8.2.3.a with 239 violations.
The window in Figure has an error regarding the use of Frames within Frames. A guideline in GNOME HIG 2.0 Section 6.19 states that frames should not be nested to avoid visual clutter. The purpose of a Frame in Glade is to group widgets together. However the window in Figure appears to use a Frame as a placeholder for placing other interface content to be loaded in at run-time (shown in figure ). Note that the dynamically placed content in the Frame shown Figure is validated because it is common content defined in separate Glade files. It is arguable that the inner-frame error is not really a violation because it does not seem to be the designers intent to use the Frames in the ways described under Section 6.19 in GNOME HIG 2.0. That is, the context of the Frame in the window in Figure is different to the context of a Frame covered in the guideline.
MySQL Query Browser
The MySQL Query Browser is visual tool for creating, executing, and optimizing SQL queries for MySQL Database Servers. Figure shows an annotated screen-shot of MySQL Query Browser’s main window in the Glade Interface Designer. It highlights some of the violations detected by GUAT.
The MySQL Query Browser contains seven windows within four glade files (excluding common interfaces shared with other tools). Over all of the Glade interfaces 292 errors and five warnings, violating guidelines 377 times. The guideline that was violated the most was GNOME HIG 2.0 section 6.7.a with 77 violations.
The toolbars window Figure are specified at horizontal box containers containing buttons with labels and/or icons, as well as separators for grouping and aesthetics. GUAT however does not interpret these buttons and labels as toolbars: only toolbar and toolbutton widgets are considered to be toolbar elements. Notice that in Figure there are no guidelines violated under the GNOME HIG 2.0 toolbar section (section 5) because MySQL chooses to specify all their toolbars in this way. A Toolbar is a UI concept which can be composed of other UI concepts (for example strategically aligned buttons) in many ways. This is a limitation of GUAT: as it does not understand what the purpose/intent of a widget. This issue can be abstracted for other concepts in other Interface specification systems.
A keyboard navigation guideline in GNOME HIG 2.0 is violated because the menu item “Bookmark” in the “work area” window assigns a keyboard accelerated reserved for the common application menu item “Add Bookmark”. Clearly the designer was on the right track, it is arguable that it is not really an error because there is no “Remove Bookmark” present therefore it might be obvious that invoking “Bookmark” will add a bookmark. A better error message would be to notify the designer that they should rename the label instead of raising the error based on the keyboard navigation guideline.
MySQL Workbench provides facilities such as visual database design, generation and documentation. Figure shows an annotated screen-shot of MySQL Workbench’s main window in the Glade Interface Designer. It highlights some of the violations detected by GUAT.
The MySQL Workbench contains four glade files specify one window per file (excluding common interfaces shared with other tools). Over all of the Glade interfaces 371 errors and seven warnings, violating guidelines 531 times. The guideline that was violated the most was GNOME HIG 2.0 section 11.1.1.d with 77 violations, mostly due to menu items not following the appropriate capitalization rules. GUAT detects 55 failures based from the GNOME HIG 2.0 Window Layout section (a visual design subsection) for the window in Figure .
The combobox controls horizontally aligned on the left of the “Tool Options:” in Figure actually reside in a tabbed notebook control (a Gtk control that manages tab pages). There are six different tab pages within this notebook: where each tab page is missing a label. The missing labels are considered an error as a guideline in the notebook controls section of the GNOME HIG 2.0 guidelines states that tab pages must provide labels (section 6.16). The designer however intends to select the tabs through another mechanism thus have hidden labels. This use of tabbed notebooks was not considered when developing GUAT and would be a quick fix to only raise an error if the tabs are visible.
The combobox controls horizontally aligned on the left of the “Tool Options:” in Figure do not contain any items. This violates a GNOME HIG 2.0 guideline that states that comboboxes should not be used for storing less than three items. However these are combo boxes are populated at run-time thus may not violate the guideline. An exception could be introduced to ignore cases where comboboxes are empty since it is likely that the contents such comboboxes are not statically defined.
Another case with the violation of a keyboard navigation guideline as defined in section occurred where a menu item called “Delete Selected” assigns a keyboard accelerated assigned for the common menu item called “Delete”. Once again it could be argued that this guideline is not breached, and in this particular case by changing it to just “Delete” might be less intuitive and thus worse off in terms on usability that if it did not use the standard labelling.
The MySQL Migration Toolkit provides facilities to migrate proprietary databases to MySQL. Figure is a screen-shot of a window for the Migration Toolkit in the Glade Interface Designer. It highlights all five of the violations detected by GUAT.
The migration toolkit only consisted of a single window (as shown in Figure ). Most of the migration toolkit content is defined in the common interfaces shared by multiple tools. The next section covers such common interfaces.
Results for interfaces shared by multiple tools
The MySQL suite of graphical database tools share many interfaces: there are nine Glade-files/windows that are reused amongst all of the suite. Figure shows an annotated screen-shot of a connect dialog shared amongst most of the tools.
Over all of the Glade interfaces 608 errors and eight warnings, violating guidelines 809 times. The guideline that was violated the most was GNOME HIG 2.0 section 8.2.3.a with 110 violations.
Glade uses the Accessibility Toolkit (ATK) which could open up more areas to do with automating accessibility based guidelines.Glade interfaces specify signals to point to methods in code. Use of signals could be the road to dynamic checking with Glade. For example a external tool could invoke the signals per widget and wait for a response. However some signals might lead to more complex responses that requires human intervention, for example an input dialog that leads to a range of many different responses.
Support for multiple sets of guidelines is another area of future work. With the more guidelines supported the more problems are faced with conflicting guidelines thus yielding unsatisfiable errors . Even duplication of errors might become overkill in identify usability or accessibility error in interfaces.
Summary of unimplemented guidelines for Gnome HIG 2.0
|2||Covers how an application should integrate with the Desktop once installed.||These guidelines are all out of scope since such information is not included in Glade markup.|
|3||Covers all aspects of windows: how they should look, what buttons there should be and what types of titles should be used – all depending on the type of window and/or type of application.||Most of this section cannot be automated because information is out of scope: window types are not specified (although there are window hints, there are not still specific enough). Many guidelines are also impossible to breach because of Glade limiting the way windows can be customized. Naming of windows require human judgment to determine whether or not a title fits the application according to the type of application. This section also covers modality: stating that application should not use system modal windows, however such guidelines are impossible to violate due to Glade only supporting Application modal windows.|
|6.3||Covers control sensitivity in terms of enabled/locked states for controls.||These guidelines are classified as requiring dynamic assessment.|
|7||This section is about feedback. It has the areas: including characteristics of responsive applications, acceptable response times, responding to user requests, types of visual feedback, choosing appropriate feedback and allowing interruptions.||The majority of the feedback sections were unable to be statically automated because they were dynamic. This entire section could not be codified because they all were classified as either or a combination of requiring human judgement or dynamic assessment.|
|8.1||Covers some visual accessibility topics. Defines a recommended color palette to be used for all elements of an interface. Mentions they color should not be the only means to convey information.||These guidelines essentially require visual human judgement. Although tools like vischeckREFERENCE NEEDEDcan simulate color-related visual impairments, such tools still require human intervention to determine whether or not an interface passes such guidelines. All widget colors except for Pango text and images are defined by user-themes, thus many guidelines cannot be violated for the majority of glade interfaces.|
|9||Covers many aspects of icons. Including styles of icons (perspectives and lighting), kinds of icons, designing effective icons and making icons accessible.||This entire section can be classified as requiring visual human judgement. The accessibility guidelines could partially be automated since it states that high and low contrast icons should be provided, however a high/low contrast design could still be no better off than a color design. This section is very opinionated, for example accessibility icons should be a metaphor for the original icons.|
|10.1||Covers aspects of mouse interaction such as drag and drop behavior or widget/data selection.||Some of this section was codable (for example checking for small mouse-targets), but the majority of guidelines were classified as being dynamic due to most of the content related to behaviour of interface elements with the mouse.|
|11.1||Covers language aspects of labelling controls, such as keeping descriptions concise.||Many of these guidelines are classified as textual human judgment. Some guidelines require human judgement to determine whether a text label is related to a control or not.|
|11.2||Covers error and warning messages: essentially defining what is considered a good warning or error message.||These guidelines are classified as textual human judgment. They are also classified as dynamic because errors messages and warnings are usually created at runtime.|
Summary of standards not implemented for Section 508
|1194.21||Technical standards for software applications and operating systems.||Most of these require a combination of human visual judgement and the ability to dynamically assess the interfaces. Note that standard A in this section, which states “When software is designed to run on a system that has a keyboard, product functions shall be executable from a keyboard where the function itself or the result of performing a function can be discerned textually.”, is almost impossible to breach since all Gnome elements are accessible with a keyboard by default. However it is conceivable that functions could be implemented without a graphical interface which would be out of scope and require dynamic assessment anyway.|
|1194.31||All of subpart C: Functional performance criteria. Covers how modes of operation and information retrieval should be accessible in different ways to accommodate for users who are visually or orally impaired.||These are a mix of dynamic and visual human judgement classed guidelines.|