A Plan for the Current & Desired Future States of Digital Services for Readability, Usability, Language Access, & Accessibility (RULA)

MEMORANDUM

DATE: August 23rd, 2019
TO: Office of New Americans
FROM: Design Office
RE: Department of Innovation & Technology : Readability, Usability, Language Access, & Accessibility (RULA)  Policy Implementation Current & Future state

EXECUTIVE SUMMARY

The Mayor’s Office asked DoIT to review the city’s website and digital communications for usability, language, disability access, and other under-utilized opportunities. This work has dovetailed with ongoing work to respond to the findings of the Inspector General’s report “Language Access Ordinance Compliance Audit Follow-up Inquiry.”

This document is not a substitute for the language access plan of any City of Chicago department, including the Department of Innovation & Technology. Instead, it details the current and future state of DoIT’s capacity to support four aspects of a person’s experience of our digital information and services: readability, usability, language access, and accessibility. It details the products, the teams, and the workflows that succeed and those which can be improved upon to provide a better experience for all Chicagoans.

Due to recent events, accessibility is of critical importance. In August the U.S. Chamber of Commerce requested the Supreme Court to decide whether or not the Americans with Disabilities Act should also apply equally to the internet. With the possibility of the Department of Justice creating obligatory guidance to comply with Title III of the Act, it would mean accessibility standards could become the law of the land.

CURRENT STATE

Our current state varies widely from platform to platform on all four aspects of customer experience, some competing with leading edge cities like New York or Seattle while others are built without consideration for all our communities. All can be improved to provide a better, more consistent experience across products. Overall, Chicago is a policy leader in accessibility and digital equity but is uneven in its implementation of process and product to support that policy. We made significant ad hoc improvements in 2018, and have been working diligently to analyze and build a pilot case for our translation options in 2019 with our work to make 311 multilingual.

The ‘Readability’ category shows a range of values from several pages where possible.

READABILITY

Readability is a key concern for our website, as best practices for government communications suggest targeting the reading level of your audience. Oftentimes people will suggest writing to an 8th grade level, but our content is intended for a wide variety of people and can cover topics of interest to people from many different backgrounds and education levels. The important thing is to write appropriately for our audience.

For many of our sites, the reading level is appropriate but the content is unclear, wordy, and disorganized across multiple pages due to the patchwork nature of content updates over time.

Content updates for chicago.gov and 311 are prepared by more than a hundred content authors across the city, payment platforms are managed by Finance, and other properties are often built, written, deployed, and managed without involvement from DoIT. With uneven training and experience and no consistent editorial process tone and readability vary greatly from team to team and project to project.

USABILITY

Usability has been a core component of new and ongoing City of Chicago DoIT technology projects when there is budget available, like the 311 modernization effort. On those projects we have worked with residents throughout the process to ensure their feedback made it into the final product. We have occasionally performed usability testing with volunteer testers and volunteer test proctors from local universities like Northwestern and DePaul University on this and other projects, but usability and user-centered design are not a consistent portion of the development process.

LANGUAGE ACCESS

Language support for digital communications varies from machine translation via Google, to no translation, to high quality human translation into one or more languages. For chicago.gov, Google retired their current (free) translation widget, allowing it to be used where it is in place currently, so our site is compliant with the law but the translation quality of that particular product is not high.

311 will soon display a short message at the top of the web page prompting non-English speakers to call 311 instead. We have been working with Salesforce, our technology vendor, Catalyst Group, and Language Line to scope and plan a project to produce a translated version of the 311 site later this year. As it stands the plan would be for a one-time translation into our core supported languages at a cost of more than half a million dollars. There will be an additional need to produce updates and modify the 311 workflow to account for language translation in content creation.

Both our payment sites do not support multiple languages today. Individual microsite or custom built sites like https://chistreetwork.chicago.gov/map or https://sustainchicago.cityofchicago.org/ by individual departments vary widely but most do not support multiple languages.

ACCESSIBILITY

We seek to meet or exceed Federal, State, and City of Chicago accessibility law with all our projects. Specifically, we support the Web Content Accessibility Guidelines (WCAG) 2.0 AA standard. If the federal case pending before the Supreme Court is settled for accessibility advocates, WCAG2.0 A would become the law of the land, and we would be serving residents above that standard consistently.

Our support for accessibility follows a similar pattern as the other factors for user experience. When given time and resources, like the 311 modernization effort, DoIT and MOPD have been able to work with volunteers to produce apps and sites that support our community. Most often, though, there is limited time and even fewer resources to plan or facilitate our own testing for accessibility, so we react to correct issues instead of working to avoid them.

Given the dozens of authors and the many departments launching hundreds of digital projects every year, a portion of which DoIT delivers, delivering a consistent and well-designed user experience is challenging. Here are some steps we can take to improve delivery quality.

FUTURE STATE

In this section we look at actions currently underway operationally by DoIT and make recommendations for improvements to quality of each aspect of constituent experience. In addition, we review the efforts underway for each platform to bring it to our standards.

In our master consulting agreement for all technology projects sponsored by DoIT, we have updated our requirements to ensure all vendors take the appropriate steps to ensure readability, usability, language access, and accessibility. These steps include designing with users, taking advantage of City of Chicago and Federal government (18F & United States Digital Service) frameworks, and using our accessibility, usability, readability and language access guidelines in their development process. In places where vendors have their own framework or methods, we certify they have done the work to make our products as inclusive and welcoming as if they had used our methods.

READABILITY

The department originating the project or responsible for the content should follow the conventions of their department’s Language Access Plan to gather, edit, and test content for readability and overall comprehension. They should make use of resources like https://www.plainlanguage.gov/guidelines/ to implement methods to learn what their audience needs, test if their content is understandable, and if it supports people with the information or abilities they need to accomplish their task.

As a radical departure from business as usual, we suggest making the Mayor’s Office Communications team or another executive team responsible for ensuring quality of content, tone, and voice. The goal is not to police the message, but to ensure consistency of quality communicate for residents. Perhaps there is an opportunity to partner with CPS resources, students, or universities here?

We recommend the Mayor’s Office set a baseline standard for readability as “some high school” reading level for content, and that it supports community engagement work to test common digital product usage scenarios.

USABILITY & ACCESSIBILITY

Budgeting time and resources for community engagement and research with constituents reduces risk and improves project outcomes to make sure our products work well for everyone. For accessibility, this is even more accurate and acutely felt. With accessibility testing it is imperative to test with community members to ensure equality of access, as many accommodations are individually tailored to that person.

Using the Chicago Design System or another framework can help build in accessibility from the start. Working with volunteers and constituents is time-intensive work. Compensating constituents for their time comes with a host of ethical issues for city staff. However, finding ways to remove barriers to resident engagement as part of technology delivery is key to representing an unfiltered lens into their experience and needs.

This is especially true with accessibility work, where testing with constituents should be part of the software development lifecycle. There are several automated solutions to accessibility testing sold as a panacea, but there are no silver bullets for all content types and contexts.

We have been working with the Mayor’s Office for People with Disabilities to get sufficient resources for accessibility testing, and to find volunteers to help us test to date. Ideally, the city would hire an Accessibility Champion from the disabled community to focus on digital communications and testing coordination with the community, and create partnerships with nonprofits like the Great Lakes ADA Center to provide training on accessible document creation as well as community feedback from people with a wide array of challenges. This feedback needn’t be limited to digital communications. Sessions, once piloted and running successfully, could easily include representation from departments working within the built environment, Smart Cities, hiring, and other community touch points.

LANGUAGE ACCESS

It is the City of Chicago’s policy is to support English, Spanish, Polish, Chinese Mandarin (Simplified), Tagalog, and Arabic languages at a minimum in all communications. There are different approaches a project can take to comply with the language access ordinance, each of which impacts budget, timeline, and quality. It is DoIT’s goal to produce a consistent set of turnkey-like options for project teams to evaluate when scoping, planning, and budgeting digital projects.

For language access via other channels, like phone or in person, Language Line offers translation services support to many city departments. For digital communication, there are several options provided by Language Line, large tech companies, and smaller tech players.

We are complete with the majority of vendor evaluation, having worked with Amazon, Microsoft’s Bing team, Google’s Cloud AutoML Translate, and Language Line to understand costs and capabilities for human and human/machine hybrid translation. Each approach and next steps are detailed here for consideration.

Option 1

Google’s Cloud AutoML Translate had the richest feature set of the machine learning-based translation tools. This approach to translation works on demand after initial training and guidance from linguists. It can also learn from community suggestions to improve future translations. It is a similar experience to the Google widget currently providing translation services on chicago.gov, though more complex to implement.

Large sections of a site or content can be batch translated ahead of time or less frequently used content can be translated at a user’s request. People could translate content into any of the 100+ languages Google supports.

Once implemented, the cost and time are minimal at $20/million characters translated instantly. The quality will not be as good as human translation immediately, but with sufficient training this model has provided successful for New York City, Los Angeles, and many other municipalities. The advantages are cost, breadth of coverage, breadth of languages, and the ability to translate in the moment. This last ability means there is no need for a disruption to content authors’ workflow outside of when content is flagged for training the model.

We have created a proof of concept on GitHub, and will be creating a Salesforce proof of concept by 8/30.

Option 2

Language Line can offer three types of translation help to us:

A one time translation of content. We hand over content to them, they hand translated content back, we implement the changes. We would be responsible for managing changes between live and newly authored content, and looping in Language Line to re-translate. It is the highest quality translation, but it is time consuming and expensive.  To translate all 311 knowledge articles, flex questions, and interface copy to the minimum supported six languages will take up to six months and will cost over half a million dollars, then implementation would begin. Language Line offers content management system (CMS) connectors to help automate some of the workflow for translation. There are tools like Contentful that can help manage content workflow and translation as well. We are still investigating whether Language Line’s offering is compatible with chicago.gov’s CMS, for example. A 3rd-party partnership with Smartling, a workflow tool that provides automated translation management through a proxy server solution with Language Line staffing and managing content through that tool. This seems the best cost and time to translation option from Language Line, but it is also the most risky technically. We’d be introducing a new piece of infrastructure into any of our current products, so we would need to do due diligence to understand which platforms and content systems each of these tools support and their timeline for implementation.

We are continuing to explore each option by creating prototypes and estimates for each approach, detailed below by platform. Today, our recommended approach would be to use Google’s Cloud AutoML Translate with support from a Language Line or other translator to help set up the engine and manage community engagement. Additionally, we would empower and resource the Office of New Americans to recruit and engage community members fluent in our core languages to give feedback and help us improve the experience of the city for immigrants and non-English speakers.

Ideally, we could consider resourcing some of the design solutions produced by Northwestern University’s Segal Design Institute, like Dear Chicago. Dear Chicago is a program designed to help longtime Chicagoans welcome new members to their community and overcome the challenges of making a foreign place into a new home through connection to city resources.

RULA ROADMAP FOR KEY PLATFORMS

Looking ahead to the remainder of 2019 and into 2020, here are the high-level roadmaps for each of the key platforms which support residents and employees.

CHICAGO.GOV

Major updates should be published beginning later in 2019, and wrapping up in early 2020. A task order request (TOR) to redesign this website will be issued shortly. This will be the first time we will be commencing on a content redesign of this scale with clearly defined RULA requirements. The goal of this redesign is to simplify access to common information and services.

This redesign effort will be similar in scope to the effort required to gather content for 311. If that project serves as an example, this will be challenging to get department support.Success will be determined by clearly communicating expectations and the fact that this effort has the support of the Mayor and the Mayor’s Office.

We have tens of thousands of web pages that receive very little traffic. Do we retire all of them or keep them for posterity? We have ten times more PDFs. Google AutoML can translate them all. Is that the best user experience? DoIT is creating proof of concept solutions using the Google Translate API to understand and demonstrate the capability.

The reality of the project dictates this is our recommendation, as we would prefer to spend the majority of the project’s budget on refining content, implementing workflow controls on content production, creating easy access paths to common needs like paying a ticket or getting a job, and leveraging the Chicago Design System brand for accessibility and inclusion.

If we attempted to translate our website with Language Line it could easily exceed the budget of the entire project. We would prefer to engage as mentioned above with language access: engaging internal stakeholders to understand which content is critical, ask Language Line for help with the key issues, and engage the community for feedback and help translating.

311.CHICAGO.GOV

We have been working diligently to understand language access requirements, technology and human capabilities, and vendor strengths and weaknesses as the next priority after launch in December 2018. It has been a long journey, summarized briefly here.

We began by attempting to extend the same Google Translate widget experience on chicago.gov to 311, only to learn we would not allowed to by Google. Now, we have vetted a half dozen different translation solutions to understand our best options for cost and value to Chicagoans.

Now, we have a budgeted translation solution to translate the entirety of 311 by a hybrid machine learning/human team plan (the most efficient form of Option 2A, hand translation) into the minimum necessary supported languages. This would be the highest quality 311 translation in the land, it will also require a minimum of six months for a one time translation of 311 at a cost of over $600,000.

DoIT’s technology partner is in the process of putting together a proof of concept prototype to Google AutoML Translate integration into the Salesforce-based CHI311 platform. The past version of Google translate only worked on static content, whereas most of the interesting stuff in 311 (knowledge articles, flex questions) is dynamic. We should know more by Friday, 8/30.

If this option is not viable, we would explore options 2B & 2C (extensions to CMS or proxy server translation as a service) to understand future implications of maintenance cost and effort for this entire option path. That is to say, if we follow the path of hand translation, it will require either a repeated hand translation effort at regular intervals with manual tracking of content changes, or the implementation of some form of workflow or technology solution to ensure consistent updates.

PAY.CHICAGO.GOV/PARKING TICKETS PAYMENT PLAN/PAYMENT KIOSKS

We are working with our technology vendors to plan and prioritize language access and accessibility support against the many competing priorities of payment launches in 2019 and 2020. These teams are currently under stress, with many initiatives launching soon, but that is all the more reason to lean into these issues now so we launch strong and improve from a good base.

EVERYTHING ELSE

“Everything else” has many different types of technology products within it. Let’s discuss the categories broadly.

Employee systems. Most modern systems are accessibility compliant. As far as I know, none of our internal systems have been internationalized. Generally speaking, we purchase many systems like ServiceNow or the Salesforce platform which meet our accessibility standards. Older systems are often not accessible. We deal with exceptions on a case by case basis, and do accessibility testing on SharePoint and other intranet projects we launch internally.

DoIT-managed resident-facing systems. There’s hundred of applications like this that accomplish tasks for a smaller community of people. For example, the Heath Action Network, the Health Atlas, Chicago Business Direct, the Ticket Photo site, the Red Light video site, Resilient Chicago, Census 2020 and more. Many of these applications are often bespoke, and do not receive much consideration for RULA. Our data science and mapping tools, (PowerBI, WindyGrid, Socrata, and others) are generally accessible given the platform support for accessibility, but again with little oversight. We will attempt to address these through frameworks and a consistent Google AutoML approach on a priority basis.

Unmanaged resident facing-systems. There are also hundreds of resident-facing systems that receive little review or oversight from anyone outside of the originating department or team. These projects include efforts like Sustain Chicago, ChiStreetWork, and many others. We can make a best effort, but in many cases it is a stretch just to support such projects launching. Triaging any RULA to create a better experience is difficult if not impossible.

Overall, our focus is on getting the big systems right: implementing an updated and improved experience on chicago.gov and CHI311 first, then rolling those lessons into payments as their roadmaps allow.

Once we have provided a good experience across those core scenarios, take the learnings and programmatic support developed over the next 6-9 months and use it to drive change. The remaining 90% of systems that provide the other 20% of features and functions will then be addresses as resources and time permit.

DoIT alone cannot enforce readability, usability, language access, or accessibility, nor can it provide the resources for each department to meet its Language Access Plan goals. To deliver a world class resident, business, and visitor experience requires coordination, communication, and shared values between departments, technology teams, and the Mayor’s Office.

Thank you for reading this memo on the learnings and experiments of the past year or so. We have many answers about technology, but more questions about how we can collaborate to support this policy. We look forward to working together with you to make Chicago more accessible to all.