Technology

Vaccine website frustrations point to larger government tech problems

New York had months to prepare for rolling out the COVID-19 vaccine, and yet signing up for a vaccination is complicated and difficult. Here’s why.

COVID-19 Vaccine

COVID-19 Vaccine Ed Reed/Mayoral Photography Office

New Yorkers have waited almost 10 months for a COVID-19 vaccine. Now that it’s finally here, all that stands between them and what will hopefully be the pandemic’s antidote is a maddening, sometimes hours-long online registration process.

As eligibility for the COVID-19 vaccine expanded on Monday to phase 1B – a cohort of more than four million people statewide, including some essential workers and people ages 75 and over – a number of issues quickly surfaced with the systems used to find info about where to get vaccinated and sign up for appointments. On Tuesday, Gov. Andrew Cuomo announced that eligibility would be expanded to people 65 years old and up and people who are immunocompromised. However, he said it was unclear which categories of immunocompromised people were covered by the Centers for Disease Control guidance. 

Miriam Grossman, a Connecticut resident who has been trying to sign up her father – who is over 75 years old and lives in Queens – for a vaccine appointment, estimated that she spent about four hours on Sunday and Monday trying to secure a slot. The Health + Hospitals vaccine scheduler would show her no available appointments and then direct her to find COVID-19 testing availability, which she wasn’t looking for. After filling out the many fields of questions that the city’s vaccine hub asked her, Grossman finished the long process only to find no appointments available. The city also has a vaccine finder to connect users to providers administering the vaccine, but Grossman encountered problems here too. A number she tried for a provider listed on the city’s website told her to schedule an appointment online, but the provider’s website had no online scheduling function. (On Tuesday, the New York City vaccination sign-up websites allowed New Yorkers 65 and older to sign up, along with various essential workers, but not immunocompromised people under 65.) 

“I thought, maybe I’m doing something wrong,” Grossman recalled. “But then when I saw on Twitter that other people were noting the same thing, I felt kind of vindicated. It’s not just me.” Many of the troubles New Yorkers reported with signing up for vaccines this week can be found in replies to a Sunday tweet from New York City Comptroller Scott Stringer, calling attention to what he said were “twin failures of outreach and technology” on the part of the city. “This is an impossible tech bureaucracy that is going to hurt seniors, hurt people without internet access or people who are just not digitally literate, and people who have language barriers,” Stringer told City & State on Monday. 

Many of the technical issues New Yorkers had signing up for vaccines have to do with design elements of the websites and a lack of adequate planning to make the experience user-friendly, technologists said. More broadly, issues with online sign-ups for vaccine appointments reflect a greater problem in New York: A failure to tap the vast technology resources at the city and state’s disposal to offer accessible essential services to New Yorkers. “It’s a lack of attention to the fact that digital technology is a critical tool in how government services are delivered in the 21st Century,” said Noel Hidalgo, co-founder of the technology nonprofit BetaNYC. That’s especially true during a pandemic, he said.

“Like our entire public health apparatus, our vaccine roll out is a massive undertaking requiring the resources of multiple agencies working together,” New York City Department of Health spokesperson Patrick Gallahue wrote in an email on Tuesday. Representatives for Cuomo did not respond before publication to a request for comment about problems reported with the state’s website. 

Despite New York City Mayor Bill de Blasio’s administration being the one to create new technology roles, including the city’s chief technology officer and New York City Cyber Command, Hidalgo said the administration has come up short in coordinating to create innovative tools that make the process of signing up for vaccines, for example, as easy as possible. “It’s missing that additive component where there’s very clear leadership and coordination on how technology, data and design can be used to address the real, present needs during the pandemic,” he said.

New York City isn’t alone in this though. The state has had its own high-profile tech issues during the pandemic, including the state Department of Labor’s unemployment insurance application – an outdated and bug-filled system that made the process of getting unemployment checks unnecessarily difficult for the more than one million suddenly jobless New Yorkers this spring. The state’s vaccination website appeared to fail some users on Monday too, with the site reportedly timing out for some. New York City and New York state have now both launched phone lines where eligible people can schedule an appointment to get the vaccine. It’s an important alternative to online scheduling, but callers may experience wait times there too. 

New York’s vaccine rollout has been the subject of non-technology related criticism too. Before eligibility was opened up to include non-health care essential workers and older New Yorkers, vaccine doses were being thrown away because government couldn’t connect eligible New Yorkers to providers. And leading up to this week’s expansion to phase 1B, there was still confusion about which essential workers would be eligible, as many types of occupations – grocery store clerks as well as firefighters – have been classified as “essential” throughout the pandemic.

Tracey, a social worker in Brooklyn who asked to be identified by first name only to preserve her privacy with clients, said that she has received conflicting messages about whether she’s eligible as someone working remotely who would hopefully be able to eventually transition back to in-person work after being vaccinated. The state and city websites haven’t provided much help in clarifying that for her. Instead, she’s resorted to seeking information on a Facebook page for New York City social workers. When the state’s unemployment insurance system failed to work this spring and summer, unemployed New Yorkers took to a Facebook group with over 47,000 members to seek answers and commiserate too.

A well-functioning government website or application needs to not only provide users with all the relevant information and functions, but be easy to use and intuitive. That means that seniors can use it without needing help to upload a photo of their insurance card or knowing to clear their browser’s cookies to circumvent error messages. The tools need to be as accessible to non-native English speakers as they are to English speakers. A website needs to be as easy to use on a mobile phone as it is on a desktop computer, so that people with only mobile internet access can use it. Nearly 30% of New York City households don’t have a home broadband subscription, according to the city’s own figures. (Offline options also need to be available for the 18% of city residents who lack both home and mobile broadband.)

For New York City’s two main vaccine scheduling websites – the one run by the Department of Health and the other run by Health + Hospitals – user experience and design is a big shortcoming, critics said. The Health Department website asks many questions – Stringer reported as many as 51 questions and fields – before an appointment can be made. Not only does that take a long time, but one crash or interruption along the way can force users to start the long process from scratch.

Sign-up websites should be asking for a minimal amount of data to remove as many hurdles that could trip people up – like remembering their insurance ID number – said Emil Skandul, founder of Capitol Foundry, a digital innovation firm. Rather than asking users to answer dozens of questions, the site could ask for a few main data points – name, age, eligibility, health risks and contact info, for example. Insurance and other information could be collected later, in person. There is concern within the city, however, that this would lead to bottlenecks at the vaccination sites, when the goal is to get as many people in and out of the sites quickly.

Others questioned why frustrations and technical problems with the websites weren’t foreseen before people started signing up for appointments en masse this week. It’s not clear how much user testing was done of New York City’s vaccine websites, but when designing a well-functioning site, testing it out on every kind of user has to be a part of the planning. “Did they run usability tests? Did they run load tests on these websites?” Skandul asked. The design process should include testing the product with different user “personas” – people who only have access to a smartphone, Spanish speakers and seniors, as just a few examples – he said. 

In a press conference on Monday, de Blasio addressed complaints about the city’s website, saying that the city planned to continue to make improvements. “Folks who are going through a lot, we want them to have as easy an experience as possible making their appointments,” he said. “What we do find as we put any system into play (is) that we improve it as we go along based on customer feedback, so I'm certain that's going to happen here.”

The Health + Hospitals website, while more streamlined, also experienced some slowness on Monday because of a high volume of users. Hidalgo said that the website also uses a web design element called an “iframe” for its scheduling framework, which he said makes it difficult for a translator to work with the site. “If somebody is using a language translator, the content in the actual scheduling app doesn’t get translated,” he said. That means that the Health + Hospitals scheduler is currently only offered in English. 

“Our scheduling system was able to connect New Yorkers yesterday to an appointment,” Health + Hospitals spokesperson Christopher Miller wrote in an email on Tuesday. “In the coming days more appointments will become available – we urge those eligible to check back frequently. As vaccine supply increases more appointments will become readily available."

There are easier ways to sign New Yorkers up to get the vaccine, experts said. Some lawmakers have called for New York City to have a single sign-up system to avoid confusion over which of the current options to choose from. Skandul said that New York state should be working with a single site through which residents can pre-register for a vaccine. He cited New Jersey’s pre-registration system as a model to follow here. Users could register their basic information with the state and be notified by text or other means when they are eligible and make an appointment on a system that integrated the different scheduling mechanisms used for public hospitals, pharmacies and all other vaccine providers. Instead of New Yorkers having to seek out a vaccine appointment, the opportunity to schedule an appointment would come to them. 

While a low number of available vaccine appointments on these websites is a sign that vaccination is speeding up, some are nervous that technical issues could mean wasted vaccines. In addition to the problems she encountered with the government-run websites, Grossman had problems getting through to private providers that the city and state’s vaccine sites directed her to. “I think New York City and New York state need to be not only on top of their own sites and web scheduling, but the private places that they’ve entrusted the vaccine to,” Grossman wrote in a message later on Monday. “If New York City says a clinic is a vaccine site but I can’t get through to make an appointment, where is the vaccine going, knowing (the) vaccine expires when not used in time?” By Monday night, Grossman had finally secured an appointment for her father at a Health + Hospitals site, though not in Queens, but Manhattan. 

While experts said that designing websites that could integrate all of these elements is no easy task, it’s work that could have been done long in advance of a vaccine actually arriving. “You’ve had nine months to prepare for this particular moment,” Hidalgo said. “It is a massive shortcoming that was preventable.”

It’s also not as if New York doesn’t have the brainpower to accomplish a system like that. Hidalgo cited several divisions of city agencies that already exist that could be a part of creating a more user-friendly and accessible system, including a civic service design studio within the Office for Economic Opportunity and user experience unit within the Department of Information Technology and Telecommunications. A spokesperson for the city did not comment on what agencies and resources were involved in designing its vaccine website or who led the effort, so it’s not clear if any of these resources were tapped. 

“This city needs government to succeed in this pandemic, because it is the only entity that is going to get us out of this pandemic,” Hidalgo said. “It’s frustrating as a technologist to see these things happen when you know there are good people inside of government who want to get us there.”

Correction: The vaccine finder website Grossman used is run by New York City and she did not have to upload photos of an insurance card.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.