How to detect clicks from VoiceOver
VoiceOver clicks and mouse clicks are the same to the browser—so how can we differentiate them from each other?
2 min read
·
By Magnus Rand
·
December 6, 2023
The problem
A couple months ago I was coding away at some new dropdown components for our design system and came across an annoying issue. When using VoiceOver and focusing the input of my dropdown, VoiceOver announced: “… to display a list of options, press Control-Option-Space.” However, my dropdown did nothing when pressing Control-Option-Space—which obviously isn't good.
My next issue then was: How do I know when the user presses Control-Option-Space? Some quick event investigation teaches me that Control-Option-Space sends out a normal click event. This means displaying my list with options when receiving a click event will work, but in my case I don't want to show the list when the user clicks with a mouse.
My new issue now was: How do I know when the user clicks using VoiceOver? And this didn't seem to be as trivial. Web browsers can't detect screen readers[1] and VoiceOver sends normal click events … Well, after a lot of looking around I found one hacky way to solve my problem—it's not elegant but it works and is still better than nothing:
The solution
When sending click events with VoiceOver, the click position (i.e. the x and y coordinates) is always set to the center of the clicked element, halfway from the top and halfway from the left. An actual mouse click is very rarely placed exactly in the middle, so I used this to my advantage. I made a function that takes in a click event and tells me wether the click is within a small perimeter of the exact center or not. This tells me if the click was (most likely) a VoiceOver click or not.
/**A VoiceOver click is always preformed in the center of the clicked element.
This functions expolits that to check if the performed click likely is
made by VoiceOver. */
export const isVoiceOverClick = (clickEvent: React.MouseEvent) => {
// the element the user or VoiceOver has clicked
const targetElementRect = clickEvent.currentTarget.getBoundingClientRect();
// the hoizontal center of the clicked element
const targetElementMiddleX = Math.floor(
targetElementRect.x + targetElementRect.width / 2,
);
// the vertical center of the clicked element
const targetElementMiddleY = Math.floor(
targetElementRect.y + targetElementRect.height / 2,
);
// the coordinates of the click event
const clickPositionX = clickEvent.clientX;
const clickPositionY = clickEvent.clientY;
// lastly we check weather the click event coordinates are
// within 1 px of the clicked elements center. The 1 px
// extra radius is needed to accound for rounding differences
return (
Math.abs(targetElementMiddleX - clickPositionX) <= 1 &&
Math.abs(targetElementMiddleY - clickPositionY) <= 1
);
};It's by no means a perfect solution and could break without notice, but for now it solves this very specific problem.
A last notice
As a last notice, remember that VoiceOver isn't the only screen reader out there. This solution will likely not work for other screen readers and you should test your solution thoroughly. If you can solve your problem using traditional solutions you should always prefer that. In this case, the problem is VoiceOver-specific. The announced message did not reflect real usage, so we added extra functionality on top of an accessible component. Something like isVoiceOverClick should never be used to build core accessibility functionality.
