Remove an element using tabindex="-1". For example:
tabindex="-1"
<button id="foo" tabindex="-1">I'm not keyboard focusable</button>