Couldn’t you just create a custom .ttf that only has the ampersand character in it? I vaguely recall some tutorials suggesting that in the past.