True, this was my problem with vimium as well. The characters of the link would change even for relatively stable UI functionality like going fullscreen on Youtube. So, there was no muscle memory. Ironically, the only time i would remember i had vimium installed was when i would try to press f and then press a couple more characters while wondering why i installed the plugin in the first place.