If you're still on your original gearbox oil, 54k is EXACTLY where mine became pretty unbearable. Also keep in mind that as your car ages, no matter how well maintained it is, you're going to have to be a little more patient with your shifting (the 02J in particular doesn't like to be rushed IMO)
If you've got fresh gear oil and it doesn't seem to be a temperature or overfill/underfill phenomenon or something you can smooth out with a smoother/gentler shifting technique, then you can start looking at synchros. If you do a gearbox oil change and you're suspicious that parts inside your gearbox are wearing or grenading, a Blackstone Labs oil analysis could give you some hints that something bad is happening without you having to pull your gearbox or open it up.
Best of luck!
Ah ha excellent point, yes bad mounts or a failed clip at the end of your shift cable will also screw things up. The more you can button down your drivetrain the better it feels (love how much sharper my shifting is with the 034 Motorsport street line density engine mounts and the ECS tuning dogbone) so it makes perfect sense that a blown or damaged pendulum or engine mount would make it significantly worse! Cheers!