It helps against stupid automated attacks though.

If someone has changed the port it’s likely that they have set up a great password or disabled password auth all together.

It’s worth it for just having cleaner logs and fewer attempts.