It looks like this was a larger decision from the lemmy development community in an attempt to eliminate karma farming. They say it’s psychologically damaging, and as someone who looks at them a lot, they may be right.
Here’s a GitHub thread discussing it where our Voyager dev weighs in:
https://github.com/LemmyNet/lemmy/issues/3393#issuecomment-1779400639
Voyager could probably add a hidden feature, like long press post count label to switch to karma and compute karma, maybe for last 500 or so posts & comments (10 total requests).
Because it would be a hidden feature requiring manual activation I doubt it would add server load. And being hidden means it also wouldn’t be an issue with karma farming.