Im not sure how it would fair as a regular workstation. Apparently the Ampere Altra is already being used for remote gaming workloads in China and it utilizes RTX 4090 for that.
https://www.ipi.wiki/products/com-hpc-ampere-altra?variant=43315757121698
It looks like a pretty solid value for 128 cores at 3000 bucks. Maybe even 32 cores for 2000. It is even competitive with M1/M2 macs and Epyc/threadripper.
What kind of workstation? Is your software fully compatible with arm and the gpu stack?
I would verify that the new Nvidia gpus work well with the arm-kernel; Though I cant possibly imagine why it would differ to x86_64. I don’t know though.
Personally I would not consider any Nvidia card for linux. Simply due to the fact that nvidia may hide capabilities after the purchase. If you need e.g. virtualization in the future the flag may already hidden or change afterwards by nvidia (happend before). They will force a non-consumer card at you for this.
I am a happy linux-arm user.