At least for desktop computers, you have the power switch on the back of the PSU. Assuming your PSU is actually ATX compliant and not some proprietary or otherwise non-standard bullshit.

That switch is inline with the AC input and will kill power to the device completely.