I completely agree, but (1) most identity providers don't issue SSH certificates, and (2) a lot of services won't accept SSH certificates for auth. But we're actually using this approach for access to some of our internal components (we issue a short-lived cert that logs you in as a generic user but still gives an audit trail), for the advantages you describe.