Windows does not let you export a private key certificate without a password, but there does not seem to be any way to specify a password when using ARM. I'm not sure why the ARM resource requires a base64 string--it would make much more sense to just reference a certificate in an existing Key Vault, like the Compute team does.
Thanks for the valid suggestion. Your feedback is now open for the user community to upvote & comment on. This allows us to effectively prioritize your request against our existing feature backlog and also gives us insight into the potential impact of implementing the suggested feature.