Deleting an App Service Plan does not delete Alerts
One of my devs had created an App Service Plan with a typo in the name. Whether this was the right course or not, we deleted the App Service Plan and created a new one. Later I discovered that there were still Alerts defined (Disabled) referencing the delete App Service Plan.
Closing due to lack of feedback. Please post on the MSDN or Stack Overflow forums for Web Apps if there is still an open question.