Deleted wp app on website and I keep getting UP/DOWN alerts

I recently deleted a WP page/app from a website keeping it only for email but I keep getting UP/DOWN alerts constantly.

Should I delete it from monitor pro's domain list? What would the correct procedure for this be? Shouldn't it auto detect?

MonitorPro: Website Monitoring Alerts Detected!

If the website itself is removed, MonitorPro will detect that immediately.

You can mark the site as ignored directly from the MonitorPro page in the WebUI.