Marking a risky sign in as "Confirmed Safe" in the ID protection blade should factor in to the algorithm for future sign ins
In the risky sign ins report or risky users report in AD Identity Protection you can mark a risky sign in as "confirmed safe." However this does not allow future sign ins from this IP. If an administrator confirms that the sign in is not risky, future sign ins for this user from this location should not be considered risky.
Thank you for your feedback. We are reviewing options for integrating feedback provided by confirm safe/compromised. In the interim, if you want to mark specific IPs as safe for Identity Protection in your tenant, you can do so my marking them as trusted locations. More information is available here (make sure to check the “mark as trusted location” checkbox): https://docs.microsoft.com/en-us/azure/active-directory/reports-monitoring/quickstart-configure-named-locations
Andrew Bryant commented
To clarify, "Confirmed safe" should provide a different function than "dismiss." for example, if a user risk level is high and an administrator resets the password and then dismisses the user's risk events, those risk events should not be considered "safe."