AAD currently requires a strict directory scoping. We can investigate options if this is a popular request.
You can remove directories by contacting the organization that owns the directory and asking them to remove you.
We can add the ability to hide them, if this is a popular request.
We have done a lot of progress on this area in last few years, and we would like to know if there is something scenario specific that you guys are looking for?
Azure Portal Team