An Azure VM configured in a dual-stack IPv6 subnet with two IP configurations (IPv4 and IPv6) by default has IPv4-only connectivity.
The VM will get IPv6 internet connectivity only if a public IPv6 address is assigned to it. Once a public IPv6 is assigned to the machine, IF the machine does NOT have a public IPv4 assigned to it, the VM will loose IPv4 internet connectivity and will only be able to browse ipv6-only sites.
When a public IPv4 address is assigned to the VM, VM will be able to browse IPv4 and IPv6 sites.

Such behavior is unexpected (assigning a public IPv6 should have no impact on IPv4 traffic) and undocumented.
Associated case # 120032421002062