Speaker diarization for more than 2 speakers.

See this one: https://cognitive.uservoice.com/forums/555925-speaker-recognition/suggestions/34823824-add-support-for-speaker-diarization-for-untrained

I dont feel this should be marked as resolved. Would expect support for at least 10 speakers. Additionally its currently really poor and switches between speaker 1 and 2 almost randomly. Please make this more intelligent. Its a deal breaker for us and I'm sure many others. Especially considering the google alternative can handle unlimited speakers and is far more accurate at identifying them.

https://cloud.google.com/speech-to-text/docs/multiple-voices

And no... expecting a sample to train it for each voice is not an option. We literally just need it to assign a number to the speaker and not identify who the speaker is.

(Original post https://cognitive.uservoice.com/forums/555925-speaker-recognition/suggestions/40702504-speaker-diarization-for-more-than-2-speakers)

The JSON and/or result object needs to have the offset and duration of the whole normalized word.
I've reviewed the JSON and it still doesn't solve the problem. I need to know the relationship of the DisplayText words to the Word Timings in the detail When the DisplayText outputs 007 and the Word Timings output "double" "oh" "seven" as 3 different words I don't know that 007 = those three words as there is no reference. There needs to be a display word reference to the audio word to track offset/duration of an underlying audio file. The only option that we would have is to only use the words and build a post processor to do the conversion that the model is already doing in the formatted DisplayText to allow that reference to be maintained. Ideally, there should be two word lists. The current one with the Lexical version (that it appears to be) and a new one that would be the final formatted words that would combine the Offset/Duration for the formatted phrase. This would allow to have to exact offset/duration for the outputted word. Example:
Option for new list of words
Word = 007, Offset = 1000, Duration = 30000
Current
Word = "double", Offset=1000, Duration = 10000
Word = "oh", Offset=11000, Duration = 5000
Word = "seven", Offset=16000, Duration = 15000

Twice now the Speech portal has been broken by the owning Product Group.

Twice now I have wasted hours of my time as well as MS support personnel time trying to debug something only to find out that the portal (and associated APIs) were broken and it was known by the group.

Twice now the fix has been weeks in the deploying so god knows how many other customer's time has been wasted.

If you have a known issue that affects your customers, especially given the woeful error messaging on the portal, then please add a banner on the site proactively telling your customers that there are issues that may affect them and the workarounds.

The speech service is great but it is being badly let down by the speech training side of things.

When a Dataset upload fails the error messaging is literally "Failed" and clicking on the Dataset displays "Failed to upload data. Please check your data format and try to upload again."

This is not actionable error messaging. I have checked the data multiple times. I have been uploading this data, with additions, using an automated process for a year without issue.

Tell us why it failed. Give us a hint. I have 15,000 files and entries in the Trans.txt file. "It failed" is not useful information. Especially when it could easily be a problem server-side and Microsoft provides no validation tool that can be used.

You are wasting your customer's time with such vague error messaging and making their only recource having to open a support ticket to get any sort of useful answer (which is something that requires $$$ to get).

Related StackOverflow: https://stackoverflow.com/questions/65596225/cannot-upload-speech-dataset-because-failed

I have a degree in Exercise and Sports Science and just got into coding 8 months ago, and I now want to create an Exercise App that uses voice command to run the app. For example, I'd like for the user to be able to use the commands "What's today's workout?", "What's the first exercise?", "What's the next exercise?", and "I'm finished with the workout?". I've used C++ before for simple projects, but I've never used it to create an app with voice commands. I'd really like to start from scratch and have somebody guide me through a Teams meeting where I share my scree. Your help will be greatly appreciated.

The sound of the neural voices speaking some single letters such as would be done when speaking multiple choice test options does not match with other utterances by the same voice. One particular example of this is the letter "A" sounds very very short and also of a lower volume than "B" "C" and "D". It sticks out like a sore thumb especially since much of the rest of the utterances of words and sentences sounds so good. Compare the single letter utterances of Guy (neural) with Noah and you will find the latter are much more natural sounding and balanced. The letter "E" in the neural voices may not be quite as bad as the letter "A" but it is still not good. In trying to correct for the flaw, one can slow down only the speed of the letter "A" but then the volume is still too low and also the silence after the letter is spoken is too long compared to the other letters spoken at normal speed.

Problem you have encountered:
<prosody rate="108%"> does not work as per the W3C spec for SSML.
Neither does <prosody rate="100%">

These result in the TTS being spoken at about twice normal rate - Which is not right.

What you expected to happen:
I expect the speaking rate to be DEFAULT with rate="100%", as the W3C spec your documentation references at: https://www.w3.org/TR/speech-synthesis/#S3.3.2
literally says: "For example, a value of 100% means no change in speaking rate"

However if instead we used: <prosody rate="+100%"> (With a '+') - THEN the speed should be doubled. The "+" and "-" are critical for relative changes. Your SSML engine is not properly handling percentages without +/-. - It treats "100%" the same as "+100%", which is wrong.

Steps to reproduce:
On this page: https://azure.microsoft.com/en-au/services/cognitive-services/text-to-speech

Put in all this SSML:

<speak xmlns="http://www.w3.org/2001/10/synthesis&quot; xmlns:mstts="http://www.w3.org/2001/mstts&quot; xmlns:emo="http://www.w3.org/2009/10/emotionml&quot; version="1.0" xml:lang="en-US"><voice name="en-US-AriaNeural">
<prosody rate="100%">
This is spoken at double speed. But should not be!
</prosody>
</voice></speak>

And click Play. The audio speaks for itself!

Other information (workarounds you have tried, documentation consulted, etc):
Exclusively using relative changes, like rate="+8%" or "-5%" works as it should, so it a workable work-around.

But the current behaviour is definitely a bug and not W3C compliant!