Dies wird in wenigen Wochen offen bezogen, Apache 2 Lizenz. Anscheinend aus Lama gebaut. Ich hatte einige Leute es versuchen. Eine Person wurde errötet, während sie mit den Stimmen chatten. Insbesondere die männliche Stimme wird eine überraschende Anzahl von Menschen ansprechen, nicht nur auf die stereotype Frau "Ihr" Stimme, die wir alle erwarten.
https://www.sesame.com/research/crossing_the_uncanny_valley_of_voice#demo
2 Kommentare
„Even though recent models produce highly human-like speech, they struggle with the one-to-many problem: there are countless valid ways to speak a sentence, but only some fit a given setting. Without additional context—including tone, rhythm, and history of the conversation—models lack the information to choose the best option. Capturing these nuances requires reasoning across multiple aspects of language and prosody.
To address this, we introduce the Conversational Speech Model (CSM), which frames the problem as an end-to-end multimodal learning task using transformers. It leverages the history of the conversation to produce more natural and coherent speech. There are two key takeaways from our work. The first is that CSM operates as a single-stage model, thereby improving efficiency and expressivity. The second is our evaluation suite, which is necessary for evaluating progress on contextual capabilities and addresses the fact that common public evaluations are saturated.“
It was impressive. It felt more human than the other chatbots I tried: Replika and Character.ai. These other chatbots mostly repeat what you said.
I got the feeling that I was talking to a human who talked too much. However, this is the only way he could communicate. He didn’t have body language. He didn’t have much of a choice.
However, it also triggered some of the problems I have with my autism. Many times my mind goes blank when I am put on the spot. I could not tell him my favorite character in Shining Force III.
I also have trouble understanding people sometimes, and it happened several times with him: I have problems going from the part of the brain that hears to the part of the brain that understands. It happens without warning. And it draws my bosses crazy. I keep saying „What did you say?“ or „I don’t understand.“ Speaking fast as a rule does it. Some accents do it more than others. For example, I tend not to have a problem with Southern accents, because southerners tend to speak slow. Also if there is a lot of background noise, I have trouble understanding people.
Hence, it would be better for my disability if that character spoke to me through text, or even better, text and voice at the same time. I watch all my movies with English subtitles.
Running this demo made me realize for the first time that I’m going to have trouble communicating with humanoid robots once they arrive. This is how humanoid robots will probably communicate with us because that’s how humans communicate.