Kanade: A Simple Disentangled Tokenizer
for Spoken Language Modeling

Zhijie Huang, Stephen McIntosh, Daisuke Saito, Nobuaki Minematsu

Kanade is a single-layer disentangled speech tokenizer that extracts compact tokens suitable for both generative and discriminative modeling. It starts with SSL features that already represent speech in a structured latent space, enabling effective disentanglement and high data efficiency. It uses reconstruction losses on both SSL and acoustic features, encouraging it to capture both phonetics and prosody. A tight codebook-free quantization bottleneck, relieved of the need to encode acoustic constants by a global branch, tokenizes linguistic content without the need for auxiliary disentanglement methods. Despite this simple design, Kanade achieves superior disentanglement, as measured by VC and speaker discrimination. Moreover, it reaches best-in-class lexical availability as measured by downstream ASR and TTS, exhibits SLM performance competitive with SSL tokens, and maintains speech quality matching multi-layer codecs. Kanade demonstrates that a single-layer codec can provide the desirable properties of both SSL tokens and codec tokens for spoken language modeling.

Note: Kanade 25Hz (Clean) uses a decoder separately trained on LibriTTS-R with HiFT vocoder for better audio quality, different than the original decoder evaluated in the paper. However, the content tokens it uses are the same as Kanade 25Hz.

Speech Reconstruction

Example Info	Original	Kanade 25Hz (Clean) 341 bps	Kanade 25Hz 341 bps	Kanade 12.5Hz 171 bps	BiCodec 650 bps	FACodec 4800 bps	TiCodec 3000 bps	SpeechTokenizer 4000 bps	Mimi 1100 bps	PAST 4000 bps	DualCodec 1225 bps	StableCodec 388 bps	WavTokenizer 480 bps	XCodec2 800 bps
260-123286-0028 Its jaw is enormous and according to naturalists it is armed with no less than one hundred and eighty two teeth.
2300-131720-0005 Why if we erect a station at the falls it is a great economy to get it up to the city.
8555-284447-0005 The room of the great knife was high and big and around it ran rows of benches for the spectators to sit upon.
7176-92135-0036 The crowd drifts off leaving the hero and heroine alone in the middle of the stage and then you can begin.
61-70970-0015 Will cried he softly and stuteley who had chosen his couch across the door of his young master's chamber sprang up at once in answer.
3570-5694-0005 Under the tabu certain victuals and more particularly certain beverages are strictly reserved for the use of the superior class.
672-122797-0023 He well knew that he should never see his dear old comrades the little bushes and flowers around him anymore perhaps not even the birds.
2830-3980-0062 Not for a crown or a kingdom or our goodness but for our sins.

Zero-shot Voice Conversion

Example Info	Source	Reference	Kanade 25Hz (Clean) 341 bps	Kanade 25Hz 341 bps	Kanade 12.5Hz 171 bps	BiCodec 650 bps	FACodec 4800 bps	TiCodec 3000 bps	SpeechTokenizer 4000 bps	Mimi 1100 bps	PAST 4000 bps	DualCodec 1225 bps	CosyVoice 2	FreeVC	LinearVC
017_p270_p282 Others have tried to explain the phenomenon physically. `p270` → `p282`
017_p278_p260 Others have tried to explain the phenomenon physically. `p278` → `p260`
006_p227_p284 When the sunlight strikes raindrops in the air, they act as a prism and form a rainbow. `p227` → `p284`
006_p287_p244 When the sunlight strikes raindrops in the air, they act as a prism and form a rainbow. `p287` → `p244`
014_p274_p230 To the hebrews it was a token that there would be no more universal floods. `p274` → `p230`
014_p276_p246 To the hebrews it was a token that there would be no more universal floods. `p276` → `p246`
002_p236_p272 Ask her to bring these things with her from the store. `p236` → `p272`
002_p236_p263 Ask her to bring these things with her from the store. `p236` → `p263`
020_p276_p260 Many complicated ideas about the rainbow have been formed. `p276` → `p260`
020_p287_p277 Many complicated ideas about the rainbow have been formed. `p287` → `p277`

Zero-shot Text-to-Speech

Example Info	Original	Reference	Kanade 25Hz (Clean) 341 bps	Kanade 25Hz 341 bps	Kanade 12.5Hz 171 bps	BiCodec 650 bps	TiCodec 3000 bps	SpeechTokenizer 4000 bps	Mimi 1100 bps	PAST 4000 bps	DualCodec 1225 bps	StableCodec 388 bps	WavTokenizer 480 bps	XCodec2 800 bps	CosyVoice 2
260_123288_000016_000002 We cannot even make ourselves heard by approaching our mouth close to the ear.
8555_284449_000059_000001 They marched to their chairs and seated themselves to enjoy the cruel scene their father was about to enact, and Cap'n Bill bowed to them politely and said:
5142_36600_000006_000004 When one kidney ceases to act from disease, the other increases in size, and does double work.
4992_41806_000034_000003 To night there was no need of extra heat, and there were great ceremonies to be observed in lighting the fires on the hearthstones.
4992_41797_000006_000000 Oh, mother, there's always so much spare room in your heart—for you took in Julia and yet we never felt the difference—won't you make a place for Olive?
7127_75947_000077_000002 My affairs concern myself alone, monsieur; and I have already told you I transact them without interference.
8230_279154_000011_000001 It does not involve a definite memory of a definite past event, but only the knowledge that something happening now is similar to something that happened before.
1320_122612_000033_000001 The little knot of Indians drew back in a body, and suffered, as they thought, the conjurer and his inspired assistant to proceed.
7021_79740_000010_000003 Seeing that I am so fine, I may as well go and visit the King.
4507_16021_000035_000007 He would be like a philologist refusing to examine a fact in language, a philosopher hesitating to scrutinize a fact in humanity.

Spoken Language Modeling

The first 2.4s is prompt from LibriSpeech test-clean.

Example Info	Kanade 12.5Hz 171 bps	k-means 12.5Hz 171 bps
672-122797-22
1284-1180-14
1320-122617-39
1995-1837-12
3570-5696-2
3570-5696-10
4446-2273-36
6930-75918-18
7729-102255-18
8463-287645-9

Diverse Speech Reconstruction (GigaSpeech)

Example Info	Original	Kanade 25Hz (Clean) 341 bps	Kanade 25Hz 341 bps	Kanade 12.5Hz 171 bps	BiCodec 650 bps	FACodec 4800 bps	TiCodec 3000 bps	SpeechTokenizer 4000 bps	Mimi 1100 bps	PAST 4000 bps	DualCodec 1225 bps	StableCodec 388 bps	WavTokenizer 480 bps	XCodec2 800 bps
POD1000000022_S0000119
YOU1000000035_S0000148
YOU1000000044_S0000112
YOU1000000038_S0000088
YOU1000000020_S0000085
POD1000000017_S0000241
YOU1000000020_S0000246