Class: WordTokenizer

libs/cursorless-engine/scopeHandlers/WordScopeHandler/WordTokenizer.WordTokenizer

This class just encapsulates the word-splitting logic from WordScopeHandler. We could probably just inline it into that class, but for now we need it here because we can't yet properly mock away vscode for the unit tests in subtoken.test.ts.

Constructors

constructor

• new WordTokenizer(languageId)

Parameters

Name	Type
languageId	string

Defined in

libs/cursorless-engine/scopeHandlers/WordScopeHandler/WordTokenizer.ts:15

Properties

wordRegex

• Private wordRegex: RegExp

Defined in

libs/cursorless-engine/scopeHandlers/WordScopeHandler/WordTokenizer.ts:13

Methods

splitIdentifier

▸ splitIdentifier(text): MatchedText[]

Parameters

Name	Type
text	string

Returns

MatchedText[]

Defined in

libs/cursorless-engine/scopeHandlers/WordScopeHandler/WordTokenizer.ts:19