Module: libs/cursorless-engine/tokenizer/tokenizer
Namespaces
Variables
IDENTIFIER_WORD_REGEXES
• IDENTIFIER_WORD_REGEXES: string[]
Defined in
libs/cursorless-engine/tokenizer/tokenizer.ts:43
Functions
getMatcher
▸ getMatcher(languageId): Matcher
Parameters
| Name | Type |
|---|---|
languageId | string |
Returns
Defined in
libs/cursorless-engine/tokenizer/tokenizer.ts:96
tokenize
▸ tokenize<T>(text, languageId, mapfn): T[]
Type parameters
| Name |
|---|
T |
Parameters
| Name | Type |
|---|---|
text | string |
languageId | string |
mapfn | (v: RegExpMatchArray, k: number) => T |
Returns
T[]
Defined in
libs/cursorless-engine/tokenizer/tokenizer.ts:123