_generate_tokens_from_c_tokenizer (reference) in projects: third_party - OpenGrok search results

Repository(s)

Full Search		Search through all text tokens(words,strings,identifiers,numbers) in index.
Definition		Only finds symbol definitions(where e.g a variable(function,...) is defined).
Symbol		Only finds symbol(e.g. methods classes,function,variables).
File Path		Path of the source file(use "/").If you want just exact path,enclose it in "".Source files end with: .jar/.bz2/.a/.h/.java...
History		History log comments.
Type		Type of analyzer used to filter file types include with selected(e.g. just C sources).

Help

Searched refs:_generate_tokens_from_c_tokenizer (Results 1 - 2 of 2) sorted by relevance

/third_party/python/Lib/
H A D	tokenize.py	`685 def _generate_tokens_from_c_tokenizer(source): function`
/third_party/python/Lib/test/
H A D	test_tokenize.py	`6 NEWLINE, _generate_tokens_from_c_tokenizer, DEDENT) 1664 _generate_tokens_from_c_tokenizer(s), s 2479 return list(_generate_tokens_from_c_tokenizer(string)) 2526 tokens = list(_generate_tokens_from_c_tokenizer(valid)) 2531 tokens = list(_generate_tokens_from_c_tokenizer(invalid)) 2539 return [(kind, string) for (kind, string, *_) in _generate_tokens_from_c_tokenizer(string)]`

Completed in 6 milliseconds