Top "Tokenize" questions

Parse (split) a string in C++ using string delimiter (standard C++)

I am parsing a string in C++ using the following: using namespace std; string parsed,input="text to be parsed"; …

c++ parsing split token tokenize

How do I tokenize a string in C++?

Java has a convenient split method: String str = "The quick brown fox"; String[] results = str.split(" "); Is there an easy …

c++ string split tokenize

What is the easiest/best/most correct way to iterate through the characters of a string in Java?

StringTokenizer? Convert the String to a char[] and iterate over that? Something else?

java string iteration character tokenize

How to split a string in shell and get the last field

Suppose I have the string 1:2:3:4:5 and I want to get its last field (5 in this case). How do I do …

bash split tokenize cut

Splitting string into multiple rows in Oracle

I know this has been answered to some degree with PHP and MYSQL, but I was wondering if someone could …

sql string oracle plsql tokenize

Split string with PowerShell and do something with each token

I want to split each line of a pipe on spaces, and then print each token on its own line. …

string powershell tokenize

How do I read input character-by-character in Java?

I am used to the c-style getchar(), but it seems like there is nothing comparable for java. I am building …

java character tokenize

How to use stringstream to separate comma separated strings

I've got the following code: std::string str = "abc def,ghi"; std::stringstream ss(str); string token; while (ss >&…

c++ tokenize stringstream

Is there a function to split a string in PL/SQL?

I need to write a procedure to normalize a record that have multiple tokens concatenated by one char. I need …

string oracle plsql split tokenize

How to get rid of punctuation using NLTK tokenizer?

I'm just starting to use NLTK and I don't quite understand how to get a list of words from text. …

python nlp tokenize nltk