tokenizing a string twice in c with strtok()

c csv tokenize strtok
SummerCodin picture SummerCodin · Dec 28, 2010 · Viewed 11.7k times · Source

I'm using strtok() in c to parse a csv string. First I tokenize it to just find out how many tokens there are so I can allocate a string of the correct size. Then I go through using the same variable I used last time for tokenization. Every time I do it a second time though it strtok(NULL, ",") returns NULL even though there are still more tokens to parse. Can somebody tell me what I'm doing wrong?

char* tok;
int count = 0;
tok = strtok(buffer, ",");
while(tok != NULL) {
    count++;
    tok = strtok(NULL, ",");
}

//allocate array

tok = strtok(buffer, ",");
while(tok != NULL) {
    //do other stuff
    tok = strtok(NULL, ",");
}

So on that second while loop it always ends after the first token is found even though there are more tokens. Does anybody know what I'm doing wrong?

Answer

Fred Larson picture Fred Larson · Dec 28, 2010

strtok() modifies the string it operates on, replacing delimiter characters with nulls. So if you want to use it more than once, you'll have to make a copy.

Related questions

read comma-separated input with `scanf()`

I have the following input: AG23,VU,Blablublablu,8 IE22,VU,FooBlaFooBlaFoo,3 and so on... I want it to "parse" with scanf() using some code like this: char sem[5]; char type[5]; char title[80]; int value; while(scanf("%s,%s,%s,%d", …

Read More
How do I parse out the fields in a comma separated string using sscanf while supporting empty fields?

I have a comma separated string which might contain empty fields. For example: 1,2,,4 Using a basic sscanf(string,"%[^,],%[^,],%[^,],%[^,],%[^,]", &val1, &val2, &val3, &val4); I get all the values prior to the empty field, and unexpected results from …

Read More
Obtaining zero-length string from strtok()

I have a CSV file containing data such as value;name;test;etc which I'm trying to split by using strtok(string, ";"). However, this file can contain zero-length data, like this: value;;test;etc which strtok() skips. Is there a …

Read More