R-invalid multibyte string 1

r utf-8 text-mining multibyte
Damien picture Damien · Nov 5, 2014 · Viewed 13.5k times · Source

I'm new to R software

Now,studying text mining using "tm"package"

I have a ploblem on mapping text to lower case

sms_raw<-read.csv(............)
sms_corpus<-Corpus(VectorSource(sms_raw$text)) 
sms_corpus<-Corpus(VectorSource(sms_raw$text))  
tm_map(sms_corpus,content_transformer(tolower))   
error:invalid multubytes string 1

I thought my csv file could be not utf-8 so I restored as utf-8 but it didn't work.

my OS is win8.1

Anyone have solution on this problem please let me know.

Answer

Damien picture Damien · Nov 7, 2014

The error I had easily solved by encoding function

In my file's column which name is text contains multibyte character

So I type

sms_raw$text <- iconv(enc2utf8(sms_raw$text),sub="byte")

This command converts the 'text' column (multibyte) to utf8 form

Related questions

R tm package invalid input in 'utf8towcs'

I'm trying to use the tm package in R to perform some text analysis. I tied the following: require(tm) dataSet <- Corpus(DirSource('tmp/')) dataSet <- tm_map(dataSet, tolower) Error in FUN(X[[6L]], ...) : invalid …

Read More
How to use Greek symbols in ggplot2?

My categories need to be named with Greek letters. I am using ggplot2, and it works beautifully with the data. Unfortunately I cannot figure out how to put those greek symbols on the x axis (at the tick marks) and …

Read More
read.csv() with UTF-8 encoding

I am trying to read in data from a csv file and specify the encoding of the characters to be UTF-8. From reading through the ?read.csv() instructions, it seems that fileEncoding set equal to UTF-8 should accomplish this, however, …

Read More