Is there maximum size of string data type in Hive?

hadoop hive bigdata
xxks-kkk picture xxks-kkk · Jan 27, 2016 · Viewed 24.7k times · Source

Google a ton but haven't found it anywhere. Or does that mean Hive can support arbitrary large string data type as long as cluster is allowed? If so, where I can find what is the largest size of string data type that my cluster can support?

Thanks in advance!

Answer

Tom Harrison picture Tom Harrison · Apr 17, 2016

The current documentation for Hive lists STRING as a valid datatype, distinct from VARCHAR and CHAR See official apache doc here: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-Strings

It wasn't immediately apparent to me that STRING was indeed it's own type, but if you scroll down you'll see several cases where it's used distinctly from the others.

While perhaps not authoritative, this page indicates the max length of a STRING is 2GB. http://www.folkstalk.com/2011/11/data-types-in-hive.html

Related questions

Hive ParseException - cannot recognize input near 'end' 'string'

I am getting the following error when trying to create a Hive table from an existing DynamoDB table: NoViableAltException(88@[]) at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:9123) at org.apache.hadoop.hive.ql.parse.HiveParser.…

Read More
How to convert a Date String from UTC to Specific TimeZone in HIVE?

My Hive table has a date column with UTC date strings. I want to get all rows for a specific EST date. I am trying to do something like the below: Select * from TableName T where TO_DATE(ConvertToESTTimeZone(T.…

Read More
Books to start learning big data

I would like to start learning about the big data technologies. I want to work in this area in the future. Does anyone know good books to start learning about it? Hadoop, HBase. Beginner - intermediate - advanced - Thanks …

Read More