Does JDBC have a maximum ResultSet size?
Is there a maximum number of rows that a JDBC will put into a ResultSet specifically from a Hive query? I am not talking about fetch size or paging, but the total number of rows returned in a ResultSet.
Correct me if I'm wrong, but the fetch size sets the number of rows the jdbc looks at to process with each pass in the database, inserting appropriate responses into the ResultSet. When it has gone through all the records in the table, it returns the ResultSet to the Java code. I am asking if there is a limit to the number of rows returned to the Java code.
If it doesn't have a maximum number of rows, is there anything inherent with the class that may cause some records to be trimmed off?
Answer
No, it does not work that way. JDBC is just a wrapper around native databases. But either JDBC or database cursors work the same :
- you send a query (via JDBC) to the database
- the database analyses the query and initializes the cursor (ResulSet in JDBC)
- while you fetch data from the ResultSet, the databases software walks in the database to get new rows and populates the ResultSet
So there is no limit to the number of rows that a ResultSet can contains, nor to the number of rows that a java client program can process. The only limit comes if you try to load all the rows in memory to populate a list for example and exhaust the client application memory. But if you process rows and do not keep them in memory, there is no limit (exactly like what happens when you read a file).