Parsing json directly using input stream

Vaibs picture Vaibs · Jan 28, 2013 · Viewed 27.4k times · Source

I am doing a api call and in response i am getting json. So for that I want to parse it directly through input stream so that there would not be need of storing it in memory. For this I am trying to use JSONReader but that i am unable use for api's less than 11. So i dont know how to proceed with. I want it to be done from 2.0 version onwards. Even parsing through JsonReader is not working. I was thing of having GSON parser but i am not getting how to implement the same with Inputstream.

EDIT: My code for the same:

HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);

try {
          HttpResponse response = client.execute(httpGet);
          StatusLine statusLine = response.getStatusLine();
          int statusCode = statusLine.getStatusCode();
          Log.e("123", "Status code ----------- "+statusCode);
          if (statusCode == 200) { 
            HttpEntity entity = response.getEntity();
            InputStream content = entity.getContent();


            // for showing it on textview i am storing in in builder
            BufferedReader bReader = new BufferedReader(new InputStreamReader(content));
            String line;
            while ((line = bReader.readLine()) != null) {
              builder.append(line);
            }

//////////// 
            JsonReader reader = new JsonReader(new InputStreamReader(content));
            reader.setLenient(true);
            readGson(reader);
          } else {
            Log.e("TAG", "Failed to download file");
          }
        } catch (ClientProtocolException e) {
          e.printStackTrace();
        } catch (IOException e) {
          e.printStackTrace();
        }
        return builder.toString();
      }

private void readGson(JsonReader reader) {
    // TODO Auto-generated method stub
    Log.e("TAG", "inidde readgson"+reader);

                 try {
                     Log.e("TAG", "inside try");
                    reader.beginObject();
                    Log.e("123", "inside try  ");
                     while(reader.hasNext()){
                         Log.e("TAG", "reader values"+reader);
                         String name = reader.nextName();
                         if (name.equals("max_id")) {
                             long max_id = reader.nextLong();
                             Log.e("TAG", "sdbfhsajfgsdjbgksdfjv------------------"+max_id);
                           }  else {
                               Log.e("TAG", "c skfnvklsfvn skip value");
                             reader.skipValue();
                           }
                     }
                     reader.endObject();
                } catch (IOException e) {
                    // TODO Auto-generated catch block
                    Log.e("TAG", "inside catch");
                    e.printStackTrace();
                }
}

output:

    01-28 10:20:53.519: E/TAG(420): Status code ----------- 200
01-28 10:20:53.679: E/TAG(420): inidde readgsonJsonReader at line 1 column 1
01-28 10:20:53.679: E/TAG(420): inside try
01-28 10:20:53.679: E/TAG(420): inside catch
01-28 10:20:53.679: W/System.err(420): java.io.EOFException: End of input at line 1 column 1
01-28 10:20:53.689: W/System.err(420):  at com.google.gson.stream.JsonReader.nextNonWhitespace(JsonReader.java:954)
01-28 10:20:53.689: W/System.err(420):  at com.google.gson.stream.JsonReader.consumeNonExecutePrefix(JsonReader.java:405)
01-28 10:20:53.689: W/System.err(420):  at com.google.gson.stream.JsonReader.peek(JsonReader.java:364)
01-28 10:20:53.689: W/System.err(420):  at com.google.gson.stream.JsonReader.expect(JsonReader.java:337)
01-28 10:20:53.689: W/System.err(420):  at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:322)
01-28 10:20:53.689: W/System.err(420):  at com.example.httpconnectiondemo.MainActivity.readGson(MainActivity.java:136)
01-28 10:20:53.699: W/System.err(420):  at com.example.httpconnectiondemo.MainActivity.readTwitterFeed(MainActivity.java:116)
01-28 10:20:53.699: W/System.err(420):  at com.example.httpconnectiondemo.MainActivity.onCreate(MainActivity.java:64)
01-28 10:20:53.699: W/System.err(420):  at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1047)
01-28 10:20:53.699: W/System.err(420):  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1611)
01-28 10:20:53.699: W/System.err(420):  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1663)
01-28 10:20:53.699: W/System.err(420):  at android.app.ActivityThread.access$1500(ActivityThread.java:117)
01-28 10:20:53.699: W/System.err(420):  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:931)
01-28 10:20:53.699: W/System.err(420):  at android.os.Handler.dispatchMessage(Handler.java:99)
01-28 10:20:53.699: W/System.err(420):  at android.os.Looper.loop(Looper.java:123)
01-28 10:20:53.699: W/System.err(420):  at android.app.ActivityThread.main(ActivityThread.java:3683)
01-28 10:20:53.710: W/System.err(420):  at java.lang.reflect.Method.invokeNative(Native Method)
01-28 10:20:53.710: W/System.err(420):  at java.lang.reflect.Method.invoke(Method.java:507)
01-28 10:20:53.710: W/System.err(420):  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
01-28 10:20:53.710: W/System.err(420):  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
01-28 10:20:53.710: W/System.err(420):  at dalvik.system.NativeStart.main(Native Method)

Thanks in advance.

Answer

andr picture andr · Jan 28, 2013

You're using the content stream in a wrong way. Look at the marked lines:

InputStream content = entity.getContent();

// (1)
BufferedReader bReader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = bReader.readLine()) != null) {
    builder.append(line);
}

// (2)
JsonReader reader = new JsonReader(new InputStreamReader(content));
reader.setLenient(true);
readGson(reader);

The issue here is that you wrap the same stream twice (on marked line).

  • After first wrap, you go through the stream collecting the lines and concatenating them in a builder. You must be aware that effectively the bReader is just a wrapper around the content stream. So while you are collecting the lines, the content from the content stream is eaten. So, the condition (line = ...) != null is false when the content stream is at the end of input.
  • Then you wrap the content stream again - for parsing JSON contents. But the stream is already at the end of input here, so there is nothing to consume by the JSON reader. And that exactly is the meaning of the line: java.io.EOFException: End of input at line 1 column 1 of your exception.

What you have to do is to read through the content stream only once. So, you have several options here:

  • Don't save received contents to a String. You delete the first wrap and the loop which builds the response string using a builder. If that's not an option then I recommend:
  • Save the whole response to a String. Note: you can use the EntityUtils class for this since it will do the work for you:

    HttpEntity entity = response.getEntity();
    
    // All the work is done for you here :)
    String jsonContent = EntityUtils.toString(entity);
    
    // Create a Reader from String
    Reader stringReader = new StringReader(jsonContent);
    
    // Pass the string reader to JsonReader constructor
    JsonReader reader = new JsonReader(stringReader);
    reader.setLenient(true);
    readGson(reader);
    
    ...
    // at the end of method return the JSON response
    return jsonContent;