Annoying lags/stutters in an android game
I just started with game development in android, and I'm working on a super simple game.
The game is basically like flappy bird.
I managed to get everything to work, but I get a lot of stutters and lags.
The phone I'm using for testing is LG G2, so it should and does run games much heavier and complex than this.
Basically there are 4 'obstacles' that are a full screen width apart from each other.
When the game starts, the obstacles start to move (toward the character) at a constant speed. The player character's x value is consistent throughout the whole game, while its y value changes.
The lag occurs mainly when the character passes through an obstacle (and sometimes a little after that obstacle too). What happens is that there are uneven delays in each drawing of the game state causing stutters in the movements.
- GC does not run according to the log.
- The stutters ARE NOT caused by the speed being too high (I know that because at the beginning of the game when the obstacles are out of view the character moves smoothly)
- I don't think the problem is FPS related too, because even when the MAX_FPS field is set to 100 there are still stutters.
My thought is that there is a line or multiple lines of code that cause some kinda of delay to happen (and thus frames skipped). I also think that these lines should be around the update() and draw() methods of PlayerCharacter, Obstacle, and MainGameBoard.
The problem is, I'm still new to android development and android game development specifically, so I have no idea what could cause such delay.
I tried looking online for answers... Unfortunately, all of which I found pointed over to the GC being to blame. However, such I don't believe it it the case (correct me if I'm being wrong) those answers do no apply to me. I also read the android developer's Performance Tips page, but couldn't find anything that helped.
So, please, help me find the answer to solving these annoying lags!
Some code
MainThread.java:
public class MainThread extends Thread {
public static final String TAG = MainThread.class.getSimpleName();
private final static int MAX_FPS = 60; // desired fps
private final static int MAX_FRAME_SKIPS = 5; // maximum number of frames to be skipped
private final static int FRAME_PERIOD = 1000 / MAX_FPS; // the frame period
private boolean running;
public void setRunning(boolean running) {
this.running = running;
}
private SurfaceHolder mSurfaceHolder;
private MainGameBoard mMainGameBoard;
public MainThread(SurfaceHolder surfaceHolder, MainGameBoard gameBoard) {
super();
mSurfaceHolder = surfaceHolder;
mMainGameBoard = gameBoard;
}
@Override
public void run() {
Canvas mCanvas;
Log.d(TAG, "Starting game loop");
long beginTime; // the time when the cycle begun
long timeDiff; // the time it took for the cycle to execute
int sleepTime; // ms to sleep (<0 if we're behind)
int framesSkipped; // number of frames being skipped
sleepTime = 0;
while(running) {
mCanvas = null;
try {
mCanvas = this.mSurfaceHolder.lockCanvas();
synchronized (mSurfaceHolder) {
beginTime = System.currentTimeMillis();
framesSkipped = 0;
this.mMainGameBoard.update();
this.mMainGameBoard.render(mCanvas);
timeDiff = System.currentTimeMillis() - beginTime;
sleepTime = (int) (FRAME_PERIOD - timeDiff);
if(sleepTime > 0) {
try {
Thread.sleep(sleepTime);
} catch (InterruptedException e) {}
}
while(sleepTime < 0 && framesSkipped < MAX_FRAME_SKIPS) {
// catch up - update w/o render
this.mMainGameBoard.update();
sleepTime += FRAME_PERIOD;
framesSkipped++;
}
}
} finally {
if(mCanvas != null)
mSurfaceHolder.unlockCanvasAndPost(mCanvas);
}
}
}
}
MainGameBoard.java:
public class MainGameBoard extends SurfaceView implements
SurfaceHolder.Callback {
private MainThread mThread;
private PlayerCharacter mPlayer;
private Obstacle[] mObstacleArray = new Obstacle[4];
public static final String TAG = MainGameBoard.class.getSimpleName();
private long width, height;
private boolean gameStartedFlag = false, gameOver = false, update = true;
private Paint textPaint = new Paint();
private int scoreCount = 0;
private Obstacle collidedObs;
public MainGameBoard(Context context) {
super(context);
getHolder().addCallback(this);
DisplayMetrics displaymetrics = new DisplayMetrics();
((Activity) getContext()).getWindowManager().getDefaultDisplay().getMetrics(displaymetrics);
height = displaymetrics.heightPixels;
width = displaymetrics.widthPixels;
mPlayer = new PlayerCharacter(BitmapFactory.decodeResource(getResources(), R.drawable.ic_launcher), width/2, height/2);
for (int i = 1; i <= 4; i++) {
mObstacleArray[i-1] = new Obstacle(width*(i+1) - 200, height, i);
}
mThread = new MainThread(getHolder(), this);
setFocusable(true);
}
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width,
int height) {
}
@Override
public void surfaceCreated(SurfaceHolder holder) {
mThread.setRunning(true);
mThread.start();
}
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
Log.d(TAG, "Surface is being destroyed");
// tell the thread to shut down and wait for it to finish
// this is a clean shutdown
boolean retry = true;
while (retry) {
try {
mThread.join();
retry = false;
} catch (InterruptedException e) {
// try again shutting down the thread
}
}
Log.d(TAG, "Thread was shut down cleanly");
}
@Override
public boolean onTouchEvent(MotionEvent event) {
if(event.getAction() == MotionEvent.ACTION_DOWN) {
if(update && !gameOver) {
if(gameStartedFlag) {
mPlayer.cancelJump();
mPlayer.setJumping(true);
}
if(!gameStartedFlag)
gameStartedFlag = true;
}
}
return true;
}
@SuppressLint("WrongCall")
public void render(Canvas canvas) {
onDraw(canvas);
}
@Override
protected void onDraw(Canvas canvas) {
canvas.drawColor(Color.GRAY);
mPlayer.draw(canvas);
for (Obstacle obs : mObstacleArray) {
obs.draw(canvas);
}
if(gameStartedFlag) {
textPaint.reset();
textPaint.setColor(Color.WHITE);
textPaint.setTextAlign(Paint.Align.CENTER);
textPaint.setTextSize(100);
canvas.drawText(String.valueOf(scoreCount), width/2, 400, textPaint);
}
if(!gameStartedFlag && !gameOver) {
textPaint.reset();
textPaint.setColor(Color.WHITE);
textPaint.setTextAlign(Paint.Align.CENTER);
textPaint.setTextSize(72);
canvas.drawText("Tap to start", width/2, 200, textPaint);
}
if(gameOver) {
textPaint.reset();
textPaint.setColor(Color.WHITE);
textPaint.setTextAlign(Paint.Align.CENTER);
textPaint.setTextSize(86);
canvas.drawText("GAME OVER", width/2, 200, textPaint);
}
}
public void update() {
if(gameStartedFlag && !gameOver) {
for (Obstacle obs : mObstacleArray) {
if(update) {
if(obs.isColidingWith(mPlayer)) {
collidedObs = obs;
update = false;
gameOver = true;
return;
} else {
obs.update(width);
if(obs.isScore(mPlayer))
scoreCount++;
}
}
}
if(!mPlayer.update() || !update)
gameOver = true;
}
}
}
PlayerCharacter.java:
public void draw(Canvas canvas) {
canvas.drawBitmap(mBitmap, (float) x - (mBitmap.getWidth() / 2), (float) y - (mBitmap.getHeight() / 2), null);
}
public boolean update() {
if(jumping) {
y -= jumpSpeed;
jumpSpeed -= startJumpSpd/20f;
jumpTick--;
} else if(!jumping) {
if(getBottomY() >= startY*2)
return false;
y += speed;
speed += startSpd/25f;
}
if(jumpTick == 0) {
jumping = false;
cancelJump(); //rename
}
return true;
}
public void cancelJump() { //also called when the user touches the screen in order to stop a jump and start a new jump
jumpTick = 20;
speed = Math.abs(jumpSpeed);
jumpSpeed = 20f;
}
Obstacle.java:
public void draw(Canvas canvas) {
Paint pnt = new Paint();
pnt.setColor(Color.CYAN);
canvas.drawRect(x, 0, x+200, ySpaceStart, pnt);
canvas.drawRect(x, ySpaceStart+500, x+200, y, pnt);
pnt.setColor(Color.RED);
canvas.drawCircle(x, y, 20f, pnt);
}
public void update(long width) {
x -= speed;
if(x+200 <= 0) {
x = ((startX+200)/(index+1))*4 - 200;
ySpaceStart = r.nextInt((int) (y-750-250+1)) + 250;
scoreGiven = false;
}
}
public boolean isColidingWith(PlayerCharacter mPlayer) {
if(mPlayer.getRightX() >= x && mPlayer.getLeftX() <= x+20)
if(mPlayer.getTopY() <= ySpaceStart || mPlayer.getBottomY() >= ySpaceStart+500)
return true;
return false;
}
public boolean isScore(PlayerCharacter mPlayer) {
if(mPlayer.getRightX() >= x+100 && !scoreGiven) {
scoreGiven = true;
return true;
}
return false;
}
Answer
Update: As detailed as this was, it barely scratched the surface. A more detailed explanation is now available. The game loop advice is in Appendix A. If you really want to understand what's going on, start with that.
Original post follows...
I'm going to start with a capsule summary of how the graphics pipeline in Android works. You can find more thorough treatments (e.g. some nicely detailed Google I/O talks), so I'm just hitting the high points. This turned out rather longer than I expected, but I've been wanting to write some of this up for a while.
SurfaceFlinger
Your application does not draw on The Framebuffer. Some devices don't even have The Framebuffer. Your application holds the "producer" side of a BufferQueue object. When it has completed rendering a frame, it calls unlockCanvasAndPost() or eglSwapBuffers(), which queues up the completed buffer for display. (Technically, rendering may not even begin until you tell it to swap and can continue while the buffer is moving through the pipeline, but that's a story for another time.)
The buffer gets sent to the "consumer" side of the queue, which in this case is SurfaceFlinger, the system surface compositor. Buffers are passed by handle; the contents are not copied. Every time the display refresh (let's call it "VSYNC") starts, SurfaceFlinger looks at all the various queues to see what buffers are available. If it finds new content, it latches the next buffer from that queue. If it doesn't, it uses whatever it got previously.
The collection of windows (or "layers") that have visible content are then composited together. This may be done by SurfaceFlinger (using OpenGL ES to render the layers into a new buffer) or through the Hardware Composer HAL. The hardware composer (available on most recent devices) is provided by the hardware OEM, and can provide a number of "overlay" planes. If SurfaceFlinger has three windows to display, and the HWC has three overlay planes available, it puts each window into one overlay, and does the composition as the frame is being displayed. There is never a buffer that holds all the data. This is generally more efficient than doing the same thing in GLES. (Incidentally, this is why you can't grab a screen shot on most recent devices by simply opening the framebuffer dev entry and reading pixels.)
So that's what the consumer side looks like. You can admire it for yourself with adb shell dumpsys SurfaceFlinger. Let's go back to the producer (i.e. your app).
the producer
You're using a SurfaceView, which has two parts: a transparent View that lives with the system UI, and a separate Surface layer all its own. The SurfaceView's surface goes directly to SurfaceFlinger, which is why it has much less overhead than other approaches (like TextureView).
The BufferQueue for the SurfaceView's surface is triple-buffered. That means you can have one buffer being scanned out for the display, one buffer that is sitting at SurfaceFlinger waiting for the next VSYNC, and one buffer for your app to draw on. Having more buffers improves throughput and smooths out bumps, but increases the latency between when you touch the screen and when you see an update. Adding additional buffering of whole frames on top of this won't generally do you much good.
If you draw faster than the display can render frames, you will eventually fill up the queue, and your buffer-swap call (unlockCanvasAndPost()) will pause. This is an easy way to make your game's update rate the same as the display rate -- draw as fast as you can, and let the system slow you down. Each frame, you advance state according to how much time has elapsed. (I used this approach in Android Breakout.) It's not quite right, but at 60fps you won't really notice the imperfections. You'll get the same effect with sleep() calls if you don't sleep for long enough -- you'll wake up only to wait on the queue. In this case there's no advantage to sleeping, because sleeping on the queue is equally efficient.
If you draw slower than the display can render frames, the queue will eventually run dry, and SurfaceFlinger will display the same frame on two consecutive display refreshes. This will happen periodically if you're trying to pace your game with sleep() calls and you're sleeping for too long. It is impossible to precisely match the display refresh rate, for theoretical reasons (it's hard to implement a PLL without a feedback mechanism) and practical reasons (the refresh rate can change over time, e.g. I've seen it vary from 58fps to 62fps on a given device).
Using sleep() calls in a game loop to pace your animation is a bad idea.
going without sleep
You have a couple of choices. You can use the "draw as fast as you can until the buffer-swap call backs up" approach, which is what a lot of apps based on GLSurfaceView#onDraw() do (whether they know it or not). Or you can use Choreographer.
Choreographer allows you to set a callback that fires on the next VSYNC. Importantly, the argument to the callback is the actual VSYNC time. So even if your app doesn't wake up right away, you still have an accurate picture of when the display refresh began. This turns out to be very useful when updating your game state.
The code that updates game state should never be designed to advance "one frame". Given the variety of devices, and the variety of refresh rates that a single device can use, you can't know what a "frame" is. Your game will play slightly slow or slightly fast -- or if you get lucky and somebody tries to play it on a TV locked to 48Hz over HDMI, you'll be seriously sluggish. You need to determine the time difference between the previous frame and the current frame, and advance the game state appropriately.
This may require a bit of a mental reshuffle, but it's worth it.
You can see this in action in Breakout, which advances the ball position based on elapsed time. It cuts big jumps in time into smaller pieces to keep the collision detection simple. The trouble with Breakout is that it's using the stuff-the-queue-full approach, the timestamps are subject to variations in the time required for SurfaceFlinger to do work. Also, when the buffer queue is initially empty you can submit frames very quickly. (This means you compute two frames with nearly zero time delta, but they're still sent to the display at 60fps. In practice you don't see this, because the time stamp difference is so small that it just looks like the same frame drawn twice, and it only happens when transitioning from non-animating to animating so you don't see anything stutter.)
With Choreographer, you get the actual VSYNC time, so you get a nice regular clock to base your time intervals off of. Because you're using the display refresh time as your clock source, you never get out of sync with the display.
Of course, you still have to be prepared to drop frames.
no frame left behind
A while back I added a screen recording demo to Grafika ("Record GL app") that does very simple animation -- just a flat-shaded bouncing rectangle and a spinning triangle. It advances state and draws when Choreographer signals. I coded it up, ran it... and started to notice Choreographer callbacks backing up.
After digging at it with systrace, I discovered that the framework UI was occasionally doing some layout work (probably to do with the buttons and text in the UI layer, which sits on top of the SurfaceView surface). Normally this took 6ms, but if I wasn't actively moving my finger around the screen, my Nexus 5 slowed the various clocks to reduce power consumption and improve battery life. The re-layout took 28ms instead. Bear in mind that a 60fps frame is 16.7ms.
The GL rendering was nearly instantaneous, but the Choreographer update was being delivered to the UI thread, which was grinding away at the layout, so my renderer thread didn't get the signal until much later. (You could have Choreographer deliver the signal directly to the renderer thread, but there's a bug in Choreographer that will cause a memory leak if you do.) The fix was to drop frames when the current time is more than 15ms after the VSYNC time. The app still does the state update -- the collision detection is so rudimentary that weird stuff happens if you let the time gap grow too large -- but it doesn't submit a buffer to SurfaceFlinger.
While running the app you can tell when frames are being dropped, because Grafika flashes the border red and updates a counter on screen. You can't tell by watching the animation. Because the state updates are based on time intervals, not frame counts, everything moves just as fast as it would whether the frame was dropped or not, and at 60fps you won't notice a single dropped frame. (Depends to some extent on your eyes, the game, and the characteristics of the display hardware.)
Key lessons:
- Frame drops can be caused by external factors -- dependency on another thread, CPU clock speeds, background gmail sync, etc.
- You can't avoid all frame drops.
- If you set your draw loop up right, nobody will notice.
Drawing
Rendering to a Canvas can be very efficient if it's hardware-accelerated. If it's not, and you're doing the drawing in software, it can take a while -- especially if you're touching lots of pixels.
Two important bits of reading: learn about hardware-accelerated rendering, and using the hardware scaler to reduce the number of pixels your app needs to touch. The "Hardware scaler exerciser" in Grafika will give you a sense for what happens when you reduce the size of your drawing surface -- you can get pretty small before the effects are noticeable. (I find it oddly amusing to watch GL render a spinning triangle on a 100x64 surface.)
You can also take some of the mystery out of the rendering by using OpenGL ES directly. There's a bit of a bump learning how things work, but Breakout (and, for a more elaborate example, Replica Island) show everything you need for a simple game.