ExoPlayer Analytics for Android TV

August 5, 2019 |by Diego Perini

In our latest post, we discussed how to implement an analytics collector for vanilla media player and its adapter libraries in Android. It was trivial but lengthy in terms of loc not to break existing functionality in those tools and in our opinion, those libraries didn’t age well. Fortunately, Google published a new open source media player named ExoPlayer with magical abilities that works in all kinds of Android devices last year. Its modern API allows us to do more with less code. In this post, we will write an analytics collector for ExoPlayer using your favorite testing tool.

Android TV Blog Post Series

These posts build on top of each other. If you plan to learn other cool stuff as well, feel free to take a look.

1. Testing Android TV Apps
2. Tracking Video Events in Android TV (with MediaPlayer)
3. ExoPlayer Analytics for Android TV

Design Goals

Let’s rephrase our design goals from the previous post to make them more aligned with ExoPlayer’s capabilities.

1. Support for ExoPlayer
2. A one liner API that looks like exoPlayer.addAnalyticsListener(new TestFairyExoPlayerAnalyticsListener(exoPlayer));
3. Sending analytic events for buffering and playback events
4. Keeping track of play/pause/error states
5. Not breaking existing functionality with ExoPlayer classes
6. Exposing existing public API from the wrapped classes to allow further extensions by users

In vanilla media player, not breaking existing functionality while listening all the desired events required us to build wrappers that listens all the necessary events before forwarding them to outside delegates. ExoPlayer provides a huge improvement to this workflow by supporting multiple listeners on the same player using its player.addAnalyticsListener() api.

The entire events list provided by the AnalyticsListener interface is huge. We will be mostly focusing on playback states, error cases and buffering/progress events.

Just a quick reminder: You can replace every line that calls TestFairy SDK methods in this tutorial with your testing tool of choice without any problem.

Implementing the Listener

Let’s define a class that implements the listener.

import android.os.Handler;
import android.view.Surface;
import androidx.annotation.Nullable;
import com.google.android.exoplayer2.ExoPlaybackException;
import com.google.android.exoplayer2.ExoPlayer;
import com.google.android.exoplayer2.Format;
import com.google.android.exoplayer2.PlaybackParameters;
import com.google.android.exoplayer2.Player;
import com.google.android.exoplayer2.SimpleExoPlayer;
import com.google.android.exoplayer2.analytics.AnalyticsListener;
import com.google.android.exoplayer2.audio.AudioAttributes;
import com.google.android.exoplayer2.decoder.DecoderCounters;
import com.google.android.exoplayer2.metadata.Metadata;
import com.google.android.exoplayer2.source.MediaSourceEventListener;
import com.google.android.exoplayer2.source.TrackGroupArray;
import com.google.android.exoplayer2.trackselection.TrackSelectionArray;
import com.google.android.exoplayer2.upstream.HttpDataSource;
import com.testfairy.TestFairy;
import java.io.IOException;
import java.util.Locale;

public class TestFairyExoPlayerAnalyticsListener implements AnalyticsListener {
    // TODO : implement methods from interface
}

Don’t get scared by the import list. In most cases, your IDE will automatically handle those statements for you.

This class won’t compile without specifying overrides for each listener method. We will put the ones we are interested in together and leave the rest of the overrides as empty blocks. Here is an organized version of the event list.

public class TestFairyExoPlayerAnalyticsListener implements AnalyticsListener {

  ////////////////////THESE WE NEED/////////////////////////////
  @Override
  public void onPlayerStateChanged(EventTime eventTime, boolean playWhenReady,
      int playbackState) {
    // TODO
  }

  @Override
  public void onPositionDiscontinuity(EventTime eventTime, int reason) {
    // TODO
  }

  @Override
  public void onSeekProcessed(EventTime eventTime) {
    // TODO
  }

  @Override
  public void onRepeatModeChanged(EventTime eventTime, int repeatMode) {
    // TODO
  }

  @Override
  public void onShuffleModeChanged(EventTime eventTime, boolean shuffleModeEnabled) {
    // TODO
  }

  @Override
  public void onPlayerError(EventTime eventTime, ExoPlaybackException error) {
    // TODO
  }

  @Override
  public void onTracksChanged(EventTime eventTime, TrackGroupArray trackGroups,
      TrackSelectionArray trackSelections) {
    // TODO
  }

  @Override
  public void onLoadCompleted(EventTime eventTime,
      MediaSourceEventListener.LoadEventInfo loadEventInfo,
      MediaSourceEventListener.MediaLoadData mediaLoadData) {
    // TODO
  }

  @Override
  public void onLoadError(EventTime eventTime,
      MediaSourceEventListener.LoadEventInfo loadEventInfo,
      MediaSourceEventListener.MediaLoadData mediaLoadData, IOException error,
      boolean wasCanceled) {
    // TODO
  }

  @Override
  public void onVolumeChanged(EventTime eventTime, float volume) {
    // TODO
  }

  @Override
  public void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) {
    // TODO
  }

  @Override
  public void onDrmSessionManagerError(EventTime eventTime, Exception error) {
    // TODO
  }

  ////////////////////THESE WE DON'T NEED/////////////////////////////
  @Override
  public void onAudioUnderrun(EventTime eventTime, int bufferSize, long bufferSizeMs,
      long elapsedSinceLastFeedMs) {
  }

  @Override
  public void onVideoSizeChanged(EventTime eventTime, int width, int height,
      int unappliedRotationDegrees, float pixelWidthHeightRatio) {
  }

  @Override
  public void onRenderedFirstFrame(EventTime eventTime, @Nullable Surface surface) {
  }

  @Override
  public void onDrmSessionAcquired(EventTime eventTime) {
  }

  @Override
  public void onDrmKeysLoaded(EventTime eventTime) {
  }

  @Override
  public void onDownstreamFormatChanged(EventTime eventTime,
      MediaSourceEventListener.MediaLoadData mediaLoadData) {
  }

  @Override
  public void onUpstreamDiscarded(EventTime eventTime,
      MediaSourceEventListener.MediaLoadData mediaLoadData) {
  }

  @Override
  public void onMediaPeriodCreated(EventTime eventTime) {
  }

  @Override
  public void onMediaPeriodReleased(EventTime eventTime) {
  }

  @Override
  public void onReadingStarted(EventTime eventTime) {
  }

  @Override
  public void onBandwidthEstimate(EventTime eventTime, int totalLoadTimeMs,
      long totalBytesLoaded, long bitrateEstimate) {
  }

  @Override
  public void onSurfaceSizeChanged(EventTime eventTime, int width, int height) {
  }

  @Override
  public void onMetadata(EventTime eventTime, Metadata metadata) {
  }

  @Override
  public void onDecoderEnabled(EventTime eventTime, int trackType,
      DecoderCounters decoderCounters) {
  }

  @Override
  public void onDecoderInitialized(EventTime eventTime, int trackType, String decoderName,
      long initializationDurationMs) {
  }

  @Override
  public void onDecoderInputFormatChanged(EventTime eventTime, int trackType, Format format) {
  }

  @Override
  public void onDecoderDisabled(EventTime eventTime, int trackType,
      DecoderCounters decoderCounters) {
  }

  @Override
  public void onAudioSessionId(EventTime eventTime, int audioSessionId) {
  }

  @Override
  public void onAudioAttributesChanged(EventTime eventTime, AudioAttributes audioAttributes) {
  }

  @Override
  public void onLoadCanceled(EventTime eventTime,
      MediaSourceEventListener.LoadEventInfo loadEventInfo,
      MediaSourceEventListener.MediaLoadData mediaLoadData) {
  }

  @Override
  public void onLoadStarted(EventTime eventTime,
      MediaSourceEventListener.LoadEventInfo loadEventInfo,
      MediaSourceEventListener.MediaLoadData mediaLoadData) {
  }

  @Override
  public void onLoadingChanged(EventTime eventTime, boolean isLoading) {
  }

  @Override
  public void onPlaybackParametersChanged(EventTime eventTime,
      PlaybackParameters playbackParameters) {
  }

  @Override
  public void onSeekStarted(EventTime eventTime) {
  }

  @Override
  public void onTimelineChanged(EventTime eventTime, int reason) {
  }

  @Override
  public void onDrmKeysRestored(EventTime eventTime) {
  }

  @Override
  public void onDrmKeysRemoved(EventTime eventTime) {
  }

  @Override
  public void onDrmSessionReleased(EventTime eventTime) {
  }
}

ExoPlayer among all these elaborate events, doesn’t publish any playback progress events. Luckily, the player has necessary getter methods we can periodically query to calculate seek position in video. Just like we did in the previous tutorial, we will provide an on/off switch for our custom handler which will publish this event manually.

First, we need to ask for the player in our listener constructor and initialize a handler which will be used to run small blocks in the main thread asynchronously.

private final Handler handler = new Handler();
private final ExoPlayer player;

private Runnable currentPositionTracker;
private long lastKnownPlaybackPercent = -1;

public TestFairyExoPlayerAnalyticsListener(ExoPlayer player) {
  this.player = player;
}

Our handler will run currentPositionTracker every 100 miliseconds and ask for playback progress. It will compare this progress with the stored value of its previous query to decide if a new event should be generated. Below is the boilerplate that enables and disables this behavior as desired.

 private void registerCurrentPositionTracker() {
  unRegisterCurrentPositionTracker();

  currentPositionTracker = new Runnable() {
    @Override
    public void run() {
      long currentPosition = player.getCurrentPosition();
      long percent = (currentPosition * 100) / player.getDuration();

      if (lastKnownPlaybackPercent != percent) {
        lastKnownPlaybackPercent = percent;
        TestFairy.addEvent(String.format(Locale.ENGLISH, "Playback position %d%%", percent));
      }

      handler.postDelayed(currentPositionTracker, 100);
    }
  };

  handler.postDelayed(currentPositionTracker, 100);
}

private void unRegisterCurrentPositionTracker() {
  if (currentPositionTracker != null) {
    handler.removeCallbacks(currentPositionTracker);
    currentPositionTracker = null;
  }
}

In our listener code, when we learn that a playback is resumed, we will register this tracker. Whenever the playback is paused or completed, we will unregister the tracker to leave the CPU idle.

Playback events reported by ExoPlayer are quite straightforward and also includs pause events occured due to slow buffering or activity lifecycle.

@Override
public void onPlayerStateChanged(EventTime eventTime, boolean playWhenReady,
    int playbackState) {
  if (playWhenReady && playbackState == Player.STATE_READY) {
    if (currentPositionTracker == null) {
      registerCurrentPositionTracker();
    }

    TestFairy.addEvent("Playback is playing");
  } else if (playWhenReady) {
    unRegisterCurrentPositionTracker();

    if (player.getDuration() != 0 && player.getDuration() <= player.getCurrentPosition()) {
      TestFairy.addEvent("Playback has completed");
    } else {
      TestFairy.addEvent("Playback is buffering or paused automatically");
    }
  } else {
    unRegisterCurrentPositionTracker();

    TestFairy.addEvent("Playback is paused");
  }
}

If video playback stutters without pausing entirely, we can deduce the reason with the following event.

@Override
public void onPositionDiscontinuity(EventTime eventTime, int reason) {
  switch (reason) {
    case SimpleExoPlayer.DISCONTINUITY_REASON_PERIOD_TRANSITION:
      TestFairy.addEvent("Video stutters due to period transition");
      break;
    case SimpleExoPlayer.DISCONTINUITY_REASON_SEEK:
      TestFairy.addEvent("Video stutters due to a seek");
      break;
    case SimpleExoPlayer.DISCONTINUITY_REASON_SEEK_ADJUSTMENT:
      TestFairy.addEvent("Video stutters due to seek adjustment");
      break;
    case SimpleExoPlayer.DISCONTINUITY_REASON_AD_INSERTION:
      TestFairy.addEvent("Video stutters due to an inserted ad");
      break;
    case SimpleExoPlayer.DISCONTINUITY_REASON_INTERNAL:
      TestFairy.addEvent("Video stutters due to an internal problem");
      break;
  }
}

If a user skips a portion of the video, we have the following event generated to get the newly seeked playback position.

@Override
public void onSeekProcessed(EventTime eventTime) {
  long currentPosition = player.getCurrentPosition();
  long percent = (currentPosition * 100) / player.getDuration();

  lastKnownPlaybackPercent = percent;

  TestFairy
      .addEvent(String.format(Locale.ENGLISH, "Playback seeks to position %d%%", percent));
}

ExoPlayer is a rich library with built-in shuffle and repeat functionality. These toggles also report relevant events if utilized properly.

@Override
public void onRepeatModeChanged(EventTime eventTime, int repeatMode) {
  switch (repeatMode) {
    case ExoPlayer.REPEAT_MODE_OFF:
      TestFairy.addEvent("Repeat mode has been changed to OFF");
    case ExoPlayer.REPEAT_MODE_ONE:
      TestFairy.addEvent("Repeat mode has been changed to ONE");
    case ExoPlayer.REPEAT_MODE_ALL:
      TestFairy.addEvent("Repeat mode has been changed to ALL");
      break;
  }
}

@Override
public void onShuffleModeChanged(EventTime eventTime, boolean shuffleModeEnabled) {
  if (shuffleModeEnabled) {
    TestFairy.addEvent("Shuffle mode is enabled");
  } else {
    TestFairy.addEvent("Shuffle mode is disabled");
  }
}

If we change tracks in the same player, the following event notifies the change for us.

@Override
public void onTracksChanged(EventTime eventTime, TrackGroupArray trackGroups,
    TrackSelectionArray trackSelections) {
  lastKnownPlaybackPercent = -1;
  TestFairy.addEvent("A new video has been loaded");
}

ExoPlayer buffers the video in chunks, sized according to connection quality. If the file is read from a stream, it will benchmark the connection in real time to decide the best chunk size. If it is read from the file system, a small amount of large chunks will be utilized instead. In both cases, this architecture allows loading very long videos without any issue. The event defined below works for both scenarios.

@Override
public void onLoadCompleted(EventTime eventTime,
    MediaSourceEventListener.LoadEventInfo loadEventInfo,
    MediaSourceEventListener.MediaLoadData mediaLoadData) {
  TestFairy.addEvent(
      String.format(Locale.ENGLISH, "Playback is buffering %d%%", player.getBufferedPercentage())
  );
}

Listening for volume level events will allow us to detect videos with problematic audio configurations.

@Override
public void onVolumeChanged(EventTime eventTime, float volume) {
  TestFairy.addEvent(String.format(Locale.ENGLISH, "Volume level has changed to %s",
      Float.valueOf(volume * 100f).intValue()));
}

Badly encoded or streamed videos are likely to drop frames to be able to stay in sync with the played audio. The event generated below will notify us if such issues occur.

@Override
public void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) {
  TestFairy.addEvent(
      String.format(Locale.ENGLISH, "Video has dropped %d frames in %dms", droppedFrames,
          elapsedMs));
}

Error Events

Errors in ExoPlayer come in three distinct categories.

• Loading errors which occur if the loaded file is corrupted
• Drm errors that occur due to a failed decryption of a copyright protected video
• The remaining IOException errors due to http connection or storage issues

TestFairy SDK provides a convenient logThrowable() utility to report stack traces gathered from these events. It will help us recognize recurring errors in our session dashboard without needing to navigate through multiple session manually.

@Override
public void onLoadError(EventTime eventTime,
    MediaSourceEventListener.LoadEventInfo loadEventInfo,
    MediaSourceEventListener.MediaLoadData mediaLoadData, IOException error,
    boolean wasCanceled) {
  TestFairy.addEvent("Error during loading");
  TestFairy.logThrowable(error);
}

@Override
public void onDrmSessionManagerError(EventTime eventTime, Exception error) {
  TestFairy.addEvent("Drm session manager error occured");
  TestFairy.logThrowable(error);
}

Finally, using the suggested deduction logic from ExoPlayer’s documentation, we can distinguish http errors from the rest using a catch-all event generated for all kinds of errors. If an http error is found, the http error code returned from the server will also reported as a seperate event for convenience.

public void onPlayerError(EventTime eventTime, ExoPlaybackException error) {
  if (error.type == ExoPlaybackException.TYPE_SOURCE) {
    IOException cause = error.getSourceException();

    if (cause instanceof HttpDataSource.HttpDataSourceException) {
      // An HTTP error occurred.
      HttpDataSource.HttpDataSourceException httpError = (HttpDataSource.HttpDataSourceException) cause;

      // It's possible to find out more about the error both by casting and by
      // querying the cause.
      if (httpError instanceof HttpDataSource.InvalidResponseCodeException) {
        // Cast to InvalidResponseCodeException and retrieve the response code,
        // message and headers.
        HttpDataSource.InvalidResponseCodeException ex = (HttpDataSource.InvalidResponseCodeException) httpError;

        TestFairy.addEvent(
            String.format(Locale.ENGLISH, "Http error during playback - %d", ex.responseCode));
        TestFairy.logThrowable(cause);
      } else {
        // Try calling httpError.getCause() to retrieve the underlying cause,
        // although note that it may be null.
        TestFairy.addEvent("Http error during playback before response");

        Throwable innerCause = httpError.getCause();
        if (innerCause != null) {
          TestFairy.logThrowable(innerCause);
        } else {
          TestFairy.logThrowable(cause);
        }
      }
    } else {
      TestFairy.addEvent(String.format(Locale.ENGLISH, "Player error - %s", error.toString()));
      TestFairy.logThrowable(cause);
    }
  }
}

Give it a run!

If everything went smoothly, you can start listening for all the mentioned events with the following code.

exoPlayer.addAnalyticsListener(new TestFairyExoPlayerAnalyticsListener(exoPlayer));

If you’d like to jump right in without needing to glue all these code blocks manually, simply copy this file into your project and enjoy your new analytics in your session dashboard.

References

• ExoPlayer website