Sending Messages to Android Auto

    In November of 2014, Google provided developers with two simulators, one for media apps and another for message notifications, to start updating and testing apps to work with the new Android Auto platform. In a previous post, I shared how to create a service that works with Android Auto in order to play media, and in this post I will go over how to interact with and send messages to the Android Auto notification screen. All source code for this tutorial can be found on GitHub. The previous post (linked above) has instructions for installing the Auto simulators, so I'm going to skip over that in this post for brevity.

Main message screen on Android Auto

    To start, we're going to want to get everything set up. We'll do this by first making sure our app is targeting SDK version 21 or higher in build.gradle. Next we'll move over to adding in a new xml file under /res/xml called automotive_app_desc.xml and add in the following content to let Android Auto know that this app supports notifications

<automotiveApp>
    <uses name="notification"/>
</automotiveApp>

    Next we're going to want to go into AndroidManifest.xml and add a metadata tag within the application node to direct the OS towards our previously created xml file.

<meta-data android:name="com.google.android.gms.car.application"
     android:resource="@xml/automotive_app_desc" />

    In addition to our metadata tag, we're going to add two broadcast receivers with intent-filters that look for specific actions for 'read' or 'reply' user actions. While I'm using broadcast receivers here, it should be noted that you could also create Services to handle the read and reply situations instead, which I'll talk about a little more when we start tying in these broadcast receivers in code.

<receiver android:name=".AutoMessageReadReceiver" android:exported="false">
    <intent-filter>
        <action android:name="com.ptrprograms.androidautomessenger.ACTION_MESSAGE_READ"/>
    </intent-filter>
</receiver>

<receiver android:name=".AutoMessageReplyReceiver" android:exported="false">
    <intent-filter>
        <action android:name="com.ptrprograms.androidautomessenger.ACTION_MESSAGE_REPLY"/>
    </intent-filter>
</receiver>

    Now that the general setup is done, we can jump into our Java code. The way Android Auto's messaging system works is an Android application connected to Auto runs all of the logic for when a notification should be constructed, then it sends that notification to the Auto dashboard. To keep things simple, I'm going to build out the notification in our application's MainActivity.java file in order to point out what needs to be done. The first part of this should seem familiar if you've dealt with Android notifications before (and if not, I've written other posts on creating notifications): we're going to create a general NotificationCompat.Builder object.

NotificationCompat.Builder notificationBuilder =
    new NotificationCompat.Builder( getApplicationContext() )
            .setSmallIcon( R.drawable.ic_launcher )
            .setLargeIcon( BitmapFactory.decodeResource( getResources(), R.drawable.ic_launcher ) )
            .setContentText( "content text" )
            .setWhen( Calendar.getInstance().get( Calendar.SECOND ) )
            .setContentTitle( "content title" );

    With the NotificationCompat.Builder created, we can start adding in the Android Auto portions of the code. To do this, we need to use the .extend() method of NotificationCompat.Builder to add functionality from NotificationCompat.CarExtender(), which also uses a builder pattern to add functionality to our app (more options are available than shown here, such as setting text color for the Auto notification, but I'll leave that off for now). Each message group on the Auto dashboard is called a Conversation, so we also need to create an UnreadConversation to add to our notification.

notificationBuilder.extend( new NotificationCompat.CarExtender()
        .setUnreadConversation( getUnreadConversation() ) );

   getUnreadConversation() handles creating the messages that we will display, the actions taken when the user has read them and if they reply

private NotificationCompat.CarExtender.UnreadConversation getUnreadConversation() {
    NotificationCompat.CarExtender.UnreadConversation.Builder unreadConversationBuilder =
            new NotificationCompat.CarExtender.UnreadConversation.Builder( UNREAD_CONVERSATION_BUILDER_NAME );

    unreadConversationBuilder
        .setReadPendingIntent( getMessageReadPendingIntent() )
        .setReplyAction( getMessageReplyPendingIntent(), getVoiceReplyRemoteInput() )
        .addMessage( "Message 1")
        .addMessage( "Message 2" )
        .addMessage( "Message 3" )
        .setLatestTimestamp( Calendar.getInstance().get( Calendar.SECOND ) );

    return unreadConversationBuilder.build();
}

    The important things to pay attention to here are getMessageReadPendingIntent(), getMessageReplyPendingIntent() and getVoiceReplyRemoteInput(). As mentioned before, I'm using a set of BroadcastReceivers to handle responding to when the user reads or replies to a message, but since we're using a system of PendingIntents here, we could just as easily create a set of intents that go to a service to deal with these actions. getMessageReadPendingIntent() is pretty straight forward, creating a pending intent with an action that will be caught by AutoMessageReadReceiver.java after the user has tapped on the message and it has been read aloud by the system.

private Intent getMessageReadIntent() {
    return new Intent()
        .addFlags( Intent.FLAG_INCLUDE_STOPPED_PACKAGES )
        .setAction( MESSAGE_READ_ACTION )
        .putExtra( MESSAGE_CONVERSATION_ID_KEY, 1 );
}

private PendingIntent getMessageReadPendingIntent() {
    return PendingIntent.getBroadcast( getApplicationContext(),
            1,
            getMessageReadIntent(),
            PendingIntent.FLAG_UPDATE_CURRENT );
}

    AutoMessageReadReceiver simply listens for our intent with the MESSAGE_READ_ACTION and dismisses the notification associated with the intent.

public class AutoMessageReadReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        int conversationId = intent.getIntExtra( MainActivity.MESSAGE_CONVERSATION_ID_KEY, -1 );
        Log.d( "Message", "id: " + conversationId );
        NotificationManagerCompat.from( context ).cancel( conversationId );
    }
}

Messages before they have been listened to

Message notification after all 'UnreadConversations' have been read

    The reply action is a little more interesting, though equally straight forward. We create another PendingIntent for a separate BroadcastReceiver, and also attach a RemoteInput for a voice reply with the setReplyAction method of our UnreadConversationBuilder.

private Intent getMessageReplyIntent() {
    return new Intent()
            .addFlags( Intent.FLAG_INCLUDE_STOPPED_PACKAGES )
            .setAction( MESSAGE_REPLY_ACTION )
            .putExtra( MESSAGE_CONVERSATION_ID_KEY, 1 );
}

private PendingIntent getMessageReplyPendingIntent() {
    return PendingIntent.getBroadcast( getApplicationContext(),
            1,
            getMessageReplyIntent(),
            PendingIntent.FLAG_UPDATE_CURRENT );
}

private RemoteInput getVoiceReplyRemoteInput() {
    return new RemoteInput.Builder( VOICE_REPLY_KEY )
            .setLabel( "Reply" )
            .build();
}

    where our BroadcastReceiver, AutoMessageReplyReceiver.java, not only marks a message as read, but will also extract a voice reply provided by the user in order to use it within our app.

public class AutoMessageReplyReceiver extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        Toast.makeText( context, "Message Received", Toast.LENGTH_LONG ).show();

        int conversationId = intent.getIntExtra( MainActivity.MESSAGE_CONVERSATION_ID_KEY, -1 );
        Log.d( "Message", "id: " + conversationId );
        NotificationManagerCompat.from(context).cancel( conversationId );

        String message = getMessageFromIntent( intent );
    }

    private String getMessageFromIntent( Intent intent ) {
        //Note that Android Auto does not currently allow voice responses in their simulator
        Bundle remoteInput = RemoteInput.getResultsFromIntent( intent );
        if( remoteInput != null && remoteInput.containsKey( "extra_voice_reply" ) ) {
            return remoteInput.getCharSequence( "extra_voice_reply" ).toString();
        }

        return null;
    }
}

Reply button pressed. Currently the Android Auto simulator does not allow for voice responses, so it sends an empty string value with the RemoteInput and posts a Toast with "Canned response sent".

    Once the notification is properly set up for displaying on Auto, we use NotificationManagerCompat to display it on our device and Auto from MainActivity.java

NotificationManagerCompat.from( this ).notify( 1, notificationBuilder.build() );

    Once the notification has been sent, it will be visible to the user on the Android Auto dashboard. While this system may be simple to work with, given a proper location or context aware app, it could be an invaluable feature for users as they travel. While there are still a few things missing from the official documentation that I'd like to see, such as how to hide the reply button, I'm definitely excited to see what else becomes available as Auto is released and matures, and hope this tutorial helps others create great apps.

Using the Android Auto Media Browser

    During the 2014 I/O, Google made the announcement that Android would be available for in-vehicle systems, but left everyone hanging on the major details. Luckily, in November 2014 they released two simulator APKs for developers to begin testing different features of their apps for Android Auto: audio and messaging services. For this tutorial, I will go over using the Android Auto framework to browse media and create a simple media player (which can obviously be expanded on for a real application). Source code for this tutorial can be found on GitHub.

   In order to start, we'll first need to install the Media Browser Simulator APK for Android Auto onto a Lollipop device. I ended up using a 2013 Nexus 7, though a Lollipop emulator should also work for our purposes. This can be found under your Android SDK folder along the path /extras/google/simulators/media-browser-simulator.apk after installing Android Auto API Simulators from the SDK Manger. You can install this through the Android Device Bridge (ADB) with the following command:

adb install media-browser-simulator.apk

    Once this is loaded up, we should be able to open the application on our device to see a screen similar to this (minus the AndroidAutoMedia item that we'll build through this tutorial):

    Next we'll want to get going on our actual app. We'll start by creating an app in Android Studio for phone/tablet. Once that's done and we have our standard "Hello World" template together, we'll need to go under /res/xml and create a new xml file - under my source code it's called automotive_app_desc.xml, though you can call it whatever you'd like. This file will be used by our application to let Android Auto know that we're building components to work with it. For now, simply copy the following source code into the XML file that tells Android Auto that we're making a media Auto plugin.

<automotiveApp>
    <uses name="media" />
</automotiveApp>

    With our XML file created, we'll move over and open AndroidManifest.xml to make a few additions to work with Auto. Within the application tag, we're going to want to add two meta-data items - one points to the XML file we just created, and the other provides the Android Auto launcher icon used in the media browser shown above (I just used the standard ic_launcher.png).

<meta-data android:name="com.google.android.gms.car.application"
    android:resource="@xml/automotive_app_desc"/>

<meta-data android:name="com.google.android.gms.car.notification.SmallIcon"
    android:resource="@drawable/ic_launcher" />

    After the meta-data nodes are made, we'll want to add a new Service node to the manifest, and include an intent-filter for "android.media.browse.MediaBrowserService."

<service android:name="com.ptrprograms.androidautomedia.service.AutoMediaBrowserService">
    <intent-filter>
        <action android:name="android.media.browse.MediaBrowserService" />
    </intent-filter>
</service>

    Now that the manifest is filled out, we can start working on our Java code. We'll create a service (I called mine AutoMediaBrowserService, as seen above) and extend the MediaBrowserService class provided in the SDK. There are two methods that we will want to override here in order to get started. The first is onGetRoot( String clientPackageName, int clientUid, Bundle rootHints ), which is called by the Auto media browser application when first interacting with our service. In this method, we're only going to add one line to return a new BrowserRoot object using our root identifier string (I've defined mine at the top of the class as a final variable of BROWSEABLE_ROOT), though you can also use this method to validate the calling package to verify that it should have access to your media assets and return null if it fails validation.

@Override
public BrowserRoot onGetRoot(String clientPackageName, int clientUid, Bundle rootHints) {
    return new BrowserRoot(BROWSEABLE_ROOT, null);
}

    The other method that we'll need to override is onLoadChildren(String parentId, Result<List<MediaBrowser.MediaItem>> result). This method is called when the root item or any subsequent items are clicked on in the Android Auto media browser. This is where you will check the parentId parameter and build out the list of MediaBrowser.MediaItems based on that id in order to provide a file structure for your app. In order to simplify things, I've moved this logic into a separate method so that onLoadChildren() is easily digestible.

@Override
public void onLoadChildren(String parentId, Result<List<MediaBrowser.MediaItem>> result) {

    List<MediaBrowser.MediaItem> items = getMediaItemsById( parentId );
    if( items != null ) {
        result.sendResult( items );
    }

}

    Now before I go into explaining how getMediaItemsById() works for providing media items and folders, I need to go over a bit of the prep code that I placed into onCreate() for this service. I created a model object called Song that consists of general information about the media object, such as artist, title, genre, etc. and a quick generator class to create the media objects that will be used for this sample. How you create and work with your own media objects will depend on your implementation and applications, so I tried to keep mine as simple as possible The Song model object source can be found here, and the generator helper class can be found here. With that in mind, my onCreate() looks like this

    @Override
    public void onCreate() {
        super.onCreate();

        mSongs = SongGenerator.generateSongs();

        initMediaSession();
    }

    where initMediaSession simply builds out the MediaSession and Token used for this sample:

private void initMediaSession() {
    mMediaSession = new MediaSession( this, "Android Auto Audio Demo" );
    mMediaSession.setActive( true );
    mMediaSession.setCallback( mMediaSessionCallback );

    mMediaSessionToken = mMediaSession.getSessionToken();
    setSessionToken( mMediaSessionToken );
}

    With this foundation in mind, we can move over to looking at getMediaItemsById(). This method will take an id, either our root id or one associated with a clicked item in the Auto browser, and create a folder or file structure to display to the user.

    private List<MediaBrowser.MediaItem> getMediaItemsById( String id ) {
        List<MediaBrowser.MediaItem> mediaItems = new ArrayList<MediaBrowser.MediaItem>();
        if( BROWSEABLE_ROOT.equalsIgnoreCase( id ) ) {
            mediaItems.add( generateBrowseableMediaItemByGenre(BROWSEABLE_CAJUN) );
            mediaItems.add( generateBrowseableMediaItemByGenre(BROWSEABLE_JAZZ) );
            mediaItems.add( generateBrowseableMediaItemByGenre(BROWSEABLE_ROCK) );
        } else if( !TextUtils.isEmpty( id ) ) {
            return getPlayableMediaItemsByGenre( id );
        }

        return mediaItems;
    }

    As you can see, if BROWSEABLE_ROOT is the clicked item, we create three media items with the method generateBrowseableMediaItemByGenre() and pass in one of three strings defined at the top of the class, and then return those items as part of our MediaItems list. generateBrowseableMediaItemByGenre() simply creates a MediaItem with the FLAG_BROWSEABLE flag set and some basic information to define the folders, as shown here:

private MediaBrowser.MediaItem generateBrowseableMediaItemByGenre( String genre ) {
    MediaDescription.Builder mediaDescriptionBuilder = new MediaDescription.Builder();
    mediaDescriptionBuilder.setMediaId( genre );
    mediaDescriptionBuilder.setTitle( genre );
    mediaDescriptionBuilder.setIconBitmap( BitmapFactory.decodeResource( getResources(), R.drawable.folder ) );

    return new MediaBrowser.MediaItem( mediaDescriptionBuilder.build(), MediaBrowser.MediaItem.FLAG_BROWSABLE );
}

    This will provide us with three folders for three different genres under our root structure, like so

    When one of these three folders is tapped on by a user, we return to onLoadChildren(), which in turn calls getMediaItemsById() again, but this getMediaItemsById() will have an id that is not our root folder, so getPlayableMediaItemsByGenre( String genre ) is called. This will loop through all of our songs and create a list of MediaItems based on the passed genre

private List<MediaBrowser.MediaItem> getPlayableMediaItemsByGenre( String genre ) {
    if( TextUtils.isEmpty( genre ) )
        return null;

    List<MediaBrowser.MediaItem> mediaItems = new ArrayList();

    for( Song song : mSongs ) {
        if( !TextUtils.isEmpty( song.getGenre() ) && genre.equalsIgnoreCase( song.getGenre() ) ) {
            mediaItems.add( generatePlayableMediaItem( song ) );
        }
    }
    return mediaItems;
}

    where generatePlayableMediaItem( song ) creates a MediaItem with properties based on the song and the FLAG_PLAYABLE flag set

private MediaBrowser.MediaItem generatePlayableMediaItem( Song song ) {
    if( song == null )
        return null;

    MediaDescription.Builder mediaDescriptionBuilder = new MediaDescription.Builder();
    mediaDescriptionBuilder.setMediaId( song.getuId() );

    if( !TextUtils.isEmpty( song.getTitle() ) )
        mediaDescriptionBuilder.setTitle( song.getTitle() );

    if( !TextUtils.isEmpty( song.getArtist() ) )
        mediaDescriptionBuilder.setSubtitle( song.getArtist() );

    if( !TextUtils.isEmpty( song.getThumbnailUrl() ) )
        mediaDescriptionBuilder.setIconUri( Uri.parse( song.getThumbnailUrl() ) );

    return new MediaBrowser.MediaItem( mediaDescriptionBuilder.build(), MediaBrowser.MediaItem.FLAG_PLAYABLE );
}

    At this point we should be able to click through the Auto browser to get to a screen similar to this:

    Now if you remember, back in initMediaSession() we had this line:

mMediaSession.setCallback( mMediaSessionCallback );

    mMediaSessionCallback is defined at the top of our class and is meant to handle actions from the buttons available in the Auto media player. This is where we start and stop our media, define which controls are available and set the metadata for our media that will be displayed by Android Auto.

private MediaSession.Callback mMediaSessionCallback = new MediaSession.Callback() {
    @Override
    public void onPlay() {
        super.onPlay();

        toggleMediaPlaybackState( true );
        playMedia( PreferenceManager.getDefaultSharedPreferences( getApplicationContext() ).getInt( CURRENT_MEDIA_POSITION, 0 ), null );
    }

    //This is called when the pause button is pressed, or when onPlayFromMediaId is called in
    //order to pause any currently playing media
    @Override
    public void onPause() {
        super.onPause();

        toggleMediaPlaybackState( false );
        pauseMedia();
    }

    @Override
    public void onPlayFromMediaId(String mediaId, Bundle extras) {
        super.onPlayFromMediaId(mediaId, extras);

        initMediaMetaData( mediaId );
        toggleMediaPlaybackState( true );
        playMedia( 0, mediaId );
    }
};

    As you can see, I have a few helper methods here that I'll go over next. onPlayFromMediaId( String mediaId, Bundle extras ) is the callback that the system uses when a MediaItem with the FLAG_PLAYABLE property is clicked. In this callback we first set up our metadata for displaying information about the MediaItem with initMediaMetaData()

private void initMediaMetaData( String id ) {

    for( Song song : mSongs ) {
        if( !TextUtils.isEmpty( song.getuId() ) && song.getuId().equalsIgnoreCase( id ) ) {
            MediaMetadata.Builder builder = new MediaMetadata.Builder();

            if( !TextUtils.isEmpty( song.getTitle() ) )
                builder.putText( MediaMetadata.METADATA_KEY_TITLE, song.getTitle() );

            if( !TextUtils.isEmpty( song.getArtist() ) )
                builder.putText( MediaMetadata.METADATA_KEY_ARTIST, song.getArtist() );

            if( !TextUtils.isEmpty( song.getGenre() ) )
                builder.putText( MediaMetadata.METADATA_KEY_GENRE, song.getGenre() );

            if( !TextUtils.isEmpty( song.getAlbum() ) )
                builder.putText( MediaMetadata.METADATA_KEY_ALBUM, song.getAlbum() );

            if( !TextUtils.isEmpty( song.getAlbumUrl() ) )
                builder.putText( MediaMetadata.METADATA_KEY_ALBUM_ART_URI, song.getAlbumUrl() );

            mMediaSession.setMetadata( builder.build() );
        }
    }
}

     Then we toggle our MediaState and set our controls with toggleMediaPlaybackState(). Note that the controls are set using setAction on our PlaybackState.Builder() by bitwise ORing our actions, so when our media is playing we'll display the Pause, Skip to Next and Skip to Previous, and when our media is paused we'll only display the Play button.

private void toggleMediaPlaybackState( boolean playing ) {
    PlaybackState playbackState;
    if( playing ) {
        playbackState = new PlaybackState.Builder()
                .setActions( PlaybackState.ACTION_PLAY_PAUSE | PlaybackState.ACTION_SKIP_TO_NEXT | PlaybackState.ACTION_SKIP_TO_PREVIOUS )
                .setState( PlaybackState.STATE_PLAYING, 0, 1 )
                .build();
    } else {
        playbackState = new PlaybackState.Builder()
                .setActions( PlaybackState.ACTION_PLAY_PAUSE )
                .setState(PlaybackState.STATE_PAUSED, 0, 1)
                .build();
    }

    mMediaSession.setPlaybackState( playbackState );
}

 

        The final methods in our service simply control starting, pausing and resuming our media. For this sample I only have one MP3 available to keep things simple, though you can set what plays based on the media id passed to the callback methods. The MP3 I'm using here was provided by a good friend of mine, Geoff Ledak, who you can catch DJing online at AfterHoursDJs.org.

private void playMedia( int position, String id ) {
    if( mMediaPlayer != null )
        mMediaPlayer.reset();

    //Should check id to determine what to play in a real app
    int songId = getApplicationContext().getResources().getIdentifier("geoff_ledak_dust_array_preview", "raw", getApplicationContext().getPackageName());
    mMediaPlayer = MediaPlayer.create(getApplicationContext(), songId);

    if( position > 0 )
        mMediaPlayer.seekTo( position );
    mMediaPlayer.start();

}

private void pauseMedia() {
    if( mMediaPlayer != null ) {
        mMediaPlayer.pause();
        PreferenceManager.getDefaultSharedPreferences( this ).edit().putInt( CURRENT_MEDIA_POSITION,
                mMediaPlayer.getCurrentPosition() ).commit();
    }
}

@Override
public void onDestroy() {
    super.onDestroy();
    if( mMediaPlayer != null ) {
        pauseMedia();
        mMediaPlayer.release();
        PreferenceManager.getDefaultSharedPreferences( this ).edit().putInt( CURRENT_MEDIA_POSITION,
                0 ).commit();
    }

}

    One more thing we can do is add a colorAccent to our app theme in order to add a bit of personalization to fit our app under /res/values-v21

<style name="AppTheme" parent="android:Theme.Material.Light">
    <item name="android:colorAccent">@android:color/holo_green_dark</item>
</style>

    And with that, we should have a working Android Auto browser service plugin. I hope you've enjoyed the tutorial, and good luck building your own implementations for this new platform!