Saturday, May 31, 2014

Ford Sync AppLink Android Demo - Playing Online Audio

*** I have this TDK available for sale. Anyone interested, shoot me an email or a comment to discuss ***

    Back in March I won a Ford Sync Test Development Kit (TDK), a device for testing mobile integration with Ford vehicles. As of January, 2014, there are 1.5 million vehicles in North America that support AppLink, and the AppLink functionality will be released in Europe and Asia this year. 

    For this tutorial I will go over creating an app that plays an online audio stream (Denver's Comedy 103.1 icecast stream) from a mobile device over the Ford Sync system. The app supports audio playback, console and steering wheel button interactions, displaying text on the console and opening with a voice command. Below is a video I recorded today showing the app in use. Notice that the app itself is never brought to the foreground on the phone, as the only requirement is that the phone be on and paired with the system. All code for this demo is on GitHub.


    The first thing that we need to do is grab the AppLinkSDKAndroid-2-2 .jar file from http://developer.ford.com and place it into the project's lib folder. We then need to open the manifest to register required permissions, a receiver for the Sync bluetooth events and the service that controls the interactions between the Android device and the Ford Sync console. 

Permissions:
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.RECEIVE_BOOT_COMPLETED" />

Service and Receiver:
<service android:name=".Service.AppLinkService"></service>

<receiver android:name=".Receiver.AppLinkReceiver">
    <intent-filter>
        <action android:name="android.bluetooth.adapter.action.STATE_CHANGED" />
        <action android:name="android.bluetooth.device.action.ACL_CONNECTED" />
        <action android:name="android.bluetooth.device.action.ACL_DISCONNECTED"/>
        <action android:name="android.intent.action.BOOT_COMPLETED" />
        <action android:name="android.media.AUDIO_BECOMING_NOISY" />
    </intent-filter>
</receiver>

    It should be noted that the receiver should listen for five different actions pertaining to connection and signal noise. These actions help us determine when the audio service should be stopped or started.

    Once the manifest has been filled in, we can move over to our MainActivity file. In onCreate, we just need to call a method to start the AppLink service and proxy, which listens for events from the Ford Sync console. This allows the app to account for when the app is opened and then the user enters their vehicle, whereas our receiver will handle starting the service when the app is not open.

private void startSyncProxyService() {
    boolean isPaired = false;
    BluetoothAdapter btAdapter = BluetoothAdapter.getDefaultAdapter();

    if( btAdapter != null ) {
        if( btAdapter.isEnabled() && btAdapter.getBondedDevices() != null && !btAdapter.getBondedDevices().isEmpty() ) {
            for( BluetoothDevice device : btAdapter.getBondedDevices() ) {
                if( device.getName() != null && device.getName().contains( getString( R.string.device_name ) ) ) {
                    isPaired = true;
                    break;
                }
            }
        }

        if( isPaired ) {
            if( AppLinkService.getInstance() == null ) {
                Intent appLinkServiceIntent = new Intent( this, AppLinkService.class );
                startService( appLinkServiceIntent );
            } else {
                SyncProxyALM proxyInstance = AppLinkService.getInstance().getProxy();
                if( proxyInstance == null ) {
                    AppLinkService.getInstance().startProxy();
                }
            }
        }
    }
}

    The inverse to startSyncProxyService in onCreate is endSyncProxyService, which is called when MainActivity is destroyed.

private void endSyncProxyInstance() {
    if( AppLinkService.getInstance() != null ) {
        SyncProxyALM proxy = AppLinkService.getInstance().getProxy();
        if( proxy != null ) {
            AppLinkService.getInstance().reset();
        } else {
            AppLinkService.getInstance().startProxy();
        }
    }
}

    Next we need to fill in the AppLinkReceiver class, which extends BroadcastReceiver. The only method we need from the receiver is onReceive, which takes an intent and checks the actions associated with that intent in order to determine if the AppLinkService should be started or stopped.

@Override
public void onReceive(Context context, Intent intent) {
    if( intent == null || intent.getAction() == null || context == null )
        return;

    BluetoothDevice device = intent.getParcelableExtra( BluetoothDevice.EXTRA_DEVICE );
    String action = intent.getAction();

    Intent serviceIntent = new Intent( context, AppLinkService.class );
    serviceIntent.putExtras( intent );


    //Should start service
    if( action.compareTo(BluetoothDevice.ACTION_ACL_CONNECTED) == 0 &&
            device != null &&
            device.getName() != null &&
            device.getName().contains( context.getString( R.string.device_name ) ) &&
            AppLinkService.getInstance() == null )
    {
        context.startService(serviceIntent);
    }

    else if( action.equals( Intent.ACTION_BOOT_COMPLETED ) &&
            BluetoothAdapter.getDefaultAdapter() != null &&
            BluetoothAdapter.getDefaultAdapter().isEnabled() ) {
        context.startService(serviceIntent);

    }

    //Should stop service
    else if( action.equals( BluetoothDevice.ACTION_ACL_DISCONNECTED ) &&
            device != null &&
            device.getName() != null &&
            device.getName().contains( context.getString( R.string.device_name ) ) &&
            AppLinkService.getInstance() != null )
    {
        context.stopService( intent );
    }

    else if( action.equals(BluetoothAdapter.ACTION_STATE_CHANGED ) &&
        intent.getIntExtra(BluetoothAdapter.EXTRA_STATE, -1) == BluetoothAdapter.STATE_TURNING_OFF &&
        AppLinkService.getInstance() != null )
    {
        context.stopService( serviceIntent );
    }

    else if( action.equals( AudioManager.ACTION_AUDIO_BECOMING_NOISY ) ) {
        context.stopService( serviceIntent );
    }
}

    Now that we have the infrastructure for starting and stopping our AppLinkService, it's time to build the service file. We start off by extending Service and implementing Ford's IProxyListenerALM interface, then importing all of the required stub methods associated with it. At the top of the class we should define our class variables:

private static AppLinkService mInstance = null;
private MediaPlayer mPlayer = null;
private SyncProxyALM mProxy = null;
private int mCorrelationId = 0;

    Because of the way Ford handles the AppLinkService and the requirement that there only be one AppLinkService and SyncProxyALM to control all interactions between an app and the console, we must keep an instance of our service and proxy. We also need to keep track of an integer value (mCorrelationId) to differentiate messages between the console and app. When the service is started in onStartCommand, we need to check that bluetooth is available and enabled, and then start the proxy.

@Override
public int onStartCommand(Intent intent, int flags, int startId) {
    if( intent != null &&
        BluetoothAdapter.getDefaultAdapter() != null &&
        BluetoothAdapter.getDefaultAdapter().isEnabled() ) {
        startProxy();
    }

    return START_STICKY;
}

    The startProxy method simply creates a SyncProxyALM object with a title and number for identifying the app. The title used here is what's used for starting the app with vocal commands from the console.

public void startProxy() {
    if( mProxy == null ) {
        try {
            mProxy = new SyncProxyALM( this, getString( R.string.display_title ), true, getString( R.string.app_link_id ) );
        } catch( SyncException e ) {
            if( mProxy == null ) {
                stopSelf();
            }
        }
    }
}

    When the app connects to Sync and starts on the console, the onOnHMIStatus method is called with an OnHMIStatus object passed in. This is where we determine if audio should be played or stopped, display initial text on the screen and we determine what buttons to subscribe to.

@Override
public void onOnHMIStatus(OnHMIStatus onHMIStatus) {
    switch( onHMIStatus.getSystemContext() ) {
        case SYSCTXT_MAIN:
        case SYSCTXT_VRSESSION:
        case SYSCTXT_MENU:
            break;
        default:
            return;
    }

    switch( onHMIStatus.getAudioStreamingState() ) {
        case AUDIBLE: {
            playAudio();
            break;
        }
        case NOT_AUDIBLE: {
            stopAudio();
            break;
        }
    }

    if( mProxy == null )
        return;

    if( onHMIStatus.getHmiLevel().equals( HMILevel.HMI_FULL ) && onHMIStatus.getFirstRun() ) {
        //setup app with SYNC
        try {
            mProxy.show( "Welcome to Paul's", "Ford AppLink Demo", TextAlignment.CENTERED, mCorrelationId++ );
        } catch( SyncException e ) {}
        subscribeToButtons();
    }
}

    subscribeToButtons simply subscribes through the control proxy, and the service listens for button presses and performs actions based on which button comes through Sync's event listener

private void subscribeToButtons() {
    if( mProxy == null )
        return;

    try {
        mProxy.subscribeButton( ButtonName.OK, mCorrelationId++ );
    } catch( SyncException e ) {}
}

@Override
public void onOnButtonPress(OnButtonPress notification) {
    if( ButtonName.OK == notification.getButtonName() ) {
        if( mPlayer != null ) {
            if( mPlayer.isPlaying() ) {
                stopAudio();
            } else {
                playAudio();
            }
        }
    }
}

    The final steps to implementing audio over AppLink is playing and stopping the audio, which is handled exactly as it would be in any other app, where the MediaPlayer object is created and set with a URL, then started once the stream has loaded. The only difference here is that the proxy is used to display text to the user's console as the operations happen.

private void playAudio() {
    String url = "http://173.231.136.91:8060/";
    if( mPlayer == null )
        mPlayer = new MediaPlayer();

    try {
        mProxy.show("Loading...", "", TextAlignment.CENTERED, mCorrelationId++);
    } catch( SyncException e ) {}

    mPlayer.reset();
    mPlayer.setAudioStreamType( AudioManager.STREAM_MUSIC );
    try {
        mPlayer.setDataSource(url);
        mPlayer.setOnPreparedListener( new MediaPlayer.OnPreparedListener() {
            @Override
            public void onPrepared( MediaPlayer mediaPlayer ) {
                mediaPlayer.start();
                try {
                    mProxy.show("Playing online audio", "", TextAlignment.CENTERED, mCorrelationId++);
                } catch( SyncException e ) {}
            }
        });
        mPlayer.prepare();
    } catch (IllegalArgumentException e) {
    } catch (SecurityException e) {
    } catch (IllegalStateException e) {
    } catch (IOException e) {
    }
}

private void stopAudio() {
    if( mPlayer == null )
        return;
    mPlayer.pause();
    try {
        mProxy.show("Press OK", "to play audio", TextAlignment.CENTERED, mCorrelationId++);
    } catch( SyncException e ) {}
}

    And with that, we have a simple audio service integrated with Ford Sync AppLink. There's a lot more that can be done with the system, such as text to speech, using GPS, and integrating any information available from the phone via sensors or online data. As more vehicles end up with this system in the world, it'll be interesting to see what else people do with it.