⑤NuPlayer播放框架之GenericSource源码分析

[时间：2017-01] [状态：Open]

[关键词：android，nuplayer，开源播放器，播放框架，GenericSource]

0 导读

GenericSource是NuPlayer::Source的一个子类，主要功能是负责本地多媒体文件的读取解析，功能类似FFmpeg的libavformt。

通常GenericSource有以下功能：

• 多媒体文件格式探测
• 多媒体文件解析

本文是我的NuPlayer播放框架的第五篇。

1 GenericSource对外接口及主要成员

NuPlayer::Source抽象类

先从父类开始，接口如下：

// code from ~/frameworks/av/media/libmediaplayerservice/nuplayer/NuPlayerSource.h
struct NuPlayer::Source : public AHandler {
    enum Flags {
        FLAG_CAN_PAUSE          = 1,
        FLAG_CAN_SEEK_BACKWARD  = 2,  // the "10 sec back button"
        FLAG_CAN_SEEK_FORWARD   = 4,  // the "10 sec forward button"
        FLAG_CAN_SEEK           = 8,  // the "seek bar"
        FLAG_DYNAMIC_DURATION   = 16,
        FLAG_SECURE             = 32,
        FLAG_PROTECTED          = 64,
    };

    // The provides message is used to notify the player about various events.
    Source(const sp<AMessage> &notify): mNotify(notify) {}

    virtual void prepareAsync() = 0; // 常规接口，播放、停止、暂停、恢复
    virtual void start() = 0;
    virtual void stop() {}
    virtual void pause() {}
    virtual void resume() {}

    // Explicitly disconnect the underling data source
    virtual void disconnect() {}

    // Returns OK iff more data was available, an error or ERROR_END_OF_STREAM if not.
    virtual status_t feedMoreTSData() = 0;
	// 获取音视频格式
    virtual sp<AMessage> getFormat(bool audio);
    virtual sp<MetaData> getFormatMeta(bool /* audio */) { return NULL; }
    virtual sp<MetaData> getFileFormatMeta() const { return NULL; }

    virtual status_t dequeueAccessUnit(bool audio, sp<ABuffer> *accessUnit) = 0;

    virtual status_t getDuration(int64_t * /* durationUs */) {...}
	// 返回实际解析之后的节目数目及信息，这里将每个节目称为一个Track
    virtual size_t getTrackCount() const {return 0;}
    virtual sp<AMessage> getTrackInfo(size_t /* trackIndex */) const {...}
    virtual ssize_t getSelectedTrack(media_track_type /* type */) const {...}
    virtual status_t selectTrack(size_t /* trackIndex */, bool /* select */, int64_t /* timeUs*/) {...}
	// seek操作的主要执行函数
    virtual status_t seekTo(int64_t /* seekTimeUs */) {...}

    virtual status_t setBuffers(bool /* audio */, Vector<MediaBuffer *> &/* buffers */) {...}

    virtual bool isRealTime() const {return false;}

    virtual bool isStreaming() const {return true;}

    virtual void setOffloadAudio(bool /* offload */) {}

protected:
    virtual ~Source() {}

    virtual void onMessageReceived(const sp<AMessage> &msg);

    sp<AMessage> dupNotify() const { return mNotify->dup(); }

    void notifyFlagsChanged(uint32_t flags);
    void notifyVideoSizeChanged(const sp<AMessage> &format = NULL);
    void notifyInstantiateSecureDecoders(const sp<AMessage> &reply);
    void notifyPrepared(status_t err = OK);

private:
    sp<AMessage> mNotify;
};

从类层次结构来看，NuPlayer::Source的子类有：GenericSource、HTTPLiveSource、RTSPSource、StreamingSource。

本文先从最简单的GenericSource开始分析。

GenericSource对外接口及主要成员

鉴于存在继承关系，这里仅给出GenericSource特有的接口，及其主要成员函数：

// 注意这里有部分代码删减，并不是全部
struct NuPlayer::GenericSource : public NuPlayer::Source {
    GenericSource(const sp<AMessage> &notify, bool uidValid, uid_t uid);

    status_t setDataSource(const sp<IMediaHTTPService> &httpService,
            const char *url, const KeyedVector<String8, String8> *headers);
    status_t setDataSource(int fd, int64_t offset, int64_t length);
    status_t setDataSource(const sp<DataSource>& dataSource);
private:
	struct Track {
        size_t mIndex;
        sp<IMediaSource> mSource;
        sp<AnotherPacketSource> mPackets;
    };
	// Helper to monitor buffering status. The polling happens every second.
    // When necessary, it will send out buffering events to the player.
    struct BufferingMonitor : public AHandler { ... };

	Vector<sp<IMediaSource> > mSources;
    Track mAudioTrack; // 音频流
    int64_t mAudioTimeUs;
    int64_t mAudioLastDequeueTimeUs;
    Track mVideoTrack; // 视频流
    int64_t mVideoTimeUs;
    int64_t mVideoLastDequeueTimeUs;
    Track mSubtitleTrack; // 字幕流
    Track mTimedTextTrack;

    sp<IMediaHTTPService> mHTTPService;
    AString mUri;
    KeyedVector<String8, String8> mUriHeaders;
    int mFd;
    int64_t mOffset;
    int64_t mLength;

    sp<DataSource> mDataSource;
    sp<NuCachedSource2> mCachedSource;
    sp<DataSource> mHttpSource;
    sp<WVMExtractor> mWVMExtractor;
    sp<MetaData> mFileMeta;
    DrmManagerClient *mDrmManagerClient;
    sp<DecryptHandle> mDecryptHandle;
    bool mStarted;
    bool mStopRead;
    int64_t mBitrate;
    sp<BufferingMonitor> mBufferingMonitor;
    uint32_t mPendingReadBufferTypes;
    sp<ABuffer> mGlobalTimedText;

    sp<ALooper> mLooper;
    sp<ALooper> mBufferingMonitorLooper;
};

从这里可以看到GenericSource添加了setDataSource接口，并包含多个Track和各种DataSource/NuCachedSource2。

2 NuPlayer中调用的Source接口

先回顾下NuPlayer源码解析中的调用的NuPlayer::Source接口。

• 构建部分接口——构造/析构函数/setDataSource
• 基本播放控制接口——prepareAsync/stop/start/pause/resume/seekTo/disconnect
• 节目信息相关——getTrackCount/getTrackInfo/getSelectedTrack/selectTrack/getFormat
• 辅助信息获取及设置——getDuration/isRealTime/getFormatMeta/isStreaming/setOffloadAudio/setBuffers/setBuffers/feedMoreTSData

一个典型的GenericS初始化逻辑如下：

    sp<AMessage> notify = new AMessage(kWhatSourceNotify, this);
    sp<GenericSource> source = new GenericSource(notify, mUIDValid, mUID);
    status_t err = source->setDataSource(fd, offset, length);

3 构建部分接口

构造函数

GenericSource的构造函数相对简单，代码如下：

void NuPlayer::GenericSource::resetDataSource() {
    mHTTPService.clear();
    mHttpSource.clear();
    mUri.clear();
    mUriHeaders.clear();
    if (mFd >= 0) {
        close(mFd);
        mFd = -1;
    }
    mOffset = 0;
    mLength = 0;
    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);
    mDecryptHandle = NULL;
    mDrmManagerClient = NULL;
    mStarted = false;
    mStopRead = true;

    if (mBufferingMonitorLooper != NULL) {
        mBufferingMonitorLooper->unregisterHandler(mBufferingMonitor->id());
        mBufferingMonitorLooper->stop();
        mBufferingMonitorLooper = NULL;
    }
    mBufferingMonitor->stop();
}
NuPlayer::GenericSource::GenericSource(
        const sp<AMessage> &notify,
        bool uidValid,
        uid_t uid)
    : Source(notify),
      mAudioTimeUs(0),
      mAudioLastDequeueTimeUs(0),
      mVideoTimeUs(0),
      mVideoLastDequeueTimeUs(0),
      mFetchSubtitleDataGeneration(0),
      mFetchTimedTextDataGeneration(0),
      mDurationUs(-1ll),
      mAudioIsVorbis(false),
      mIsWidevine(false),
      mIsSecure(false),
      mIsStreaming(false),
      mUIDValid(uidValid),
      mUID(uid),
      mFd(-1),
      mDrmManagerClient(NULL),
      mBitrate(-1ll),
      mPendingReadBufferTypes(0) {
    mBufferingMonitor = new BufferingMonitor(notify);
    resetDataSource();
    DataSource::RegisterDefaultSniffers(); // 这部分注意下，各种格式的探测链就是在这里初始化的
}

逻辑比较简单都是一些变量及参数的初始化。比较有意思的是关于Sniffer的注册。我们看一下对应代码：

// code from ~/frameworks/av/media/libstagefright/DataSource.cpp
Mutex DataSource::gSnifferMutex;
List<DataSource::SnifferFunc> DataSource::gSniffers;
bool DataSource::gSniffersRegistered = false;

// static
void DataSource::RegisterSniffer_l(SnifferFunc func) {
    for (List<SnifferFunc>::iterator it = gSniffers.begin();
         it != gSniffers.end(); ++it) {
        if (*it == func) {
            return;
        }
    }

    gSniffers.push_back(func);
}

// static
void DataSource::RegisterDefaultSniffers() {
    Mutex::Autolock autoLock(gSnifferMutex);
    if (gSniffersRegistered) {
        return;
    }

    RegisterSniffer_l(SniffMPEG4); // mpeg4
    RegisterSniffer_l(SniffMatroska); // mkv
    RegisterSniffer_l(SniffOgg); // ogg
    RegisterSniffer_l(SniffWAV); // wav
    RegisterSniffer_l(SniffFLAC); // flac
    RegisterSniffer_l(SniffAMR); // amr
    RegisterSniffer_l(SniffMPEG2TS); // mpeg-ts
    RegisterSniffer_l(SniffMP3); // mp3
    RegisterSniffer_l(SniffAAC); // aac
    RegisterSniffer_l(SniffMPEG2PS); // mpeg-ps
    if (getuid() == AID_MEDIA) {
        // WVM only in the media server process
        RegisterSniffer_l(SniffWVM);
    }
    RegisterSniffer_l(SniffMidi);

    char value[PROPERTY_VALUE_MAX];
    if (property_get("drm.service.enabled", value, NULL)
            && (!strcmp(value, "1") || !strcasecmp(value, "true"))) {
        RegisterSniffer_l(SniffDRM);
    }
    gSniffersRegistered = true;
}

上面代码就是将所有支持的容器格式放到一个List中。后面会用到的。

析构函数

析构函数相对简单，直接销毁Looper，重置DataSource即可，代码如下：

NuPlayer::GenericSource::~GenericSource() {
    if (mLooper != NULL) {
        mLooper->unregisterHandler(id());
        mLooper->stop();
    }
    resetDataSource();
}

setDataSource接口

设置数据源的接口，有三个重载函数，这里以file_descriptor的接口为例给出，实现非常简单，就是保存下参数就算完成了。代码如下：

status_t NuPlayer::GenericSource::setDataSource(
        int fd, int64_t offset, int64_t length) {
    resetDataSource();

    mFd = dup(fd);
    mOffset = offset;
    mLength = length;

    // delay data source creation to prepareAsync() to avoid blocking
    // the calling thread in setDataSource for any significant time.
    return OK;
}

4 基本播放控制接口

这一部分的接口可以认为是一个多媒体文件播放必然会调用的接口。

prepareAsync

代码里边主要是发送kWhatPrepareAsync消息，如下：

void NuPlayer::GenericSource::prepareAsync() {
    sp<AMessage> msg = new AMessage(kWhatPrepareAsync, this);
    msg->post();
}

具体消息处理是在onPrepareAsync中。这个函数完成了格式探测和Metadata提取等操作，处理代码如下：

// 注意代码有删减
void NuPlayer::GenericSource::onPrepareAsync() {
    // delayed data source creation 创建DataSource
    if (mDataSource == NULL) {
		{
            mIsWidevine = false;

            mDataSource = new FileSource(mFd, mOffset, mLength);
            mFd = -1;
        }

        if (mDataSource == NULL) {
            ALOGE("Failed to create data source!");
            notifyPreparedAndCleanup(UNKNOWN_ERROR);
            return;
        }
    }

    if (mDataSource->flags() & DataSource::kIsCachingDataSource) {
        mCachedSource = static_cast<NuCachedSource2 *>(mDataSource.get());
    }

    // init extractor from data source
    status_t err = initFromDataSource();

    if (err != OK) {
        ALOGE("Failed to init from data source!");
        notifyPreparedAndCleanup(err); // 上报处理结果
        return;
    }

    if (mVideoTrack.mSource != NULL) {
        sp<MetaData> meta = doGetFormatMeta(false /* audio */);
        sp<AMessage> msg = new AMessage;
        err = convertMetaDataToMessage(meta, &msg);
        if(err != OK) {
            notifyPreparedAndCleanup(err);
            return;
        }
        notifyVideoSizeChanged(msg); // 上报视频分辨率
    }
	// 上报流状态
    notifyFlagsChanged(
            (mIsSecure ? FLAG_SECURE : 0)
            | (mDecryptHandle != NULL ? FLAG_PROTECTED : 0)
            | FLAG_CAN_PAUSE
            | FLAG_CAN_SEEK_BACKWARD
            | FLAG_CAN_SEEK_FORWARD
            | FLAG_CAN_SEEK);

   	finishPrepareAsync();// 上报函数调用正常结束
}

这里重点关注下initFromDataSource函数，因为这里面包含多媒体文件格式探测，代码如下：

status_t NuPlayer::GenericSource::initFromDataSource() {
    sp<IMediaExtractor> extractor;
    String8 mimeType;
    float confidence;
    sp<AMessage> dummy;

    CHECK(mDataSource != NULL);

 	{
        extractor = MediaExtractor::Create(mDataSource,
                mimeType.isEmpty() ? NULL : mimeType.string());
    }

    if (extractor == NULL) {
        return UNKNOWN_ERROR;
    }

    if (extractor->getDrmFlag()) {
        checkDrmStatus(mDataSource);
    }

    mFileMeta = extractor->getMetaData();
    if (mFileMeta != NULL) {
        int64_t duration;
        if (mFileMeta->findInt64(kKeyDuration, &duration)) {
            mDurationUs = duration;
        }
    }

    int32_t totalBitrate = 0;
    size_t numtracks = extractor->countTracks();
    if (numtracks == 0) {
        return UNKNOWN_ERROR;
    }
	// 读取多媒体文件的全部Track信息
    for (size_t i = 0; i < numtracks; ++i) {
        sp<IMediaSource> track = extractor->getTrack(i);
        if (track == NULL) {
            continue;
        }

        sp<MetaData> meta = extractor->getTrackMetaData(i);
        if (meta == NULL) {
            ALOGE("no metadata for track %zu", i);
            return UNKNOWN_ERROR;
        }

        const char *mime;
        CHECK(meta->findCString(kKeyMIMEType, &mime));

        // 处理下音视频信息，并保存 ... 省略部分代码
        mSources.push(track);
        int64_t durationUs;
        if (meta->findInt64(kKeyDuration, &durationUs)) {
            if (durationUs > mDurationUs) {
                mDurationUs = durationUs;
            }
        }

        int32_t bitrate;
        if (totalBitrate >= 0 && meta->findInt32(kKeyBitRate, &bitrate)) {
            totalBitrate += bitrate;
        } else {
            totalBitrate = -1;
        }
    }

    if (mSources.size() == 0) {
        ALOGE("b/23705695");
        return UNKNOWN_ERROR;
    }

    mBitrate = totalBitrate;

    return OK;
}

最终通过MediaExtractor::CreateFromService调用DataSource::sniff函数来判断具体类型。sniff的实现代码如下：

bool DataSource::sniff(String8 *mimeType, float *confidence, sp<AMessage> *meta) {
    *mimeType = "";
    *confidence = 0.0f;
    meta->clear();
	// 遍历，找得分最高的，注意需要遍历全部支持的格式
    for (List<SnifferFunc>::iterator it = gSniffers.begin();
         it != gSniffers.end(); ++it) {
        String8 newMimeType;
        float newConfidence;
        sp<AMessage> newMeta;
        if ((*it)(this, &newMimeType, &newConfidence, &newMeta)) {
            if (newConfidence > *confidence) {
                *mimeType = newMimeType;
                *confidence = newConfidence;
                *meta = newMeta;
            }
        }
    }

    return *confidence > 0.0;
}

stop/start

stop函数实现相对简单，直接修改当前状态。代码如下：

void NuPlayer::GenericSource::stop() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);
    mStarted = false;
	// ...
}

start函数主要发送kWhatStart消息，代码如下：

void NuPlayer::GenericSource::start() {
    mStopRead = false;
    if (mAudioTrack.mSource != NULL) { // 启动音频包读取
        postReadBuffer(MEDIA_TRACK_TYPE_AUDIO);
    }

    if (mVideoTrack.mSource != NULL) { // 启动视频包读取
        postReadBuffer(MEDIA_TRACK_TYPE_VIDEO);
    }

    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);
    mStarted = true;

    (new AMessage(kWhatStart, this))->post();
}

实际消息响应函数比较简单，如下：

case kWhatStart:
case kWhatResume:
{
  mBufferingMonitor->restartPollBuffering();
  break;
}

pause/resume

这两个函数跟start/pause类似，直接设置状态值，代码如下：

void NuPlayer::GenericSource::pause() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);
    mStarted = false;
}

void NuPlayer::GenericSource::resume() {
    // nothing to do, just account for DRM playback status
    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);
    mStarted = true;

    (new AMessage(kWhatResume, this))->post();
}

seekTo

seekTo是实现多媒体文件seek的主要函数，其实现跟kWhatSeek消息有关，代码如下：

status_t NuPlayer::GenericSource::seekTo(int64_t seekTimeUs) {
    sp<AMessage> msg = new AMessage(kWhatSeek, this);
    msg->setInt64("seekTimeUs", seekTimeUs);

    sp<AMessage> response;
    status_t err = msg->postAndAwaitResponse(&response);
    if (err == OK && response != NULL) {
        CHECK(response->findInt32("err", &err));
    }

    return err;
}

实际消息响应函数在onSeek中，代码如下：

void NuPlayer::GenericSource::onSeek(sp<AMessage> msg) {
    int64_t seekTimeUs;
    CHECK(msg->findInt64("seekTimeUs", &seekTimeUs));

    sp<AMessage> response = new AMessage;
    status_t err = doSeek(seekTimeUs); // 这是实际作seek的
    response->setInt32("err", err);

    sp<AReplyToken> replyID;
    CHECK(msg->senderAwaitsResponse(&replyID));
    response->postReply(replyID);
}

status_t NuPlayer::GenericSource::doSeek(int64_t seekTimeUs) {
    mBufferingMonitor->updateDequeuedBufferTime(-1ll);

    // If the Widevine source is stopped, do not attempt to read any more buffers.
    if (mStopRead) {
        return INVALID_OPERATION;
    }
    if (mVideoTrack.mSource != NULL) { // 调整视频读取时间
        int64_t actualTimeUs;
        readBuffer(MEDIA_TRACK_TYPE_VIDEO, seekTimeUs, &actualTimeUs);

        seekTimeUs = actualTimeUs;
        mVideoLastDequeueTimeUs = seekTimeUs;
    }

    if (mAudioTrack.mSource != NULL) { // 调整音频读取时间
        readBuffer(MEDIA_TRACK_TYPE_AUDIO, seekTimeUs);
        mAudioLastDequeueTimeUs = seekTimeUs;
    }

    setDrmPlaybackStatusIfNeeded(Playback::START, seekTimeUs / 1000);
    if (!mStarted) {
        setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);
    }

    // If currently buffering, post kWhatBufferingEnd first, so that
    // NuPlayer resumes. Otherwise, if cache hits high watermark
    // before new polling happens, no one will resume the playback.
    mBufferingMonitor->stopBufferingIfNecessary();
    mBufferingMonitor->restartPollBuffering();

    return OK;
}

disconnect

这个函数主要断开DataSource和GenericSource之间的关联，保证后续可用，代码如下：

void NuPlayer::GenericSource::disconnect() {
    sp<DataSource> dataSource, httpSource;
    {
        Mutex::Autolock _l(mDisconnectLock);
        dataSource = mDataSource;
        httpSource = mHttpSource;
    }

    if (dataSource != NULL) {
        // disconnect data source
        if (dataSource->flags() & DataSource::kIsCachingDataSource) {
            static_cast<NuCachedSource2 *>(dataSource.get())->disconnect();
        }
    } else if (httpSource != NULL) {
        static_cast<HTTPBase *>(httpSource.get())->disconnect();
    }
}

5 节目信息相关

getTrackCount、getTrackInfo、selectTrack和getSelectedTrack

这几个函数都是跟节目选择有关的，getTrackCount返回当前Source中包含的Track数目（一个Track可以是音频、视频、字幕或者文本），getTrackInfo则返回对应索引的详细信息。getSelectedTrack则返回当前选择或者正在播放的Track信息。selectTrack则用于选定特定的读取Track，也可用于取消读取。

getFormat

这个接口用于获取音频或者视频格式，实现如下：

sp<AMessage> NuPlayer::Source::getFormat(bool audio) {
    sp<MetaData> meta = getFormatMeta(audio);

    if (meta == NULL) {
        return NULL;
    }

    sp<AMessage> msg = new AMessage;

    if(convertMetaDataToMessage(meta, &msg) == OK) {
        return msg;
    }
    return NULL;
}

也就是说可以看看getFormatMeta的实现逻辑，代码如下：

sp<MetaData> NuPlayer::GenericSource::getFormatMeta(bool audio) {
    sp<AMessage> msg = new AMessage(kWhatGetFormat, this);
    msg->setInt32("audio", audio);

    sp<AMessage> response;
    sp<RefBase> format;
    status_t err = msg->postAndAwaitResponse(&response);
    if (err == OK && response != NULL) {
        CHECK(response->findObject("format", &format));
        return static_cast<MetaData*>(format.get());
    } else {
        return NULL;
    }
}

主要是发送kWhatGetFormat消息，然后交给DataSource处理。

6 辅助信息获取及设置

这里面几个函数都比较简单，多数信息都是在prepareAsync函数中读取的。这里仅列出代码：

getDuration

status_t NuPlayer::GenericSource::getDuration(int64_t *durationUs) {
    *durationUs = mDurationUs;
    return OK;
}

isRealTime、isStreaming

virtual bool isRealTime() const {
    return false;
}

bool NuPlayer::GenericSource::isStreaming() const {
    return mIsStreaming;
}

setOffloadAudio/setBuffers

void NuPlayer::GenericSource::setOffloadAudio(bool offload) {
    mBufferingMonitor->setOffloadAudio(offload);
}

status_t NuPlayer::GenericSource::setBuffers(
        bool audio, Vector<MediaBuffer *> &buffers) {
    if (mIsSecure && !audio && mVideoTrack.mSource != NULL) {
        return mVideoTrack.mSource->setBuffers(buffers);
    }
    return INVALID_OPERATION;
}

feedMoreTSData

// 用于测试是否处理完所有数据
status_t NuPlayer::GenericSource::feedMoreTSData() {
    return OK;
}

7 小结

断断续续的把本文整理完成，算是基本整清楚了针对文件读取的解析过程和调用逻辑。本文代码居多，有点乱。不建议参考，如果有问题，还是直接阅读代码吧。