CaptureManagerSDK - 秒客网

Simple SDK for capturing, recording and streaming video and audio from web-cams on Windows OS by Windows Media Foundation.

CaptureManager SDK Versions:

The Commercial version:

CaptureManager SDK Demo programs

Capture Manager Topology Editor - Capture Manager Store

Updates:

Introduction

This article presents my project for capturing of video and audio sources on Windows OS by Microsoft Media Foundation.

I have spent much time on resolving of different tasks with processing of video and audio data, and researched techniques for capturing of video and audio on Windows family OSs. DirectShow was the main technique for capturing live-video on Windows family OSs for long time. However, since Vista OS Microsoft has introduced a new framework for working with video and audio - Microsoft Media Foundation, and since Windows7 OS it has features for working with web-cam via USB Video Class driver and line audio input. I included that technology into many projects and wrote two articles about it: Capturing Live-video from Web-camera on Windows 7 and Windows 8 and Capturing Video from Web-camera on Windows 7 and 8 by using Media Foundation. Those articles present simple libraries for capturing web-cams and grabbing of image frames, but Microsoft Media Foundation is much more powerful framework. After some thinking, I have decided to write a new solution for working with live-video and audio which can be more flexible than my previous solutions, and can take much more power (or force :)) of Microsoft Media Foundation. The solution, which can compete with Microsoft solution. The solution, which can be called the best anyone have seen before by any developer.

Background

I had got an idea to write a new solution for working with web-cams on basement of Microsoft Media Foundation while faced with one unusual task. So, the task was not resolved, but I had wrote some code and had decided to continue development of the solution.

At beginning the solution included only few classes and allowed to execute only few functions, but after adding of some demands for this solution I had decided to write a simple SDK which allows to make capture configuration for the new tasks easy and to inject a new developed code into it by implementation of the Microsoft Media Foundation's and CaptureManager's interfaces.

As a result, I have got this SDK for capturing, recording and steaming of live-video and audio from web-cams only by Microsoft Media Foundation.

Using the code

CaptureManager SDK Demo programs

OpenGLWebCamViewerViaCOMServer

This demo program presents only simple configuration of the web-cam capturing and viewing via OpenGL rendering.

For using of Capture Manager it needs call the suitable interfaces from CaptureManager COM server. For setting of print output log destination it needs to get ILogPrintOutControl interface:

Hide Copy Code

// get COM log interface

CComPtrCustom<iclassfactory> lCoLogPrintOut;

HRESULT lhresult = CoGetClassObject(CLSID_CoLogPrintOut, CLSCTX_INPROC_SERVER, nullptr, IID_PPV_ARGS(&lCoLogPrintOut));

if (FAILED(lhresult))

    return lhresult;

CComPtrCustom<ilogprintoutcontrol> lLogPrintOutControl;

lCoLogPrintOut->LockServer(true);

lhresult = lCoLogPrintOut->CreateInstance(

    nullptr,

    IID_PPV_ARGS(&lLogPrintOutControl));

if (FAILED(lhresult))

    return lhresult;

// set log file for info

lhresult = lLogPrintOutControl->addPrintOutDestination(

    (DWORD)INFO_LEVEL,

    L"Log.txt");

if (FAILED(lhresult))

    return lhresult;

For getting main methods of the CaptureManager it needs get ICaptureManagerControl interface:

Hide Copy Code

CComPtrCustom<IClassFactory> lCoCaptureManager;

lhresult = CoGetClassObject(CLSID_CoCaptureManager, CLSCTX_INPROC_SERVER, nullptr, IID_PPV_ARGS(&lCoCaptureManager));

if (FAILED(lhresult))

    return lhresult;

lCoCaptureManager->LockServer(true);

// get ICaptureManagerControl interfrace

CComPtrCustom<ICaptureManagerControl> lCaptureManagerControl;

lhresult = lCoCaptureManager->CreateInstance(

    nullptr,

    IID_PPV_ARGS(&lCaptureManagerControl));

if (FAILED(lhresult))

    return lhresult;

The topology of this demo program is presented in the next schema:

CaptureManagerSDK

This demo program can be gotten by link: Download OpenGLWebCamViewerViaCOMServer.zip

EVRWebCapViewerViaCOMServer

This demo program presents the way of using of Enhanced Video Renderer for rendering of live video via CaptureManager COM server.

This code allows easy integrate this CaptureManager SDK into already created Windows application - it needs only handler on element of GUI which is used as web-cam display. This code controls resizing of GUI element and modify renderer for changing of video size and proportion.

This demo program can be gotten by link: Download EVRWebCapViewerViaCOMServer.zip

CaptureManagerToCSharpProxy

This demo program presents the way of linking CaptureManager.dll with C# project. In this project CaptureManagerSDK is wrapped by C# class CaptureManager. This class hides direct working with COM interface and marshaling of image data from unmanaged code. For flexibility of solutions the class upload CaptureManager.dll directly into the process and get COM objects WITHOUT of calling of COM's infrastructure - it allows to use CaptureManager COM server WITHOUT any registration in system.

This demo program can be gotten by link: Download CaptureManagerToCSharpProxy.zip

WPFSourceInfoViewer

This demo program presents the way of collecting information from sources and presenting it in the readable form. Information from the sources is presented in form of XML document. There are some reasons for using of XML format for transfering infofmation from COM server:

1. simple structure - Microsoft Media Foundation uses huge amount of GUID constant and types which need to know for understanding of the info, but in this solution all of them presented into the friendly form;

2. easy transfare from COM server;

3. easy parsing - XPath expression allows to get almost any needed information and present it, it is more easy than working with static defined classes and lists of info;

4. In the most cases, user needs to read which features are supported by source and select one of it;

5. XML document is easy integrated source into code of different presentation models (for instance - MVVM).

CaptureManagerSDK

This demo program can be gotten by link: Download WPFSourceInfoViewer.zip

WPFWebViewerCall

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by calling of video frames from WPF thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code launchs grabbing of video frames via ISampleGrabber interface and works in sync mode.

CaptureManagerSDK

This demo program can be gotten by link: Download WPFWebViewerCall.zip

WPFWebViewerCallback

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by calling of video frames from CaptureManager thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code registers update method into the class with ISampleGrabberCallback interface and inject this class into COM Server. While CaptureManager gets new video frame it calls WPF update method and post message to update of frame. In this case, the thread of WPF is not overloaded by grabbing task.

CaptureManagerSDK

This demo program can be gotten by link: Download WPFWebViewerCallback.zip

WPFWebViewerEVR

This demo program presents the way viewing live video from web cam via COM Sever on C# - WPF by using of Enhanced Video Renderer from CaptureManager thread. This code gets XML document from CaptureManager and pares it for getting some information: name of device, type of data stream and list of supported resolutions. The code gets EVR node from CaptureManager by setting of HWND of the integrated WindowForms panel. In this case, all working with video frames is executed behind of WPF thread.

CaptureManagerSDK

This demo program can be gotten by link: Download WPFWebViewerEVR.zip

WPFRecorder

This demo program presents the way of working with CaptureManager SDK for capturing, encodring and recording / broadcasting video - audio from web cameras, microphones, desktop screens and speakers via COM Sever on C# - WPF.

CaptureManagerSDK

Code of this demo program presents the correct algorithm of working with CaptureManager SDK via COM Server interfaces. The final point of data stream can be file sink or byte stream sink. The last one is more interesting opportunity then simple saving into the file. In this demo program byte stream sink is implemented in form of TCP Server with Loopback address and http port: 8080. The simular implementation can be find in demo proram on C++, but this one has one significant difference - in this demo program the TCP Server is implemented on C#-dotNet framework. CaptureManager SDK is written on C++ and uses Windows Media Foundation on C, however, there is a good news - CaptureManager SDK needs implementation Windows Media Foundation interface IMFByteStream for streaming of bytes. Implementation C interface IMFByteStream on C# does not need any specific Windows Media Foundation function - it needs defining interfaces: IMFByteStream, IMFAsyncCallback, IMFAsyncResult; and enum constants:

MFAsyncCallbackQueue,
MFASync,
MFByteStreamSeekingFlags,
MFByteStreamSeekOrigin,
MFByteStreamCapabilities,
UnmanagedNameAttribute.

Implementation of those interfaces can be found in file NetworkStreamSink.cs, but I would like to point your attention on the next moments:

1. in IMFByteStream interface there are four implemented methods: GetCapabilities, Write, BeginWrite, EndWrite. In the first one the code sets type of implementation of IMFByteStream interface - writteable, but not seekable. Method Write is used for sync writing data into the stream, while methods BeginWrite, EndWrite are used for async writing. However, there are some important moments: method Write is called once ant the start - it writes the head of medta streams: type of encoder, amount of streams, names of streams and other metadata. Async writing needs to execute the methods in the next order: BeginWrite, argument IMFAsyncCallback pCallback.Invoke, EndWrite, but Calling of methods BeginWrite, EndWrite can be locked by the same mutex. It means that argument IMFAsyncCallback pCallback.Invoke must be executed in the separated thread - for example by ThreadPool.QueueUserWorkItem.

2. In the implementation of TCP Server I have used async calling BeginAcceptTcpClient, and writing head data at the start of each connection - it allows to connect any mount of client to the media stream server.

Hide Shrink CaptureManagerSDK

Copy Code

public void Start()

{

try

{

tcpListener = new TcpListener(Configuration.IPAddress, Configuration.Port);

tcpListener.Start();

tcpListener.BeginAcceptTcpClient(

new AsyncCallback(callBack),

tcpListener);

}

catch (Exception e)

{

}

}

/// <summary>

/// Stops the WebServer thread

/// </summary>

public void Stop()

{

try

{

tcpListener.Stop();

foreach (var item in mClientBag)

{

item.Value.Client.Close();

item.Value.Client.Dispose();

item.Value.Close();

}

tcpListener.Server.Dispose();

}

catch (Exception e)

{

}

}

private void callBack(IAsyncResult aIAsyncResult)

{

TcpListener ltcpListener = (TcpListener)aIAsyncResult.AsyncState;

if (ltcpListener == null)

return;

TcpClient lclient = null;

try

{

lclient = ltcpListener.EndAcceptTcpClient(aIAsyncResult);

}

catch (Exception exc)

{

return;

}

if (lclient != null && lclient.Client.Connected)

{

StreamReader streamReader = new StreamReader(lclient.GetStream());

// Read full request with client header

StringBuilder receivedData = new StringBuilder();

while (streamReader.Peek() > -1)

receivedData.Append(streamReader.ReadLine());

string request = GetRequest(receivedData.ToString());

if (!SuportedMethod(request))

{

SendError(StatusCode.BadRequest, "Only GET is supported.", lclient);

lclient.Client.Close();

lclient.Close();

}

else

{

Socket socket = lclient.Client;

if (socket.Connected)

{

SendHeader(StatusCode.OK, lclient);

lock (this)

{

if (mHeaderMemory != null)

{

int sentBytes = socket.Send(mHeaderMemory);

}

mClientBag[lclient] = lclient;

}

}

}

}

ltcpListener.BeginAcceptTcpClient(

new AsyncCallback(callBack),

ltcpListener);

}

3. The head includes MIME type of byte stream, which allows to use in the future release the same solution for any type of media container - ASF, MP4, MKV.

Hide Copy Code

private void SendHeader(string mimeType, long totalBytes, StatusCode statusCode, TcpClient aTcpClient)

{

StringBuilder header = new StringBuilder();

header.Append(string.Format("HTTP/1.1 {0}\r\n", GetStatusCode(statusCode)));

header.Append(string.Format("Content-Type: {0}\r\n", mimeType));

header.Append(string.Format("Accept-Ranges: bytes\r\n"));

header.Append(string.Format("Server: {0}\r\n", Configuration.ServerName));

header.Append(string.Format("Connection: close\r\n"));

header.Append(string.Format("Content-Length: {0}\r\n", totalBytes));

header.Append("\r\n");

SendToClient(header.ToString(), aTcpClient);

}

This demo program can be gotten by link: Download WPFRecording.zip

QtMinGWDemo

This demo program presents the way of working with CaptureManager SDK for capturing, encoding and recording / broadcasting live video and audio from web cameras, microphones, desktop screens and speakers via COM Sever on Qt framework. On Windows OS there is a version of Qt for Visual Studio Compiler, but this demo presents using of CaptureManager SDK with Qt version for MinGW Compiler. Of course, this demo can be recompiled for Visual Studio Compiler, but version for MinGW Compiler shows flexibility of this SDK and compaliblness with many other compilers. This demo include code for capturing live video from web cameras and viewing by calling of samples, by callback of view update code from Capture Manager SDK inner thread or by direct drawing image via HWND of widget.

CaptureManagerSDK

Another example in this demo presents way for connection of sources, encoders and sinks into the one pipe line of processing. This demo presents way for recording video and audio into the file and real code for writing of network stream broadcasting into the Internet code by Qt framework QTcpServer and QTcpSocket classes.

CaptureManagerSDK

This demo program can be gotten by link: Download QtMinGWDemo.

CaptureManagerSDKWindowsStoreDemo

This demo program presents the way of working with CaptureManager SDK for capturing of live video from web cameras in Windows Store application. CaptureManager is built with using of Microsoft Media Foundation on Desktop Windows OSs and as any Desktop solution it cannot be migrated in Windows Store application in easy way. However, after some research I have found some solutions which can resolve this problem.

CaptureManagerSDK

Code of this demo program is not difficult for understanding, but I would like to point your attention on two important moments:

1. This solution is created for using in Desktop Windows Store applications which is on top of Desktop Windows OSs. It allows use CaptureManager SDK COM-DLL and work with this SDK in the same way like in regular Desktop application. The way of connecting of the CaptureManager SDK COM-DLL I found in the next article: Using Free-COM DLL in Windows Store C++ Project. This way allows load CaptureManager SDK COM-DLL into the process of Windows Store application by the correct way and calls CaptureManager SDK interfaces.

2. Working with resources in Windows Store application needs defining of capabilities for application. The same rules there are for the web cameras and microphones. However, Microsoft Media Foundation does not support such functionality. I have resolved this problem by the next code:

Hide Shrink CaptureManagerSDK

Copy Code

auto lFindAllTask = create_task(DeviceInformation::FindAllAsync(Windows::Devices::Enumeration::DeviceClass::VideoCapture));

lFindAllTask.then([this](DeviceInformationCollection^ op) {

    String^ lIdString;

    for (int lIndex = 0;

        lIndex < op->Size;

        lIndex++)

    {

        auto litem = op->GetAt(lIndex);

        auto lname = litem->Name;

        auto kk = litem->Properties;

        lIdString = litem->Id;

        lDeviceCollection->Insert(lIdString, lname);

    }

    for (auto& litem : lDeviceCollection)

    {

        auto mediaCapture = ref new MediaCapture();

        auto settings = ref new MediaCaptureInitializationSettings();

        settings->VideoDeviceId = litem->Key;

        auto lSymbolicLink = litem->Key;

        auto lInitializeTask = mediaCapture->InitializeAsync(settings);

        lInitializeTask->Completed = ref new AsyncActionCompletedHandler([this, lSymbolicLink](

            IAsyncAction^ asyncInfo,

            AsyncStatus asyncStatus)

        {

            if (asyncStatus == AsyncStatus::Completed)

            {

                lAccessableDeviceCollection->Insert(lSymbolicLink, lDeviceCollection->Lookup(lSymbolicLink));

            }

            else

            {

            }

            if (++lDeviceCount == lDeviceCollection->Size)

            {

                Dispatcher->RunAsync(CoreDispatcherPriority::Normal,

                    ref new DispatchedHandler([this]() {

                    auto lresult = CaptureManagerProxy::getInstance().isInitialized();

                    if (lresult)

                    {

                        fillSourceComboBox();

                        fillRendererComboBox();

                    }

                    else

                    {

                    }

                }));

            }

        });

    }

});

In this code I found all compatible video devices:

Hide Copy Code

auto lFindAllTask = create_task(DeviceInformation::FindAllAsync(Windows::Devices::Enumeration::DeviceClass::VideoCapture));

and initialize all of them:

Hide Copy Code

auto lInitializeTask = mediaCapture->InitializeAsync(settings);

It allows set for the process of Windows Store application the needed rights for working with these devices and then it does not mater which part of process will get access for the video cameras - API of Windows Runtime apps or Media Foundation from CaptureManager and low level Windows API.

This demo program can be gotten by link: Download CaptureManagerSDKWindowsStoreDemo

CaptureManagerSDKPythonDemo

This demo program presents the way of working with CaptureManager SDK for capturing of live video from web cameras in Python application on Windows OS. CaptureManager is built with using of Microsoft Media Foundation on Desktop Windows OSs and COM technology. As any COM server CaptureManager can be intergated into the project via direct calling interfaces or type library. However, dynamic type programming languages have some problems with correct using type libraries. For resolving such problems I have included IDispatch and wrote implementation for many classes, but some projects need working with pointers on massive of memory and it could be a problem with dynamic types. For simplifing solution I have implemented IDispatch interface only for limited functionality. This demo program presents functionality for selections of sources, encoders and rendering video via HWND and saving into the file,

CaptureManagerSDK

This demo program can be gotten by link: Download CaptureManagerSDKPythonDemo

WPFScreenStreamer

This demo program presents the way of working with CaptureManager SDK for streaming live video of desktop screen and audio loop back of speaker's audio stream. This implementation is founded on MP4 media container. The modern web browsers support playing media content of format MP4 with video encoding by H264 and audio encoding by AAC and MP3. Since presenting Windows 8, Microsoft Media Foundation allows create MP4 media container which can be transmitted via LAN in format which compalible with HTML5. Such unique opportunity is implemented in this demo in the form of the simplest screen streaming web server. After launching of demo program it ready to connect with the client browser.

CaptureManagerSDK

Working of this demo can be tested via HTML5 page. For example like this:

Hide Copy Code

<html>

<video controls><source src="http://localhost:8080/anyfile.mp4" type="video/mp4" /> I'm sorry; your browser doesn't support HTML5 video in MP4 with H.264.</video></html>

This demo program can be gotten by link: Download ScreenStreaming_Bin.zip and Download WPFScreenStreaming.zip

For preventing of questions about licensing for using of the format MP4 I can say that CaptureManager SDK DOES NOT INCLUDE code for working with MP4 container. It DOES NOT read or write MP4 container. All these actions are executed by modules of Windows OS which is written by Microsoft. As a result, all licensed royalty for using of MP4 is already paid by Microsoft and users of Windows OSs have already paid for using those modules as part of Windows OSs. So, In my view, It allows to use Microsoft Media Foundation implementation MP4 writer in commercial projects without any problems.

WPFWebCamShot

This demo program presents the way of working with CaptureManager SDK for capturing single frame from video stream. CaptureManager SDK includes new mode for SampleGrabberCallSinkFactory - PULL.

Hide Shrink CaptureManagerSDK

Copy Code

This mode is differ from SYNC and ASYNC modes. SYNC and ASYNC modes work in continues mode - they send request for next sample after receiving of the current one in automatic mode without blocking. It leads queue of requests and samples like on image:

CaptureManagerSDK

This type is useful for presenting of video, but the task of capturing the single frame has faced with the next dificulty - 99 percents of frames are captured, decoded and delivered to the sink, but customer's code does not take it from sink. This task can be in reality - for example, taking single frame each one second while camera produces 30 frames per second - it this case CPU wastes time and power on 29 frames which will throw out into the garbge. For such task new mode - PULL can be more useful. The sink sends request only while customer's code calls for the new single frame:

CaptureManagerSDK

It allows utilize CPU power more effective only for one task. The new mode can be useful also for many projects of image recognition - such projects usually process frames with frame rate low than it can be produced by cameras, but video pipeline will spend much CPU power on frames, which will not be processed. PULL mode allows release some CPU power from video pipeline for other parts of program.

This demo program can be gotten by link: Download WPFWebCamShot.zip

WPFWebCamSerialShots

This demo program presents the way of working with CaptureManager SDK for capturing the last serial frames from video stream. CaptureManager SDK 1.2.0 includes new type of stream control node - sample accumulator node:

Hide Shrink CaptureManagerSDK

Copy Code

It includes two sample accumulator nodes on 5 and 10 samples which collect/accumulate 5 or 10 LAST SAMPLES. Idea is that in the real world taking of photo does not executed in the moment of viewing of event. Human reaction on event, process of pressing on take photo button, sending event from hardware layer into layer of customer's code and requesting of the single frame take some time - from 500 ms til 1 second. It leads to loose of important moment which was the reason of taking of photo. Sample accumulator nodes can compensate such time delay by accumulating the last samples in video stream. These samples can be gotten by PULL mode of SampleGrabberCallSinkFactory. It sends the request:

CaptureManagerSDK

and receives the last samples:

CaptureManagerSDK

It can be useful for resolving of some problems, but accumulation of samples needs memory buffer which can be expanded fast - for example for keeping of 10 last samples of RGB32 image of 1080p format it needs about 60 MByte - one second accumulation in such video can need about 200 MByte or more. Of course, it is possible create sample accumulator with different schema - for example one by third - it means keeping only each third sample from video stream.

This demo program can be gotten by link: Download WPFWebCamSerialShots.zip

CaptureManagerToWinRTProxy

This demo program presents the way of linking CaptureManager.dll with WindowsStore project on x86_64 platform. In this project CaptureManagerSDK is wrapped by C++/CX class CaptureManager. This class hides direct working with COM interface and marshaling of image data from unmanaged code. For flexibility of solutions the class upload CaptureManager.dll directly into the process and get COM objects WITHOUT of calling of COM's infrastructure - it allows to use CaptureManager COM server WITHOUT any registration in system.

This demo program can be gotten by link: Download CaptureManagerToWinRTProxy.zip

WindowsStoreCam

This demo program presents the way of working with CaptureManager SDK in WindowsStore project on x86_64 platform. This demo program implements wide functionality of Microsoft Media Foundation via calling of COM server of CaptureManager. It allows write code for working with camera which can be used in Windows OSs Desktop applications and WindowsStore x86_64 platform application without rewriting of code.

This demo program can be gotten by link: Download WindowsStoreCam.zip

CaptureManager SDK Versions:

Version 1.0.0

Capture Manager SDK v1.0.0 – open release. I include interfaces with the stable definition:

via class CoLogPrintOut – interface ILogPrintOut

via class CoCaptureManager – interface ICaptureManagerControl:

method createControl – create the main control objects with interfaces:

ISourceControl,
ISinkColtrol,
IEncoderControl,
ISessionControl,
IStreamControl;

method createMisc – create a miscellaneous object with interfaces:

IMediaTypeParser,
IStrideForBitmap,
IVersionControl;

via object with interface ISourceControl:

IWebCamControl;

via object with interface ISinkControl:

IFileSinkFactory,
ISampleGrabberCallSinkFactory,
IEVRSinkFactory,
ISampleGrabberCallbackSinkFactory,
IByteStreamSinkFactory;

via object with interface ISession:

ISessionCallback;

via object with interface ISampleGrabberCallSinkFactory:

ISampleGrabberCall;

via object with interface ISampleGrabberCallbackSinkFactory:

ISampleGrabberCallback;

These interfaces do the next work:

ILogPrintOut – it manages writing of log information;
ICaptureManagerControl – it manages creating of all capturing controls and miscellaneous objects;
ISourceControl – it manages and creates sources of video and audio signals;
ISinkColtrol – it manages and creates sinks of video and audio streams;
IEncoderControl – it manages and creates video and audio encoders;
ISessionControl – it creates object for management of recording session;
IStreamControl – it creates object for controlling of media streams;
IMediaTypeParser – it creates text representation of media type in XML format;
IStrideForBitmap – it computes stride of memory for the specific bitmap format;
IVersionControl – it manages information about current version of CaptureManager;
IWebCamControl – it manages options of web camera;
IFileSinkFactory – it creates media sink nodes which are linked with media file;
ISampleGrabberCallSinkFactory – it creates media sink for grabbing of one sample by direct calling object with interface ISessionCallback;
ISampleGrabberCall – it manages grabbing of one sample;
ISampleGrabberCallbackSinkFactory – it creates media sink for grabbing of one sample by calling object with interface ISampleGrabberCallback from CaptureManager inner thread;
ISampleGrabberCallback – it manages grabbing of one sample from CaptureManager inner thread;
IEVRSinkFactory – it creates media sink for rendering of video stream;
IByteStreamSinkFactory – it creates media sink nodes which are linked with customised byte stream object with interface IMFByteStream;
ISession – it manages the recording session;
ISessionCallback – it manages the result state of current recording session from CaptureManager inner thread;

The definitions of these interfaces are presented in SDK files and in CaptureManagerToCSharpProxy project. For effective marshalling information via border of unmanaged-managed code CaptureManager uses eight simple XML documents with the next structures:

Hide Copy Code

<!--?xml version="1.0" ?-->

<!--XML Document of sources-->

These documents contain information about sources, encoders, sinks, stream controls. Most types and GUID is taken from Microsoft Media Foundation MSDN: Microsoft Media Foundation, but there are some rules for definition of nodes in these XML documents: SingleValue - node for presenting only one value - name, integer, type; RatioValue - node for presenting of value in float point format with Numerator and Denominator; Value.ValueParts - node for storing collection of ValuePart nodes with the same type. Examples of parsing of these XML documents can be found in code of WPF examples.

Capture Manager SDK can be downloaded by link: Download CaptureManagerSDK.zip

Version 1.1.0 with HEVC(H265) encoder

Capture Manager SDK v1.1.0 – open release. This release includes the next changings:

Add supporting of automating registration CaptureManager as COM server with Type Library
Add supporting of dynamic language Python 2.7
Add supporting of HEVC(H265) encoder in Windows 10: Download HEVCEncoder_Windows10

Capture Manager SDK v1.1.0 can be downloaded by link: Download CaptureManagerSDK-110.zip

Version 1.2.0 Beta

Capture Manager SDK v1.2.0 – beta release. This release includes the next changings:

Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
Lazy binding of Microsoft Media Foundation functions.
Replace web camera properties functionality from DirectShow on DeviceIoControl.
Add resizing in EVR.

Lazy binding of functions is implemented by loading of Microsoft Media Foundation libraries at runtime. Functions which cannot be found in Microsoft Media Foundation libraries are replaced by stub functions. It allows prevent crushing of applications by using of unsupported functions of Microsoft Media Foundation libraries.

Using of DeviceIoControl for changing of web camera properties allows resolve problem with delay of web camera driver initialization. It allows expand workable set of web camera properties by : "Amount of digital zoom", "Upper limit for the amount of digital zoom", "Power line frequency".

Resizing in EVR resolves problem with unchanging image size of rendered video. The new implementation controls current image size of renderer GUI and changes image size and position for keeping proportion of video.

Capture Manager SDK v1.2.0 – beta release can be downloaded by link: Download CaptureManagerSDK-1.2.0_beta.zip

Version 1.2.0

Capture Manager SDK v1.2.0 – release. This release includes the next changings:

Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
Lazy binding of Microsoft Media Foundation functions.
Replace web camera properties functionality from DirectShow on DeviceIoControl.
Add resizing in EVR.
Add PULL mode in SampleGrabberCallSinkFactory - the new mode which allows to take a single sample.
Add sample accumulator nodes for storing of 5 or 10 last samples in media stream.

Capture Manager SDK v1.2.0 – release can be downloaded by link: Download CaptureManagerSDK-1.2.0-Freeware.zip

Version 1.3.0 Beta with DirectShow Crossbar capture

Capture Manager SDK v1.3.0 – beta. This release includes the next changings:

Add supporting of video capturing via DirectShow Crossbar technique for the next inputs: Composite, SVideo, USB, 1394 (FireWire).

On MSDN - Audio/Video Capture in Media Foundation you can find that "Video capture devices are supported through the UVC class driver and must be compatible with UVC 1.1". It means that Microsoft Media Foundation can capture video only from devices with supporting of UVC driver - USB Video Class driver. It is usually USB web cameras - other types of video capture devices are not supported. From my experience I can say that the target platform of Microsoft Media Foundation is WindowsStore applications. WindowsStore applications are originally targeted on portable devices like "Pad" - "Surface" - such devices DO NOT have external ports for connecting of capturing cards. They can work only with embedded web cameras which are connected via inner USB ports. I think that it is a correct idea, but I have got many questions about using of CaptureManager with capture cards which support DirectShow Crossbar and answered that it is impossible. However, it is not full answer - the full answer is that it is possible see capture card which supports DirectShow Crossbar via Microsoft Media Foundation functionality, but captured video is not accessable. After some research I found the way to get access for such capture cards. Info for working with capture card can be selected from XML string of sources via checking of attributes nodes of Source with name 'MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY': "Source.Attributes/Attribute[@Name='MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_CATEGORY']

/SingleValue[@Value='CLSID_VideoInputDeviceCategory']" - Microsoft Media Foundation define CLSID_WebcamInterfaceDeviceCategory - {E5323777-F976-4F5B-9B55-B94699C46E44} for device with UVC supporting and CLSID_VideoInputDeviceCategory - {860BB310-5D01-11D0-BD3B-00A0C911CE86} for DirectShow capture devices. It allows easy select only DirectShow video capture devices. Input connection of DirectShow according to physical type can be selected by chosing of the suitable media type - each media type node has child MediaTypeItem node with name DirectShowPhysicalType - {09FCAE6B-8F00-4C0A-BBCA-C6820B576C99}. It allows to group the meda types according physical type of input connection.

Hide Copy Code

<MediaTypeItem Name="DirectShowPhysicalType" GUID="{09FCAE6B-8F00-4C0A-BBCA-C6820B576C99}" Title="Physical type of input pin of Crossbar." Description="Physical type of input pin of Crossbar.">

  <SingleValue Value="PhysConn_Video_Composite" />

  </MediaTypeItem>

Capture Manager SDK v1.3.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware-beta.zip

Capture Manager Topology Editor

This example of implementation of CaptureManager SDK can be found by link: Capture Manager Store. It presents simple program for capturin live video and audio from web cameras via building simple graph by "Visual Programming" way. This solution has name - Capture Manager Topology Editor.

CaptureManagerSDK

The commercial versions:

Version 1.0.0

In addition for version of SDK under CodeProject policy, there is a commercial version with additional features. The commercial version is based on the same COM interfaces as freeware CaptureManager SDK, but includes additional encoders and media containers. It includes video encoder:

Microsoft H264;

and audio encoders:

Microsoft MP3,
Microsoft MPEG2 Audio,
Microsoft AAC,
Microsoft DolbyDigital;

File container includes:

AVI format.

All these features are base on the libraries which already exist in core of Windows 7 and Windows 8. It allows develop simple solution with supporting of the most popular formats.

CaptureManagerSDK

However, there are two disadvantages of using of these features:

poor campatibility Microsoft implementation of encoders with some decoders. Testing on Linux "Mint" showed that only combinations H264 with MP3 or MPEG 2 Audio have good campatibility. With the other codecs there are some problems;
poor campatibility of AVI container with some live video sources. The problem is that many web cameras have unstable frame rate video stream. At the moment of changing the brightness of scene many web cameras change inner options of frame capture and it leads to increase of the duration of frames. However, in AVI container the frame duration is computed on base of general frame per second rate of video stream and duration of presenting each frame is defined as constant. As a result, it is possible to view the problem with sync recording video and audio.

Of course, using of encoders and file container which are campatibile with other OSs is useful (especially AVI container, because it is possible to write raw video data of YVY2 or RGB24 into the file and easy extracts seperated frames without decoding with using of very simple code), but it needs to pay attention on some limitations.

Thr features of commercial version can be tested on commercial trial version which is limitated only by 30 seconds duration of capturing session.

Version 1.1.0 with MP4 streaming

Microsoft H264;
Microsoft HEVC(H264);

and audio encoders:

Microsoft MP3,
Microsoft MPEG2 Audio,
Microsoft AAC,
Microsoft DolbyDigital;

File container includes:

AVI format,
MP4 format

Webcamera properties are extended:

Power line frequency

All these features are base on the libraries which already exist in core of Windows 7, Windows 8 and Windows 10. It allows develop simple solution with supporting of the most popular formats.

This version supports using MP4 writer of Windows 8 and Windows 10 for writing of MP4 files and streaming. This implementation supports writing only one video stream and one audio stream, but it can be enough for most purposes - Download ScreenStreaming_Bin.zip, WPFScreenStreamer.

The features of commercial version can be tested on CaptureManagerSDK-110_trial.zip which is limitated only by 60 second duration of capturing session.

Capture Manager Store: Application License - $119.99

Version 1.2.0

Microsoft H264;
Microsoft HEVC(H265);

and audio encoders:

Microsoft MP3,
Microsoft MPEG2 Audio,
Microsoft AAC,
Microsoft DolbyDigital;

File container includes:

AVI format,
MP4 format

Webcamera properties are extended:

Power line frequency.
Amount of digital zoom.
Upper limit for the amount of digital zoom.

This release includes the next changings:

Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
Lazy binding of Microsoft Media Foundation functions.
Replace web camera properties functionality from DirectShow on DeviceIoControl.
Add resizing in EVR.
Add PULL mode in SampleGrabberCallSinkFactory - the new mode which allows to take a single sample.
Add sample accumulator nodes for storing of 5 or 10 last samples in media stream.

All these features are base on the libraries which already exist in core of Windows 7, Windows 8 and Windows 10. It allows develop simple solution with supporting of the most popular formats.

The features of commercial version can be tested on Download CaptureManagerSDK-1.2.0-Commercial-Trial.zip which is limitated only by 60 second duration of capturing session.

Capture Manager Store: Application License - $119.99

Points of Interest

Previously I wrote that the was one unusual task which was the reason to start of development of CaptureManager. The task included recording live-video of industrial process from two sources and recording of data from two sensors. I have started to write a new solution for Microsoft Media Foundation and thought to use Microsoft ASF format due to existence of MFMediaType_Binary Major Type. However, after some time I found that implementation of Microsoft ASF format for Microsoft Media Foundation for recording supports only MFMediaType_Video, MFMediaType_Audio, and MFMediaType_Script. It was the main reason for stopping to resolve that task.

Updates:

The first update on 21/08/2015:

improved quality of the AudioLoopback capture;

resolved problem with syncronization of the Screen Capture Source - GDIScreenCapture and AudioLoopback;

added new Media Types for Screen Capture Source - GDIScreenCapture with the next frame rates: 20 fps, 25 fps and 30 fps;

added into Screen Capture Source - GDIScreenCapture supporting for capturing of cursor and drawing it into the captured video.

The second update on 31/08/2015:

fixed loosing of samples in the AudioLoopback capture;

added new Media Types for Screen Capture Source - GDIScreenCapture with the next frame rates: 1 fps, 5 fps and 10 fps;

added supporting of WM Speech Encoder DMO;

added supporting of WMVideo9 Screen Encoder MFT - FOURCC "MSS2";

added version of CaptureManager SDK for x64 Windows OS.

The third update on 14/09/2015:

added new Screen Capture Source - DirectX9ScreenCapture with the next frame rates: 1 fps, 5 fps, 10 fps, 15fps, 20 fps, 25 fps, 30 fps;

added library CaptureManagerProxy.dll for calling CaptureManagerSDK descriptors in "C" style;

added supporting of 3rdParty MediaSink solutions - RecordInto3rdPartyMediaSink;

added demo program for capturing of screen in DirectX9 video games - ScreenCaptureWithDirect3D9APIHooks;

The fourth update on 30/09/2015:

added supporting of Enhanced Video Renderer - the demo program: EnhancedVideoRendererWebCapViewer

The fifth update on 13/11/2015:

added limited supporting of COM Server;

added CaptureManager.tlb;

added demo programs for presenting of working with CaptureManagerSDK via COM on C#-WPF:

CaptureManagerToCSharpProxy,

WPFSourceInfoViewer,

WPFWebViewerCall,

WPFWebViewerCallback,

WPFWebViewerEVR

The sixth update on 07/03/2016:

first stable release 1.0.0;

implemented supporting of all SDK functionality in COM Server;

wrote new COM Server interface;

stopped development of C interface;

replace GDIScreenCapture and DirectX9ScreenCapture on the Screen Capture;

included full support of play, pause, stop and close functinality;

updated code of demo programs for presenting of working with CaptureManagerSDK via COM on C#-WPF:

CaptureManagerToCSharpProxy,

WPFSourceInfoViewer,

WPFWebViewerCall,

WPFWebViewerCallback,

WPFWebViewerEVR;

added new demo program for demonstration of functinality of recording and network streaming media content:

WPFRecorder;

added freeware software for creating of the suitable video and audio capture configuration for recording and network broadcatsing - Capture Manager Topology Editor - Capture Manager Store;

developed commercial version of CaptureManager SDK Capture Manager Store: with supporting the next encoders:

H264,

MP3,

MPEG2 Audio,

AAC,

DolbyDigital;

added supporing of AVI container into the commercial version of CaptureManager SDK;

added commercial trial version of CaptureManager SDK with 30 second duration of capture session - Download Commercial_trial.zip.

The seventh update on 21/03/2016:

Add demo program for working with Capture Manager SDK in Qt framework programs on MinGW compiler: QtMinGWDemo

The eighth update on 12/04/2016

Add demo program for working with CaptureManager SDK in Windows Store application: CaptureManagerSDKWindowsStoreDemo

The nineth update on 13/06/2016

Stable release 1.1.0:

Add supporting of automating registration CaptureManager as COM server with Type Library
Add supporting of dynamic language Python 2.7
Add supporting of HEVC(H265) encoder in Windows 10: Download HEVCEncoder_Windows10

Capture Manager SDK 1.1.0 can be downloaded by link: Download CaptureManagerSDK-110.zip

Webcamera properties are extended - Power line frequency
Add supporting of dynamic language Python 2.7
MP4 file container format
MP4 stream container format - Download ScreenStreaming_Bin.zip, WPFScreenStreamer

All these features are base on the libraries which already exist in core of Windows 7, Windows 8 and Windows 10. It allows develop simple solution with supporting of the most popular formats.

The features of commercial version can be tested on CaptureManagerSDK-110_trial.zip which is limitated only by 60 second duration of capturing session.

Capture Manager Store: Application License - $119.99

The tenth update on 01/08/2016

Beta release 1.2.0:

Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
Lazy binding of Microsoft Media Foundation functions.
Replace web camera properties functionality from DirectShow on DeviceIoControl.
Add resizing in EVR.

Capture Manager SDK v1.2.0 beta can be downloaded by link: Download CaptureManagerSDK-1.2.0_beta.zip

The eleventh update on 05/09/2016:

Release 1.2.0:

Deleted old functionality of working with CaptureManager via library linking. Old demo programs are moved in old demos.
Lazy binding of Microsoft Media Foundation functions.
Replace web camera properties functionality from DirectShow on DeviceIoControl.
Add resizing in EVR.
Add PULL mode in SampleGrabberCallSinkFactory - the new mode which allows to take a single sample.
Add sample accumulator nodes for storing of 5 or 10 last samples in media stream.
Add four new demo programms:

Capture Manager SDK v1.2.0 can be downloaded by link: Download CaptureManagerSDK-1.2.0-Freeware.zip

The twelveth update on 03/10/2016:

Beta release 1.3.0:

Add supporting of video capturing via DirectShow Crossbar technique for the next inputs: Composite, SVideo, USB, 1394 (FireWire).

Capture Manager SDK v1.3.0 – beta can be downloaded by link: Download CaptureManagerSDK-1.3.0-Freeware-beta.zip

History

The first version was published on 11/08/2015.

The second version was published on 21/08/2015.

The third version was published on 31/08/2015.

The fifth version was published on 13/11/2015.

The sixth version was published on 07/03/2016.

The seventh version was published on 21/03/2016.

The eighth version was published on 12/04/2016.

The nineth version was published on 13/06/2016.

The tenth version was published on 01/08/2016.

The eleventh version was published on 05/09/2016.

The twelveth version was published on 03/10/2016.

&amp;lt;a href="https://pubads.g.doubleclick.net/gampad/jump?iu=/6839/lqm.codeproject.site/Multimedia/Audio-and-Video/General&amp;amp;sz=300x250&amp;amp;c=723176"&amp;gt;&amp;lt;img src="https://pubads.g.doubleclick.net/gampad/jump?iu=/6839/lqm.codeproject.site/Multimedia/Audio-and-Video/General&amp;amp;sz=300x250&amp;amp;c=723176" width="300px" height="250px" target="_blank"/&amp;gt;&amp;lt;/a&amp;gt;

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)