如何在c++中在一个独立的posix线程中运行OpenCV Videocapture进行跨平台运行?

In my code, I want to run a Videocapture in a separate pthread continuously, while the main thread will act as a socket server to communicate to any client (or vice versa).

在我的代码中，我希望在一个单独的pthread中连续运行一个Videocapture，而主线程作为套接字服务器与任何客户端进行通信(反之亦然)。

When the client connects to the server, the server will immediately send the latest Videocapture frame to the client and close the connection and again wait for another connection.

当客户端连接到服务器时，服务器将立即将最新的Videocapture框架发送给客户端，关闭连接，再次等待另一个连接。

I am currently using OpenCV 2.3.0 in mingw32 environment.

我目前在mingw32环境中使用OpenCV 2.3.0。

After looking some possible solutions on *, I tried following them, but, however may I place the OpenCV variables either locally or globally or swapping the threads i.e. keeping Videocapture thread in main thread or in the 2nd thread, the program just hangs silently at the mutex locked area of the VideoCapture thread when the client gets connected.

*上寻找一些可能的解决方法之后,我试着跟着他们,但是,但是我可以把OpenCV变量在本地或全球或交换主线程的线程即保持Videocapture线程或第二线程,程序只是静静地挂在Videocapture线程的互斥锁区域当客户机连接。

I suppose this is happening when the server thread starts locking the mutex after connecting to a client.

我猜想这是在服务器线程在连接到客户端后开始锁定互斥对象时发生的。

Any reasoning or possible solution will be highly appreciated. Also, I am using prebuilt OpenCV version 2.3.0 and I need the code to be cross platform. In any case, if it points to a bug, I can obviously switch to some other version.

任何推理或可能的解决办法将受到高度赞赏。另外，我正在使用预构建的OpenCV 2.3.0版本，我需要代码是跨平台的。无论如何，如果它指向一个bug，我显然可以切换到其他版本。

---

Server code

服务器代码

mynet.h

mynet.h

#ifndef __MYNET__
#define __MYNET__
#ifdef _WIN32
#ifndef _WIN32_WINNT
#define _WIN32_WINNT 0x0501  /* Windows XP. */
#endif
#include <winsock2.h>
#include <Ws2tcpip.h>
#else
/* Assume that any non-Windows platform uses POSIX-style sockets instead. */
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netdb.h>  /* Needed for getaddrinfo() and freeaddrinfo() */
#include <unistd.h> /* Needed for close() */
#endif

int sockInit(void)
{
#ifdef _WIN32
    WSADATA wsa_data;
    return WSAStartup(MAKEWORD(1,1), &wsa_data);
#else
    return 0;
#endif
}

int sockQuit(void)
{
#ifdef _WIN32
    return WSACleanup();
#else
    return 0;
#endif
}

/* Note: For POSIX, typedef SOCKET as an int. */
#ifndef _WIN32
typedef int SOCKET;
#else
typedef unsigned int SOCKET;
#endif

int sockClose(SOCKET sock)
{
    int status = 0;
#ifdef _WIN32
    status = shutdown(sock, SD_BOTH);
    if (status == 0) { status = closesocket(sock); }
#else
    status = shutdown(sock, SHUT_RDWR);
    if (status == 0) { status = close(sock); }
#endif
    return status;
}


#if(defined(_WIN32) || defined(WIN32))
#include <windows.h>
#define mysleep(x) Sleep((x))
#else
#include <unistd.h>
#define mysleep(x) usleep((x)*1000)
#endif

#endif

netserver.cpp

netserver.cpp

#define _GLIBCXX_USE_CXX11_ABI 0
#undef _GLIBCXX_DEBUG
#include "../mynet.h"
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <pthread.h>

using namespace cv;
Mat frame(120, 160, CV_8UC3);;
pthread_mutex_t mutex;
pthread_t thread;

void *myfunc(void *threadid)
{
    int n;
    int listenfd = 0,connfd = 0;
    struct sockaddr_in serv_addr;
    unsigned char sendBuff[160*120*3];
    Mat hereframe;

    listenfd = socket(AF_INET, SOCK_STREAM, 0);
    printf("socket retrieve success\n");

    memset(&serv_addr, '0', sizeof(serv_addr));
    memset(sendBuff, '0', sizeof(sendBuff));

    serv_addr.sin_family = AF_INET;
    serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
    serv_addr.sin_port = htons(5000);
    bind(listenfd, (struct sockaddr*)&serv_addr,sizeof(serv_addr));
    if(listen(listenfd, 10) == -1)
    {
        printf("Failed to listen\n");
    }
    else
    {
        connfd = accept(listenfd, (struct sockaddr*)NULL,NULL);  // accept awaiting request
        if(connfd<=0)  printf("Something went wrong with write()! %s\n", strerror(errno));
        else
        {
            while(1)
            {
                printf("Got client...\n");
                pthread_mutex_lock (&mutex);
                printf("Copying data...\n");
                frame.copyTo(hereframe);
                pthread_mutex_unlock (&mutex);
                printf("Hereframe size: %dX%d\n", hereframe.rows, hereframe.cols);
                memcpy(sendBuff, hereframe.data, 120*160*3);
                printf("Sending data...\n");
                n = send(connfd, (char*)sendBuff, 120*160*3,0);
                if(n<0)  printf("Something went wrong with write()! %s\n", strerror(errno));
                printf("Closing client after writing %d bytes...\n", n);
                printf("Closed...\n");
                mysleep(40);
                printf("Restartng data...\n");
            }
            sockClose(connfd);
        }
    }
    sockQuit();

    pthread_exit(NULL);
}

int main(void)
{
    //cvNamedWindow("Sisplay", CV_WINDOW_AUTOSIZE);
    sockInit();
    pthread_mutex_init(&mutex,NULL);
    VideoCapture cap;
    Mat hereframe1;
    cap.open(0);
    cap.set( CV_CAP_PROP_FRAME_WIDTH, 160 );
    cap.set( CV_CAP_PROP_FRAME_HEIGHT, 120 );

    if( !cap.isOpened()) printf("Fukced up\n");
    int rc;
    long t;
    rc = pthread_create(&thread, NULL, myfunc, (void *)t);
    if (rc)
    {
        printf("ERROR; return code from pthread_create() is %d\n", rc);
        exit(-1);
    }

    for(;;)
    {
        cap >> hereframe1;
        //     printf("Video size: %dX%d\n", hereframe1.rows, hereframe1.cols);

        pthread_mutex_lock(&mutex);
        hereframe1.copyTo(frame); /* hangs here*/
        pthread_mutex_unlock(&mutex);
        // cvtColor(frame, gray, CV_BGR2GRAY);

        Scalar tempVal = mean(hereframe1);
        //  printf("Mean = %f\n", tempVal.val[0]);
        mysleep(30);
        //if(waitKey(30)=='q') break;
        // std::swap(prevgray, gray);
    }
    char *b;
    pthread_join(thread,(void**)&b);
    return 0;
}

---

Client code

客户端代码

receiver.cpp

receiver.cpp

#define _GLIBCXX_USE_CXX11_ABI 0
#include "../mynet.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <opencv/cv.h>
#include <opencv/highgui.h>

using namespace cv;

int main(int argc, char *argv[])
{
    int sockfd = 0,n = 0;
    unsigned char recvBuff[120*160*3];
    struct sockaddr_in serv_addr;
    sockInit();
    memset(recvBuff, '0',sizeof(recvBuff));
    if((sockfd = socket(AF_INET, SOCK_STREAM, 0))< 0)
    {
        printf("\n Error : Could not create socket \n");
        return 1;
    }

    serv_addr.sin_family = AF_INET;
    serv_addr.sin_port = htons(5000);
    serv_addr.sin_addr.s_addr = inet_addr(argv[1]);

    if(connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr))<0)
    {
        printf("\n Error : Connect Failed \n");
        return 1;
    }
    printf("Connected to server...\n");
    Mat frame(120, 160, CV_8UC3);
    int csz = 0;
    cvNamedWindow("Display", CV_WINDOW_AUTOSIZE);
    printf("OK here 0\n");
    for(;;)
    {
        csz = 0;
        while((n = recv(sockfd, (char*)recvBuff, sizeof(recvBuff), 0)) > 0)
        {
            printf("OK here 2\n");

            printf("Read %d bytes...\n", n);
            csz += n;
            if(csz<=120*160*3)
            {
                memcpy(frame.data+(csz-n), recvBuff, n);
            }
            if(csz>= 120*160*3) break;
        }
        printf("OK here 3\n");

        printf("Got data of size %d bytes...\n", csz);
        imshow("Display", frame);
        if(csz < 120*160*3)
        {
            printf("\n Read Error \n");
        }
        if(waitKey(30)=='q') break;
        // std::swap(prevgray, gray);
    }
    sockClose(sockfd);

    sockQuit();
    return 0;
}

1 个解决方案

#1

What I think is that the OpenCV 2.3.0 functions are not thread-safe. e.g. functions like Mat::clone(), Mat::copyTo, etc. (need to clarify).

我认为OpenCV 2.3.0函数不是线程安全的。如Mat::克隆()，Mat::copyTo等(需要澄清)。

So, instead of using those OpenCV functions in mutex locked regions and OpenCV variables as a whole shared in multiple threads, I am just using basic C++ functions and raw buffers.

因此，我没有在互斥锁区域和OpenCV变量中使用OpenCV函数，而是使用基本的c++函数和原始缓冲区。

Using OpenCV functions in mutex locked regions and OpenCV variables as a whole shared in multiple threads does not seem to be friendly at all.

在互斥锁区域中使用OpenCV函数和在多个线程*享的OpenCV变量看起来一点都不友好。

So, I have made a change in the server thread as

因此，我对服务器线程as做了更改

pthread_mutex_lock (&mutex);
//printf("Copying data...\n");
memcpy(sendBuff, frame.data, 120*160*3);
//frame.copyTo(hereframe);  buggy here removed 
pthread_mutex_unlock (&mutex);

And now Its working as expected. Uffh! Now, I need to clean up all the mess in my code.

现在它的工作和预期一样。Uffh !现在，我需要清理代码中的所有混乱。

In fact, I will also replace the OpenCV frame variable with a simple unsigned char buffer for the sharing.

实际上，我还将用一个简单的无符号字符缓冲区替换OpenCV框架变量，以便共享。