如何在c++中实现Coroutines

时间:2022-09-02 09:31:44

I doubt it can be done portably, but are there any solutions out there? I think it could be done by creating an alternate stack and reseting SP,BP, and IP on function entry, and having yield save IP and restore SP+BP. Destructors and exception safety seem tricky but solvable.

我怀疑它能否很好地移植,但有没有解决办法呢?我认为可以通过创建一个备用堆栈并在函数入口上重新设置SP、BP和IP,并具有yield save IP和恢复SP+BP来实现。析构函数和异常安全看起来很棘手,但却是可以解决的。

Has it been done? Is it impossible?

它已经做了什么?这是不可能的吗?

17 个解决方案

#1


86  

Yes it can be done without a problem. All you need is a little assembly code to move the call stack to a newly allocated stack on the heap.

是的,这是可以做到的,没有问题。您需要的只是一个小的汇编代码,将调用堆栈移动到堆上新分配的堆栈上。

I would look at the boost::coroutine library.

我会查看boost:::coroutine library。

The one thing that you should watch out for is a stack overflow. On most operating systems overflowing the stack will cause a segfault because virtual memory page is not mapped. However if you allocate the stack on the heap you don't get any guarantee. Just keep that in mind.

需要注意的是堆栈溢出。在大多数操作系统中,溢出栈将导致一个segfault,因为虚拟内存页没有被映射。但是,如果您在堆上分配堆栈,则不会得到任何保证。记住这一点。

#2


17  

On POSIX, you can use makecontext()/swapcontext() routines to portably switch execution contexts. On Windows, you can use the fiber API. Otherwise, all you need is a bit of glue assembly code that switches the machine context. I have implemented coroutines both with ASM (for AMD64) and with swapcontext(); neither is very hard.

在POSIX上,可以使用makecontext()/swapcontext()例程可移植地切换执行上下文。在Windows上,可以使用fiber API。否则,您所需要的只是一些胶水组装代码,用于切换机器上下文。我使用ASM(用于AMD64)和swapcontext()实现了coroutines;都是非常困难的。

#3


12  

For posterity,

为后代,

Dmitry Vyukov's wondeful web site has a clever trick using ucontext and setjump to simulated coroutines in c++.

德米特里•维尤科夫(Dmitry Vyukov)设计的奇妙网站有一个聪明的诀窍:使用ucontext,并在c++中设置到模拟的coroutines。

Also, Oliver Kowalke's context library was recently accepted into Boost, so hopefully we'll be seeing an updated version of boost.coroutine that works on x86_64 soon.

另外,Oliver Kowalke的上下文库最近被Boost所接受,所以我们希望看到Boost的更新版本。在x86_64上工作的coroutine。

#4


9  

There's no easy way to implement coroutine. Because coroutine itself is out of C/C++'s stack abstraction just like thread. So it cannot be supported without language level changes to support.

实现coroutine没有简单的方法。因为coroutine本身就像线程一样脱离了C/ c++的堆栈抽象。因此,如果没有对支持进行语言级别的更改,就不能支持它。

Currently(C++11), all existing C++ coroutine implementations are all based on assembly level hacking which is hard to be safe and reliable crossing over platforms. To be reliable it needs to be standard, and handled by compilers rather than hacking.

目前(c++ 11),所有现有的c++ coroutine实现都是基于汇编级的黑客攻击,很难在平台上安全可靠地交叉。要可靠,它必须是标准的,由编译器处理,而不是黑客。

There's a standard proposal - N3708 for this. Check it out if you're interested.

有一个标准的提案——N3708。如果你有兴趣的话可以去看看。

#5


7  

You might be better off with an iterator than a coroutine if possible. That way you can keep calling next() to get the next value, but you can keep your state as member variables instead of local variables.

如果可能的话,使用迭代器比使用coroutine要好。这样,您就可以继续调用next()来获得下一个值,但是您可以将您的状态保持为成员变量而不是局部变量。

It might make things more maintainable. Another C++ developer might not immediately understand the coroutine whereas they might be more familiar with an iterator.

它可能使事情更易于维护。另一个c++开发人员可能不会马上理解coroutine,但是他们可能更熟悉迭代器。

#6


5  

I dont think there are many full-blown, clean implementations in C++. One try that I like is protothread library.

我不认为c++有很多成熟的、干净的实现。我喜欢的一个尝试是prototype线程库。

#7


5  

Does COROUTINE a portable C++ library for coroutine sequencing point you in the right direction? It seems like an elegant solution that has lasted the test of time.....it's 9 years old!

COROUTINE是一个便携式的c++库,用于COROUTINE测序,它是否为你指明了正确的方向?这似乎是一个优雅的解决方案,它持续了时间的考验……这是9岁!

In the DOC folder is a pdf of the paper A Portable C++ Library for Coroutine Sequencing by Keld Helsgaun which describes the library and provides short examples using it.

在DOC文件夹中是一份pdf文件,是由Keld Helsgaun提供的便携式c++库,用于Coroutine测序。

[update] I'm actually making successful use of it myself. Curiosity got the better of me, so I looked into this solution, and found it was a good fit for a problem I've been working on for some time!

[更新]我自己也成功地利用了它。好奇心战胜了我,所以我研究了这个解决方案,发现它很适合我研究了一段时间的问题。

#8


4  

For those who want to know how they may leverage Coroutines in a portable way in C++ you will have to wait for C++17. The standards committee is working on the feature see the N3722 paper. To summarize the current draft of the paper, instead of Async and Await, the keywords will be resumable, and await.

对于那些想知道如何在c++中以一种可移植的方式利用Coroutines的人来说,您将不得不等待c++ 17。标准委员会正在研究这一特性,请参阅N3722论文。为了总结本文的当前草稿,本文将不使用异步等待,而是将关键字恢复并等待。

Take a look at the experimental implementation in Visual Studio 2015 to play with Microsoft's experimental implementation. It doesn't look like clang has a implementation yet.

看看Visual Studio 2015的实验性实现,看看微软的实验性实现。看起来clang还没有实现。

There is a good talk from Cppcon Coroutines a negative overhead abstraction outline the benefits of using Coroutines in C++ and how it affects simplicity and performance of the code.

Cppcon Coroutines有一个很好的讨论,一个负面的开销抽象概述了在c++中使用Coroutines的好处,以及它如何影响代码的简单性和性能。

At present we still have to use library implementations, but in the near future, we will have coroutines as a core C++ feature.

目前,我们仍然需要使用库实现,但是在不久的将来,我们将把coroutines作为一个核心c++特性。

Update: Looks like the coroutine implementation isn't making it into C++17, but it will be a technical specification (p0057r2). On the upside, it looks like their is support in clang with the -fcoroutines_ts flag and in Visual Studio 2015 Update 2. The keywords also have a co_ prepended to them. So co_await, co_yield etc.

更新:看起来coroutine实现并没有把它变成c++ 17,但是它将是一个技术规范(p0057r2)。从好的方面来看,他们的is支持在clang中带有-fcoroutines_ts标志,在Visual Studio 2015 Update 2中也有。关键词也有一个前缀。所以co_await co_yield等等。

#9


3  

A new library, Boost.Context, was released today with portable features for implementing coroutines.

一个新的图书馆,提高。Context,今天发布了实现coroutines的可移植特性。

#10


3  

This is an old thread, but I would like to suggest a hack using Duff's device that is not os-dependent (as far as I remember):

这是一条古老的线索,但我想建议使用Duff的设备,而不是依赖于os的(据我所知):

C coroutines using Duff's device

用达夫的设备

And as an example, here is a telnet library I modified to use coroutines instead of fork/threads: Telnet cli library using coroutines

举个例子,我修改了一个telnet库来使用coroutines而不是fork/threads: telnet cli库使用coroutines

And since standard C prior to C99 is essentially a true subset of C++, this works well in C++ too.

由于C99之前的标准C本质上是c++的一个子集,这在c++中也适用。

#11


2  

It is based on (cringe) macros, but the following site provides an easy-to-use generator implementation: http://www.codeproject.com/KB/cpp/cpp_generators.aspx

它基于(cringe)宏,但是以下站点提供了一个易于使用的生成器实现:http://www.codeproject.com/KB/cpp/cpp_generators.aspx

#12


2  

I've come up with an implementation without asm code. The idea is to use the system's thread creating function to initialize stack and context, and use setjmp/longjmp to switch context. But It's not portable, see the tricky pthread version if you are interested.

我提出了一个没有asm代码的实现。其思想是使用系统的线程创建函数来初始化堆栈和上下文,并使用setjmp/longjmp来切换上下文。但它不是可移植的,如果你感兴趣的话,可以看看棘手的pthread版本。

#13


1  

https://github.com/tonbit/coroutine is C++11 single .h asymmetric coroutine implementation supporting resume/yield/await primitives and Channel model. It's implementing via ucontext / fiber, not depending on boost, running on linux/windows/macOS. It's a good starting point to learn implementing coroutine in c++.

https://github.com/tonbit/coroutine是c++ 11单。h不对称的coroutine实现,支持恢复/收益/等待原语和通道模型。它通过ucontext / fiber方式实现,而不是依靠boost,在linux/windows/macOS上运行。学习在c++中实现coroutine是一个很好的起点。

#14


0  

You should always consider using threads instead; especially in modern hardware. If you have work that can be logically separated in Co-routines, using threads means the work might actually be done concurrently, by separate execution units (processor cores).

您应该始终考虑使用线程;尤其是在现代的硬件。如果您有可以在逻辑上在共同例程中分离的工作,那么使用线程意味着工作实际上可以通过单独的执行单元(处理器内核)并发地完成。

But, maybe you do want to use coroutines, perhaps because you have an well tested algorithm that has already been written and tested that way, or because you are porting code written that way.

但是,也许您确实想要使用coroutines,也许是因为您有一个经过良好测试的算法,它已经被以这种方式编写和测试过,或者因为您正在移植以这种方式编写的代码。

If you work within Windows, you should take a look at fibers. Fibers will give you a coroutine-like framework with support from the OS.

如果你在窗户里工作,你应该看看纤维。纤程将提供一个支持OS的类coroutine框架。

I am not familiar with other OS's to recommend alternatives there.

我不熟悉其他操作系统在那里推荐替代方案。

#15


0  

WvCont is a part of WvStreams that implements so-called semi-coroutines. These are a little easier to handle than full-on coroutines: you call into it, and it yields back to the person who called it.

WvCont是WvStreams的一部分,它实现了所谓的半coroutines。这比全面的警戒线更容易处理:你调用它,它会返回给调用它的人。

It's implemented using the more flexible WvTask, which supports full-on coroutines; you can find it in the same library.

它是使用更灵活的WvTask实现的,WvTask支持全面的coroutines;你可以在同一个图书馆找到它。

Works on win32 and Linux, at least, and probably any other Unix system.

至少可以在win32和Linux上工作,可能还可以使用其他任何Unix系统。

#16


0  

Check out my implementation, it illustrates the asm hacking point and is simple:

看看我的实现,它演示了asm黑客点,很简单:

https://github.com/user1095108/generic/blob/master/coroutine.hpp

https://github.com/user1095108/generic/blob/master/coroutine.hpp

#17


-2  

I've tried to implement coroutines myself using C++11 and threads:

我尝试过使用c++ 11和线程实现coroutines:

#include <iostream>
#include <thread>

class InterruptedException : public std::exception {
};

class AsyncThread {
public:
    AsyncThread() {
        std::unique_lock<std::mutex> lock(mutex);
        thread.reset(new std::thread(std::bind(&AsyncThread::run, this)));
        conditionVar.wait(lock); // wait for the thread to start
    }
    ~AsyncThread() {
        {
            std::lock_guard<std::mutex> _(mutex);
            quit = true;
        }
        conditionVar.notify_all();
        thread->join();
    }
    void run() {
        try {
            yield();
            for (int i = 0; i < 7; ++i) {
                std::cout << i << std::endl;
                yield();
            }
        } catch (InterruptedException& e) {
            return;
        }
        std::lock_guard<std::mutex> lock(mutex);
        quit = true;
        conditionVar.notify_all();
    }
    void yield() {
        std::unique_lock<std::mutex> lock(mutex);
        conditionVar.notify_all();
        conditionVar.wait(lock);
        if (quit) {
            throw InterruptedException();
        }
    }
    void step() {
        std::unique_lock<std::mutex> lock(mutex);
        if (!quit) {
            conditionVar.notify_all();
            conditionVar.wait(lock);
        }
    }
private:
    std::unique_ptr<std::thread> thread;
    std::condition_variable conditionVar;
    std::mutex mutex;
    bool quit = false;
};

int main() {
    AsyncThread asyncThread;
    for (int i = 0; i < 3; ++i) {
        std::cout << "main: " << i << std::endl;
        asyncThread.step();
    }
}

#1


86  

Yes it can be done without a problem. All you need is a little assembly code to move the call stack to a newly allocated stack on the heap.

是的,这是可以做到的,没有问题。您需要的只是一个小的汇编代码,将调用堆栈移动到堆上新分配的堆栈上。

I would look at the boost::coroutine library.

我会查看boost:::coroutine library。

The one thing that you should watch out for is a stack overflow. On most operating systems overflowing the stack will cause a segfault because virtual memory page is not mapped. However if you allocate the stack on the heap you don't get any guarantee. Just keep that in mind.

需要注意的是堆栈溢出。在大多数操作系统中,溢出栈将导致一个segfault,因为虚拟内存页没有被映射。但是,如果您在堆上分配堆栈,则不会得到任何保证。记住这一点。

#2


17  

On POSIX, you can use makecontext()/swapcontext() routines to portably switch execution contexts. On Windows, you can use the fiber API. Otherwise, all you need is a bit of glue assembly code that switches the machine context. I have implemented coroutines both with ASM (for AMD64) and with swapcontext(); neither is very hard.

在POSIX上,可以使用makecontext()/swapcontext()例程可移植地切换执行上下文。在Windows上,可以使用fiber API。否则,您所需要的只是一些胶水组装代码,用于切换机器上下文。我使用ASM(用于AMD64)和swapcontext()实现了coroutines;都是非常困难的。

#3


12  

For posterity,

为后代,

Dmitry Vyukov's wondeful web site has a clever trick using ucontext and setjump to simulated coroutines in c++.

德米特里•维尤科夫(Dmitry Vyukov)设计的奇妙网站有一个聪明的诀窍:使用ucontext,并在c++中设置到模拟的coroutines。

Also, Oliver Kowalke's context library was recently accepted into Boost, so hopefully we'll be seeing an updated version of boost.coroutine that works on x86_64 soon.

另外,Oliver Kowalke的上下文库最近被Boost所接受,所以我们希望看到Boost的更新版本。在x86_64上工作的coroutine。

#4


9  

There's no easy way to implement coroutine. Because coroutine itself is out of C/C++'s stack abstraction just like thread. So it cannot be supported without language level changes to support.

实现coroutine没有简单的方法。因为coroutine本身就像线程一样脱离了C/ c++的堆栈抽象。因此,如果没有对支持进行语言级别的更改,就不能支持它。

Currently(C++11), all existing C++ coroutine implementations are all based on assembly level hacking which is hard to be safe and reliable crossing over platforms. To be reliable it needs to be standard, and handled by compilers rather than hacking.

目前(c++ 11),所有现有的c++ coroutine实现都是基于汇编级的黑客攻击,很难在平台上安全可靠地交叉。要可靠,它必须是标准的,由编译器处理,而不是黑客。

There's a standard proposal - N3708 for this. Check it out if you're interested.

有一个标准的提案——N3708。如果你有兴趣的话可以去看看。

#5


7  

You might be better off with an iterator than a coroutine if possible. That way you can keep calling next() to get the next value, but you can keep your state as member variables instead of local variables.

如果可能的话,使用迭代器比使用coroutine要好。这样,您就可以继续调用next()来获得下一个值,但是您可以将您的状态保持为成员变量而不是局部变量。

It might make things more maintainable. Another C++ developer might not immediately understand the coroutine whereas they might be more familiar with an iterator.

它可能使事情更易于维护。另一个c++开发人员可能不会马上理解coroutine,但是他们可能更熟悉迭代器。

#6


5  

I dont think there are many full-blown, clean implementations in C++. One try that I like is protothread library.

我不认为c++有很多成熟的、干净的实现。我喜欢的一个尝试是prototype线程库。

#7


5  

Does COROUTINE a portable C++ library for coroutine sequencing point you in the right direction? It seems like an elegant solution that has lasted the test of time.....it's 9 years old!

COROUTINE是一个便携式的c++库,用于COROUTINE测序,它是否为你指明了正确的方向?这似乎是一个优雅的解决方案,它持续了时间的考验……这是9岁!

In the DOC folder is a pdf of the paper A Portable C++ Library for Coroutine Sequencing by Keld Helsgaun which describes the library and provides short examples using it.

在DOC文件夹中是一份pdf文件,是由Keld Helsgaun提供的便携式c++库,用于Coroutine测序。

[update] I'm actually making successful use of it myself. Curiosity got the better of me, so I looked into this solution, and found it was a good fit for a problem I've been working on for some time!

[更新]我自己也成功地利用了它。好奇心战胜了我,所以我研究了这个解决方案,发现它很适合我研究了一段时间的问题。

#8


4  

For those who want to know how they may leverage Coroutines in a portable way in C++ you will have to wait for C++17. The standards committee is working on the feature see the N3722 paper. To summarize the current draft of the paper, instead of Async and Await, the keywords will be resumable, and await.

对于那些想知道如何在c++中以一种可移植的方式利用Coroutines的人来说,您将不得不等待c++ 17。标准委员会正在研究这一特性,请参阅N3722论文。为了总结本文的当前草稿,本文将不使用异步等待,而是将关键字恢复并等待。

Take a look at the experimental implementation in Visual Studio 2015 to play with Microsoft's experimental implementation. It doesn't look like clang has a implementation yet.

看看Visual Studio 2015的实验性实现,看看微软的实验性实现。看起来clang还没有实现。

There is a good talk from Cppcon Coroutines a negative overhead abstraction outline the benefits of using Coroutines in C++ and how it affects simplicity and performance of the code.

Cppcon Coroutines有一个很好的讨论,一个负面的开销抽象概述了在c++中使用Coroutines的好处,以及它如何影响代码的简单性和性能。

At present we still have to use library implementations, but in the near future, we will have coroutines as a core C++ feature.

目前,我们仍然需要使用库实现,但是在不久的将来,我们将把coroutines作为一个核心c++特性。

Update: Looks like the coroutine implementation isn't making it into C++17, but it will be a technical specification (p0057r2). On the upside, it looks like their is support in clang with the -fcoroutines_ts flag and in Visual Studio 2015 Update 2. The keywords also have a co_ prepended to them. So co_await, co_yield etc.

更新:看起来coroutine实现并没有把它变成c++ 17,但是它将是一个技术规范(p0057r2)。从好的方面来看,他们的is支持在clang中带有-fcoroutines_ts标志,在Visual Studio 2015 Update 2中也有。关键词也有一个前缀。所以co_await co_yield等等。

#9


3  

A new library, Boost.Context, was released today with portable features for implementing coroutines.

一个新的图书馆,提高。Context,今天发布了实现coroutines的可移植特性。

#10


3  

This is an old thread, but I would like to suggest a hack using Duff's device that is not os-dependent (as far as I remember):

这是一条古老的线索,但我想建议使用Duff的设备,而不是依赖于os的(据我所知):

C coroutines using Duff's device

用达夫的设备

And as an example, here is a telnet library I modified to use coroutines instead of fork/threads: Telnet cli library using coroutines

举个例子,我修改了一个telnet库来使用coroutines而不是fork/threads: telnet cli库使用coroutines

And since standard C prior to C99 is essentially a true subset of C++, this works well in C++ too.

由于C99之前的标准C本质上是c++的一个子集,这在c++中也适用。

#11


2  

It is based on (cringe) macros, but the following site provides an easy-to-use generator implementation: http://www.codeproject.com/KB/cpp/cpp_generators.aspx

它基于(cringe)宏,但是以下站点提供了一个易于使用的生成器实现:http://www.codeproject.com/KB/cpp/cpp_generators.aspx

#12


2  

I've come up with an implementation without asm code. The idea is to use the system's thread creating function to initialize stack and context, and use setjmp/longjmp to switch context. But It's not portable, see the tricky pthread version if you are interested.

我提出了一个没有asm代码的实现。其思想是使用系统的线程创建函数来初始化堆栈和上下文,并使用setjmp/longjmp来切换上下文。但它不是可移植的,如果你感兴趣的话,可以看看棘手的pthread版本。

#13


1  

https://github.com/tonbit/coroutine is C++11 single .h asymmetric coroutine implementation supporting resume/yield/await primitives and Channel model. It's implementing via ucontext / fiber, not depending on boost, running on linux/windows/macOS. It's a good starting point to learn implementing coroutine in c++.

https://github.com/tonbit/coroutine是c++ 11单。h不对称的coroutine实现,支持恢复/收益/等待原语和通道模型。它通过ucontext / fiber方式实现,而不是依靠boost,在linux/windows/macOS上运行。学习在c++中实现coroutine是一个很好的起点。

#14


0  

You should always consider using threads instead; especially in modern hardware. If you have work that can be logically separated in Co-routines, using threads means the work might actually be done concurrently, by separate execution units (processor cores).

您应该始终考虑使用线程;尤其是在现代的硬件。如果您有可以在逻辑上在共同例程中分离的工作,那么使用线程意味着工作实际上可以通过单独的执行单元(处理器内核)并发地完成。

But, maybe you do want to use coroutines, perhaps because you have an well tested algorithm that has already been written and tested that way, or because you are porting code written that way.

但是,也许您确实想要使用coroutines,也许是因为您有一个经过良好测试的算法,它已经被以这种方式编写和测试过,或者因为您正在移植以这种方式编写的代码。

If you work within Windows, you should take a look at fibers. Fibers will give you a coroutine-like framework with support from the OS.

如果你在窗户里工作,你应该看看纤维。纤程将提供一个支持OS的类coroutine框架。

I am not familiar with other OS's to recommend alternatives there.

我不熟悉其他操作系统在那里推荐替代方案。

#15


0  

WvCont is a part of WvStreams that implements so-called semi-coroutines. These are a little easier to handle than full-on coroutines: you call into it, and it yields back to the person who called it.

WvCont是WvStreams的一部分,它实现了所谓的半coroutines。这比全面的警戒线更容易处理:你调用它,它会返回给调用它的人。

It's implemented using the more flexible WvTask, which supports full-on coroutines; you can find it in the same library.

它是使用更灵活的WvTask实现的,WvTask支持全面的coroutines;你可以在同一个图书馆找到它。

Works on win32 and Linux, at least, and probably any other Unix system.

至少可以在win32和Linux上工作,可能还可以使用其他任何Unix系统。

#16


0  

Check out my implementation, it illustrates the asm hacking point and is simple:

看看我的实现,它演示了asm黑客点,很简单:

https://github.com/user1095108/generic/blob/master/coroutine.hpp

https://github.com/user1095108/generic/blob/master/coroutine.hpp

#17


-2  

I've tried to implement coroutines myself using C++11 and threads:

我尝试过使用c++ 11和线程实现coroutines:

#include <iostream>
#include <thread>

class InterruptedException : public std::exception {
};

class AsyncThread {
public:
    AsyncThread() {
        std::unique_lock<std::mutex> lock(mutex);
        thread.reset(new std::thread(std::bind(&AsyncThread::run, this)));
        conditionVar.wait(lock); // wait for the thread to start
    }
    ~AsyncThread() {
        {
            std::lock_guard<std::mutex> _(mutex);
            quit = true;
        }
        conditionVar.notify_all();
        thread->join();
    }
    void run() {
        try {
            yield();
            for (int i = 0; i < 7; ++i) {
                std::cout << i << std::endl;
                yield();
            }
        } catch (InterruptedException& e) {
            return;
        }
        std::lock_guard<std::mutex> lock(mutex);
        quit = true;
        conditionVar.notify_all();
    }
    void yield() {
        std::unique_lock<std::mutex> lock(mutex);
        conditionVar.notify_all();
        conditionVar.wait(lock);
        if (quit) {
            throw InterruptedException();
        }
    }
    void step() {
        std::unique_lock<std::mutex> lock(mutex);
        if (!quit) {
            conditionVar.notify_all();
            conditionVar.wait(lock);
        }
    }
private:
    std::unique_ptr<std::thread> thread;
    std::condition_variable conditionVar;
    std::mutex mutex;
    bool quit = false;
};

int main() {
    AsyncThread asyncThread;
    for (int i = 0; i < 3; ++i) {
        std::cout << "main: " << i << std::endl;
        asyncThread.step();
    }
}