OBSE and Boost.Asio: Threaded async UDP server with deadline_timer on the same io_service

Question

Platform: Windows 7 Professional 64 bit
Compiler: VS2010 Express
Boost: Version 1.49
Plugin System: OBSE 20 (for the Oblivion game by Bethesda)

I have a class based upon the async udp examples. I run the io service itself as a thread. Here is the code for the class:

    // udp buffer queues
    extern concurrent_queue<udp_packet> udp_input_queue; // input from external processes
    extern concurrent_queue<udp_packet> udp_output_queue; // output to external processes

    using boost::asio::ip::udp;
    class udp_server
    {
    public:
        udp_server(boost::asio::io_service& io_service, short port)
            : io_service_(io_service),
              socket_(io_service_, udp::endpoint(boost::asio::ip::address_v4::from_string(current_address), port))//, // udp::v4()  
        {
            // start udp receive
            socket_.async_receive_from(
                boost::asio::buffer(recv_buf), sender_endpoint_,
                boost::bind(&udp_server::handle_receive_from, this,
                  boost::asio::placeholders::error,
                  boost::asio::placeholders::bytes_transferred));  

            send_timer_ = NULL;            
        }

        ~udp_server(){
            io_service_.stop();
            if(send_timer_){
                send_timer_->cancel();
                delete send_timer_;
            }
        }

        void start(){
            // start send timer                
            send_timer_ = new boost::asio::deadline_timer(io_service_, boost::posix_time::milliseconds(500));
            send_timer_restart();
        }

        void handle_send_to(const boost::system::error_code& error, size_t bytes_recvd);
        void handle_receive_from(const boost::system::error_code& error, size_t bytes_recvd);

        //void handle_send_timer(const boost::system::error_code& error);
        void handle_send_timer();
        void send_timer_restart();

        void stop()
        {
            io_service_.stop();
        }

        private:
            boost::asio::io_service& io_service_;
            udp::socket socket_;
            udp::endpoint sender_endpoint_; 
            std::vector<udp::endpoint> clientList;
            //std::auto_ptr<boost::asio::io_service::work> busy_work;
            udp_buffer recv_buf;
            boost::asio::deadline_timer* send_timer_;
    };

Now I instantiate the class and thread like this:

    udp_server *udp_server_ptr=NULL;
    boost::asio::deadline_timer* dlineTimer=NULL;
    static void PluginInit_PostLoadCallback()
    {   
        _MESSAGE("NetworkPipe: PluginInit_PostLoadCallback called");

        if(!g_Interface->isEditor)
        {
            _MESSAGE("NetworkPipe: Starting UDP");
            udp_server_ptr = new udp_server(io_service, current_port);
            //dlineTimer = new boost::asio::deadline_timer(io_service);
            udp_thread = new boost::thread(boost::bind(&boost::asio::io_service::run, &io_service));         
            //
            _MESSAGE("NetworkPipe: UDP Started");
            NetworkPipeEnable = true;
        }
        else
        {
            _MESSAGE("NetworkPipe: Running in editor, not starting UDP");
        }    
    }

Now notice that dlineTimer is commented out above. If I enable that it ceases to function. The only way I can get the dlineTimer to function with this io service is to create it during the udp_server::handle_receive_from call. I think this is because it is running inside the other thread. So for some reason the deadline_timer object does not like being created outside the thread it needs to run inside.

Now, in order to communicate to the main thread I use concurrent_queue objects. So these allow me to send messages in and out of the thread pretty simply. I could theoretically run the dlineTimer inside its own thread and use the output queue to manage its activity. However, I like the simplicity of having is in the same thread as the udp_server. For instance the udp_server object keeps track of clients in a vector. When the deadline_timer expires I cycle through the known clients and send them messages. Then I restart the timer. This makes my response independent of the udp packets that are sent to the server. So when packets arrive they are put on a queue for another part of the process. Then later data is placed on the output queue and the deadline_timer processes those responses and sends them to the appropriate clients.

So my main question is:

How do I more cleanly create the deadline_timer object using the same thread and same io_service as the udp_server object?

At a glance, I can't see any problem with the way you create the `deadline_timer`. Perhaps, the problem is in some other place that you don't show? What do you mean by saying "it ceases to function"? What behavior do you observe exactly? — Igor R., Mar 19 '13 at 20:46
your _MESSAGE macro invokes undefined behavior because identifiers beginning with an underscore and a capital letter are [reserved for the implementation](http://stackoverflow.com/a/228797/283302). — Sam Miller, Mar 19 '13 at 21:15
What platform? Note that Windows platforms use a [thread per io_service to emulate asynchroncity](http://www.boost.org/doc/libs/release/doc/html/boost_asio/overview/implementation.html) for deadline_timer objects. This may be related as you seem to be writing some sort of plugin. — Sam Miller, Mar 19 '13 at 21:29
@IgorR. Sorry, this is a plugin for OBSE which itself is a plugin for the game Oblivion. What I mean is when it starts with the deadline_timer started it reaches a certain point in the startup of the plugin where it fails to attach to some system DLLs. I cannot trace what happens for some reason. So I figured I had something designed wrong. — Demolishun, Mar 19 '13 at 23:05
@SamMiller, The _MESSAGE macro is a logging mechanism for the plugin API. I can't really change how that works. If it is not future proof I am not really concerned because the future of this code is tied to a proprietary code base. The platform is Windows 7 64 bit, the compiler is VS2010, and the boost version is Boost 1.49 it looks like. I really need this code to operate in a thread as that allows the networking to be asynchronous from the game engine. — Demolishun, Mar 19 '13 at 23:08
You know, I just tried commenting out the udp_server creation and it still stops the game from starting correctly. On top of that it also causes some sort of cascade failure of the game from loading any DLLs. So my guess is that it is interrupting some sort of signal handling. This may be by design and there might not be anything I can do about this. So perhaps the issue has nothing to do with boost or threading at all. I guess to really test this it would be good to setup a standalone program. I think having it work at all may be a miracle. :) — Demolishun, Mar 19 '13 at 23:47
You should edit your question with the platform, compiler, boost version, and whatever plugin environment details are relevant. It might also be useful to add a tag for this Plugin environment, if one exists on SO. — Sam Miller, Mar 20 '13 at 22:40

Demolishun · Accepted Answer · 2013-03-21T16:29:13.220

Okay, I was thinking about this really stupidly.

First the deadline_timer needs to be completely inside the thread I want it to time in. That means it needs to be created inside the thread.
Second I need to define the function called in the thread loop and not set it to the io_service::run function. So I made it the udp_server::start function. Inside the start call I create my deadline_timer.

So here is the class:

    class udp_server
    {
    public:
        udp_server(boost::asio::io_service& io_service, short port)
            : io_service_(io_service),
              socket_(io_service_, udp::endpoint(boost::asio::ip::address_v4::from_string(current_address), port))//, // udp::v4()  
        {
            // start udp receive
            socket_.async_receive_from(
                boost::asio::buffer(recv_buf), sender_endpoint_,
                boost::bind(&udp_server::handle_receive_from, this,
                  boost::asio::placeholders::error,
                  boost::asio::placeholders::bytes_transferred));  

            send_timer_ = NULL;            
        }

        ~udp_server(){
            io_service_.stop();
            if(send_timer_){
                send_timer_->cancel();
                delete send_timer_;
            }
        }

        void start();

        void startSendTimer();

        void handle_send_to(const boost::system::error_code& error, size_t bytes_recvd);
        void handle_receive_from(const boost::system::error_code& error, size_t bytes_recvd);

        void handle_send_timer();
        void send_timer_restart();

        void stop()
        {
            io_service_.stop();
        }

        private:
            boost::asio::io_service& io_service_;
            udp::socket socket_;
            udp::endpoint sender_endpoint_; 
            std::vector<udp::endpoint> clientList;               
            udp_buffer recv_buf;
            boost::asio::deadline_timer* send_timer_;                
    };

Here are the relevant functions:

    void udp_server::start(){
        // startup timer
        startSendTimer();

        // run ioservice
        io_service_.run();
    }

    void udp_server::startSendTimer(){            
        // start send timer 
        if(!send_timer_)
            send_timer_ = new boost::asio::deadline_timer(io_service_, boost::posix_time::milliseconds(500));
        send_timer_restart();
    }

    void udp_server::send_timer_restart(){    
        if(send_timer_){
            // restart send timer
            send_timer_->expires_from_now(boost::posix_time::milliseconds(500));
            send_timer_->async_wait(boost::bind(&udp_server::handle_send_timer, this));
        }
    }        

    void udp_server::handle_send_timer(){            
        for(std::vector<udp::endpoint>::iterator itr = clientList.begin(); itr != clientList.end(); ++itr){
            socket_.async_send_to(
                boost::asio::buffer("heart beat", strlen("heart beat")), *itr,              
                boost::bind(&udp_server::handle_send_to, this,
                boost::asio::placeholders::error,
                boost::asio::placeholders::bytes_transferred));
        }

        send_timer_restart(); 
    }

So I was thinking about this all wrong in the first place. I need to define my starting point of where the thread begins execution. The I can create the objects that need to reside in that thread inside the thread.

The udp_server is now started like this:

    static void PluginInit_PostLoadCallback()
    {   
        _MESSAGE("NetworkPipe: PluginInit_PostLoadCallback called");

        if(!g_Interface->isEditor)
        {
            _MESSAGE("NetworkPipe: Starting UDP");
            udp_server_ptr = new udp_server(io_service, current_port);                 
            udp_thread = new boost::thread(boost::bind(&udp_server::start, udp_server_ptr));         
            _MESSAGE("NetworkPipe: UDP Started");
            NetworkPipeEnable = true;
        }
        else
        {
            _MESSAGE("NetworkPipe: Running in editor, not starting UDP");
        }    
    }

The deadline_timer creation occurs within the udp_thread now. Creating the deadline_timer object in the main thread would cause the program to fail to load properly.

It's not obvious to me how this solves your problem. All you've done is delay the instantiation of the internal thread the Asio library uses to emulate asynchronous `deadline_timer` operations on Windows platforms. Is there some thread creation restriction with this plugin API you are using? — Sam Miller, Mar 20 '13 at 22:39
I did not explain very well so I added the thread creation code. This should explain how the deadline_timer is being created inside the thread itself. If I create/allocate the deadline_timer in the main thread it causes the program to fail to load. I really cannot explain why it works. It does however, answer the question on how to more cleanly create the object inside the new thread. — Demolishun, Mar 21 '13 at 16:32
Is there some online documentation for this plugin API that may help future readers who find this question? The restrictions here seem unrelated to Boost.Asio. — Sam Miller, Mar 25 '13 at 02:21
Not really. There is the code and example project to show how to use the API. Yes, I think the issue is related to the existing application (which I cannot change) not liking the deadline timer. I will change the title to reflect that boost is involved, but it is not the issue. — Demolishun, Apr 01 '13 at 02:46

OBSE and Boost.Asio: Threaded async UDP server with deadline_timer on the same io_service

1 Answers1