Runy Open3.popen3 Entering input into the subprocess from the command-line

Question

Goal: I am writing a workflow command-line program in ruby that sequentially executes other programs on the UNIX shell, some of which require the user to enter input.

Problem: Although I can successfully handle the stdout and stderr thanks to this helpful blog post by Nick Charlton, I am however stuck on capturing user input and passing it into the sub-processes via the command line. The code is as follows:

Method

module CMD
  def run(cmd, &block)
    Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
      Thread.new do # STDOUT
        until (line = stdout.gets).nil? do
          yield nil, line, nil, thread if block_given?
        end
      end

      Thread.new do # STDERR 
        until (line = stderr.gets).nil? do
          yield nil, nil, line, thread if block_given?
        end
      end

      Thread.new do # STDIN
        # ????? How to handle
      end

      thread.join
    end
  end
end

Calling the method

This example calls the shell command units which prompts the user to enter a unit of measurement and then prompts for a unit to convert to. This is how it would look in the shell

> units
586 units, 56 prefixes        # stdout
You have: 1 litre             # user input
You want: gallons             # user input
* 0.26417205                  # stdout
/ 3.7854118                   # stdout

When I run this from my program I expect to be able to interact with it in exactly the same way.

unix_cmd = 'units'
run unix_cmd do | stdin, stdout, stderr, thread|
  puts "stdout #{stdout.strip}" if stdout
  puts "stderr #{stderr.strip}" if stderr
  # I'm unsure how I would allow the user to
  # interact with STDIN here?
end

Note: Calling the run method this way allows the user to be able to parse the output, control process flow and add custom logging.

From what I've gathered about STDIN, the snippet below is as close as I've come in understanding how to handle STDIN, there are clearly some gaps in my knowledge because I'm still unsure how to integrate this into my run method above and pass the input into the child process.

# STDIN: Constant declared in ruby
# stdin: Parameter declared in Open3.popen3
Thread.new do 
    # Read each line from the console
    STDIN.each_line do |line|
       puts "STDIN: #{line}" # print captured input 
       stdin.write line      # write input into stdin
       stdin.sync            # sync the input into the sub process
       break if line == "\n"
    end
end

Summary: I wish to understand how to handle user input from the command-line via the Open3.popen3 method so that I can allow users to enter data into various sequence of sub-commands called from my program.

I'd also love to know how to do this – Charles Finkel Apr 18 '15 at 01:13 — Charles Finkel, Apr 18 '15 at 01:13

score 3 · Answer 1 · answered Apr 18 '15 at 02:42

Here's something that should work:

module CMD
  def run(cmd, &block)
    Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
      Thread.new do # STDOUT
        until (line = stdout.gets).nil? do
          yield nil, line, nil, thread if block_given?
        end
      end

      Thread.new do # STDERR 
        until (line = stderr.gets).nil? do
          yield nil, nil, line, thread if block_given?
        end
      end

      t = Thread.new { loop { stdin.puts gets } }

      thread.join
      t.kill
    end
  end
end

I've just added two lines to your original run method: t = Thread.new { loop { stdin.puts gets } }, and t.kill.

This doesn't seem to work I'm afraid, I tested it by calling the method `CMD.run 'units'` did you manage to get it working? — ImaginateWayne, Apr 18 '15 at 08:11

score 2 · Accepted Answer · edited May 23 '17 at 12:24

After a lot of reading about STDIN as well as some good old trial and error, I discovered an implementation not to dissimilar to Charles Finkel's answer but with some subtle differences.

require "open3"

module Cmd
  def run(cmd, &block)
    Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
      # We only need to check if the block is provided once
      # rather than every cycle of the loop as we were doing 
      # in the original question.

      if block_given?
        Thread.new do
          until (line = stdout.gets).nil? do
            yield line, nil, thread
          end
        end

        Thread.new do
          until (line = stderr.gets).nil? do
            yield nil, line, thread
          end
        end
      end

      # $stdin.gets reads from the console
      #
      # stdin.puts writes to child process
      #
      # while thread.alive? means that we keep on
      # reading input until the child process ends
      Thread.new do
        stdin.puts $stdin.gets while thread.alive?
      end

      thread.join
    end
  end
end

include Cmd

Calling the method like so:

  run './test_script.sh' do | stdout, stderr, thread|
    puts "#{thread.pid} stdout: #{stdout}" if stdout
    puts "#{thread.pid} stderr: #{stderr}" if stderr
  end

Where test_script.sh is as follows:

echo "Message to STDOUT"
>&2 echo "Message to STDERR"
echo "enter username: "
read username
echo "enter a greeting"
read greeting
echo "$greeting $username"
exit 0

Produces the following successful output:

25380 stdout: Message to STDOUT
25380 stdout: enter username:
25380 stderr: Message to STDERR
> Wayne
25380 stdout: enter a greeting
> Hello
25380 stdout: Hello Wayne

Note: You will notice the stdout and stderr don't appear in order, this is a limitation I'm yet to solve.

If you're interested in knowing more about stdin it's worth reading the following answer to the question - What is the difference between STDIN and $stdin in Ruby?

This will throw exception `closed stream (IOError)` on the second call to `run`. You need to kill the stdin thread as in Charles Finkel's answer. — FuzzY, Oct 06 '21 at 17:22

Runy Open3.popen3 Entering input into the subprocess from the command-line

2 Answers2