How to do Parallel Input and Output Simultaneously in Python3?

Question

I need to design a script that uses the top portion of the terminal as output where some lines are printed after each second in an infinite loop, and the bottom portion keeps taking user input and also printing them in the above portion (among the regular periodic outputs).

In other words, I need to design a sort of shell.

I tried multithreading with the naive approach like this:

#!/usr/bin/python3

from math import acos
from threading import Thread
from random import choice
from time import sleep
from queue import Queue, Empty

commandQueue = Queue()

def outputThreadFunc():
    outputs = ["So this is another output","Yet another output","Is this even working"] # Just for demo
    while True:
        print(choice(outputs))
        try:
            inp = commandQueue.get(timeout=0.1)
            if inp == 'exit':
                return
            else:
                print(inp)
        except Empty:
            pass        
        sleep(1)

def inputThreadFunc():
    while True:
        command = input("> ") # The shell
        if command == 'exit':
            return
        commandQueue.put(command)

# MAIN CODE
outputThread = Thread(target=outputThreadFunc)
inputThread = Thread(target=inputThreadFunc)
outputThread.start()
inputThread.start()
outputThread.join()
inputThread.join()

print("Exit")

But as obviously expected, the output lines merge with the input lines as the user keeps typing.

Any ideas?

Use tkinter to design your UI - input on the bottom and output on the top. — wwii, May 27 '21 at 18:24
Does [Take input from tkinter to python script and output from python script to tkinter](https://stackoverflow.com/questions/62155548/take-input-from-tkinter-to-python-script-and-output-from-python-script-to-tkinte) answer your question?? — wwii, May 27 '21 at 18:27
Or: [Update data in a Tkinter-GUI with data from a second Thread](https://stackoverflow.com/questions/47934144/update-data-in-a-tkinter-gui-with-data-from-a-second-thread), [Update Tkinter GUI from a separate thread running a command](https://stackoverflow.com/questions/64287940/update-tkinter-gui-from-a-separate-thread-running-a-command), [How to run a function/thread in a different terminal window in python?](https://stackoverflow.com/questions/34810652/how-to-run-a-function-thread-in-a-different-terminal-window-in-python), ...? — wwii, May 27 '21 at 18:31
`Any ideas?` - doesn't seem like that question could have *best* answer. — wwii, May 27 '21 at 18:32
Check out the [python curses module](https://docs.python.org/3/howto/curses.html) — He3lixxx, May 30 '21 at 22:55
@tobias Yes, `ncurses` is allowed. Can I see an example of how I would use that module in my project? — Sohail Saha, Jun 01 '21 at 04:29
you are not divide your terminal, so yes it will all messed, doesn't matter with parallel yet, you'll need some sort of cli graphics, says ncurse, or many better py equivalent ( or rip a basic framework from them ) , likely still to refresh whole screen, — Jack Wu, Jun 01 '21 at 13:09
Does your program have to be threaded because of other things? It might be easier to just use the event loop/dispatching approach if your program doesn't have to be threaded. — Mia, Jun 06 '21 at 03:37

Anurag Regmi · Answer 1 · 2021-06-02T14:36:49.190

As discussed in comments, used curses library.

Update

used two subwin for input and output

#!/usr/bin/python3

import curses

from math import acos
from threading import Thread
from random import choice
from time import sleep
from queue import Queue, Empty


commandQueue = Queue()

stdscr = curses.initscr()
stdscr.keypad(True)

upperwin = stdscr.subwin(2, 80, 0, 0)
lowerwin = stdscr.subwin(2,0)

def outputThreadFunc():
    outputs = ["So this is another output","Yet another output","Is this even working"] # Just for demo
    while True:
        upperwin.clear()
        upperwin.addstr(f"{choice(outputs)}")
        try:
            inp = commandQueue.get(timeout=0.1)
            if inp == 'exit':
                return
            else:
                upperwin.addch('\n')
                upperwin.addstr(inp)
        except Empty:
            pass

        upperwin.refresh()
        sleep(1)
        


def inputThreadFunc():
    while True:
        global buffer

        lowerwin.addstr("->")

        command = lowerwin.getstr()

        if command:
            command = command.decode("utf-8")
            commandQueue.put(command)
            lowerwin.clear()

            lowerwin.refresh()
            if command == 'exit':
                return

            
        


# MAIN CODE
outputThread = Thread(target=outputThreadFunc)
inputThread = Thread(target=inputThreadFunc)
outputThread.start()
inputThread.start()
outputThread.join()
inputThread.join()

stdscr.keypad(False)
curses.endwin()
print("Exit")

Old Solution

I've edited your example to use getch insted of input

#!/usr/bin/python3

import curses
import datetime

from math import acos
from threading import Thread
from random import choice
from time import sleep
from queue import Queue, Empty

INFO_REFRESH_SECONDS = 1

commandQueue = Queue()
buffer = list()  # stores your input buffer
stdscr = curses.initscr()
stdscr.keypad(True)

def outputThreadFunc():
    outputs = ["So this is another output","Yet another output","Is this even working"] # Just for demo
    info = choice(outputs), datetime.datetime.now()
    while True:

        if datetime.datetime.now() - info[1] > datetime.timedelta(seconds=INFO_REFRESH_SECONDS):
            # refresh info after certain period of time

            info = choice(outputs), datetime.datetime.now()  # timestamp which info was updated

        inp = ''
        buffer_text = ''.join(buffer)
        try:
            command = commandQueue.get(timeout=0.1)
            if command == 'exit':
                return
            inp = f"\n{command}"
        except Empty:
            pass 
        output_string = f"{info[0]}{inp}\n->{buffer_text}"
        stdscr.clear()
        stdscr.addstr(output_string)
        stdscr.refresh()
        if inp:
            # to make sure you see the command
            sleep(1)
        


def inputThreadFunc():
    while True:
        global buffer

        # get one character at a time
        key = stdscr.getch()
        curses.echo()

        if chr(key) == '\n':
            command = ''.join(buffer)
            commandQueue.put(command)
            if command == 'exit':
                return
            buffer = []
        elif key == curses.KEY_BACKSPACE:
            
            if buffer:
                buffer.pop()
        else:
            buffer.append(chr(key))

            
        


# MAIN CODE
outputThread = Thread(target=outputThreadFunc)
inputThread = Thread(target=inputThreadFunc)
outputThread.start()
inputThread.start()
outputThread.join()
inputThread.join()

stdscr.keypad(False)
curses.endwin()
print("Exit")

How do I modify this so that the random lines I chose do not get erased after each run of the loop, and the input string in its incomplete form do not get mixed up with it? — Sohail Saha, Jun 01 '21 at 18:44
@CaptainWoof I've updated the answer. There is timestamp along with info and variable `INFO_REFRESH_SECONDS`. Alternatively you can start another thread to update info value just like i updated buffer in inputThread and displayed in outputThread — Anurag Regmi, Jun 02 '21 at 01:46
@CaptainWoof **Update** Used subwin one win for output and one for input. — Anurag Regmi, Jun 02 '21 at 11:56

score 1 · Answer 2 · answered May 27 '21 at 21:03

1

The simplest solution is to use two scripts; One, a server that prints the output, and the other, a client that sends the user's input to the server. Then you can use a standard solution like tmux to open the two scripts in two panes.

answered May 27 '21 at 21:03

HappyFace

3,439
2
24
43

score 0 · Answer 3 · answered Jun 02 '21 at 04:08

The two are merging because of the way the terminal writes to the output. It collects outputs in a buffer, and when the time is right it outputs everything at once. An easy fix would be to use a '\n' before each actual statement so that each new output is on a separate line.

#!/usr/bin/python3

                    .
                    .
                    .

            if inp == 'exit':
                return
            else:
                print("\n", inp) # CHANGE OVER HERE

                    .
                    .
                    .

        command = input("\n> ") # CHANGE OVER HERE
        if command == 'exit':
            return

                    .
                    .
                    .

print("Exit")

Beware that since two threads are running in parallel, the next output will print before you are done typing and pressing enter to the input (unless you can type really fast and have really fast reflexes). Hope this answers your question!

How to do Parallel Input and Output Simultaneously in Python3?

3 Answers3

Update

Old Solution