PySide wait for signal from main thread in a worker thread

Question

I decided to add a GUI to one of my scripts. The script is a simple web scraper. I decided to use a worker thread as downloading and parsing the data can take a while. I decided to use PySide, but my knowledge of Qt in general is quite limited.

As the script is supposed to wait for user input upon coming across a captcha I decided it should wait until a QLineEdit fires returnPressed and then send it's content to the worker thread so it can send it for validation. That should be better than busy-waiting for the return key to be pressed.

It seems that waiting for a signal isn't as straight forward as I thought it would be and after searching for a while I came across several solutions similar to this. Signaling across threads and a local event loop in the worker thread make my solution a bit more complicated though.

After tinkering with it for several hours it still won't work.

What is supposed to happen:

• Download data until refered to captcha and enter a loop
• Download captcha and display it to the user, start QEventLoop by calling self.loop.exec_()
• Exit QEventLoop by calling loop.quit() in a worker threads slot which is connected via self.line_edit.returnPressed.connect(self.worker.stop_waiting) in the main_window class
• Validate captcha and loop if validation fails, otherwise retry the last url which should be downloadable now, then move on with the next url

What happens:

• ...see above...

• Exiting QEventLoop doesn't work. self.loop.isRunning() returns False after calling its exit(). self.isRunning returns True, as such the thread didn't seem to die under odd circumstances. Still the thread halts at the self.loop.exec_() line. As such the thread is stuck executing the event loop even though the event loop tells me it is not running anymore.

• The GUI responds as do the slots of the worker thread class. I can see the text beeing send to the worker thread, the status of the event loop and the thread itself, but nothing after the above mentioned line gets executed.

The code is a bit convoluted, as such I add a bit of pseudo-code-python-mix leaving out the unimportant:

class MainWindow(...):
    # couldn't find a way to send the text with the returnPressed signal, so I
    # added a helper signal, seems to work though. Doesn't work in the
    # constructor, might be a PySide bug?
    helper_signal = PySide.QtCore.Signal(str)
    def __init__(self):
        # ...setup...
        self.worker = WorkerThread()
        self.line_edit.returnPressed.connect(self.helper_slot)
        self.helper_signal.connect(self.worker.stop_waiting)

    @PySide.QtCore.Slot()
    def helper_slot(self):
        self.helper_signal.emit(self.line_edit.text())

class WorkerThread(PySide.QtCore.QThread):
    wait_for_input = PySide.QtCore.QEventLoop()

    def run(self):
        # ...download stuff...
        for url in list_of_stuff:
            self.results.append(get(url))

    @PySide.QtCore.Slot(str)
    def stop_waiting(self, text):
        self.solution = text
        # this definitely gets executed upon pressing return
        self.wait_for_input.exit()

    # a wrapper for requests.get to handle captcha
    def get(self, *args, **kwargs):
        result = requests.get(*args, **kwargs)
        while result.history: # redirect means captcha
            # ...parse and extract captcha...
            # ...display captcha to user via not shown signals to main thread...

            # wait until stop_waiting stops this event loop and as such the user
            # has entered something as a solution
            self.wait_for_input.exec_()

            # ...this part never get's executed, unless I remove the event
            # loop...

            post = { # ...whatever data necessary plus solution... }
            # send the solution
            result = requests.post('http://foo.foo/captcha_url'), data=post)
        # no captcha was there, return result
        return result

frame = MainWindow()
frame.show()
frame.worker.start()
app.exec_()

Answer 1

What you are describing looks ideal for QWaitCondition.

Simple example:

import sys
from PySide import QtCore, QtGui

waitCondition = QtCore.QWaitCondition()
mutex = QtCore.QMutex()

class Main(QtGui.QMainWindow):
    def __init__(self, parent=None):
        super(Main, self).__init__()

        self.text = QtGui.QLineEdit()
        self.text.returnPressed.connect(self.wakeup)

        self.worker = Worker(self)
        self.worker.start()

        self.setCentralWidget(self.text)

    def wakeup(self):
        waitCondition.wakeAll()

class Worker(QtCore.QThread):
    def __init__(self, parent=None):
        super(Worker, self).__init__(parent)

    def run(self):
        print "initial stuff"

        mutex.lock()
        waitCondition.wait(mutex)
        mutex.unlock()

        print "after returnPressed"

if __name__=="__main__":      
    app = QtGui.QApplication(sys.argv)
    m = Main()
    m.show()
    sys.exit(app.exec_())