Execute multiple functions in parallel in python

1

Hi everyone,

I’m trying to set up a Python script to run 2 functions in parallel (process_one_run and expt.justtask) ; I read about Thread Pool executor and tried to implement it as pasted below.

But I’m stuck as it actually run the expt.justtask and then; only if I press escape it starts running the process one run. Besides, I would like to use one of the output of the process one run and feed it into justtask.

Do you have any tips, recommendation of what I’m doing wrong?

Thank you very much in advance.

with concurrent.futures.ThreadPoolExecutor() as executor:
running_tasks = [executor.submit(task) for task in [receiver.process_one_run, expt.justtask(0,0)]]

2

Hi @Lysianne,

Here is an example of submitting multiple functions in parallel:

import concurrent.futures
import time

def function_1():
    print("Function 1 started")
    time.sleep(2)
    print("Function 1 completed")

def function_2():
    print("Function 2 started")
    time.sleep(4)
    print("Function 2 completed")

def function_3():
    print("Function 3 started")
    time.sleep(1)
    print("Function 3 completed")

# Using ThreadPoolExecutor to execute functions in parallel
with concurrent.futures.ThreadPoolExecutor() as executor:
    # Submit functions for execution
    futures = [executor.submit(function_1), executor.submit(function_2), executor.submit(function_3)]

    # Wait for all functions to complete
    concurrent.futures.wait(futures)

    # Alternatively, you can retrieve results if functions return values
    # results = [future.result() for future in futures]
    # print(results)

Without knowing more about what your function is, it may be hard for us to debug your issue precisely.

Best,
Steven

3

Oh my gosh you fixed it !!
Thank you so much it works perfectly now !

1 Like
4

Hi , I’m noticing a new issue with Psychopy , which I think is related to this parallel processes.

It does not return errors anymore, and for examples when I do a mistake in my line of code, it does not mention anything as the other piece of the parallel process keeps running. I tried to set up a log files with different level of warning but it still does not report when the step is not working. Any idea of how to solve this?

THank you !