python - Multi Threading Makes Process Slower -

this question has answer here:

• python multi-threading slower serial? 3 answers

i have following task make faster via multi threading (python3).

import threading, time  q = []  def fill_list():     global q     while true:         q.append(1)         if len(q) >= 1000000000:             return 

the first main not utilize multithreading:

t1 = time.clock() fill_list() tend = time.clock() - t1 print(tend) 

and results in 145 seconds of run time.

the second invokes 2 threads:

t1 = time.clock() thread1 = threading.thread(target=fill_list, args=()) thread2 = threading.thread(target=fill_list, args=())  thread1.start() thread2.start()  thread1.join() thread2.join()  tend = time.clock() - t1 print(tend) 

this takes 152 seconds complete.

finally, added third thread.

t1 = time.clock() thread1 = threading.thread(target=fill_list, args=()) thread2 = threading.thread(target=fill_list, args=()) thread3 = threading.thread(target=fill_list, args=())  thread1.start() thread2.start() thread3.start()  thread1.join() thread2.join() thread3.join()  tend = time.clock() - t1 print(tend) 

and took 233 seconds complete.

obviously more threads add, longer process takes, though not sure why. fundamental misunderstanding of multithreading, or there bug in code repeating task multiple times instead of contributing same task?

answers 1 , 2.

first of all, task cpu-bound, , in python process 1 thread may running cpu-bound python code @ given time (this due global interpreter lock: https://wiki.python.org/moin/globalinterpreterlock ). since costs quite bit of cpu switch threads (and more threads have, more have pay cost), program doesn't speed up: slows down.

second, no matter language you're using, you're modifying 1 object (a list) multiple threads. guarantee not corrupt object, access must synchronized. in other words, only 1 thread may modifying @ given time. python automatically (thanks in part aforementioned gil), in lower-level language c++ you'd have use lock or risk memory corruption.

the optimal way parallelize tasks across threads ensure threads isolated possible. if access shared objects, should read-only, , cross-thread writes should happen infrequently possible, through thread-aware data structures such message queues.

(which why performant parallel systems erlang , clojure place such high emphasis on immutable data structures , message-passing)