python3之多程序全解 生產者消費者模型

2021-09-14 07:27:34 字數 4681 閱讀 1562

-subprocess

-完全跳過執行緒,使用程序

-是派生程序的主要替代方案

-python2.4後引入

-multiprocessing

-使用threading藉口派生,使用子程序

-允許為多核或者多cpu派生程序,介面跟theading非常相似

-python2.6

-concurrent.futures

-新的非同步執行模組

-任務級別的操作

-python3.2後引入

-程序間通訊(interprocesscommunication,ipc)

-程序之間無任何共享狀態

-程序的建立

-直接生成process例項物件

import multiprocessing

from time import sleep,ctime

def clock(interval):

while true:

print("the time is %s" % ctime())

sleep(interval)

if __name__ == '__main__':

p = multiprocessing.process(target=clock,args=(5,))

p.start()

while true:

print("sleeping.....")

sleep(1)

執行:sleeping.....

sleeping.....

sleeping.....

the time is mon apr 1 21:06:44 2019

sleeping.....

sleeping.....

sleeping.....

sleeping.....

the time is mon apr 1 21:06:49 2019

sleeping.....

sleeping.....

sleeping.....

sleeping.....

sleeping.....

the time is mon apr 1 21:06:54 2019

sleeping.....

-派生子類

import multiprocessing

from time import sleep,ctime

class clockprocess(multiprocessing.process):

'''兩個函式比較重要

1、init建構函式

2、run

'''def __init__(self,interval):

super().__init__()

self.interval = interval

def run(self):

while true:

print("the time is %s" % ctime())

sleep(self.interval)

if __name__ == '__main__':

p = clockprocess(3)

p.start()

while true:

print("sleeping....")

sleep(1)

執行:sleeping....

the time is mon apr 1 21:31:14 2019

sleeping....

sleeping....

sleeping....

the time is mon apr 1 21:31:17 2019

sleeping....

-在os中檢視pid,ppid以及他們的關係

from multiprocessing import process

import os

def info(title):

print(title)

print('module name:',__name__)

#得到父親程序的id

print('parent process:',os.getppid())

#得到本身程序的id

print('process id :',os.getppid())

def f(name):

info('function f')

print('hello',name)

if __name__ == '__main__':

info('main line')

p = process(target=f,args=('bob',))

p.start()

執行:main line

module name: __main__

parent process: 2796

process id : 2796

function f

module name: __mp_main__

parent process: 6544

process id : 6544

hello bob

-生產者消費者模型

-joinablequeue

-設定哨兵

import multiprocessing

from time import ctime

#設定哨兵問題

def consumer(input_q):

print("into consumer:",ctime())

while true:

#處理項

item = input_q.get()

if item is none:

break

#此處替換為有用的工作

print("pull",item,'oout of q')

#發出訊號通知任務完成

# input_q.task_done()

#此句未執行,因為q.join()收集到四個task_done()訊號後,主程序啟動,未等到print此句

print("out of consumer:",ctime())

def producer(sequence,output_q):

for item in sequence:

print("into procuder:", ctime())

output_q.put(item)

print("put",item,"into q")

print("out of procuder:",ctime())

if __name__ == '__main__':

#做完了通知發個訊息~

q = multiprocessing.joinablequeue()

#執行消費者程序

cons_p1 = multiprocessing.process(target=consumer,args=(q,))

# cons_p.daemon = true

cons_p1.start()

cons_p2 = multiprocessing.process(target=consumer, args=(q,))

cons_p2.start()

#生產多個項,sequence代表要傳送給消費者的項xulie

#在實踐中,這可能是生成器的輸出或通過一些其他方式生產出來

sequence = [1,2,3,4]

producer(sequence,q)

q.put(none)

q.put(none)

#等待所有項被處理

cons_p1.join()

cons_p2.join()

執行:into procuder: mon apr 1 22:28:28 2019

put 1 into q

into procuder: mon apr 1 22:28:28 2019

put 2 into q

into procuder: mon apr 1 22:28:28 2019

put 3 into q

into procuder: mon apr 1 22:28:28 2019

put 4 into q

out of procuder: mon apr 1 22:28:28 2019

into consumer: mon apr 1 22:28:28 2019

pull 1 oout of q

pull 2 oout of q

pull 3 oout of q

pull 4 oout of q

out of consumer: mon apr 1 22:28:28 2019

into consumer: mon apr 1 22:28:28 2019

out of consumer: mon apr 1 22:28:28 2019

Python3之多程序join daemon

import os import time from multiprocessing import process,joinablequeue defproducer0 q for i in range 5 print s 生產了 s os.getpid i q.put i q.join defpr...

python之多程序

要讓python實現多程序 multiprocessing 我們先來了解作業系統相關知識。unix 和 linux 作業系統提供了乙個 fork 函式系統呼叫,它非常特殊。普通的函式,呼叫一它次,執行一次,但是 fork 函式呼叫一次執行兩次,因為作業系統自動把當前程序 稱為父程序 複製了乙份 稱為...

python併發之多程序

一 multiprocessing模組介紹 python中的多執行緒無法利用多核優勢,如果想要充分地使用多核cpu的資源 os.cpu count 檢視 在python中大部分情況需要使用多程序。python提供了multiprocessing。multiprocessing模組用來開啟子程序,並在...