私信  •  关注

Rouzip

Rouzip 最近创建的主题
Rouzip 最近回复了
7 年前
回复了 Rouzip 创建的主题 » 多进程python大文件读取
  1. 你可以阅读缓冲区 here .
  2. 如果你的记忆力很小,你可以试着强迫GC:
import gc

SIZE = 1024*768  
MEMOSIZE = 1024  # your memory size
with open('xxx', 'rb') as fp:  # open the file
    i = 0  # remember the number to gc in time
    queue = []
    while True:
        if (i*(SIZE-1) < MEMOSIZE):
            x = fp.read(SIZE)  # if your image is single channel
            queue.append(x)
            # do something
        else:
            del queue
            gc.collect()
7 年前
回复了 Rouzip 创建的主题 » 使用python的列表中元素的总和

你是说这样吗?

from collections import Counter
a = [(0.14409883378622257, 'count_90'), (0.1274820635854658, 'count_60'), (0.10362930186446877, 'count_30'), (0.017066033814948037, 'country_code_destination_ID'), (0.014104260612047153,
                                                                                                                                                                     'country_code_destination_US'), (0.011414953372550486, 'SACC_MARKET_SEGMENT_BDZ'), (0.010291762087645236, 'leading_bu_PP'), (0.009979426898654558, 'SACC_MARKET_SEGMENT_IT3')]
cnt = Counter()
for item in a:
    cnt[item[1]] += item[0]