巧みにgroupbyを用いてDataframeフィルタリングパケットの効率が遅い問題を解決する
2200 ワード
元のコード:
最適化後のコード:
for name in list_valid_perfor_inventory:
time_stamp = time.time()
df_tmp1 = df_all_performance[df_all_performance['res_ins_id'] == name] ###170 , 2S
if df_tmp1.empty:
continue
del df_tmp1['res_ins_id']
print('choose time ')
print(str(time.time() - time_stamp))
time_stamp = time.time()
df_tmp1.to_csv(path_or_buf=os.path.join(cs.max_avg_busy_dir, str(name) + '.csv'))
print(str(time.time() - time_stamp))
最適化後のコード:
groups = df_all_performance.groupby('res_ins_id') ##
for name in list_valid_perfor_inventory:
time_stamp = time.time()
df_tmp1 = groups.get_group(name) ## , dataframe
if df_tmp1.empty:
continue
del df_tmp1['res_ins_id']
df_tmp1.to_csv(path_or_buf=os.path.join(cs.max_avg_busy_dir, str(name) + '.csv'))