在Julia中使用分布式和SharedArrays

Question

使用Julia 1.2。我遇到了一个很大的问题，涉及电力市场上的许多生产者，并且希望使用分布式计算，以便更快地解决问题。但是我不确定布局应该是什么，我不希望工作人员一个接一个地写到共享数组中。而且我的代码仍然运行得非常慢...

有人可以检查下面的布局是否正确？

 using Distributed
 n_workers = workers()
 if n_workers[end]<3
    addprocs(3)
 end


 @everywhere using JuMP, Gurobi, DataFrames, CSV, DelimitedFiles, SharedArrays

 ###I initialize many shared arrays into which I want my results to go into
 Results=SharedArray{Float64,2}(length(vec4), 38)
 Injection_IndPros=SharedArray{Float64,3}(length(vec4)*length(day_steps), length(time_steps), tot_consumers)  
 Injection_Prosumers=SharedArray{Float64,2}(length(vec4)*length(day_steps), length(time_steps))
 Prosumer_caps=SharedArray{Float64,2}(length(vec4)*4, tot_households)
 Year_behav=SharedArray{Float64,2}(length(vec4)*length(time_steps),6)

 @sync @distributed for iter4=1:length(vec4)

     ###Call in a function which returns results 

     ####Then I carry out some calculations on these results (here I also use normal matrices, I do not use @everywhere here)

     ####Then I put my calculated results into the shared arrays for e.g.
     Results[iter4,:]=[w_max_with[1]/8, w_max_inj[1]/8, obj, Net_injection, Invest_PV, Totcap_PV, Invest_PVINV, Totcap_PVINV, Totgen_PV, Maxgen_PV, Curtailment_PV, Invest_BAT, Totcap_BAT, Invest_BATINV, Totcap_BATINV, Invest_WIND, Totcap_WIND, Totgen_WIND, Invest_Conv_Base, cap_conv_opt[1], Invest_Conv_Mid, cap_conv_opt[2], Invest_Conv_Peak, cap_conv_opt[3],Totgen_CONVBASE, Totgen_CONVMID, Totgen_CONVPEAK, Totgen_CONV, Totgen_ALL, Genshare_PV, Genshare_WIND, Genshare_CONVBASE, Genshare_CONVMID, Genshare_CONVPEAK, Yearavg_elec_price, Max_inj_pos, Avg_INVtoPV_ratio, Avg_INVtoBAT_ratio]


end #here ends the distributed for loop

Answer 1

问问题时，您应该提供一个最小的工作示例，而不仅仅是复制粘贴您的一些生产代码。

[这是SharedArrays使用的典型模式，类似于您可能尝试做的事情：

using SharedArrays, Distributed

addprocs(3)

@everywhere using SharedArrays, Distributed

sx = SharedArray(Array(reshape(1:30,5,6)))

@everywhere function myjob(v)
    sum(v)
end

@sync @distributed for i in 1:5
    s = myjob(sx[i,1:end-1]) + 1000*myid()
    sx[i,end] = s
end