Slurm：重命名文件夹中的特定行

Question

我正在尝试使用 for 循环来重复更改文件的行。这是我的代码。

#!/bin/bash
# Example SLURM Batch Script for Running Multiple Tasks

# SLURM Configuration
#SBATCH --nodes=1
#SBATCH --time=30-00:00:00
#SBATCH --mem-per-cpu=1gb
#SBATCH --partition=example_partition

# Define the number of tasks in the array
#SBATCH --array=1-2

# Print out some information about the environment
echo "Hostname: $(hostname)"
echo "Starting at: $(date)"
echo "Running on hosts: $SLURM_NODELIST"
echo "Running on $SLURM_NNODES nodes."
echo "Running on $SLURM_NPROCS processors."
echo "Current working directory: $(pwd)"

# Export environment variable
export MY_VARIABLE=1234

# Loop through tasks
for ((i=1; i<=$SLURM_ARRAY_TASK_COUNT; i++)); do
    # Format the index with leading zeros (e.g., 001, 002, ..., 030)
    index=$(printf "%04d" $i)

    # Modify parameters for each task
    sed -i "s/input_file = .*/input_file = task$index.in/" task_config.ini

done

echo "Program completed with exit code $? at: $(date)"

仍然没有对参数进行任何更改。我想使用Slurm，但不能使用'sed'。有兼容的 slurm 功能吗？

我的想法是使用集群并反复更改线路。我没有得到什么？有 Slurm 方法来执行此操作吗？

Answer 1

这只是

sed

部分：

task_num = $SLURM_ARRAY_TASK_COUNT
for ((i=1; i<=$task_num; i++)); 
    do echo $i; 
    index=$(printf "%04d" $i); 
    echo $index;  
    sed  "s//input_file = .*/input_file = task$index.ini/g" master_input.ini > task$index.ini; 

done

我想这仍然不是为数组作业提供工作输入的正确方法，但至少它应该解决

sed

问题。

编辑

有几种方法可以使用

SLURM_ARRAY_TASK_ID

为程序提供不同的输入/参数。

1. U。卡尔加里
2. U。佛罗里达州

#!/bin/sh
#SBATCH --job-name=mega_array   # Job name
#SBATCH --mail-type=ALL         # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH [email protected] # Where to send mail  
#SBATCH --nodes=1                   # Use one node
#SBATCH --ntasks=1                  # Run a single task
#SBATCH --mem-per-cpu=1gb           # Memory per processor
#SBATCH --time=00:10:00             # Time limit hrs:min:sec
#SBATCH --output=array_%A-%a.out    # Standard output and error log
#SBATCH --array=1-5                 # Array range
# This is an example script that combines array tasks with
# bash loops to process many short runs. Array jobs are convenient
# for running lots of tasks, but if each task is short, they
# quickly become inefficient, taking more time to schedule than
# they spend doing any work and bogging down the scheduler for
# all users. 
pwd; hostname; date

#Set the number of runs that each SLURM task should do
PER_TASK=1000

# Calculate the starting and ending values for this task based
# on the SLURM task and the number of runs per task.
START_NUM=$(( ($SLURM_ARRAY_TASK_ID - 1) * $PER_TASK + 1 ))
END_NUM=$(( $SLURM_ARRAY_TASK_ID * $PER_TASK ))

# Print the task and run range
echo This is task $SLURM_ARRAY_TASK_ID, which will do runs $START_NUM to $END_NUM

# Run the loop of runs for this task.
for (( run=$START_NUM; run<=END_NUM; run++ )); do
  echo This is SLURM task $SLURM_ARRAY_TASK_ID, run number $run
  #Do your stuff here
done

date

3. 浪人博客

#!/bin/bash
#SBATCH --job-name=myarrayjob
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --array=1-10

# Specify the path to the config file
config=/path/to/config.txt

# Extract the sample name for the current $SLURM_ARRAY_TASK_ID
sample=$(awk -v ArrayTaskID=$SLURM_ARRAY_TASK_ID '$1==ArrayTaskID {print $2}' $config)

# Extract the sex for the current $SLURM_ARRAY_TASK_ID
sex=$(awk -v ArrayTaskID=$SLURM_ARRAY_TASK_ID '$1==ArrayTaskID {print $3}' $config)

# Print to a file a message that includes the current $SLURM_ARRAY_TASK_ID, the same name, and the sex of the sample
echo "This is array task ${SLURM_ARRAY_TASK_ID}, the sample name is ${sample} and the sex is ${sex}." >> output.txt