simulation.batch_simulations

Methods for running simulations on AWS Batch.

Functions

check_and_save_job_logs(bucket, series_name, ...)	Check job status and save CloudWatch logs for successfully completed jobs.
copy_simulation_outputs(bucket, series_name, ...)	Copy simulation outputs from where they are saved to pipeline file structure.
generate_configs_from_file(bucket, ...)	Generate configs from given file for each seed and save to S3 bucket.
generate_configs_from_template(bucket, ...)	Generate configs for each given file for each seed and save to S3 bucket.
register_and_run_simulations(bucket, ...)	Register job definitions and submit jobs to AWS Batch.

generate_configs_from_file(bucket: str, series_name: str, timestamp: str, random_seeds: list[int], config_file: str) → None

Generate configs from given file for each seed and save to S3 bucket.

Parameters:

• bucket – Name of S3 bucket for input and output files.

• series_name – Name of simulation series.

• timestamp – Current timestamp used to organize input and outfile files.

• random_seeds – Random seeds for simulations.

• config_file – Path to the config file.

generate_configs_from_template(bucket: str, series_name: str, timestamp: str, random_seeds: list[int], config_files: list[str], pattern: str, key_map: dict[str, str]) → list[str]

Generate configs for each given file for each seed and save to S3 bucket.

Parameters:

• bucket – Name of S3 bucket for input and output files.

• series_name – Name of simulation series.

• timestamp – Current timestamp used to organize input and outfile files.

• random_seeds – Random seeds for simulations.

• config_files – Path to the config files.

• pattern – Regex pattern to find config condition value.

• key_map – Map of condition values to file keys.

Returns:

List of config groups.

register_and_run_simulations(bucket: str, series_name: str, timestamp: str, group_keys: list[str], aws_account: str, aws_region: str, aws_user: str, image: str, vcpus: int, memory: int, job_queue: str, job_size: int) → list[str]

Register job definitions and submit jobs to AWS Batch.

Parameters:

• bucket – Name of S3 bucket for input and output files.

• series_name – Name of simulation series.

• timestamp – Current timestamp used to organize input and outfile files.

• group_keys – List of config group keys.

• aws_account – AWS account number.

• aws_region – AWS region.

• aws_user – User name prefix for job name and image.

• image – Image name and version.

• vcpus – Number of vCPUs for each job.

• memory – Memory for each job.

• job_queue – Job queue.

• job_size – Job array size.

Returns:

List of job ARNs.

check_and_save_job_logs(bucket: str, series_name: str, job_arns: list[str], aws_region: str) → None

Check job status and save CloudWatch logs for successfully completed jobs.

Parameters:

• bucket – Name of S3 bucket for input and output files.

• series_name – Name of simulation series.

• job_arns – List of job ARNs.

• aws_region – AWS region.

copy_simulation_outputs(bucket: str, series_name: str, source_template: str, n_replicates: int, condition_keys: dict[str, str] | None = None) → None

Copy simulation outputs from where they are saved to pipeline file structure.

Parameters:

• bucket – Name of S3 bucket for input and output files.

• series_name – Name of simulation series.

• source_template – Template string for source output files.

• n_replicates – _Number of simulation replicates.

• condition_keys – Map of source to target condition keys.