Learn amendments in peppy
This vignette will show you how and why to use the amendments functionality of the peppy
package.
-
basic information about the PEP concept on the project website.
-
broader theoretical description in the amendments documentation section.
Problem/Goal
The example below demonstrates how and why to use amendments project attribute to, e.g. define numerous similar projects in a single project config file. This functionality is extremely convenient when one has to define projects with small settings discreptancies, like different attributes in the annotation sheet. For example libraries ABCD
and EFGH
instead of the original RRBS
.
examples_dir = "../tests/data/example_peps-cfg2/example_amendments1/"
sample_table = examples_dir + "sample_table.csv"
%cat $sample_table | column -t -s, | cat
sample_name protocol organism time file_path
pig_0h RRBS pig 0 source1
pig_1h RRBS pig 1 source1
frog_0h RRBS frog 0 source1
frog_1h RRBS frog 1 source1
Solution
This can be achieved by using amendments section of project_config.yaml
file (presented below). The attributes specified in the lowest levels of this section (here: sample_table
) overwrite the original ones. Consequently, a completely new set of settings is determined with just this value changed. Moreover, multiple amendments can be defined in a single config file and activated at the same time. Based on the file presented below, two subprojects will be defined: newLib
and newLib2
.
project_config_file = examples_dir + "project_config.yaml"
%cat $project_config_file
pep_version: "2.0.0"
sample_table: sample_table.csv
output_dir: $HOME/hello_looper_results
sample_modifiers:
derive:
attributes: [file_path]
sources:
source1: /data/lab/project/{organism}_{time}h.fastq
source2: /path/from/collaborator/weirdNamingScheme_{external_id}.fastq
project_modifiers:
amend:
newLib:
sample_table: sample_table_newLib.csv
newLib2:
sample_table: sample_table_newLib2.csv
Obviously, the amendments functionality can be combined with other peppy
package options, e.g. imply and derive sample modifiers. The derive modifier is used in the example considered here (derive
key in the sample_modifiers
section of the config file).
Files sample_table_newLib.csv
and sample_table_newLib2.csv
introduce different the library
attributes. They are used in the subprojects newLib
and newLib2
, respectively
sample_table = examples_dir + "sample_table_newLib.csv"
%cat $sample_table | column -t -s, | cat
sample_name protocol organism time file_path
pig_0h ABCD pig 0 source1
pig_1h ABCD pig 1 source1
frog_0h ABCD frog 0 source1
frog_1h ABCD frog 1 source1
sample_table = examples_dir + "sample_table_newLib2.csv"
%cat $sample_table | column -t -s, | cat
sample_name protocol organism time file_path
pig_0h EFGH pig 0 source1
pig_1h EFGH pig 1 source1
frog_0h EFGH frog 0 source1
frog_1h EFGH frog 1 source1
Code
Import peppy
and read in the project metadata by specifying the path to the project_config.yaml
from peppy import Project
p = Project(project_config_file)
An appropriate message is displayed, which informs you what are the names of the amendments that you have defined in the project_config.yaml
file. Nontheless, just the main project is "active".
Let's inspect it:
p.sample_table
file_path | organism | protocol | sample_name | time | |
---|---|---|---|---|---|
sample_name | |||||
pig_0h | /data/lab/project/pig_0h.fastq | pig | RRBS | pig_0h | 0 |
pig_1h | /data/lab/project/pig_1h.fastq | pig | RRBS | pig_1h | 1 |
frog_0h | /data/lab/project/frog_0h.fastq | frog | RRBS | frog_0h | 0 |
frog_1h | /data/lab/project/frog_1h.fastq | frog | RRBS | frog_1h | 1 |
The column file_path
was derived and the library
column holds the original attributes: RRBS
for each sample.
To "activate" any of the amendments just pass the names of the desired amendments to the amendments
argument in the Project
object constructor.
In case you don't remember the subproject names run the listAmendments()
metohods on the Project
object, just like that:
p.list_amendments
['newLib', 'newLib2']
p_new_lib = Project(project_config_file, amendments = "newLib")
Let's inspect it:
p_new_lib.sample_table
file_path | organism | protocol | sample_name | time | |
---|---|---|---|---|---|
sample_name | |||||
pig_0h | /data/lab/project/pig_0h.fastq | pig | ABCD | pig_0h | 0 |
pig_1h | /data/lab/project/pig_1h.fastq | pig | ABCD | pig_1h | 1 |
frog_0h | /data/lab/project/frog_0h.fastq | frog | ABCD | frog_0h | 0 |
frog_1h | /data/lab/project/frog_1h.fastq | frog | ABCD | frog_1h | 1 |
As you can see, the library
column consists of new attributes (ABCD
), which were defined in the sample_table_newLib.csv
file.
Amendments can be also activated interactively, after Project
object has been crated. Let's activate the second amendment this way:
p_new_lib2 = p.activate_amendments("newLib2")
p_new_lib2.sample_table
file_path | organism | protocol | sample_name | time | |
---|---|---|---|---|---|
sample_name | |||||
pig_0h | /data/lab/project/pig_0h.fastq | pig | EFGH | pig_0h | 0 |
pig_1h | /data/lab/project/pig_1h.fastq | pig | EFGH | pig_1h | 1 |
frog_0h | /data/lab/project/frog_0h.fastq | frog | EFGH | frog_0h | 0 |
frog_1h | /data/lab/project/frog_1h.fastq | frog | EFGH | frog_1h | 1 |