Difference between revisions of "MSblender TACC"

From Marcotte Lab
Jump to: navigation, search
(Run MSGF+)
 
(10 intermediate revisions by one user not shown)
Line 3: Line 3:
 
* This document is for [https://portal.tacc.utexas.edu/user-guides/stampede 'stampede'].  
 
* This document is for [https://portal.tacc.utexas.edu/user-guides/stampede 'stampede'].  
 
* Currently in most cases I use three search engines: comet, X!Tandem, and MS-GF+.
 
* Currently in most cases I use three search engines: comet, X!Tandem, and MS-GF+.
 +
* You don't need to run 'MSblender' modeling on TACC, because it does not take that long. I normally run all searches at TACC, then transfer the output to my local machine to run MSblender. So it only covers 'search' part. For running MSblender, please see [[MSblender]] page.
  
 
== Install MSblender (and comet, MSGF+, X!Tandem) ==  
 
== Install MSblender (and comet, MSGF+, X!Tandem) ==  
Line 28: Line 29:
 
   
 
   
 
=== DB setup for X!tandem ===
 
=== DB setup for X!tandem ===
<pre> $ $HOME/git/MSblender/extern/fasta_pro.exe my_seq.conbind.fa</pre>
+
<pre> $ $HOME/git/MSblender/extern/fasta_pro.exe my_seq.combined.fa</pre>
  
 
You may see the message like below:
 
You may see the message like below:
Line 44: Line 45:
 
It uses significant amount of computing resources (i.e. memory), so it may not be suitable to run on login node.  
 
It uses significant amount of computing resources (i.e. memory), so it may not be suitable to run on login node.  
  
<pre> $ module load jdk64
+
<pre>$ module load jdk64
$ java -Xmx4000M -cp /home1/00992/linusben/git/MSblender/extern/MSGFPlus.jar edu.ucsd.msjava.msdbsearch.BuildSA -d XenopusHybrid_xlJGIv16_xtJGIv83.combined.fa -tda 0</pre>  
+
$ java -Xmx4000M -cp /home1/00992/linusben/git/MSblender/extern/MSGFPlus.jar edu.ucsd.msjava.msdbsearch.BuildSA -d XenopusHybrid_xlJGIv16_xtJGIv83.combined.fa -tda 0</pre>
  
 
== Prepare mzXML files ==
 
== Prepare mzXML files ==
Line 52: Line 53:
  
 
== Run comet ==
 
== Run comet ==
<pre>$ ~/git/MSblender/extern/comet.linux.exe -p</pre>
+
<pre>$ cd $SCRATCH/myProject/comet
 +
$ ~/git/MSblender/extern/comet.linux.exe -p</pre>
  
 
Edit 'comet.params.new' file. Typically, you need to change the following lines.
 
Edit 'comet.params.new' file. Typically, you need to change the following lines.
<pre> num_threads = 16
+
<pre>num_threads = 16
 +
 
 
peptide_mass_tolerance = 20.0
 
peptide_mass_tolerance = 20.0
 
peptide_mass_units = 2
 
peptide_mass_units = 2
 +
 
search_enzyme_number = 2  ## See the end of param file for the type of enzymes
 
search_enzyme_number = 2  ## See the end of param file for the type of enzymes
  
Line 63: Line 67:
 
output_pepxmlfile = 0</pre>
 
output_pepxmlfile = 0</pre>
  
Then, create the launcher script as below.
+
Then, create the launcher script (called 'stampede-comet.sh') as below.
  
 
<pre>#!/bin/bash
 
<pre>#!/bin/bash
Line 87: Line 91:
 
done
 
done
 
</pre>
 
</pre>
 +
 +
Then, submit the job by typing 'sbatch stampede-comet.sh'
  
 
== Run MSGF+ ==
 
== Run MSGF+ ==
 +
 +
Create 'stampede-MSGF+.sh' file as below.
 +
<pre>#!/bin/bash
 +
#SBATCH -n 16
 +
#SBATCH -p normal
 +
#SBATCH -t 24:00:00
 +
 +
#SBATCH -o mg+.o%j
 +
set -x
 +
 +
module load jdk64
 +
 +
MSGFplus_JAR="$HOME/git/MSblender/extern/MSGFPlus.jar"
 +
 +
DB="../DB/my_seq.combined.fa"
 +
 +
DBNAME=$(basename $DB)
 +
DBNAME=${DBNAME/.fa/}
 +
 +
#SBATCH -J "mg+"
 +
for MZXML in $(ls ../mzXML/*mzXML)
 +
do
 +
  OUT=$(basename $MZXML)
 +
  OUT=${OUT/.mzXML/}"."$DBNAME".MSGF+.mzid"
 +
  TBL=${OUT/.mzid/.tsv}
 +
  time java -Xmx20000M -jar $MSGFplus_JAR -d $DB -s $MZXML -o $OUT -t 20ppm -tda 0 -ntt 2 -e 1 -inst 3
 +
  time java -Xmx20000M -cp $MSGFplus_JAR edu.ucsd.msjava.ui.MzIDToTsv -i $OUT -o $TBL -showQValue 1 -showDecoy 1 -unroll 0
 +
done</pre>
 +
 +
Then, submit the job by typing 'sbatch stampede-MSGF+.sh'
  
 
== Run X!Tandem ==
 
== Run X!Tandem ==
 +
 +
<pre>$ cd $SCRATCH/myProject/tandemK
 +
$ ~/git/MSblender/search/prepare-tandemK-high.py ../mzXML/ ../DB/my_seq.combined.fa.pro</pre>
 +
 +
First argument of prepare-tendemK-high.py is a directory for mzXML, and second one is for .pro database generated by fasta_pro.exe as above.
 +
 +
You will see *.xml files matched to your mzXML files (X!Tandem input), 'tandem-taxonomy.xml' file (another X!Tandem input), and run-tandemK.sh (a script to run X!tandem).
 +
 +
Make the following launcher ('stampede-tandemK.sh'), and submit it as 'sbatch stampede-tandemK.sh'.
 +
 +
<pre>#!/bin/bash
 +
#SBATCH -n 16
 +
#SBATCH -p normal
 +
#SBATCH -t 24:00:00
 +
 +
#SBATCH -o tK.o%j
 +
#SBATCH -J "tK"
 +
set -x
 +
 +
bash ./run-tandemK.sh</pre>
 +
 +
If you have many mzXML files, you can run it parallel by splitting run-tandemK.sh with 'split -l' command, and run individual splitted script at 'stampede-tandemK.sh'.

Latest revision as of 14:39, 29 June 2015

Contents

Before you start

  • To use this setting, your TACC account needs to be allocated to our lab project('A-cm10'). If you don't have an account, create it at https://portal.tacc.utexas.edu/. Then, ask Edward to assign your account as a member of lab project.
  • This document is for 'stampede'.
  • Currently in most cases I use three search engines: comet, X!Tandem, and MS-GF+.
  • You don't need to run 'MSblender' modeling on TACC, because it does not take that long. I normally run all searches at TACC, then transfer the output to my local machine to run MSblender. So it only covers 'search' part. For running MSblender, please see MSblender page.

Install MSblender (and comet, MSGF+, X!Tandem)

$ cd ~
$ mkdir git
$ cd git
$ git clone https://github.com/marcottelab/MSblender.git

Prepare a working space

$ module load python
$ cd $SCRATCH
$ mkdir myProject
$ cd myProject
$ mkdir mzXML
$ mkdir DB
$ mkdir comet
$ mkdir MSGF+
$ mkdir tandemK

Prepare database

  • You can run this process on any computer. If it takes longer than a minute, it would be better to process it on other than TACC login node (your account may be locked).
$ python $HOME/git/MSblender/pre/fasta-reverse.py my_seq.fa
$ cat my_seq.fa.* > my_seq.combined.fa

DB setup for X!tandem

 $ $HOME/git/MSblender/extern/fasta_pro.exe my_seq.combined.fa

You may see the message like below:

$ ~/git/MSblender/extern/fasta_pro.exe my_seq.combined.fa 
fasta_pro file conversion utility, v. 2006.09.15
 input path = my_seq.combined.fa
output path = my_seq.combined.fa.pro
db type = plain

DB setup for comet

You don't need to do anything for this.

DB setup for MSGF+

It uses significant amount of computing resources (i.e. memory), so it may not be suitable to run on login node.

$ module load jdk64
$ java -Xmx4000M -cp /home1/00992/linusben/git/MSblender/extern/MSGFPlus.jar edu.ucsd.msjava.msdbsearch.BuildSA -d XenopusHybrid_xlJGIv16_xtJGIv83.combined.fa -tda 0

Prepare mzXML files

Copy your mzXML files on this diretory ($SCRATCH/myProject/mzXML).

Run comet

$ cd $SCRATCH/myProject/comet
$ ~/git/MSblender/extern/comet.linux.exe -p

Edit 'comet.params.new' file. Typically, you need to change the following lines.

num_threads = 16

peptide_mass_tolerance = 20.0
peptide_mass_units = 2

search_enzyme_number = 2   ## See the end of param file for the type of enzymes

output_txtfile = 1
output_pepxmlfile = 0

Then, create the launcher script (called 'stampede-comet.sh') as below.

#!/bin/bash
#SBATCH -n 16
#SBATCH -p normal
#SBATCH -t 24:00:00

#SBATCH -o cmt.o%j
COMET="$HOME/git/MSblender/extern/comet.linux.exe"

DB="../DB/my_seq.combined.fa"
DBNAME=$(basename $DB)
DBNAME=${DBNAME/.fa/}

PARAM="./comet.params.new"

#SBATCH -J "cmt"
for MZXML in $(ls ../mzXML/*mzXML)
do
  OUT=$(basename $MZXML)
  OUT=${OUT/.mzXML/}"."$DBNAME".comet"
  time $COMET -P$PARAM -D$DB -N$OUT $MZXML
done

Then, submit the job by typing 'sbatch stampede-comet.sh'

Run MSGF+

Create 'stampede-MSGF+.sh' file as below.

#!/bin/bash
#SBATCH -n 16
#SBATCH -p normal
#SBATCH -t 24:00:00

#SBATCH -o mg+.o%j
set -x

module load jdk64

MSGFplus_JAR="$HOME/git/MSblender/extern/MSGFPlus.jar"

DB="../DB/my_seq.combined.fa"

DBNAME=$(basename $DB)
DBNAME=${DBNAME/.fa/}

#SBATCH -J "mg+"
for MZXML in $(ls ../mzXML/*mzXML)
do
  OUT=$(basename $MZXML)
  OUT=${OUT/.mzXML/}"."$DBNAME".MSGF+.mzid"
  TBL=${OUT/.mzid/.tsv}
  time java -Xmx20000M -jar $MSGFplus_JAR -d $DB -s $MZXML -o $OUT -t 20ppm -tda 0 -ntt 2 -e 1 -inst 3
  time java -Xmx20000M -cp $MSGFplus_JAR edu.ucsd.msjava.ui.MzIDToTsv -i $OUT -o $TBL -showQValue 1 -showDecoy 1 -unroll 0
done

Then, submit the job by typing 'sbatch stampede-MSGF+.sh'

Run X!Tandem

$ cd $SCRATCH/myProject/tandemK
$ ~/git/MSblender/search/prepare-tandemK-high.py ../mzXML/ ../DB/my_seq.combined.fa.pro

First argument of prepare-tendemK-high.py is a directory for mzXML, and second one is for .pro database generated by fasta_pro.exe as above.

You will see *.xml files matched to your mzXML files (X!Tandem input), 'tandem-taxonomy.xml' file (another X!Tandem input), and run-tandemK.sh (a script to run X!tandem).

Make the following launcher ('stampede-tandemK.sh'), and submit it as 'sbatch stampede-tandemK.sh'.

#!/bin/bash
#SBATCH -n 16
#SBATCH -p normal
#SBATCH -t 24:00:00

#SBATCH -o tK.o%j
#SBATCH -J "tK"
set -x

bash ./run-tandemK.sh

If you have many mzXML files, you can run it parallel by splitting run-tandemK.sh with 'split -l' command, and run individual splitted script at 'stampede-tandemK.sh'.