Bastien
Chevreux
bach@chevreux.org

> Solomon Short

> The universe is full of surprises - most of them nasty.

# De-novo assemblies

## Introduction

This guide assumes that you have basic working knowledge of Unix systems, know the basic principles of sequencing (and sequence assembly) and what assemblers do.

While there are step by step instructions on how to setup your data and then perform an assembly, this guide expects you to read at some point in time

- Before the assembly, to know what to do (or not to do) with the sequencing data before giving it to MIRA.

- For users with PacBio reads, has important information regarding special parameters needed.

- After the assembly, to know what to do with the results of the assembly. More specifically, , , , and .

- And also to look up how manifest files should be written ( and and ), some command line options as well as general information on what tags MIRA uses in assemblies, files it generates etc.pp

- Last but not least, you may be interested in some observations about the different sequencing technologies and the traps they may contain, see for that. For advice on what to pay attention to before going into a sequencing project, have a look at .

## General steps

This part will introduce you step by step how to get your data together for a simple mapping assembly. I'll make up an example using an imaginary bacterium: Bacillus chocorafoliensis (or short: Bchoc). You collected the strain you want to assemble somewhere in the wild, so you gave the strain the name Bchoc_wt.

Just for laughs, let's assume you sequenced that bug with lots of more or less current sequencing technologies: Sanger, 454, Illumina, Ion Torrent and Pacific Biosciences.

### Copying and naming the sequence data

You need to create (or get from your sequencing provider) the sequencing data in any supported file format. Amongst these, FASTQ and FASTA + FASTA-quality will be the most common, although the latter is well on the way out nowadays. The following walkthrough uses what most people nowadays get: FASTQ.

Create a new project directory (e.g. `myProject`) and a subdirectory of this which will hold the sequencing data (e.g. `data`).

    arcadia:/path/to mkdir myProject
    arcadia:/path/to cd myProject
    arcadia:/path/to/myProject$ mkdir data

Put the FASTQ data into that `data` directory so that it now looks perhaps like this:

    arcadia:/path/to/myProject$ ls -l data
    -rw-r--r-- 1 bach users 263985896 2008-03-28 21:49 bchocwt_lane6.solexa.fastq

I completely made up the file names above. You can name them anyway you want. And you can have them live anywhere on the hard-disk, you do not need to put them in this `data` directory. It's just the way I do it ... and it's where the example manifest files a bit further down in this chapter will look for the data files.

We're almost finished with the setup. As I like to have things neatly separated, I always create a directory called `assemblies` which will hold my assemblies (or different trials) together. Let's quickly do that:

    arcadia:/path/to/myProject$ mkdir assemblies
    arcadia:/path/to/myProject$ mkdir assemblies/1sttrial
    arcadia:/path/to/myProject$ cd assemblies/1sttrial

### Writing a simple manifest file

A manifest file is a configuration file for MIRA which tells it what type of assembly it should do and which data it should load. In this case we'll make a simple assembly with unpaired Illumina data

    # Example for a manifest describing a de-novo assembly with
    # unpaired Illumina data

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,denovo,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # here comes the unpaired Illumina data

    readgroup = SomeUnpairedIlluminaReadsIGotFromTheLab
    data = ../../data/bchocwt_lane6.solexa.fastq
    technology = solexa

Please look up the parameters of the manifest file in the main manual or the example manifest files in the following section.

The ones above basically say: make an accurate denovo assembly of unpaired Illumina reads.

### Starting the assembly

Starting the assembly is now just a matter of a simple command line:

    arcadia:/path/to/myProject/assemblies/1sttrial$ mira manifest.conf >&log_assembly.txt

For this example - if you followed the walk-through on how to prepare the data - everything you might want to adapt in the first time are the following thing in the manifest file: options:

- project= (for naming your assembly project)

Of course, you are free to change any option via the extended parameters, but this is the topic of another part of this manual.

## Manifest files for different use cases

This section will introduce you to manifest files for different use cases. It should cover the most important uses, but as always you are free to mix and match the parameters and readgroup definitions to suit your specific needs.

Taking into account that there may be a lot of combinations of sequencing technologies, sequencing libraries (shotgun, paired-end, mate-pair, etc.) and input file types (FASTQ, FASTA, GenBank, GFF3, etc.pp), the example manifest files just use Illumina and 454 as technologies, GFF3 as input file type for the reference sequence, FASTQ as input type for sequencing data ... and they do not show the multitude of more advanced features like, e.g., using ancillary clipping information in XML files, ancillary masking information in SSAHA2 or SMALT files etc.pp.

I'm sure you will be able to find your way by scanning through the corresponding section on manifest files in the reference chapter :-)

### Manifest for shotgun data

Well, we've seen that already in the section above, but here it is again ... but this time with 454 data.

    # Example for a manifest describing a denovo assembly with
    # unpaired 454 data

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,denovo,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # here's the 454 data

    readgroup = SomeUnpaired454ReadsIGotFromTheLab
    data = ../../data/some454data.fastq
    technology = 454

### Assembling with multiple sequencing technologies (hybrid assemblies)

Hybrid mapping assemblies follow the general manifest scheme: tell what you want in the first part, then simply add as separate readgroup the information MIRA needs to know to find the data and off you go. Just for laughs, here's a manifest for 454 shotgun with Illumina shotgun

    # Example for a manifest describing a denovo assembly with
    # shotgun 454 and shotgun Illumina data

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,mapping,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # now the shotgun 454 data
    readgroup = DataForShotgun454
    data = ../../data/project454data.fastq
    technology = 454

    # now the shotgun Illumina data

    readgroup = DataForShotgunIllumina
    data = ../../data/someillumina.fastq
    technology = solexa

### Manifest for data sets with paired reads

When using paired-end data, you should know

1. the orientation of the reads toward each other. This is specific to sequencing technologies and / or the sequencing library preparation.

2. at which distance these reads should be. This is specific to the sequencing library preparation and the sequencing lab should tell you this.

In case you do not know one (or any) of the above, don't panic! MIRA is able to estimate the needed values during the assembly if you tell it to.

The following manifest shows you the most laziest way to define a paired data set by simply adding autopairing as keyword to a readgroup (using Illumina just as example):

    # Example for a lazy manifest describing a denovo assembly with
    # one library of paired reads

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,denovo,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # now the Illumina paired-end data

    readgroup = DataIlluminaPairedLib
    autopairing
    data = ../../data/project_1.fastq ../../data/project_2.fastq
    technology = solexa

If you know the orientation of the reads and/or the library size, you can tell this MIRA the following way (just showing the readgroup definition here):

    readgroup = DataIlluminaPairedEnd500Lib
    data = ../../data/project_1.fastq ../../data/project_2.fastq
    technology = solexa
    template_size = 250 750
    segment_placement = ---> <---

In cases you are not 100% sure about, e.g., the size of the DNA template, you can also give a (generous) expected range and then tell MIRA to automatically refine this range during the assembly based on real, observed distances of read pairs. Do this with autorefine modifier like this:

    template_size = 50 1000 autorefine

The following manifest file is an example for assembling with several different libraries from different technologies. Do not forget you can use autopairing or autorefine :-)

    # Example for a manifest describing a denovo assembly with
    # several kinds of sequencing libraries

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,denovo,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # now the Illumina paired-end data

    readgroup = DataIlluminaForPairedEnd500bpLib
    data = ../../data/project500bp-1.fastq ../../data/project500bp-2.fastq
    technology = solexa
    strain = bchoc_se1
    template_size = 250 750
    segment_placement = ---> <---

    # now the Illumina mate-pair data

    readgroup = DataIlluminaForMatePair3kbLib
    data = ../../data/project3kb-1.fastq ../../data/project3kb-2.fastq
    technology = solexa
    strain = bchoc_se1
    template_size = 2500 3500
    segment_placement = <--- --->

    # some Sanger data (6kb library)

    readgroup = DataForSanger6kbLib
    data = ../../data/sangerdata.fastq
    technology = sanger
    template_size = 5500 6500
    segment_placement = ---> <---

    # some 454 data

    readgroup = DataFo454Pairs
    data = ../../data/454data.fastq
    technology = 454
    template_size = 8000 1200
    segment_placement = 2---> 1--->

    # some Ion Torrent data

    readgroup = DataFoIonPairs
    data = ../../data/iondata.fastq
    technology = iontor
    template_size = 1000 300
    segment_placement = 2---> 1--->

### De-novo with multiple strains

MIRA will make use of ancillary information present in the manifest file. One of these is the information to which strain (or organism or cell line etc.pp) the generated data belongs.

You just need to tell in the manifest file which data comes from which strain. Let's assume that in the example from above, the "lane6" data were from a first mutant named bchoc_se1 and the "lane7" data were from a second mutant named bchoc_se2. Here's the manifest file you would write then:

    # Example for a manifest describing a de-novo assembly with
    # unpaired Illumina data, but from multiple strains

    # First part: defining some basic things
    # In this example, we just give a name to the assembly
    #  and tell MIRA it should map a genome in accurate mode

    project = MyFirstAssembly
    job = genome,denovo,accurate

    # The second part defines the sequencing data MIRA should load and assemble
    # The data is logically divided into "readgroups"

    # now the Illumina data

    readgroup = DataForSE1
    data = ../../data/bchocse_lane6.solexa.fastq
    technology = solexa
    strain = bchoc_se1

    readgroup = DataForSE2
    data = ../../data/bchocse_lane7.solexa.fastq
    technology = solexa
    strain = bchoc_se2

While assembling de-novo (pr mapping) with multiple strains is possible, the interpretation of results may become a bit daunting in some cases. For many scenarios it might therefore be preferable to successively use the data sets in own assemblies or mappings.

This strain information for each readgroup is really the only change you need to perform to tell MIRA everything it needs for handling strains.