Skip to content

Feature: co-assemblies submission with --mode metagenomic_assembly - #66

Draft
ochkalova wants to merge 33 commits into
devfrom
feat/coassembly_submission
Draft

ochkalova wants to merge 33 commits into
devfrom
feat/coassembly_submission

Conversation

@ochkalova

@ochkalova ochkalova commented May 14, 2026 •

Copy link
Copy Markdown
Contributor

Resolves #61

What this PR does

--mode metagenomic_assemblies can now submit co-assemblies: assemblies built from the reads of several runs.

Bins and MAGs made from co-assemblies are not supported yet.

How to use it

A co-assembly is one row in the samplesheet. List all its runs in run_accession, and their read files in fastq_1 / fastq_2, separated by ;:

id,fasta,fastq_1,fastq_2,coverage,run_accession,assembler,assembler_version,assembly_type
coassembly_1,contigs.fasta.gz,run1_R1.fastq.gz;run2_R1.fastq.gz,run1_R2.fastq.gz;run2_R2.fastq.gz,,ERR011324;ERR011325,MEGAHIT,1.2.9,

Read files must be in the same order as the run accessions. If coverage is given in the samplesheet, read files can be left empty, as before.

Changes

Samplesheet

  • fastq_1, fastq_2 and run_accession accept semicolon-separated values.
  • The schema can't check that each path in a ;-separated list exists, so "format": "file-path" and "exists": true were removed from fastq_1/fastq_2. Instead, the workflow now checks each read file with file(..., checkIfExists: true), so missing files still fail early, before any process runs.
  • The pipeline fails if the number of read files doesn't match the number of run accessions.
  • Duplicated accessions in run_accession are removed.

Coverage

  • If coverage is missing, CoverM maps the reads of all runs to the assembly (coverm contig --methods mean length).
  • The new calculateCoassemblyCoverage function turns the CoverM table into one number:
    • depth of a contig = sum of its depths over all runs, because the reads were pooled to build the co-assembly;
    • assembly coverage = mean contig depth, weighted by contig length.
  • The function stops with a clear error if the CoverM table has no contigs, misses the expected columns, or gives coverage 0, which usually means the reads don't belong to the assembly.
  • ⚠️ This also changes the result for normal single-run assemblies. Before, coverage was a plain average over contigs, so short contigs counted as much as long ones. Now it's length-weighted, which gives the average depth per assembled base.
  • The formula is described in docs/methods.md.

Virtual sample for co-assemblies

ENA lets an assembly point to only one sample. The new REGISTER_COASSEMBLY_SAMPLE module (bin/register_coassembly_sample.py) handles this:

  • If all runs come from the same sample, the assembly is submitted against that sample.
  • If the runs come from different samples, it registers a new "virtual" ENA sample (default sample checklist ERC000011) with a composed of attribute that lists the source samples. The assembly is then submitted against this sample.
  • Metadata of the virtual sample is taken from the source samples:
    • collection date and country are kept if all source samples agree, otherwise they're set to not provided. Countries not allowed by ERC000011 are also set to not provided.
    • Taxon and scientific name must be the same in all source samples; otherwise the pipeline stops with an error.
  • Works with private runs and samples (--private): ENA is queried with your Webin credentials.
  • With --test_upload, the sample is registered on the ENA test server. A random suffix is added to its alias so repeated test runs don't clash.
  • Single-run assemblies skip this step and work as before.

Other changes

  • CREATE_ASSEMBLY_METADATA_CSV writes a list of run accessions in the Runs column for co-assemblies.

Tests

  • New pipeline tests: test_assembly_coassembly_with_coverage and test_assembly_coassembly_no_coverage. Their samplesheets are in nf-core/test-datasets.
  • nf-test for the REGISTER_COASSEMBLY_SAMPLE module.
  • Function tests for calculateCoassemblyCoverage: one run, three runs, header only, missing Length column, zero depth.

Docs

  • README.md and docs/usage.md: how to describe a co-assembly in the samplesheet, and how the virtual sample is made. Co-assemblies are removed from the list of unsupported features.
  • docs/methods.md: the new coverage formula.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/seqsubmit branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

🤖 Generated with Claude Code

ochkalova and others added 7 commits May 13, 2026 12:20
@ochkalova ochkalova changed the title Feat/coassembly submission Feature: co-assemblies submission with --mode metagenomic_assembly Jul 10, 2026
@github-actions

github-actions Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 9d2a06a

+| ✅ 264 tests passed       |+
#| ❔   9 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗   4 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 1.0.0
  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here

❔ Tests ignored:

  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • files_exist - File is ignored: .github/workflows/linting_comment.yml
  • nextflow_config - Config variable ignored: params.input
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/nf-core-seqsubmit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqsubmit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqsubmit_logo_dark.png
  • container_configs - container_configs

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 4.1.0
  • Run at 2026-10-29 16:16:13

@github-actions

github-actions Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 4.1.0.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the Synchronisation documentation.

@github-actions

github-actions Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

❌ nf-test failed with latest Nextflow version

Note

Tests with Nextflow's latest version failed but it will not cause a CI workflow failure.
Please check if the failure is expected with newer (edge-)releases of Nextflow or if it needs fixing.

  • ❌ docker | latest-everything | Shard 1/7

See the full run for details.

@ochkalova
ochkalova force-pushed the feat/coassembly_submission branch from 5c1c85c to 54b14fe Compare October 20, 2026 14:54

@KateSakharova KateSakharova left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A massive amount of work! Thank you, very coool!

Have you tried multiqc "" ?

Comment thread docs/usage.md Outdated
Comment thread docs/usage.md Outdated
Comment thread docs/usage.md Outdated
Comment thread docs/usage.md Outdated
id,fasta,fastq_1,fastq_2,coverage,run_accession,assembler,assembler_version,assembly_type
assembly_001,data/assembly_001.fasta.gz,data/assembly_001_R1.fastq.gz,data/assembly_001_R2.fastq.gz,,ERR011322,SPAdes,3.15.5,metagenome
assembly_002,data/assembly_002.fasta.gz,,,42.7,ERR011323,MEGAHIT,1.2.9,metatranscriptome
coassembly_001,data/coassembly_001.fasta.gz,data/run1_R1.fastq.gz;data/run2_R1.fastq.gz,data/run1_R2.fastq.gz;data/run2_R2.fastq.gz,,ERR011324;ERR011325,MEGAHIT,1.2.9,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to have PE and SE in co-assembly?
like

...,data/run1_R1.fastq.gz;data/run2_SE.fastq.gz,data/run1_R2.fastq.gz;,...

if yes - maybe add it as example. If not - we should think can we support it or not

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add those to Limitations

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱

@@ -0,0 +1,11 @@
Runs,Coverage,Assembler,Version,Filepath

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have example here pointing to one sample somewhere? if not - I think it might worth adding and check in test that no new sample created 🤔

Comment thread bin/register_coassembly_sample.py Outdated
Comment thread bin/register_coassembly_sample.py Outdated
Comment thread bin/register_coassembly_sample.py
Comment thread bin/register_coassembly_sample.py Outdated
Comment thread bin/register_coassembly_sample.py Outdated
ochkalova and others added 2 commits October 29, 2026 12:18
- Move ERC000011 checklist and ENA Portal field lists to global constants
- Validate secondary sample accessions with a regex
- Report tax_id/scientific_name of all source samples in one error
- Reword virtual sample description

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread docs/usage.md
| `fastq_1` | Conditional | Path to single-end/paired-end forward reads in FASTQ format (optionally gzipped). Required if `coverage` is not provided. |
| `fastq_2` | No | Path to the second read file in FASTQ format for paired-end data used to generate the source metagenomic assembly. Leave empty for single-end reads. |
| `coverage` | Conditional | Estimated sequencing depth of the assembly. If this value is missing, it is computed automatically during pipeline execution when reads are provided. |
| `run_accession` | Yes | ENA run accession for the reads used to generate the metagenomic assembly. Reads must already be submitted to ENA. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a note about runs for co-assembly in here as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support submission of metagenomic assemblies generated from co-assemblies

2 participants