Specimen and sequence exports

Two download jobs, Specimen Records and Sequences, export the records selected on a console as files. Both run as analysis jobs through the same five stages as the analytical methods, so a large export does not tie up your browser, and the result arrives in the Analysis panel. Neither produces a report. This page states exactly what each job writes and which of its options take effect. For the steps on the console, see Downloading records, sequences and files.

What both jobs share

• Field selection by group. The form offers fields from nine groups. Both jobs read only four of them: specimen, taxonomy, collection and sequence. Fields ticked under record organization, forensic extension, forensic extension advanced, individualization extension and feature rollup are not exported.
• Fields always included. Whatever you select, the specimen job carries record_id, processid and phylum; the sequence job carries record_id, processid, marker_code, phylum and the sequence.
• Identifier required. The specimen job requires processid or sampleid among the specimen fields; the sequence job requires processid, sampleid or record_id. Without one, the job fails at validation.
• Records with no usable sequence are dropped. The shared record filter runs with its default settings (see The methods and their shared pipeline): a record with no sequence, or with more than 10% ambiguity codes among its bases, is left out. This applies to the specimen export too, so the specimen file lists only specimens that carry a sequence.
• No report. Neither job writes analysis_report.html or a PDF.

Specimen records

Method. The column set is the union of every field present on any exported record, sorted alphabetically. Each record is one row; a field absent from a record is an empty cell. A field holding a list (for example bold_recordset_code_arr) is written as its items joined with |; a field holding a nested object is written as JSON text. The job writes CSV first and converts it to TSV when that format is chosen.

Output. output.tsv or output.csv.

Sequences

There is no marker option. Every selected record of every marker is exported; use the marker_code in each header to tell them apart.

Method. Each record becomes one FASTA entry. The header is a |-separated list of field=value pairs, in the order record_id, processid, marker_code, phylum, then the specimen, taxonomy, collection and sequence fields you selected. A missing value leaves the pair as field=. The sequence is the stored nuc value on one line, including any gaps or ambiguity codes it contains.

Output. output.fasta, for example:

>record_id=123456|processid=ABCDE001-24|marker_code=COI-5P|phylum=Chordata|identification=Panthera pardus|country/ocean=South Africa|nuc_basecount=658
AACATTATATTTTATTTTTGG...

Package contents

The result package is a ZIP of the job's working directory. For either job it holds:

See Anatomy of a result package for how packages are named and stored.

Good to know

Good to know: The number of rows or FASTA entries can be smaller than the number of records you selected. Records without a sequence, and records with more than 10% ambiguous bases, are dropped without a message. filtered_records.jsonl shows exactly which records went into the file.

Limit: Choosing Excel or JSON for a specimen export always fails. Choose TSV or CSV and convert the file yourself.

Limit: The sequence export does not align. If you need an aligned FASTA of a set of records, Diagnostic characters writes aligned_sequences.fas as part of its package.

• Field names in the header and in the column set are the BCDM names listed in the field dictionary.
• The specimen column order is alphabetical, not the order of the form.

Related

• Downloading records, sequences and files
• The methods and their shared pipeline
• Anatomy of a result package
• Field dictionary