OTU table without abundance values about swarm HOT 6 CLOSED

Hi Ashley, sorry for not answering earlier. I'll look into your data soon, and come up with a more complete answer.

from swarm.

from swarm.

Here is the code that builds the final table:

STATS="amplicons.stats"
SWARMS="amplicons.swarms"
AMPLICON_TABLE="amplicon_contingency_table.csv"
OTU_TABLE="OTU_contingency_table.csv"

# Header
echo -e "OTU\t$(head -n 1 "${AMPLICON_TABLE}")" > "${OTU_TABLE}"

# Compute "per sample abundance" for each OTU
awk -v SWARM="${SWARMS}" \
    -v TABLE="${AMPLICON_TABLE}" \
    'BEGIN {FS = " "
            while ((getline < SWARM) > 0) {
                swarms[$1] = $0
            }
            FS = "\t"
            while ((getline < TABLE) > 0) {
                table[$1] = $0
            }
           }

     {# Parse the stat file (OTUs sorted by decreasing abundance)
      seed = $3
      n = split(swarms[seed], OTU, " ")
      for (i = 1; i < n; i = i + 2) {
          s = split(table[OTU[i]], abundances, "\t")
          for (j = 1; j < s; j++) {
              samples[j] += abundances[j+1]
          }
      }
      printf "%s\t%s", NR, $3
      for (j = 1; j < s; j++) {
          printf "\t%s", samples[j]
      }
     printf "\n"
     delete samples
     }' "${STATS}" >> "${OTU_TABLE}"

I've changed the line seed = $3 "_" $4 for seed = $3 to adapt the code to the your amplicon name format.

However, your dataset seems to be incomplete. For example, amplicon TDA23-3508 forms its own cluster and is marked as representing 6 reads (see amplicons.stats), but a look at amplicon_contingency_table.csv shows that TDA23-3508 has a total of 2 reads, all from sample TDA23. So 4 reads are missing here, and the resulting OTU table will likely be wrong.

Did you perform a global dereplication of your fasta data before clustering?

from swarm.

from swarm.

I do perform a global dereplication using USEARCH before clustering, is that the reason why the OTU table lose some information?

Performing a global dereplication is strongly recommended, so it is good you did it. When dereplicating, did you use the --sizein option to take into account the per-sample abundance values?

There are count differences between your stats, swarms and table files, so something is wrong. However, it is hard for me to say exactly what step of your process went wrong.

from swarm.

Sorry for the delay. I added -sizein option and redone the whole process, but unfortunately, there is still some absence data. I'm gonna try several more times and see if I can work this out. Thank you again for your effort.

from swarm.