'K12b' and 'K7b' calculators

I am releasing two new calculators with K=12 and K=7 components, named 'K12b' and 'K7b'. You can scroll down to the bottom if you are just interested in the downloads, or read on.


New Features

The new 'K12b' calculator is an update of the previous K12a one, that was inferred using all the new samples submitted during the last submission opportunity. The 12 components are still roughly the same, although their allele frequencies may have changed by a bit, so existing participants can expect to have slightly altered results, and new participants in the Project more so, since their data are now contributing to the creation of the new tool. Non-participants can, of course, use the new calculator with DIYDodecad.

I have also taken the opportunity to do some minor tweaks. I am releasing population portraits for K12b (which were lacking in K12a); I've changed my visualization code so that the sample IDs of non-Dodecad populations can now be seen in the barplots. This may be useful for anyone else using these reference populations, by quickly identifying potential outliers in them.

I have also decided to use normalized median admixture proportions for the populations. For example, if 5 individuals in a population have 0, 0, 0.2, 0.5, 10.0% of a particular component, then the average is 2.14%, but the median is 0.2%. By using the median, the proportions become less susceptible to the presence of outliers (such as the 10%). However, if the median is calculated over every component separately, it is no longer guaranteed that the components will add up to 100%; this can be addressed by re-normalizing them (scaling them by a constant factor) so that they do. I believe that use of the normalized median will not only give better proportions that are less susceptible to outliers, but will also improve results of the new Dodecad Oracle for K12b.

At the same time I am also releasing 'K7b' which is an update of the existing 'eurasia7' calculator and which has been built on exactly the same dataset as 'K12b' but at a lower (K=7) level of detail.

Information on K7b


Information spreadsheet.

Normalized median admixture proportions barplot for all included populations (a high resolution version of this is included in the download bundle):

Table of Fst divergences:

Neighbor-joining tree (based on above):

Information on K12b


Information spreadsheet.


Normalized median admixture proportions barplot for all included populations (a high resolution version of this is included in the download bundle):

Table of Fst divergences:

Neighbor-joining tree (based on above):

Multidimensional Scaling Plots of K12b and K7b


I have created MDS plots using synthetic individuals representing the 12 ancestral components of K12b and the 7 ancestral components of K7b. By including both in the same plot, one gets an idea of the relationship of the components at different resolution. The first 10 dimensions can be seen below:

Here is a blowup of the main West Eurasian groups from the plot of the first two dimensions:

Some observations:

• The Atlantic_Med component which is bi-modal in Basques and Sardinians occupies the apex of the figure; this makes sense, since Southwest Europe is quite distant (along land routes) to both Asia and Africa.
• The Caucasus component is surrounded by most of the others; this is consistent with my theory elaborated in The womb of nations: how West Eurasians came to be.
• The Atlantic_Baltic component (from K=7) is intermediate between the Atlantic_Med and North_European components.
• Similarly, the West_Asian component (from K=7) is intermediate between the Caucasus and Gedrosia components; the Gedrosia component diverges in the direction of the Asian groups (not shown in this figure), and in particular of South Asians. This divergence can also be seen in the plot of dimension #3.
• The Northwest_African component diverges in the direction of Sub-Saharan Africans.

Technical Details


A dataset of 268 populations/3,115 individuals was assembled. A total of 265,519 SNPs are in common in the various source datasets as well as the 23andMe v2/v3 and Family Finder platforms. Iterative removal of distant relatives was performed by removing one individual from each pair within a population if that pair had a RATIO of 2.5 or greater or more than the mean and two standard deviations in IBD analysis performed in PLINK 1.07. A total of 2,675 individuals remained. 4 individuals were removed for low genotyping rate (less than 97%). 264,328 SNPs remained after removal of SNPs with less than 97% genotyping rate or 1% minor allele frequency. 166,770 SNPs remained after linkage-based disequilibrium pruning (--indep-pairwise 200 25 0.4). The final set thus consisted of 2,671 individuals/268 populations/166,770 SNPs. Ancestral populations (components) were inferred using ADMIXTURE 1.21, with K=7 and K=12 and default parameters.

No individuals were removed from the source datasets, except in the case of the Armenians_Y sample, where one individual (ID: armenia3) was dropped because he/she was the same as a Dodecad Project participant.

Downloads


K7b population portraits, spreadsheet, and DIYDodecad files.
K12b population portraits, spreadsheet, and DIYDodecad files.

Dodecad Oracle (K12b edition) can be downloaded from here. Please read the instructions of the previous Oracle on how to use this tool. Note that the number of populations is now 223.

To use either calculator with DIYDodecad, with your 23andMe or Family Finder data, follow the instructions in the README file, but substitute 'K12b' or 'K7b' for 'dv3'.

Project participant results for both K7b and K12b are found in the spreadsheets in the Individual Results tab.

Terms of Use


You are free to use K12b and K7b, including all downloaded files for any non-commercial purpose, as long as you attribute them to the Dodecad Project and to Dienekes Pontikos as follows:

The [K7b/K12b] admixture calculator is courtesy of Dienekes Pontikos and was developed as part of the Dodecad Ancestry Project; more information here.

Dodecad Oracle (K12a edition)

I have created a new version of the Dodecad Oracle for use with the K12a calculator.

You can refer to the original Dodecad Oracle for detailed usage instructions.

(The only difference in the use of the program is that the number of populations is 204, so make sure to use this if you plan to remove any reference populations, as mentioned in the instructions)

In short:

• you first load the file DodecadOracleK12a.RData in R. You can do this by double-clicking on this file in Windows, or using the File->Load Workspace menu. In Linux, you can use the "load" command, e.g., load('/home/ubuntu/Desktop/DodecadOracleK12a.RData')
• You then enter commands at the command prompt

Some examples:

Comparing a population against other populations

DodecadOracle("Somali_D")

[,1] [,2]

[1,] "Somali_D" "0"

[2,] "Ethiopian_Jews" "12.3049"

[3,] "Ethiopians" "12.3309"

[4,] "Sandawe_He" "38.2093"

[5,] "MKK25" "40.7983"

[6,] "Egyptans" "63.2307"

[7,] "Yemenese" "69.1628"

[8,] "Moroccans" "72.6233"

[9,] "Jordanians" "73.1838"

[10,] "Palestinian" "74.2867"

Comparing a population against 2-way population mixes:

DodecadOracle("Pathan",mixedmode=T)

[,1] [,2]

[1,] "Pathan" "0"

[2,] "79.5% Sindhi + 20.5% Lezgins" "3.948"

[3,] "82% Sindhi + 18% Chechens_Y" "4.0251"

[4,] "16.7% Adygei + 83.3% Sindhi" "4.5471"

[5,] "83.4% Sindhi + 16.6% Balkars_Y" "4.6487"

[6,] "80.8% Sindhi + 19.2% Kumyks_Y" "4.7067"

[7,] "83.7% Sindhi + 16.3% North_Ossetians_Y" "4.8352"

[8,] "80.9% Sindhi + 19.1% Nogais_Y" "4.8821"

[9,] "66.6% Sindhi + 33.4% Tajiks_Y" "5.6708"

[10,] "86.4% Sindhi + 13.6% Georgians" "6.2927"

Comparing an individual against populations

DodecadOracle(c(8.4, 0, 2.8, 6, 2.2, 0.1, 40.3, 25.9, 0.3, 11.9, 1.5, 0.5))

[,1] [,2]

[1,] "Iranian_D" "2.2405"

[2,] "Kurd_D" "3.8092"

[3,] "Kurds_Y" "5.4945"

[4,] "Iranians" "6.634"

[5,] "Uzbekistan_Jews" "12.8957"

[6,] "Turks" "17.3173"

[7,] "Turkmens_Y" "17.7316"

[8,] "Iranian_Jews" "18.14"

[9,] "Assyrian_D" "18.8968"

[10,] "Azerbaijan_Jews" "18.9444"

Comparing an individual against 2-way population mixes

DodecadOracle(c(28, 0.8, 1.6, 49.9, 1.9, 0, 10.6, 4.1, 0, 2.4, 0, 0.6),mixedmode=T)

[,1] [,2]

[1,] "47.7% French_D + 52.3% Mordovians_Y" "2.5849"

[2,] "48.3% French + 51.7% Mordovians_Y" "2.6012"

[3,] "36.3% Spaniards + 63.7% Mordovians_Y" "2.9985"

[4,] "36% Spanish_D + 64% Mordovians_Y" "3.0577"

[5,] "65.9% Russian_D + 34.1% Spaniards" "3.0923"

[6,] "35.9% IBS + 64.1% Mordovians_Y" "3.0943"

[7,] "40% French + 60% Ukranians_Y" "3.1662"

[8,] "66.4% Russian_D + 33.6% IBS" "3.2359"

[9,] "24.5% Swedish_D + 75.5% Hungarians" "3.3021"

[10,] "39.3% French_D + 60.7% Ukranians_Y" "3.4046"

The numbers to the right of each result represent the "goodness" of the match; the lower, the better. If you wanted to list the top-30 results, in any of the above commands, you would enter, e.g.,

DodecadOracle(c(28, 0.8, 1.6, 49.9, 1.9, 0, 10.6, 4.1, 0, 2.4, 0, 0.6),mixedmode=T, k=30)

If you recently joined the Project, please consider leaving a brief comment in the Information about Project Samples thread.

A few comments on the use of DIYDodecad 2.0

Here are some observations that might be useful to people, especially for the new byseg and target modes:

1. Finding the origin of shared segments

Until now, when you had a segment match with another customer in your testing company, you had no idea what was the origin of the shared segment. Suppose, for example, that a Russian and a German share some sequence in a region X. This could be:

• Russian-like ancestry in the German individual
• German-like ancestry in the Russian individual
• Third party ancestry in both individuals

Using the new modes, if the German saw an excess of Eastern European (relative to his usual average), then he'd pick the first scenario; if he saw nothing unremarkable, the second; if an excess of some component rare in both Russians and Germans (e.g., West_Asian), the third.

This is extremely important, as there is a noticeably confirmation bias in some individuals of interpreting the unusual as evidence of exotic ancestry. For example, an individual in search of Jewish ancestry may interpret segment matches with Jews as evidence for that ancestry: if he sees high Southwest_Asian ancestry in such segments, then that's a reasonable interpretation, but the shared segments could very well be interpreted as non-Jewish ancestry in the Jewish individual, if, they happen to be, e.g., East_European.

2. With parents' DNA

It is important to remember that each region includes both paternal and maternal DNA and you got a random draw of the segments inherited by their parents (your grandparents).

So, if you try to figure out where your region X came from, remember that it came from two places. So, if you see an unusual combination (e.g., Northeast_Asian + Northwest_African) that doesn't correspond well to any known population, this may mean that you got half of it from one parent, and the other half from the other.

Note also, that while on genomewide analysis a child's results will often be intermediate (but not necessarily so) in his ancestral components between his parents, this is not the case when looking at small segments. Suppose parent A is 50% West_Asian and 50% Mediterranean in a particular region, and parent B is 50% West_Asian and 50% West_European in the other region.

Then the child may end up with West_Asian near 100% in that region (if he happens to inherit the West_Asian segments from both parenets) or near 0% (if he happens to inherit the Mediterranean/West_European ones).

3. With Dodecad Oracle

In general, I discourage the use of Dodecad Oracle with chromosome or segment results. For two reasons:

• Small segments may appear more mixed than they are, because there may not be any informative SNPs in a particular region to distinguish between some of the ancestral components. So, the scale of the noise may be higher. As an experiment, you can average your segments, weighted by either the number of SNPs or their physical length, and you will come up with something close to your "genomewide" average, that will, however, be off, because of this factor.
• From a different perspective, segments may appear less mixed, because it is less likely that you got genetic material from all ancestral populations in a small section of your DNA. Your genomewide admixture may have several non-zero components, but you are unlikely to have many non-zero components in a small region (barring the aforementioned noise), and you could very well see >80% percentages in some of them that are very typical of a particular ancestral component.

Do-It-Yourself Dodecad v 1.0

(UDPATE: There is a new 2.0 version of the software)

I have decided to release a Do-It-Yourself calculator (and mirror), for several reasons:

1. So that people who don't want to send me their data can still get their results
2. So that people can estimate admixture proportions in all their relatives, as relatives can't be accepted in the Project
3. So that people of mixed ancestry can get their results, as there have been limited opportunities for them to submit their data to the Project so far
   
4. Most importantly, so that I won't have to do-it-myself ;-)

You need a Windows or Linux 32bit/64bit machine to run DIYDodecad. The instructions should be easy to follow, but if you encounter any bugs or have any problems, feel free to leave a comment or write to me (dodecad@gmail.com).

Of course, I will continue to ask for people to send me their data in the future: the calculator is made possible in part because of their contributions. Project participants have added benefits, such as the more specialized Clusters Galore or regional analyses.

If you are a project participant, you can still try DIYDodecad; you will get slightly different results than the ones you already have, because DIYDodecad does not use the same "random seed" as ADMIXTURE, and has a different default convergence criterion (maximum log-likelihood change of 1e-6 between successive EM iterations). You will also need the program in the future, as more "calculators" will be disseminated for it.

So, if you don't want to/can't join the Project, you can still get your Dodecad v3 results; you can also try the Dodecad Oracle with them. Also, feel free to leave a comment in this post with your results.

Related: some background on the creation of DIYDodecad

The Dodecad Oracle v1

Here is a little fun tool that tests the Dodecad v3 admixture proportions of an individual against all the reference populations, but also against the best pairwise combinations of these populations.

You need to install R to use it, and then download the program and double click on the file DodecadOracleV1.RData that can be found within the rar file. You will then be faced with a command prompt where you can enter the following commands:

Examining which populations are available

Just enter

X[,1]

You will see a list of 227 populations. You can use these population IDs in the next section.

Which populations are closest to a particular population?

Enter:

DodecadOracle("British_D")
[,1] [,2]
[1,] "British_D" "0"
[2,] "British_Isles_D" "0.9798"
[3,] "Cornwall_1KG" "1.1533"
[4,] "Kent_1KG" "2.265"
[5,] "Irish_D" "3.7643"
[6,] "Dutch_D" "4.5354"
[7,] "Mixed_Germanic_D" "6.8971"
[8,] "Norwegian_D" "11.3111"
[9,] "Orkney_1KG" "12.4652"
[10,] "Orcadian" "12.8195"

If you want to find e.g., the top-30 populations, rather than just the top-10, enter:

DodecadOracle("British_D", k=30)

Which populations are closer to a particular individual?

Enter the admixture proportions of the individual (from the "Individual results" tab of the spreadsheet) as follows:

DodecadOracle(c(4.6, 16.7, 33.6, 0, 23.2, 0.4, 0.6, 1.6, 0.7, 14.1, 4.5, 0.2))
[,1] [,2]
[1,] "Ashkenazi_D" "3.7908"
[2,] "Ashkenazy_Jews" "4.1473"
[3,] "Morocco_Jews" "6.338"
[4,] "S_Italian_Sicilian_D" "12.5443"
[5,] "Sephardic_Jews" "13.5067"
[6,] "C_Italian_D" "14.4554"
[7,] "Sicilian_D" "14.7469"
[8,] "S_Italian_D" "15.748"
[9,] "Tuscan_X" "15.9981"
[10,] "O_Italian_D" "16.1474"

Once again, you can specify k=30, if you desire the 30 top matching populations instead of the default 10.

Mixed Mode

You use mixed mode by adding mixedmode=T in any of the commands. The program then considers all pairs of populations, and for each one of them calculates the minimum distance to the sample in consideration, and the admixture proportions that produce it; population pairs where the distance to one of the two populations is smaller than to any admixture of the two are ignored.

Example:

DodecadOracle("Pathan",mixedmode=T)
[,1] [,2]
[1,] "Pathan" "0"
[2,] "84.8% Pakistani + 15.2% Urkarah" "1.075"
[3,] "84% Pakistani + 16% Stalskoe" "1.1555"
[4,] "63.9% TN_Brahmin + 36.1% Urkarah" "1.6669"
[5,] "32.4% Urkarah + 67.6% Meghawal" "2.3516"
[6,] "56.3% INS + 43.7% Urkarah" "2.4901"
[7,] "11.5% Adygei + 88.5% Pakistani" "2.6245"
[8,] "82.4% Sindhi + 17.6% Stalskoe" "2.6318"
[9,] "62.9% AP_Brahmin + 37.1% Urkarah" "2.7322"
[10,] "11.2% Lezgins + 88.8% Pakistani" "2.7749"

The mixed mode should be used with caution, and it shows, more than anything else, how similar apparent "mixes" can be achieved by different combinations of ancestry. Nonetheless, it may prove somewhat useful. For example, there is a suggestion in the above results, that Pathans can be viewed as a mix of other South Asian populations and populations from the eastern Caucasus, a suggestion that was arrived at independently by the Project using different methods.

Here is another example:

DodecadOracle("Assyrian_D",mixedmode=T)

[,1] [,2]

[1,] "Assyrian_D" "0"

[2,] "83.9% Armenians_16 + 16.1% Yemen_Jews" "1.7829"

[3,] "89.1% Armenian_D + 10.9% Saudis" "2.1624"

[4,] "84.3% Armenians_16 + 15.7% Saudis" "2.2884"

[5,] "88.9% Armenian_D + 11.1% Yemen_Jews" "2.2983"

[6,] "83.8% Armenian_D + 16.2% Bedouin" "4.1579"

[7,] "72.2% Armenian_D + 27.8% Syrians" "4.1841"

[8,] "23.4% Georgians + 76.6% Iraq_Jews" "4.2418"

[9,] "76.2% Armenians_16 + 23.8% Bedouin" "4.332"

[10,] "61.5% Armenians_16 + 38.5% Syrians" "4.4019"

This reaffirms the close relationship of Assyrians to Armenians that has been noticed in the project and by others, and it also shows that Assyrians differ from Armenians in a Southwestern Asian direction, consistent with their Semitic language.

Or, African Americans:

DodecadOracle("ASW",mixedmode=T)

[,1] [,2]

[1,] "ASW" "0"

[2,] "81.3% Hausa + 18.7% N._European" "2.3891"

[3,] "18.4% Orkney_1KG + 81.6% Hausa" "2.4031"

[4,] "18.5% Argyll_1KG + 81.5% Hausa" "2.4268"

[5,] "18.4% Orcadian + 81.6% Hausa" "2.4657"

[6,] "80.5% Igbo + 19.5% N._European" "2.5031"

[7,] "80.6% Brong + 19.4% N._European" "2.523"

[8,] "18.6% CEU + 81.4% Hausa" "2.5938"

[9,] "19.1% Argyll_1KG + 80.9% Brong" "2.6197"

[10,] "19% Orkney_1KG + 81% Brong" "2.6274"

I don't know that much about the slave trade, but I believe that Ghana was an important part of it?

Another thing to watch, is that some populations tend to have more than one sample available, so they appear to be mixtures of themselves, which is not really very informative, e.g., Spanish_D

DodecadOracle("Spanish_D",mixedmode=T)

[,1] [,2]

[1,] "Spanish_D" "0"

[2,] "7.9% French_Basque + 92.1% IBS" "0.8713"

[3,] "68.9% IBS + 31.1% Spaniards" "1.0377"

[4,] "98.8% IBS + 1.2% Irish_D" "1.2959"

[5,] "1.2% British_Isles_D + 98.8% IBS" "1.3018"

[6,] "1.2% British_D + 98.8% IBS" "1.3019"

[7,] "99% IBS + 1% Norwegian_D" "1.3046"

[8,] "1.2% Cornwall_1KG + 98.8% IBS" "1.3048"

[9,] "98.8% IBS + 1.2% Kent_1KG" "1.3142"

[10,] "2.2% French_D + 97.8% IBS" "1.3179"

To deal with these problems, you must "edit" the X matrix if you want to exclude some populations. For example, if you want to exclude "Spaniards" and "IBS", you must enter:

X <- X[setdiff(1:227,which(X[,1]=="IBS" | X[,1]=="Spaniards")),]

but notice, that you must relaunch the program, if you want to get the original matrix, or alternatively save it like this:

Z<-X

and then retrieve it like this:

X<-Z