Home LiteratureArticle Details
PMID: 26901648 Published · ppublish English Journal Article Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't

Inferring causal molecular networks: empirical assessment through a community-based effort.

Nature methods ·Vol. 13 ·No. 4 ·2016-04-00 ·Pages 310-8

Hill SM, Heiser LM, Cokelaer T, Unger M, Nesser NK, Carlin DE, Zhang Y, Sokolov A, Paull EO, Wong CK, Graim K, Bivol A, Wang H, Zhu F, Afsari B, Danilova LV, Favorov AV, Lee WS, Taylor D, Hu CW, Long BL, Noren DP, Bisberg AJ, HPN-DREAM Consortium, Mills GB, Gray JW, Kellen M, Norman T, Friend S, Qutub AA, Fertig EJ, Guan Y, Song M, Stuart JM, Spellman PT, Koeppl H, Stolovitzky G, Saez-Rodriguez J, Mukherjee S

Abstract

It remains unclear whether causal, rather than merely correlational, relationships in molecular networks can be inferred in complex biological settings. Here we describe the HPN-DREAM network inference challenge, which focused on learning causal influences in signaling networks. We used phosphoprotein data from cancer cell lines as well as in silico data from a nonlinear dynamical model. Using the phosphoprotein data, we scored more than 2,000 networks submitted by challenge participants. The networks spanned 32 biological contexts and were scored in terms of causal validity with respect to unseen interventional data. A number of approaches were effective, and incorporating known biology was generally advantageous. Additional sub-challenges considered time-course prediction and visualization. Our results suggest that learning causal relationships may be feasible in complex settings such as disease states. Furthermore, our scoring approach provides a practical way to empirically assess inferred molecular networks in a causal sense.

MeSH Terms
Algorithms Causality Computational Biology Computer Simulation Gene Expression Profiling Gene Regulatory Networks Humans Models, Biological Neoplasms/genetics Protein Interaction Mapping/methods Signal Transduction Software Systems Biology Tumor Cells, Cultured
Authors & Affiliations
39 authors, click to expand affiliations / ORCID
Hill Steven M
MRC Biostatistics Unit, Cambridge Institute of Public Health, Cambridge, UK.
Heiser Laura M
Department of Biomedical Engineering, Oregon Health and Science University, Portland, Oregon, USA. | Center for Spatial Systems Biomedicine, Oregon Health and Science University, Portland, Oregon, USA. | Knight Cancer Institute, Oregon Health and Science University, Portland, Oregon, USA.
Cokelaer Thomas
European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome Campus, Hinxton, UK.
Unger Michael
Automatic Control Laboratory, ETH Zurich, Zurich, Switzerland. | Institute of Biochemistry, ETH Zurich, Zurich, Switzerland.
Nesser Nicole K
Department of Molecular and Medical Genetics, Oregon Health and Science University, Portland, Oregon, USA.
Carlin Daniel E
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Zhang Yang
Department of Computer Science, New Mexico State University, Las Cruces, New Mexico, USA.
Sokolov Artem
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Paull Evan O
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Wong Chris K
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Graim Kiley ORCID
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Bivol Adrian
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Wang Haizhou
Department of Computer Science, New Mexico State University, Las Cruces, New Mexico, USA.
Zhu Fan
Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA.
Afsari Bahman
Department of Oncology, Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Maryland, USA.
Danilova Ludmila V ORCID
Department of Oncology, Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Maryland, USA. | Laboratory of Systems Biology and Computational Genetics, Vavilov Institute of General Genetics, Russian Academy of Sciences, Moscow, Russia.
Favorov Alexander V
Department of Oncology, Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Maryland, USA. | Laboratory of Systems Biology and Computational Genetics, Vavilov Institute of General Genetics, Russian Academy of Sciences, Moscow, Russia. | Laboratory of Bioinformatics, Research Institute of Genetics and Selection of Industrial Microorganisms, Moscow, Russia.
Lee Wai Shing
Department of Oncology, Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Maryland, USA.
Taylor Dane ORCID
Statistical and Applied Mathematical Sciences Institute, Research Triangle Park, North Carolina, USA. | Department of Mathematics, University of North Carolina, Chapel Hill, North Carolina, USA.
Hu Chenyue W
Department of Bioengineering, Rice University, Houston, Texas, USA.
Long Byron L
Department of Bioengineering, Rice University, Houston, Texas, USA.
Noren David P
Department of Bioengineering, Rice University, Houston, Texas, USA.
Bisberg Alexander J
Department of Bioengineering, Rice University, Houston, Texas, USA.
HPN-DREAM Consortium
Mills Gordon B
Department of Systems Biology, MD Anderson Cancer Center, Houston, Texas, USA.
Gray Joe W
Department of Biomedical Engineering, Oregon Health and Science University, Portland, Oregon, USA. | Center for Spatial Systems Biomedicine, Oregon Health and Science University, Portland, Oregon, USA. | Knight Cancer Institute, Oregon Health and Science University, Portland, Oregon, USA.
Kellen Michael ORCID
Sage Bionetworks, Seattle, Washington, USA.
Norman Thea
Sage Bionetworks, Seattle, Washington, USA.
Friend Stephen ORCID
Sage Bionetworks, Seattle, Washington, USA.
Qutub Amina A
Department of Bioengineering, Rice University, Houston, Texas, USA.
Fertig Elana J
Department of Oncology, Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Maryland, USA.
Guan Yuanfang
Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA. | Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan, USA. | Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, Michigan, USA.
Song Mingzhou ORCID
Department of Computer Science, New Mexico State University, Las Cruces, New Mexico, USA.
Stuart Joshua M
Department of Biomolecular Engineering, University of California Santa Cruz, Santa Cruz, California, USA.
Spellman Paul T
Department of Molecular and Medical Genetics, Oregon Health and Science University, Portland, Oregon, USA.
Koeppl Heinz
Automatic Control Laboratory, ETH Zurich, Zurich, Switzerland. | Institute of Biochemistry, ETH Zurich, Zurich, Switzerland.
Stolovitzky Gustavo
IBM Translational Systems Biology and Nanobiotechnology, Yorktown Heights, New York, USA.
Saez-Rodriguez Julio ORCID
European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome Campus, Hinxton, UK. | RWTH-Aachen University Hospital, Joint Research Centre for Computational Biomedicine (JRC-COMBINE), Aachen, Germany.
Mukherjee Sach
MRC Biostatistics Unit, Cambridge Institute of Public Health, Cambridge, UK. | School of Clinical Medicine, University of Cambridge, Cambridge, UK. | German Centre for Neurodegenerative Diseases (DZNE), Bonn, Germany.
Investigators
136 investigators, click to expand
Afsari Bahman
Al-Ouran Rami
Anton Bernat
Arodz Tomasz
Sichani Omid Askari
Bagheri Neda
Berlow Noah
Bisberg Alexander J
Bivol Adrian
Bohler Anwesha
Bonet Jaume
Bonneau Richard
Budak Gungor
Bunescu Razvan
Caglar Mehmet
Cai Binghuang
Cai Chunhui
Carlin Daniel E
Carlon Azzurra
Chen Lujia
Ciaccio Mark F
Cokelaer Thomas
Cooper Gregory
Coort Susan
Creighton Chad J
Daneshmand Seyed-Mohammad-Hadi
de la Fuente Alberto
Di Camillo Barbara
Danilova Ludmila V
Dutta-Moscato Joyeeta
Emmett Kevin
Evelo Chris
Fassia Mohammad-Kasim H
Favorov Alexander V
Fertig Elana J
Finkle Justin D
Finotello Francesca
Friend Stephen
Gao Xi
Gao Jean
Garcia-Garcia Javier
Ghosh Samik
Giaretta Alberto
Graim Kiley
Gray Joe W
Großeholz Ruth
Guan Yuanfang
Guinney Justin
Hafemeister Christoph
Hahn Oliver
Haider Saad
Hase Takeshi
Heiser Laura M
Hill Steven M
Hodgson Jay
Hoff Bruce
Hsu Chih Hao
Hu Chenyue W
Hu Ying
Huang Xun
Jalili Mahdi
Jiang Xia
Kacprowski Tim
Kaderali Lars
Kang Mingon
Kannan Venkateshan
Kellen Michael
Kikuchi Kaito
Kim Dong-Chul
Kitano Hiroaki
Knapp Bettina
Komatsoulis George
Koeppl Heinz
Krämer Andreas
Kursa Miron Bartosz
Kutmon Martina
Lee Wai Shing
Li Yichao
Liang Xiaoyu
Liu Zhaoqi
Liu Yu
Long Byron L
Lu Songjian
Lu Xinghua
Manfrini Marco
Matos Marta R A
Meerzaman Daoud
Mills Gordon B
Min Wenwen
Mukherjee Sach
Müller Christian Lorenz
Neapolitan Richard E
Nesser Nicole K
Noren David P
Norman Thea
Oliva Baldo
Opiyo Stephen Obol
Pal Ranadip
Palinkas Aljoscha
Paull Evan O
Planas-Iglesias Joan
Poglayen Daniel
Qutub Amina A
Saez-Rodriguez Julio
Sambo Francesco
Sanavia Tiziana
Sharifi-Zarchi Ali
Slawek Janusz
Sokolov Artem
Song Mingzhou
Spellman Paul T
Streck Adam
Stolovitzky Gustavo
Strunz Sonja
Stuart Joshua M
Taylor Dane
Tegnér Jesper
Thobe Kirste
Toffolo Gianna Maria
Trifoglio Emanuele
Unger Michael
Wan Qian
Wang Haizhou
Welch Lonnie
Wong Chris K
Wu Jia J
Xue Albert Y
Yamanaka Ryota
Yan Chunhua
Zairis Sakellarios
Zengerling Michael
Zenil Hector
Zhang Shihua
Zhang Yang
Zhu Fan
Zi Zhike
References (43)
43 references, click to expand
  1. Perturbation biology: inferring signaling networks in cellular systems.
    PLoS Comput Biol. 2013;9(12):e1003290 PMID: 24367245
  2. The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity.
    Nature. 2012 Mar 28;483(7391):603-7 PMID: 22460905
  3. Comparing signaling networks between normal and transformed hepatocytes using discrete logical models.
    Cancer Res. 2011 Aug 15;71(16):5400-11 PMID: 21742771
  4. Developing predictive molecular maps of human disease through community-based modeling.
    Nat Genet. 2012 Jan 27;44(2):127-30 PMID: 22281773
  5. Bayesian inference of signaling network topology in a cancer cell line.
    Bioinformatics. 2012 Nov 1;28(21):2804-10 PMID: 22923301
  6. Systematic identification of genomic markers of drug sensitivity in cancer cells.
    Nature. 2012 Mar 28;483(7391):570-5 PMID: 22460902
  7. Crowdsourcing network inference: the DREAM predictive signaling network challenge.
    Sci Signal. 2011 Aug 30;4(189):mr7 PMID: 21900204
  8. The Cyni framework for network inference in Cytoscape.
    Bioinformatics. 2015 May 1;31(9):1499-501 PMID: 25527096
  9. Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach.
    BMC Syst Biol. 2014 Feb 07;8:13 PMID: 24507381
  10. Systematic analysis of challenge-driven improvements in molecular prognostic models for breast cancer.
    Sci Transl Med. 2013 Apr 17;5(181):181re1 PMID: 23596205
  11. Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks.
    Bioinformatics. 2003 Nov 22;19(17):2271-82 PMID: 14630656
  12. Revealing strengths and weaknesses of methods for gene network inference.
    Proc Natl Acad Sci U S A. 2010 Apr 6;107(14):6286-91 PMID: 20308593
  13. Predicting causal effects in large-scale systems from observational data.
    Nat Methods. 2010 Apr;7(4):247-8 PMID: 20354511
  14. Using Bayesian networks to analyze expression data.
    J Comput Biol. 2000;7(3-4):601-20 PMID: 11108481
  15. Ckmeans.1d.dp: Optimal k-means Clustering in One Dimension by Dynamic Programming.
    R J. 2011 Dec;3(2):29-33 PMID: 27942416
  16. Advantages and limitations of current network inference methods.
    Nat Rev Microbiol. 2010 Oct;8(10):717-29 PMID: 20805835
  17. Integrating literature-constrained and data-driven inference of signalling networks.
    Bioinformatics. 2012 Sep 15;28(18):2311-7 PMID: 22734019
  18. Prediction of human population responses to toxic compounds by a collaborative competition.
    Nat Biotechnol. 2015 Sep;33(9):933-40 PMID: 26258538
  19. Supervised, semi-supervised and unsupervised inference of gene regulatory networks.
    Brief Bioinform. 2014 Mar;15(2):195-211 PMID: 23698722
  20. Dialogue on reverse-engineering assessment and methods: the DREAM of high-throughput pathway inference.
    Ann N Y Acad Sci. 2007 Dec;1115:1-22 PMID: 17925349
  21. Causal protein-signaling networks derived from multiparameter single-cell data.
    Science. 2005 Apr 22;308(5721):523-9 PMID: 15845847
  22. Context Specificity in Causal Signaling Networks Revealed by Phosphoprotein Profiling.
    Cell Syst. 2017 Jan 25;4(1):73-83.e10 PMID: 28017544
  23. AZD8055 is a potent, selective, and orally bioavailable ATP-competitive mammalian target of rapamycin kinase inhibitor with in vitro and in vivo antitumor activity.
    Cancer Res. 2010 Jan 1;70(1):288-98 PMID: 20028854
  24. Lessons from the DREAM2 Challenges.
    Ann N Y Acad Sci. 2009 Mar;1158:159-95 PMID: 19348640
  25. Ischemia in tumors induces early and sustained phosphorylation changes in stress kinase pathways but does not affect global protein levels.
    Mol Cell Proteomics. 2014 Jul;13(7):1690-704 PMID: 24719451
  26. Pathway Commons, a web resource for biological pathway data.
    Nucleic Acids Res. 2011 Jan;39(Database issue):D685-90 PMID: 21071392
  27. Cytoscape: a software environment for integrated models of biomolecular interaction networks.
    Genome Res. 2003 Nov;13(11):2498-504 PMID: 14597658
  28. Input-output behavior of ErbB signaling pathways as revealed by a mass action model trained against dynamic data.
    Mol Syst Biol. 2009;5:239 PMID: 19156131
  29. A Technical Assessment of the Utility of Reverse Phase Protein Arrays for the Study of the Functional Proteome in Non-microdissected Human Breast Cancers.
    Clin Proteomics. 2010 Dec;6(4):129-51 PMID: 21691416
  30. Differential network biology.
    Mol Syst Biol. 2012 Jan 17;8:565 PMID: 22252388
  31. Gene regulatory network inference: data integration in dynamic models-a review.
    Biosystems. 2009 Apr;96(1):86-103 PMID: 19150482
  32. Wisdom of crowds for robust gene network inference.
    Nat Methods. 2012 Jul 15;9(8):796-804 PMID: 22796662
  33. A collection of breast cancer cell lines for the study of functionally distinct cancer subtypes.
    Cancer Cell. 2006 Dec;10(6):515-27 PMID: 17157791
  34. A pan-cancer proteomic perspective on The Cancer Genome Atlas.
    Nat Commun. 2014 May 29;5:3887 PMID: 24871328
  35. From 'differential expression' to 'differential networking' - identification of dysfunctional regulatory networks in diseases.
    Trends Genet. 2010 Jul;26(7):326-33 PMID: 20570387
  36. Inferring cellular networks--a review.
    BMC Bioinformatics. 2007 Sep 27;8 Suppl 6:S5 PMID: 17903286
  37. A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches.
    Cell. 2009 Apr 3;137(1):172-81 PMID: 19327819
  38. Reverse phase protein array: validation of a novel proteomic technology and utility for analysis of primary leukemia specimens and hematopoietic stem cells.
    Mol Cancer Ther. 2006 Oct;5(10):2512-21 PMID: 17041095
  39. DREAMTools: a Python package for scoring collaborative challenges.
    F1000Res. 2015 Oct 09;4:1030 PMID: 27134723
  40. A community effort to assess and improve drug sensitivity prediction algorithms.
    Nat Biotechnol. 2014 Dec;32(12):1202-12 PMID: 24880487
  41. How to infer gene networks from expression profiles.
    Mol Syst Biol. 2007;3:78 PMID: 17299415
  42. Towards a rigorous assessment of systems biology models: the DREAM3 challenges.
    PLoS One. 2010 Feb 23;5(2):e9202 PMID: 20186320
  43. Inference and validation of predictive gene networks from biomedical literature and gene expression data.
    Genomics. 2014 May-Jun;103(5-6):329-36 PMID: 24691108
Article Info
Journal
Nature methods
Abbr.
Nat Methods
ISSN
1548-7105
Published
2016-04-00
Epub
2016-00-22
Pages
310-8
Language
English
Region
United States
NLM ID
101215604
PMCID
PMC4854847
Subset
IM
Grants
NCI NIH HHS · P30 CA016672 · United States
NIGMS NIH HHS · R01 GM109031 · United States
Medical Research Council · MC_UP_1302/1 · United Kingdom
NLM NIH HHS · R00LM010822 · United States
NLM NIH HHS · R01LM011663 · United States
NIGMS NIH HHS · 1R01GM109031 · United States
NCI NIH HHS · U54CA143869 · United States
NCI NIH HHS · U54 CA112970 · United States
NLM NIH HHS · R01 LM011663 · United States
NCI NIH HHS · U54 CA143869 · United States
NCI NIH HHS · 5R01CA180778 · United States
NCI NIH HHS · K25 CA141053 · United States
NLM NIH HHS · R00 LM010822 · United States
NCI NIH HHS · R01 CA180778 · United States
NHGRI NIH HHS · U54 HG007990 · United States
NCI NIH HHS · R01 CA177669 · United States
NLM NIH HHS · R00 LM011673 · United States
NLM NIH HHS · R01 LM012011 · United States
Analysis Services
Analysis Services

Contact

No. 2 Wenbo Road, Zhangqiu District, Jinan, Shandong

Qilu Normal University · Genelibs Bioinformatics Lab

750 Shunhua Rd, Jinan

2F, Bldg F, University Science Park

Tel: 0531-88819269

WeChat Official Account

Follow our WeChat subscription account for real-time updates and the latest in medical and biological research.


Business Email

E-mail: [email protected]