User menu

Accès à distance ? S'identifier sur le proxy UCLouvain

On-Demand Service-Based Big Data Integration: Optimized for Research Collaboration

Primary tabs

download
  • Open access
  • PDF
  • 763.34 K
Kathiravelu, Pradeeban [UCL] Van Roy, Peter [UCL]

Biomedical research requires distributed access, analysis, and sharing of data from various disperse sources in the Internet scale. Due to the volume and variety of big data, materialized data integration is often infeasible or too expensive including the costs of bandwidth, storage, maintenance, and management. Óbidos (On-demand Big Data Integration, Distribution, and Orchestration System) provides a novel on-demand integration approach for heterogeneous distributed data. Instead of integrating data from the data sources to build a complete data warehouse as the initial step, Óbidos employs a hybrid approach of virtual and materialized data integrations. By allocating unique identifiers as pointers to virtually integrated data sets, Óbidos supports efficient data sharing among data consumers. We design Óbidos as a generic service-based data integration system, and implement and evaluate a prototype for multimodal medical data.

  1. Lee George, Doyle Scott, Monaco James, Madabhushi Anant, Feldman Michael D., Master Stephen R., Tomaszewski John E., A knowledge representation framework for integration, classification of multi-scale imaging and non-imaging data: Preliminary results in predicting prostate cancer recurrence by fusing mass spectrometry and histology, 10.1109/isbi.2009.5192987
  2. Huang, Z.: Data Integration For Urban Transport Planning. Citeseer (2003)
  3. Sujansky Walter, Heterogeneous Database Integration in Biomedicine, 10.1006/jbin.2001.1024
  4. Mildenberger Peter, Eichelberg Marco, Martin Eric, Introduction to the DICOM standard, 10.1007/s003300101100
  5. Whitcher Brandon, Schmid Volker J., Thornton Andrew, Working with the DICOM and NIfTI Data Standards inR, 10.18637/jss.v044.i06
  6. Kargín Yağíz, Ivanova Milena, Zhang Ying, Manegold Stefan, Kersten Martin, Lazy ETL in action : ETL technology dates scientific data, 10.14778/2536274.2536297
  7. Dong X. L., Srivastava D., Big data integration, 10.1109/icde.2013.6544914
  8. Rustagi, A.: Parallel processing for ETL processes. US Patent App. 11/682,815 (2007)
  9. Porter, D.L., Swanholm, D.E.: Distributed extract, transfer, and load (ETL) computer method. US Patent 7,051,334 (2006)
  10. Rimal Bhaskar Prasad, Choi Eunmi, Lumb Ian, A Taxonomy and Survey of Cloud Computing Systems, 10.1109/ncm.2009.218
  11. Kathiravelu Pradeeban, Galhardas Helena, Veiga Luís, $$\partial u\partial u$$ Multi-Tenanted Framework: Distributed Near Duplicate Detection for Big Data, Lecture Notes in Computer Science (2015) ISBN:9783319261478 p.237-256, 10.1007/978-3-319-26148-5_14
  12. White, T.: Hadoop: The Definitive Guide. O’Reilly Media Inc., Sebastopol (2012)
  13. Marchioni, F., Surtani, M.: Infinispan Data Grid Platform. Packt Publishing Ltd., Birmingham (2012)
  14. Thusoo Ashish, Sarma Joydeep Sen, Jain Namit, Shao Zheng, Chakka Prasad, Anthony Suresh, Liu Hao, Wyckoff Pete, Murthy Raghotham, Hive : a warehousing solution over a map-reduce framework, 10.14778/1687553.1687609
  15. Hausenblas Michael, Nadeau Jacques, Apache Drill: Interactive Ad-Hoc Analysis at Scale, 10.1089/big.2013.0011
  16. Veiga L., Ferreira P., Incremental replication for mobility support in OBIWAN, 10.1109/icdcs.2002.1022262
  17. Xiao Chuan, Wang Wei, Lin Xuemin, Yu Jeffrey Xu, Wang Guoren, Efficient similarity joins for near-duplicate detection, 10.1145/2000824.2000825
  18. Clark Kenneth, Vendt Bruce, Smith Kirk, Freymann John, Kirby Justin, Koppel Paul, Moore Stephen, Phillips Stanley, Maffitt David, Pringle Michael, Tarbox Lawrence, Prior Fred, The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository, 10.1007/s10278-013-9622-7
  19. Gradecki, J.D., Cole, J.: Mastering Apache Velocity. Wiley (2003)
  20. Antonioletti Mario, Atkinson Malcolm, Baxter Rob, Borley Andrew, Chue Hong Neil P., Collins Brian, Hardman Neil, Hume Alastair C., Knox Alan, Jackson Mike, Krause Amy, Laws Simon, Magowan James, Paton Norman W., Pearson Dave, Sugden Tom, Watson Paul, Westhead Martin, The design and implementation of Grid database services in OGSA-DAI, 10.1002/cpe.939
  21. Borckholder Chris, Heinzel Andreas, Kaniovskyi Yuriy, Benkner Siegfried, Lukas Arno, Mayer Bernd, A Generic, Service-based Data Integration Framework Applied to Linking Drugs & Clinical Trials, 10.1016/j.procs.2013.10.005
  22. Lecarpentier Damien, Wittenburg Peter, Elbers Willem, Michelini Alberto, Kanso Riam, Coveney Peter, Baxter Rob, EUDAT: A New Cross-Disciplinary Data Infrastructure for Science, 10.2218/ijdc.v8i1.260
  23. Widmann, H., Thiemann, H.: EUDAT B2FIND: a cross-discipline metadata service and discovery portal. In: EGU General Assembly Conference Abstracts, vol. 18, p. 8562 (2016)
  24. Ardestani, S.B., Håkansson, C.J., Laure, E., Livenson, I., Stranák, P., Dima, E., Blommesteijn, D., van de Sanden, M.: B2SHARE: an open eScience data sharing platform. In: 2015 IEEE 11th International Conference on e-Science (e-Science), pp. 448–453. IEEE (2015)
  25. Hairong Qi, Iyengar S., Chakrabarty K., Multiresolution data integration using mobile agents in distributed sensor networks, 10.1109/5326.971666
  26. Ahern, T., Casey, R., Barnes, D., Benson, R., Knight, T.: Seed standard for the exchange of earthquake data reference manual format version 2.4. Incorporated Research Institutions for Seismology (IRIS), Seattle (2007)
  27. Milchevski, E., Michel, S.: ligDB-online query processing without (almost) any storage. In: EDBT, pp. 683–688 (2015)
  28. Lyu Dao-Ming, Tian Yu, Wang Yu, Tong Dan-Yang, Yin Wei-Wei, Li Jing-Song, Design and Implementation of Clinical Data Integration and Management System Based on Hadoop Platform, 10.1109/itme.2015.86
  29. Kathiravelu Pradeeban, Sharma Ashish, A Dynamic Data Warehousing Platform for Creating and Accessing Biomedical Data Lakes, Data Management and Analytics for Medicine and Healthcare (2017) ISBN:9783319577401 p.101-120, 10.1007/978-3-319-57741-8_7
Bibliographic reference Kathiravelu, Pradeeban ; Van Roy, Peter ; et. al. On-Demand Service-Based Big Data Integration: Optimized for Research Collaboration. (2017)
Permanent URL http://hdl.handle.net/2078.1/196145