Performance Evaluation of the Karma Provenance Framework for Scientific Workflows
Y. Simmhan, B. Plale, and D. Gannon. International Provenance and Annotation Workshop (IPAW), volume 4145 of Lecture Notes in Computer Science (LNCS), page 222--236. Springer Berlin / Heidelberg, (2006)
DOI: 10.1007/11890850_23
Abstract
Provenance about workflow executions and data derivations in scientific applications help estimate data quality, track resources, and validate in silico experiments. The Karma provenance framework provides a means to collect workflow, process, and data provenance from data-driven scientific workflows and is used in the Linked Environments for Atmospheric Discovery (LEAD) project. This paper presents a performance analysis of the Karma service as compared against the contemporary PReServ provenance service. Our study finds that Karma scales exceedingly well for collecting and querying provenance records, showing linear or sub-linear scaling with increasing number of provenance records and clients when tested against workloads in the order of 10,000 application-service invocations and over 36 concurrent clients.
%0 Conference Paper
%1 Simmhan:ipaw:2006
%A Simmhan, Yogesh L.
%A Plale, Beth
%A Gannon, Dennis
%B International Provenance and Annotation Workshop (IPAW)
%D 2006
%E Moreau, Luc
%E Foster, Ian
%I Springer Berlin / Heidelberg
%K escience, iu, karma, peer provenance, reviewed workflows,
%P 222--236
%R 10.1007/11890850_23
%T Performance Evaluation of the Karma Provenance Framework for Scientific Workflows
%V 4145
%X Provenance about workflow executions and data derivations in scientific applications help estimate data quality, track resources, and validate in silico experiments. The Karma provenance framework provides a means to collect workflow, process, and data provenance from data-driven scientific workflows and is used in the Linked Environments for Atmospheric Discovery (LEAD) project. This paper presents a performance analysis of the Karma service as compared against the contemporary PReServ provenance service. Our study finds that Karma scales exceedingly well for collecting and querying provenance records, showing linear or sub-linear scaling with increasing number of provenance records and clients when tested against workloads in the order of 10,000 application-service invocations and over 36 concurrent clients.
@inproceedings{Simmhan:ipaw:2006,
abstract = {Provenance about workflow executions and data derivations in scientific applications help estimate data quality, track resources, and validate in silico experiments. The Karma provenance framework provides a means to collect workflow, process, and data provenance from data-driven scientific workflows and is used in the Linked Environments for Atmospheric Discovery (LEAD) project. This paper presents a performance analysis of the Karma service as compared against the contemporary PReServ provenance service. Our study finds that Karma scales exceedingly well for collecting and querying provenance records, showing linear or sub-linear scaling with increasing number of provenance records and clients when tested against workloads in the order of 10,000 application-service invocations and over 36 concurrent clients.},
added-at = {2023-04-07T07:37:58.000+0200},
author = {Simmhan, Yogesh L. and Plale, Beth and Gannon, Dennis},
biburl = {https://www.bibsonomy.org/bibtex/2f7e694d48cb9e068cdccdbb940ef1c07/vinayaka2000},
booktitle = {International Provenance and Annotation Workshop (IPAW)},
doi = {10.1007/11890850_23},
editor = {Moreau, Luc and Foster, Ian},
interhash = {2385d297effbd0beaf3de4ab5619f9d6},
intrahash = {f7e694d48cb9e068cdccdbb940ef1c07},
keywords = {escience, iu, karma, peer provenance, reviewed workflows,},
owner = {ysimmhan},
pages = {222--236},
publisher = {Springer Berlin / Heidelberg},
series = {Lecture Notes in Computer Science (LNCS)},
timestamp = {2023-04-07T07:37:58.000+0200},
title = {Performance Evaluation of the Karma Provenance Framework for Scientific Workflows},
volume = 4145,
year = 2006
}