Commit cf4c0fb7 authored by Jerome Mariette's avatar Jerome Mariette
Browse files

speling correction

parent c8e3a563
...@@ -25,18 +25,19 @@ Castanet-Tolosan Cedex, France.} ...@@ -25,18 +25,19 @@ Castanet-Tolosan Cedex, France.}
\section{Summary:} \section{Summary:}
Biologists produce large data sets and are in demand of rich and simple Biologists produce large data sets and are in demand of rich and simple
web portals in which they can upload and analyse their files. Providing such web portals in which they can upload and analyze their files. Providing such
tools requires to mask the complexity induced by the needed High Perfomance tools requires to mask the complexity induced by the needed High Performance
Computing (HPC) environment. The connexion between interface and computing Computing (HPC) environment. The connexion between interface and computing
infrastructure is usually specific to each portal. With Jflow, we introduce infrastructure is usually specific to each portal. With Jflow, we introduce
a Workflow Management System (WMS), composed of Jquery plugins which can easily be a Workflow Management System (WMS), composed of Jquery plug-ins which can easily
be
embedded in any web application and a Python library providing all requested embedded in any web application and a Python library providing all requested
features to setup, run and monitor workflows. features to setup, run and monitor workflows.
\section{Availability:} \section{Availability:}
Jflow is available under the GNU General Public License (GPL) at Jflow is available under the GNU General Public License (GPL) at
http://bioinfo.genotoul.fr/jflow. The package is comming with full http://bioinfo.genotoul.fr/jflow. The package is coming with full
documentation, quickstart and a running test portal. documentation, quick start and a running test portal.
\section{Contact:} \section{Contact:}
\href{support.genopole@toulouse.inra.fr}{support.genopole@toulouse.inra.fr} \href{support.genopole@toulouse.inra.fr}{support.genopole@toulouse.inra.fr}
...@@ -44,7 +45,7 @@ documentation, quickstart and a running test portal. ...@@ -44,7 +45,7 @@ documentation, quickstart and a running test portal.
\section{Introduction} \section{Introduction}
Building rich web environments aimed at helping scientists analyse their data is Building rich web environments aimed at helping scientists analyze their data is
a common trend in bioinformatics. Specialized web portals such as MG-RAST (Meyer a common trend in bioinformatics. Specialized web portals such as MG-RAST (Meyer
et al., 2008), MetaVir (Roux et al., 2011) or NG6 (Mariette et al., 2012) provide et al., 2008), MetaVir (Roux et al., 2011) or NG6 (Mariette et al., 2012) provide
multiple services and analysis tools in an integrated manner for specific multiple services and analysis tools in an integrated manner for specific
...@@ -65,12 +66,12 @@ definition or check file formats. ...@@ -65,12 +66,12 @@ definition or check file formats.
To our knowledge, Jflow, presented in this article, is the only WMS designed to To our knowledge, Jflow, presented in this article, is the only WMS designed to
be embedded in any web application, thanks to its organization as five Jquery be embedded in any web application, thanks to its organization as five Jquery
(http://jquery.com/) plugins. (http://jquery.com/) plug-ins.
\section{Methods} \section{Methods}
Jflow user interface gathers five Jquery plugins providing user oriented views. Jflow user interface gathers five Jquery plug-ins providing user oriented views.
\begin{itemize} \begin{itemize}
\item \textit{availablewf} lists all runnable workflows accessible to users, \item \textit{availablewf} lists all runnable workflows accessible to users,
...@@ -80,23 +81,23 @@ Jflow user interface gathers five Jquery plugins providing user oriented views. ...@@ -80,23 +81,23 @@ Jflow user interface gathers five Jquery plugins providing user oriented views.
\item \textit{wfoutputs} displays all outputs produced by the workflow \item \textit{wfoutputs} displays all outputs produced by the workflow
organized per component, organized per component,
\item \textit{wfstatus} shows the workflow execution state as a list or an \item \textit{wfstatus} shows the workflow execution state as a list or an
execution graph. The graph visualisation uses the Cytoscape web JavaScript execution graph. The graph visualization uses the Cytoscape web JavaScript
plugin (Lopes et al., 2010). plug-in (Lopes et al., 2010).
\end{itemize} \end{itemize}
The plugins give access to multiple communication methods and events. They The plug ins give access to multiple communication methods and events. They
interact with the server side through Jflow's REST API, running under a cherrypy interact with the server side through Jflow's REST API, running under a cherrypy
(http://www.cherrypy.org/) web server. The included server uses the JSONP (http://www.cherrypy.org/) web server. The included server uses the JSONP
communication technique enabling cross-domain requests. communication technique enabling cross-domain requests.
To be available from the different Jquery plugins, the workflows have to be To be available from the different Jquery plug-ins, the workflows have to be
implemented using the Jflow API. A Jflow component is in charge of an execution implemented using the Jflow API. A Jflow component is in charge of an execution
step. Adding a component to the system requires to write a Python step. Adding a component to the system requires to write a Python
\textit{Component} subclass. In Jflow, different solutions are available to ease \textit{Component} subclass. In Jflow, different solutions are available to ease
component creation. To wrap a single command line, the developer can give a component creation. To wrap a single command line, the developer can give a
position or a flag for each parameter. Jflow also embeds an XML parser which position or a flag for each parameter. Jflow also embeds an XML parser which
allows to run geniune Mobyle (Neron et al., 2009) components. Finally, to allows to run genuine Mobyle (Neron et al., 2009) components. Finally, to
allow developpers to integrate components from other WMS, Jflow provides a class allow developers to integrate components from other WMS, Jflow provides a class
skeleton. This class only requires to implement the parsing step. A workflow skeleton. This class only requires to implement the parsing step. A workflow
chains several components. A Jflow workflow is built as a \textit{Workflow} chains several components. A Jflow workflow is built as a \textit{Workflow}
subclass. Components are added to the workflow as variables and chained linking subclass. Components are added to the workflow as variables and chained linking
...@@ -112,14 +113,14 @@ data type compliance for each parameter. Job submission, status ...@@ -112,14 +113,14 @@ data type compliance for each parameter. Job submission, status
checking and error handling, rely on Makeflow (Albrecht et al., 2012) and checking and error handling, rely on Makeflow (Albrecht et al., 2012) and
weaver (Bui et al., 2012). Therefore Jflow manages error recovery and supports weaver (Bui et al., 2012). Therefore Jflow manages error recovery and supports
most distributed resource management systems (Condor, SGE, Work Queue or a most distributed resource management systems (Condor, SGE, Work Queue or a
single multicore machine, \ldots). Replacing Makeflow by an other job submiter single multi core machine, \ldots). Replacing Makeflow by an other job submitter
requires to implement a new \textit{Engine} subclass. This class creates and requires to implement a new \textit{Engine} subclass. This class creates and
execute the workflow directed acyclic graph (DAG). execute the workflow directed acyclic graph (DAG).
\section{Example} \section{Example}
Jflow user interface has been designed to allow an easy integration in mashup Jflow user interface has been designed to allow an easy integration in mash up
web applications. Hereunder, we present its integration in NG6, which provides a web applications. Hereunder, we present its integration in NG6, which provides a
user-friendly interface to process, store and download high-throughput user-friendly interface to process, store and download high-throughput
sequencing data. The environment displays sequencing runs as a table. From sequencing data. The environment displays sequencing runs as a table. From
...@@ -127,21 +128,22 @@ this view, the user can add new data by running workflows in charge of loading ...@@ -127,21 +128,22 @@ this view, the user can add new data by running workflows in charge of loading
the data and checking its quality. Different workflows are available considering the data and checking its quality. Different workflows are available considering
data type and sequencing technology. data type and sequencing technology.
Workflows are listed by the \textit{availablewf} plugin built within a Workflows are listed by the \textit{availablewf} plug-in built within a
NG6 modal box. It requests the server to get the workflows implemented by the NG6 modal box. It requests the server to get the workflows implemented by the
developer. A \textit{select.availablewf} event thrown by the developer. A \textit{select.availablewf} event thrown by the
\textit{availablewf} plugin is listened and catched to generate the parameter \textit{availablewf} plug-in is listened and caught to generate the parameter
form using the \textit{wfform} plugin. Considering the parameter type, Jflow form using the \textit{wfform} plug-in. Considering the parameter type, Jflow
adapts its display. For example, a date is displayed as a calendar and a boolean adapts its display. For example, a date is displayed as a calendar and a boolean
as a checkbox. as a check box.
Biologists use NG6 to check sequencing reads quality, including experimental Biologists use NG6 to check sequencing reads quality, including experimental
samples contamination measure (quantification/search). The first input of this samples contamination measure (quantification/search). The first input of this
analysis is the contaminant reference genome fasta file, displayed as a file analysis is the contaminant reference genome fasta file, displayed as a file
selector. The second input is a parameter set describing the biological samples. selector. The second input is a parameter set describing the biological samples.
It includes the read files and metadata such as sample name, tissue and It includes the read files and meta data such as sample name, tissue and
developpement stage. To help biologists populate it, Jflow uses a structured development stage. To help biologists populate it, Jflow uses a structured
data input rendered by the \textit{wfform} plugin as a spreadsheet. It allows to data input rendered by the \textit{wfform} plug-in as a spreadsheet. It allows
to
copy and paste multiple lines. Jflow iterates then on the table content to copy and paste multiple lines. Jflow iterates then on the table content to
launch each sample processing. launch each sample processing.
...@@ -149,22 +151,25 @@ launch each sample processing. ...@@ -149,22 +151,25 @@ launch each sample processing.
\centering \centering
\includegraphics[width=\linewidth]{jflow_example.png} \includegraphics[width=\linewidth]{jflow_example.png}
\caption{\textbf{Jflow integration:} (a) A piece of the NG6 HTML code source in \caption{\textbf{Jflow integration:} (a) A piece of the NG6 HTML code source in
which is positioned an empty div to build the \textit{activewf} plugin and a which is positioned an empty div to build the \textit{activewf} plug-in
modal box for the \textit{wfstatus} plugin. (b) The Jquery code in charge to and a
build Jflow plugins and manage user action. When the \textit{select.activewf} modal box for the \textit{wfstatus} plug-in. (b) The Jquery code in
charge to
build Jflow plug-ins and manage user action. When the
\textit{select.activewf}
event is thrown from \textit{activewf-div}, a function is called with two event is thrown from \textit{activewf-div}, a function is called with two
parameters: \textit{event} and \textit{workflow}. The last parameter stores parameters: \textit{event} and \textit{workflow}. The last parameter stores
all the workflow's information, such as its name and its id, used in this all the workflow's information, such as its name and its id, used in this
example to update the modal box title and to build the \textit{wfstatus} example to update the modal box title and to build the \textit{wfstatus}
plugin. (c) The workflow status displayed as a graph.} plug-in. (c) The workflow status displayed as a graph.}
\label{fig::jflow_example} \label{fig::jflow_example}
\end{figure} \end{figure}
To monitor running workflows, NG6 provides a table in a specific page. The To monitor running workflows, NG6 provides a table in a specific page. The
table is filled by the \textit{activewf} plugin. In the same way as table is filled by the \textit{activewf} plug-in. In the same way as
described above, the \textit{wfstatus} is built on a modal box when a described above, the \textit{wfstatus} is built on a modal box when a
\textit{select.activewf} event is thrown by the \textit{activewf} plugin, as \textit{select.activewf} event is thrown by the \textit{activewf} plug-in, as
presented in Figure~\ref{fig::jflow_example}. This view shows the workflow's presented in Figure~\ref{fig::jflow_example}. This view shows the workflow's
execution graph where nodes represent components and edges links between inputs execution graph where nodes represent components and edges links between inputs
and outputs. and outputs.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment