Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
genotoul-bioinfo
jflow
Commits
cf4c0fb7
Commit
cf4c0fb7
authored
Aug 31, 2015
by
Jerome Mariette
Browse files
speling correction
parent
c8e3a563
Changes
1
Hide whitespace changes
Inline
Side-by-side
communication/Bioinformatics/jflow_BIOINF.tex
View file @
cf4c0fb7
...
...
@@ -25,18 +25,19 @@ Castanet-Tolosan Cedex, France.}
\section
{
Summary:
}
Biologists produce large data sets and are in demand of rich and simple
web portals in which they can upload and analy
s
e their files. Providing such
tools requires to mask the complexity induced by the needed High Perfomance
web portals in which they can upload and analy
z
e their files. Providing such
tools requires to mask the complexity induced by the needed High Perfo
r
mance
Computing (HPC) environment. The connexion between interface and computing
infrastructure is usually specific to each portal. With Jflow, we introduce
a Workflow Management System (WMS), composed of Jquery plugins which can easily be
a Workflow Management System (WMS), composed of Jquery plug-ins which can easily
be
embedded in any web application and a Python library providing all requested
features to setup, run and monitor workflows.
\section
{
Availability:
}
Jflow is available under the GNU General Public License (GPL) at
http://bioinfo.genotoul.fr/jflow. The package is com
m
ing with full
documentation, quickstart and a running test portal.
http://bioinfo.genotoul.fr/jflow. The package is coming with full
documentation, quick
start and a running test portal.
\section
{
Contact:
}
\href
{
support.genopole@toulouse.inra.fr
}{
support.genopole@toulouse.inra.fr
}
...
...
@@ -44,7 +45,7 @@ documentation, quickstart and a running test portal.
\section
{
Introduction
}
Building rich web environments aimed at helping scientists analy
s
e their data is
Building rich web environments aimed at helping scientists analy
z
e their data is
a common trend in bioinformatics. Specialized web portals such as MG-RAST (Meyer
et al., 2008), MetaVir (Roux et al., 2011) or NG6 (Mariette et al., 2012) provide
multiple services and analysis tools in an integrated manner for specific
...
...
@@ -65,12 +66,12 @@ definition or check file formats.
To our knowledge, Jflow, presented in this article, is the only WMS designed to
be embedded in any web application, thanks to its organization as five Jquery
(http://jquery.com/) plugins.
(http://jquery.com/) plug
-
ins.
\section
{
Methods
}
Jflow user interface gathers five Jquery plugins providing user oriented views.
Jflow user interface gathers five Jquery plug
-
ins providing user oriented views.
\begin{itemize}
\item
\textit
{
availablewf
}
lists all runnable workflows accessible to users,
...
...
@@ -80,23 +81,23 @@ Jflow user interface gathers five Jquery plugins providing user oriented views.
\item
\textit
{
wfoutputs
}
displays all outputs produced by the workflow
organized per component,
\item
\textit
{
wfstatus
}
shows the workflow execution state as a list or an
execution graph. The graph visuali
s
ation uses the Cytoscape web JavaScript
plugin (Lopes et al., 2010).
execution graph. The graph visuali
z
ation uses the Cytoscape web JavaScript
plug
-
in (Lopes et al., 2010).
\end{itemize}
The plugins give access to multiple communication methods and events. They
The plug
ins give access to multiple communication methods and events. They
interact with the server side through Jflow's REST API, running under a cherrypy
(http://www.cherrypy.org/) web server. The included server uses the JSONP
communication technique enabling cross-domain requests.
To be available from the different Jquery plugins, the workflows have to be
To be available from the different Jquery plug
-
ins, the workflows have to be
implemented using the Jflow API. A Jflow component is in charge of an execution
step. Adding a component to the system requires to write a Python
\textit
{
Component
}
subclass. In Jflow, different solutions are available to ease
component creation. To wrap a single command line, the developer can give a
position or a flag for each parameter. Jflow also embeds an XML parser which
allows to run gen
i
une Mobyle (Neron et al., 2009) components. Finally, to
allow develop
p
ers to integrate components from other WMS, Jflow provides a class
allows to run genu
i
ne Mobyle (Neron et al., 2009) components. Finally, to
allow developers to integrate components from other WMS, Jflow provides a class
skeleton. This class only requires to implement the parsing step. A workflow
chains several components. A Jflow workflow is built as a
\textit
{
Workflow
}
subclass. Components are added to the workflow as variables and chained linking
...
...
@@ -112,14 +113,14 @@ data type compliance for each parameter. Job submission, status
checking and error handling, rely on Makeflow (Albrecht et al., 2012) and
weaver (Bui et al., 2012). Therefore Jflow manages error recovery and supports
most distributed resource management systems (Condor, SGE, Work Queue or a
single multicore machine,
\ldots
). Replacing Makeflow by an other job submiter
single multi
core machine,
\ldots
). Replacing Makeflow by an other job submit
t
er
requires to implement a new
\textit
{
Engine
}
subclass. This class creates and
execute the workflow directed acyclic graph (DAG).
\section
{
Example
}
Jflow user interface has been designed to allow an easy integration in mashup
Jflow user interface has been designed to allow an easy integration in mash
up
web applications. Hereunder, we present its integration in NG6, which provides a
user-friendly interface to process, store and download high-throughput
sequencing data. The environment displays sequencing runs as a table. From
...
...
@@ -127,21 +128,22 @@ this view, the user can add new data by running workflows in charge of loading
the data and checking its quality. Different workflows are available considering
data type and sequencing technology.
Workflows are listed by the
\textit
{
availablewf
}
plugin built within a
Workflows are listed by the
\textit
{
availablewf
}
plug
-
in built within a
NG6 modal box. It requests the server to get the workflows implemented by the
developer. A
\textit
{
select.availablewf
}
event thrown by the
\textit
{
availablewf
}
plugin is listened and ca
tched
to generate the parameter
form using the
\textit
{
wfform
}
plugin. Considering the parameter type, Jflow
\textit
{
availablewf
}
plug
-
in is listened and ca
ught
to generate the parameter
form using the
\textit
{
wfform
}
plug
-
in. Considering the parameter type, Jflow
adapts its display. For example, a date is displayed as a calendar and a boolean
as a checkbox.
as a check
box.
Biologists use NG6 to check sequencing reads quality, including experimental
samples contamination measure (quantification/search). The first input of this
analysis is the contaminant reference genome fasta file, displayed as a file
selector. The second input is a parameter set describing the biological samples.
It includes the read files and metadata such as sample name, tissue and
developpement stage. To help biologists populate it, Jflow uses a structured
data input rendered by the
\textit
{
wfform
}
plugin as a spreadsheet. It allows to
It includes the read files and meta data such as sample name, tissue and
development stage. To help biologists populate it, Jflow uses a structured
data input rendered by the
\textit
{
wfform
}
plug-in as a spreadsheet. It allows
to
copy and paste multiple lines. Jflow iterates then on the table content to
launch each sample processing.
...
...
@@ -149,22 +151,25 @@ launch each sample processing.
\centering
\includegraphics
[width=\linewidth]
{
jflow
_
example.png
}
\caption
{
\textbf
{
Jflow integration:
}
(a) A piece of the NG6 HTML code source in
which is positioned an empty div to build the
\textit
{
activewf
}
plugin and a
modal box for the
\textit
{
wfstatus
}
plugin. (b) The Jquery code in charge to
build Jflow plugins and manage user action. When the
\textit
{
select.activewf
}
which is positioned an empty div to build the
\textit
{
activewf
}
plug-in
and a
modal box for the
\textit
{
wfstatus
}
plug-in. (b) The Jquery code in
charge to
build Jflow plug-ins and manage user action. When the
\textit
{
select.activewf
}
event is thrown from
\textit
{
activewf-div
}
, a function is called with two
parameters:
\textit
{
event
}
and
\textit
{
workflow
}
. The last parameter stores
all the workflow's information, such as its name and its id, used in this
example to update the modal box title and to build the
\textit
{
wfstatus
}
plugin. (c) The workflow status displayed as a graph.
}
plug
-
in. (c) The workflow status displayed as a graph.
}
\label
{
fig::jflow
_
example
}
\end{figure}
To monitor running workflows, NG6 provides a table in a specific page. The
table is filled by the
\textit
{
activewf
}
plugin. In the same way as
table is filled by the
\textit
{
activewf
}
plug
-
in. In the same way as
described above, the
\textit
{
wfstatus
}
is built on a modal box when a
\textit
{
select.activewf
}
event is thrown by the
\textit
{
activewf
}
plugin, as
\textit
{
select.activewf
}
event is thrown by the
\textit
{
activewf
}
plug
-
in, as
presented in Figure~
\ref
{
fig::jflow
_
example
}
. This view shows the workflow's
execution graph where nodes represent components and edges links between inputs
and outputs.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment