Commit 734b64ec authored by Jerome Mariette's avatar Jerome Mariette
Browse files

No commit message

No commit message
parent a1fceaf0
\title[short Title]{Jflow: A fully scalable Javascript workflow management system}
\author[Mariette \textit{et~al}]{J\'{e}r\^{o}me Mariette\,$^{1,*}$,
Fr\'{e}d\'{e}ric Escudi\'{e}\,$^1$, Philippe Bardou\,$^2$, Christophe
\address{$^{1}$Plate-forme bio-informatique Genotoul, INRA, Biom\'{e}trie et
Intelligence Artificielle, BP 52627, 31326 Castanet-Tolosan Cedex, France.\\
$^{2}$Plate-forme SIGENAE, INRA, G\'{e}n\'{e}tique Cellulaire, BP 52627, 31326
Castanet-Tolosan Cedex, France.}
\history{Received on XXXXX; revised on XXXXX; accepted on XXXXX}
\editor{Associate Editor: XXXXXXX}
Building rich WEB environments aimed at helping scientists analyse their data is a common
trend in bioinformatics. These applications are often specialized WEB portals or
generic Workflow management systems (WMS). The first class provides multiple services and
analysis tools in an integrated interface for a specific experiment or data type. Quite often
these systems hide the processing steps in the back-office. The second class, for example Galaxy,
is mainly focused on workflow creation and provides a rather poor end-user interface but enable
to combine tools and data sources as desired. We introduce jflow a fully
scalable WMS that can easily be embedded in any WEB site, providing all WMS features and benefits
to your project.
The software package is available under the GNU General Public License (GPL) at
Workflow management systems (WMS) are defined as software packages managing and executing
computational pipelines. Nowadays, such systems are widely used in bioinformatics because
they enable researchers to analyse the large amount of data generated by high throughput
platforms. Some WMS like Galaxy (Giardine et al., 2005) enable users to process their data using
collection of local tools through web forms. Galaxy is probably the most used of such systems because it
is based on a comprehensive web interface designed for tool and database integration. BioMOBY
(Wilkinson et al., 2002) and Taverna (Oinn et al., 2004) differ from others WMS
because they organize and integrate multiple web service providers. The main limit of such WMS is the
network performance, service availability and I/O compatibility between providers. All these WMS have
their own user interface and can hardly be used as components of an existing web project. Nowadays, in
order to provide access to new tools, it is quite common to implement a web portal that wraps the execution
of the given software package. We describe jflow which includes on one hand the core of the WMS able to execute
workflows defined by a set of components, and on the other hand a fully scalable
jquery ( plugin able to request the jflow REST API.
Jflow core is based upon the Makeflow (Albrecht et al., 2012) workflow
engine and weaver (Bui, 2012), its Python API which is embedded within jflow. Using makeflow permits to
run jflow under different batch systems such as Condor, SGE, Work Queue or a single multicore machine.
\subsection{Adding new components and workflows}
Adding a new component in the system requires to write a Python class inheriting from the Component
class and to overwrite the process method wrapping the new tool. The class provides access to multiple
concepts such as map/reduce or multimap in order to define the way the command line pattern
should be applied to the input data. Multiple formats are available as input and output in order to
force the workflow creator to combine components the right way.
In the same way, writing a workflows consist of inheriting from the Workflow
class and overwriting the process method. In this last method, the workflow is defined as the
succession of components. Moreover, a property file is to be created to define the workflow parameters.
It gathers all the information including the data type which will be checked from the provided interfaces.
As an example, a date type will be displayed as a calendar in the graphical interface and the dd/mm/yyyy
format compliance will be checked in command line mode.
\subsection{A jquery plugin}
All information about a workflows will be accessible from both jflow command
line interface and its REST API. Thus, users can list available workflows and
their states, run and monitor them. Accessing those functionalities from the command
line interface can easily be done using the jflow command line. The same thing
can also be done from a website integrating the jflow plug-in.
To do so, jflow offers four modules to retrieve workflows information. As a
jquery plugin, these modules are fully scalable and provide multiple methods and events to
ease jflow integration. As example, the “click” event on a specific workflow triggers an event
that can be listen to and used to build a workflow form wherever the web site designer wants it
to be showed.
jflow is a simple and efficient solution to
embed workflow management systems features within any Web application.
\bibitem[Giardine B {\it et~al}., 2005]{Giardine} Giardine B, et al. (2005)
Galaxy: a platform for interactive large-scale genome analysis, {\it Genome Res.},
{\bf 15}, 1451-1455.
\bibitem[Wilkinson {\it et~al}., 2002]{Wilkinson} Wilkinson MD, Links M. (2002)
BioMOBY: an open source biological web services proposal. {\it Brief Bioinform},
{\bf3}, 331-341.
\bibitem[Oinn {\it et~al}., 2004]{Oinn} Oinn T, et al. (2004) Taverna: a tool
for the composition and enactment of bioinformatics workflows. {\it Bioinformatics},
{\bf20}, 3045-305.
\bibitem[Albrecht {\it et~al}., 2012]{Albrecht} Michael Albrecht, Patrick Donnelly,
Peter Bui, and Douglas Thain (2012) Makeflow: A Portable Abstraction for Data
Intensive Computing on Clusters, Clouds, and Grids. {\it SWEET at ACM SIGMOD}, {\bf20}.
\bibitem[Bui, 2004]{Bui} Peter Bui (2012) Compiler Toolchain For Data Intensive Scientific
Workflows. {\it Ph.D. Thesis, University of Notre Dame}.
% Are we printing crop marks?
\newif\if@cropmarkson \@cropmarksontrue
% Load all necessary packages
% Not sure if needed.
% Set twoside printing
% Marginal notes are on the outside edge
\abovedisplayskip 10\p@ \@plus2\p@ \@minus5\p@
\abovedisplayshortskip \z@ \@plus3\p@
\belowdisplayshortskip 6\p@ \@plus3\p@ \@minus3\p@
\belowdisplayskip \abovedisplayskip
\abovedisplayskip 11\p@ minus 3\p@
\belowdisplayskip \abovedisplayskip
\abovedisplayshortskip \z@ plus 2\p@
\belowdisplayshortskip 4\p@ plus 2\p@ minus2\p@
\def\@listi{\topsep 4.5\p@ plus 2\p@ minus 1\p@
\itemsep \parsep
\topsep 4\p@ plus 2\p@ minus 2\p@}}
\abovedisplayskip 6\p@ minus 3\p@
\abovedisplayshortskip \z@ plus 3\p@
\belowdisplayshortskip 6\p@ plus 3\p@ minus 3\p@
\def\@listi{\topsep 3\p@ plus 1\p@ minus 1\p@
\parsep 2\p@ plus 1\p@ minus 1\p@\itemsep \parsep}}
% Line spacing
% Paragraph dimensions and inter-para spacing
% Set inter-para skips
\setlength\smallskipamount{3\p@ \@plus 1\p@ \@minus 1\p@}
\setlength\medskipamount{6\p@ \@plus 2\p@}
\setlength\bigskipamount{12\p@ \@plus 4\p@ \@minus 4\p@}
% Page break penalties
\@lowpenalty 51
\@medpenalty 151
\@highpenalty 301
% Disallow widows and orphans
\clubpenalty 10000
\widowpenalty 10000
% Disable page breaks before equations, allow pagebreaks after
% equations and discourage widow lines before equations.
\displaywidowpenalty 100
\predisplaypenalty 10000
\postdisplaypenalty 2500
% Allow breaking the page in the middle of a paragraph
\interlinepenalty 0
% Disallow breaking the page after a hyphenated line
\brokenpenalty 10000
% Hyphenation; don't split words into less than three characters
% Set page layout dimensions
\setlength\headheight{16\p@} % height of running head
\setlength\topmargin{2.9pc} % head margin
\addtolength\topmargin{-1in} % subtract out the 1 inch driver margin
\setlength\topskip{10\p@} % height of first line of text
\setlength\headsep{19\p@} % space below running head --
\setlength\footskip{34\p@} % space above footer line
\setlength\maxdepth{.5\topskip} % pages can be short or deep by half a line?
\setlength\textwidth{42pc} % text measure excluding margins
\setlength\textheight{58\baselineskip} % 54 lines on a full page,
\addtolength\textheight{\topskip} % including the first
% line on the page
% Set the margins
\addtolength\oddsidemargin{-1in} % subtract out the 1 inch driver margin
\setlength\columnsep{1.5pc} % space between columns for double-column text
\setlength\columnseprule{0\p@} % width of rule between two columns
% Footnotes
\setlength\footnotesep{9\p@} % space between footnotes
% space between text and footnote
\setlength{\skip\footins}{12\p@ \@plus 6\p@ \@minus 1\p@}
% Float placement parameters
% The total number of floats that can be allowed on a page.
% The maximum number of floats at the top and bottom of a page.
% The maximum part of the top or bottom of a text page that can be
% occupied by floats. This is set so that at least four lines of text
% fit on the page.
% The minimum amount of a text page that must be occupied by text.
% This should accomodate four lines of text.
% The minimum amount of a float page that must be occupied by floats.
% The same parameters repeated for double column output
% Space between floats
\setlength\floatsep {12\p@ \@plus 2\p@ \@minus 2\p@}
% Space between floats and text
\setlength\textfloatsep{20\p@ \@plus 2\p@ \@minus 4\p@}
% Space above and below an inline figure
\setlength\intextsep {18\p@ \@plus 2\p@ \@minus 2\p@}
% For double column floats
\setlength\dblfloatsep {12\p@ \@plus 2\p@ \@minus 2\p@}
\setlength\dbltextfloatsep{20\p@ \@plus 2\p@ \@minus 4\p@}
% Space left at top, bottom and inbetween floats on a float page.
\setlength\@fptop{0\p@} % no space above float page figures
\setlength\@fpsep{12\p@ \@plus 1fil}
% The same for double column
\setlength\@dblfpsep{12\p@ \@plus 1fil}
% Override settings in mathtime back to TeX defaults
\DeclareMathSizes{5} {5} {5} {5}
\DeclareMathSizes{6} {6} {5} {5}
\DeclareMathSizes{7} {7} {5} {5}
\DeclareMathSizes{8} {8} {6} {5}
\DeclareMathSizes{9} {9} {6.5} {5}
\DeclareMathSizes{10} {10} {7.5} {5}
\DeclareMathSizes{12} {12} {9} {7}
% Page styles
\def\@oddfoot{\vbox to 12.5\p@{\hbox{\rule{\textwidth}{0.5\p@}}\vss
\hbox to \textwidth{\hfill\helveticabold\small\thepage}%
\def\@evenfoot{\vbox to 12.5\p@{\rule{\textwidth}{0.5\p@}\vss
\hbox to \textwidth{\helveticabold\small\thepage\hfill}%
\def\@evenhead{\vbox{\hbox to \textwidth{\fontsize{8}{10}\selectfont
\def\@oddhead{\vbox{\hbox to \textwidth{\hfill\fontsize{8}{10}\selectfont
\def\@oddfoot{\vbox to 13\p@{\hbox{\rule{\textwidth}{1\p@}}\vss
\hbox to \textwidth{\helvetica
\fontsize{7}{9}\fontshape{n}\selectfont \copyright\space Oxford University Press \@copyrightyear.%
\def\@evenfoot{\vbox to 13\p@{\rule{\textwidth}{1\p@}\vss
\hbox to \textwidth{\helvetica\thepage\hfill
\fontsize{7}{9}\fontshape{n}\selectfont \copyright\space Oxford University Press \@copyrightyear.}%
% Page range
\newif\iflastpagegiven \lastpagegivenfalse
\ClassWarning{BIO}{Increasing pagenumber to \@firstpage}%
\else \ifnum\@firstpage<\c@page
\ClassWarning{BIO}{Firstpage lower than pagenumber}\fi\fi
\global\c@page \@ne
\gdef\thepage{\csname @#1\endcsname \c@page}%
\csname @#1\endcsname \@firstpage}%
\csname @#1\endcsname \@lastpage}%
\ifx \csname \@tempa \endcsname \relax
\fi }
\iflastpagegiven \else
\ifnum\@lastpage =0 {\ \bf ???} \else
\ifnum\@lastpage = \@firstpage \ \thefirstpage\else
\thefirstpage--\thelastpage \fi\fi}
% Sectional units
% Counters
% Form of the numbers
% Form of the words
\newcommand\listfigurename{List of Figures}
\newcommand\listtablename{List of Tables}
% Clearemptydoublepage should really clear the running heads too
% Frontmatter, mainmatter and backmatter
\newif\if@mainmatter \@mainmattertrue
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% TITLE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Application Notes
% \setlength{\dropfromtop}{-2.25pc}%
\definecolor{gray}{cmyk}{0, 0, 0, 0.15}
\long\def\@makefntext##1{\parindent 3mm\noindent
% \@textsuperscript{\normalfont\@thefnmark}\raggedright##1}%
\ifnum \col@number=\@ne
\global\@topnum\z@ % Prevents figures from going at top of page.
\newlength{\aboveskipchk}%for checking oddpage or evenpage top skip
\checkoddpage\ifcpoddpage\setlength{\aboveskipchk}{-3pc}\else\setlength{\aboveskipchk}{-5pc}\fi%for checking oddpage or evenpage top skip%%
\hbox to \textwidth{%
{\helvetica\itshape\bfseries\fontsize{19}{12}\selectfont {\color{gray}BIOINFORMATICS}
\if@appnotes APPLICATIONS NOTE\hfil\fi
\enskip \parbox[b]{11.3pc}{%
Vol. 00\ no. 00 \@pubyear\\
\hfill Pages \thepagerange
\hbox to \textwidth{%
{\helveticabold\fontsize{16}{21}\selectfont\raggedright \@title \par}%
{\helvetica\fontsize{13}{15}\selectfont\raggedright \@author \par}%
{\helvetica\fontsize{10}{12}\selectfont\raggedright \@address \par}%
{\helvetica\fontsize{8}{10}\selectfont\raggedright \@history \par}
{\helvetica\fontsize{10}{12}\selectfont\raggedright \@editor \par}
\vspace{12\p@ plus 6\p@ minus 6\p@}%
%%%%%%%%%%%%%%%%%%%%%%%%%%%% Abstract %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\par\noindent{\bfseries #1}\space\ignorespaces}