Commit 4e6b2c9c authored by Nathalie Rousse's avatar Nathalie Rousse
Browse files

Adding several vle versions management : adding VleVersion, adding fields to...

Adding several vle versions management : adding VleVersion, adding fields to VpzAct (vlevlersion, vleusrpath), adding erecord_conf rule for VpzPath case...

Taking into account all current works (merging baudrimont and genotoul local codes).

=> TO BE TESTED/VALIDATED
parent 34d36ec0
......@@ -30,9 +30,15 @@ tar -zxvf erecord.tar.gz
--------------------------
or erecord from svn : !!! KEEP PREVIOUS erecord/databases,repositories !!!
svn export svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/trunk/erecord erecord_NEW
!!! TO BE REPLACED by : ... from git ...
--------------------------
only erecord/erecord from svn :
svn export svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/trunk/erecord/erecord erecord_erecord_NEW
!!! TO BE REPLACED by : ... from git ...
-------------------------
!!!+ link to /opt/databases !!!
......@@ -43,10 +49,16 @@ current databases /opt/databases/current from svn :
cd /opt/databases
svn checkout svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/branches/002_deployment/databases/current current
ln -s /opt/databases/current /opt/erecord/databases
!!! TO BE REPLACED by : ... from git ...
--------------------------
Updating current databases under svn :
cd /opt/databases/current
svn commit -m "updating from deployed erecord"
!!! TO BE REPLACED by : ... git ...
--------------------------
apache error log
......
===============================================================================
vle-x version of erecord development
===============================================================================
|
Depot SVN (mulcyber) : | Local (PC baudrimont) :
|
svnroot/recordweb/vle-x/ | /home/nrousse/workspace_svn/
| DEVELOPPEMENT_WEB/ERECORD2/
|
trunk | trunk
branches/001_history | 001_history
branches/002_deployment | 002_deployment
branches/003_help_notes | 003_help_notes
|
===============================================================================
Branch trunk:
svn checkout svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/trunk trunk_LOCALNAME
!!!
Branch history:
svn checkout svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/branches/001_history 001_history_LOCALNAME
...voir ~/workspace_git/DEVELOPPEMENT_WEB/recordweb/vle-x/trunk/erecord/erecord/install/dev_run.txt !!!
Branch deployment (DB..) :
svn checkout svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/branches/002_deployment 002_deployment_LOCALNAME
!!!
Branch help_notes:
svn checkout svn+ssh://rousse@scm.mulcyber.toulouse.inra.fr/svnroot/recordweb/vle-x/branches/003_help_notes 003_help_notes_LOCALNAME
===============================================================================
vle-x version of erecord development
===============================================================================
#cd /home/nrousse/workspace_git/DEVELOPPEMENT_WEB/
#git clone https://forgemia.inra.fr/record/recordweb.git
... export ...
=> Branch trunk: recordweb/vle-x/trunk
=> Branch history: recordweb/vle-x/branches/001_history
=> Branch deployment (DB..) : recordweb/vle-x/branches/002_deployment
=> Branch help_notes: recordweb/vle-x/branches/003_help_notes
===============================================================================
cd /home/nrousse/workspace_svn/DEVELOPPEMENT_WEB/ERECORD2/trunk/erecord/erecord/projects/ws
cd /home/nrousse/workspace_git/DEVELOPPEMENT_WEB/recordweb/vle-x/trunk/erecord/erecord/projects/ws/ws
-----------------------------------------------------------
erecordenv :
......
.. _webapi_feat_vleversion:
====================
Several vle versions
====================
Introduction
------------
erecord (*from the vle-x version of erecord*) accepts models repositories developped with **different vle versions**.
Description
-----------
A vle version is defined by :
- the **vle version name**, structured as vle-V.rev.
For example vle-1.1.3, vle-2.0.0, vle-1.1...
- the **vle usr path** where the vle version has been installed.
Containing subdirectories :
- bin
- lib
- lib/pkgconfig
How is it managed
-----------------
In case of a simulator defined/recorded as vpz (:ref:`dm_vlevpz`) :
- Each vle version (:ref:`dm_vleversion`) is recorded into the erecord database.
- Each models repository (:ref:`dm_vlerep`) is attached to a vle version (:ref:`dm_vleversion`).
In case of a simulator defined/recorded as vpzpath (:ref:`dm_vpzpath`) :
The following rule has been defined in order to be able to identify the
vle version of such simulators.
Configuration rule
++++++++++++++++++
For a simulator to be able to be declared as a VpzPath, the directory of the models repository containing it must contain a subdirectory named **"erecord_conf"**. This "erecord_conf" folder must contain :
- a file named **"VLE_VERSION"** : containing the vle version name.
- a file named **"VLE_USR_PATH"** : containing the vle install path name.
VleRep.path contains :
- erecord_conf / VLE_VERSION
- record_conf / VLE_USR_PATH
......@@ -13,6 +13,8 @@ erecord Features
feat_datafolder
feat_vleversion
More
====
.. include:: ../../../include/more.rst
......
......@@ -9,13 +9,28 @@ Data model of the erecord_db application
Schema
======
- :ref:`dm_vlerep`
- many :ref:`dm_vlepkg`
- many :ref:`dm_vlevpz`
- :ref:`dm_vleversion`
- many :ref:`dm_vlerep`
- many :ref:`dm_vlepkg`
- many :ref:`dm_vlevpz`
Data model
==========
.. _dm_vleversion:
VleVersion
----------
Vle version.
.. _dm_vlerep:
VleRep
------
Models repository.
A VleRep corresponds with a physical :term:`models repository` providing the required environment for running simulations.
.. _dm_vlepkg:
VlePkg
......@@ -29,14 +44,6 @@ It corresponds with the vle notion of package, or project.
A VlePkg is attached to a VleRep providing the physical required environnement.
.. _dm_vlerep:
VleRep
------
Models repository.
A VleRep corresponds with a physical :term:`models repository` providing the required environment for running simulations.
.. _dm_vlevpz:
VleVpz
......
......@@ -23,6 +23,9 @@ factory_path = os.path.normpath(os.path.join(PROJECT_HOME, "factory"))
REPOSITORIES_HOME = os.path.normpath(os.path.join(PROJECT_HOME, "repositories"))
"""REPOSITORIES_HOME"""
INSTALL_HOME = os.path.join(factory_path, 'install')
"""install path"""
IN_PRODUCTION = False # True or False
"""IN_PRODUCTION value False or True \n
In case of IN_PRODUCTION, no console destination for logging information ...
......@@ -71,7 +74,6 @@ RUN_HOME_APP_VPZ = os.path.join(RUN_HOME, "erecord_vpz")
REQUESTS_HOME_APP_VPZ = os.path.join(RUN_HOME_APP_VPZ, "requests")
"""requests into run workspace of erecord_vpz app"""
# Closed media files structure (from CLOSEDMEDIA_ROOT)
# (used for downloadable (and later uploaded ?) files)
......
......@@ -12,6 +12,7 @@ import shutil
from erecord_cmn.utils.vle import get_rep_pkg_data_path
from erecord_cmn.utils.vle import get_rep_pkg_path
from erecord_cmn.utils.vle import get_rep_pkg_exp_path
from erecord_cmn.utils.vle import vle_bash_cmd
from erecord_cmn.utils.process import call_subprocess
......@@ -23,7 +24,7 @@ stderr_filename = "stderr.txt"
stdout_filename = "stdout.txt"
ok_filename = "OK"
def control_in(vlehome_path, pkgname, vpzname) :
def control_in(vlehome_path, vle_version, pkgname, vpzname) :
"""Makes controls and returns a message about errors met
Controls :
......@@ -33,9 +34,10 @@ def control_in(vlehome_path, pkgname, vpzname) :
Note : the returned message is "" if no error met.
"""
pkgname_path = get_rep_pkg_path(rep_path=vlehome_path, pkgname=pkgname)
pkgname_path = get_rep_pkg_path(rep_path=vlehome_path,
vle_version=vle_version, pkgname=pkgname)
pkg_exp_path = get_rep_pkg_exp_path(rep_path=vlehome_path,
pkgname=pkgname)
vle_version=vle_version, pkgname=pkgname)
vpzname_path = os.path.join(pkg_exp_path, vpzname)
errormsg = ""
......@@ -51,10 +53,11 @@ def control_in(vlehome_path, pkgname, vpzname) :
errormsg = errormsg + msg
return errormsg
def copy_erecord_data_file(vlehome_path, src_folder, file_name):
def copy_erecord_data_file(vlehome_path, vle_version, src_folder, file_name):
"""Copies file_name file from src_folder to data folder of erecord pkg"""
erecord_data_path = get_rep_pkg_data_path(rep_path = vlehome_path,
vle_version = vle_version,
pkgname = "erecord")
shutil.copyfile(src = os.path.join(src_folder, file_name),
dst = os.path.join(erecord_data_path, file_name))
......@@ -65,9 +68,10 @@ def create_and_fill_jsonfile(content, folder_path, file_name) :
f.write(json.dumps(content)) ###f.write(json.dumps(byteify(content)))
f.close()
def run_simulation(runvle_path, vlehome_path, pkgname, vpzname) :
def run_simulation(runvle_path, vlehome_path, vle_usr_path, pkgname, vpzname) :
cmd = "cd " + runvle_path + ";"
cmd = cmd + vle_bash_cmd(vle_usr_path)
cmd = cmd + "export VLE_HOME=" + vlehome_path + ";"
cmd = cmd + "vle -P " + pkgname + " " + vpzname + ";"
......@@ -80,8 +84,12 @@ def run_simulation(runvle_path, vlehome_path, pkgname, vpzname) :
(OK_found, msg_stdout, msg_stderr) = control_out(runvle_path)
cr_ok = OK_found
if msg_stderr != "" :
cr_ok = False
# !!! a revoir !!!
# controle "enleve" car stderr non vide in case vle-2.0.0
#if msg_stderr != "" :
# cr_ok = False
# msg_stdout is not studied
errormsg = ""
......@@ -132,7 +140,8 @@ def control_out(runvle_path) :
###############################################################################
def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
def vle_modify_vpz(runvle_path, vlehome_path, vle_version, vle_usr_path,
pkgname, vpzname,
storaged_list, conditions, begin, duration):
"""Prepares and runs modify_vpz.vpz simulation of erecord pkg
......@@ -169,7 +178,7 @@ def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
if at_least_one :
modify_parameters["views"] = views
def control_in(vlehome_path, pkgname, vpzname) :
def control_in(vlehome_path, vle_version, pkgname, vpzname) :
"""Makes controls and returns a message about errors met
Controls :
......@@ -179,9 +188,10 @@ def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
Note : the returned message is "" if no error met.
"""
pkgname_path = get_rep_pkg_path(rep_path=vlehome_path, pkgname=pkgname)
pkgname_path = get_rep_pkg_path(rep_path=vlehome_path,
vle_version=vle_version, pkgname=pkgname)
pkg_exp_path = get_rep_pkg_exp_path(rep_path=vlehome_path,
pkgname=pkgname)
vle_version=vle_version, pkgname=pkgname)
vpzname_path = os.path.join(pkg_exp_path, vpzname)
errormsg = ""
......@@ -211,7 +221,8 @@ def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
return errormsg
errormsg = control_in(vlehome_path, pkgname, vpzname)
errormsg = control_in(vlehome_path=vlehome_path, vle_version=vle_version,
pkgname=pkgname, vpzname=vpzname)
if errormsg != "" :
raise Exception(errormsg)
......@@ -225,13 +236,14 @@ def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
file_name="modify_input.json")
# Install/copy modify_input.json into erecord/data
copy_erecord_data_file(vlehome_path=vlehome_path,
copy_erecord_data_file(vlehome_path=vlehome_path, vle_version=vle_version,
src_folder=runvle_path,
file_name="modify_input.json")
# Run modify_vpz.vpz simulation
(cr_ok, errormsg) = run_simulation(runvle_path=runvle_path,
vlehome_path=vlehome_path,
vle_usr_path=vle_usr_path,
pkgname="erecord",
vpzname="modify_vpz.vpz")
if not cr_ok :
......@@ -240,7 +252,8 @@ def vle_modify_vpz(runvle_path, vlehome_path, pkgname, vpzname,
###############################################################################
def vle_read_vpz(runvle_path, vlehome_path, pkgname, vpzname):
def vle_read_vpz(runvle_path, vlehome_path, vle_version, vle_usr_path,
pkgname, vpzname):
"""Prepares and runs read_vpz.vpz simulation of erecord pkg
This simulation produces (under runvle_path) the read_output.json json file
......@@ -249,7 +262,8 @@ def vle_read_vpz(runvle_path, vlehome_path, pkgname, vpzname):
Returns the input information of vpzname as json data
"""
errormsg = control_in(vlehome_path, pkgname, vpzname)
errormsg = control_in(vlehome_path=vlehome_path, vle_version=vle_version,
pkgname=pkgname, vpzname=vpzname)
if errormsg != "" :
raise Exception(errormsg)
......@@ -262,12 +276,13 @@ def vle_read_vpz(runvle_path, vlehome_path, pkgname, vpzname):
file_name="read_input.json")
# Install/copy read_input.json into erecord/data
copy_erecord_data_file(vlehome_path=vlehome_path,
copy_erecord_data_file(vlehome_path=vlehome_path, vle_version=vle_version,
src_folder=runvle_path, file_name="read_input.json")
# Run read_vpz.vpz simulation
(cr_ok, errormsg) = run_simulation(runvle_path=runvle_path,
vlehome_path=vlehome_path,
vle_usr_path=vle_usr_path,
pkgname="erecord",
vpzname="read_vpz.vpz")
if not cr_ok :
......@@ -281,7 +296,9 @@ def vle_read_vpz(runvle_path, vlehome_path, pkgname, vpzname):
###############################################################################
def vle_run_vpz(runvle_path, vlehome_path, pkgname, vpzname, plan, restype):
def vle_run_vpz(runvle_path, vlehome_path, vle_version, vle_usr_path,
pkgname, vpzname,
plan, restype):
"""Prepares and runs run_vpz.vpz simulation of erecord pkg
This simulation produces (under runvle_path) the run_output.json json file
......@@ -290,7 +307,8 @@ def vle_run_vpz(runvle_path, vlehome_path, pkgname, vpzname, plan, restype):
Returns results of vpzname simulation as json data
"""
errormsg = control_in(vlehome_path, pkgname, vpzname)
errormsg = control_in(vlehome_path=vlehome_path, vle_version=vle_version,
pkgname=pkgname, vpzname=vpzname)
if errormsg != "" :
raise Exception(errormsg)
......@@ -305,12 +323,13 @@ def vle_run_vpz(runvle_path, vlehome_path, pkgname, vpzname, plan, restype):
file_name="run_input.json")
# Install/copy run_input.json into erecord/data
copy_erecord_data_file(vlehome_path=vlehome_path,
copy_erecord_data_file(vlehome_path=vlehome_path, vle_version=vle_version,
src_folder=runvle_path, file_name="run_input.json")
# Run run_vpz.vpz simulation
(cr_ok, errormsg) = run_simulation(runvle_path=runvle_path,
vlehome_path=vlehome_path,
vle_usr_path=vle_usr_path,
pkgname="erecord",
vpzname="run_vpz.vpz")
if not cr_ok :
......
......@@ -17,19 +17,148 @@ LOGGER = get_logger(__name__) # should be passed as a parameter of methods ?
from erecord_cmn.utils.errors import logger_report_error
from erecord_cmn.utils.errors import build_error_message
def is_structured_as_vle_home( path ) :
"""determines if the path directory is structured as a VLE_HOME directory.
# vle version configuration (optional)
ERECORD_CONF_DIRNAME = "erecord_conf"
VLE_VERSION_FILENAME = "VLE_VERSION"
VLE_USR_PATH_FILENAME = "VLE_USR_PATH"
def get_vle_bin_path(usr_path):
return os.path.join(usr_path, 'bin')
def get_vle_lib_path(usr_path):
return os.path.join(usr_path, 'lib')
def get_vle_pkgconfig_path(usr_path):
return os.path.join(get_vle_lib_path(usr_path), 'pkgconfig')
def is_structured_as_vle_install(usr_path) :
"""determines if the usr_path is structured as a vle install directory.
Returns True if the path directory contains a pkgs subdirectory.
Returns True if the usr_path directory contains directories : bin, lib and
lib/pkgconfig.
"""
res = False
pkgs_path = get_rep_pkgs_path(path)
if os.path.exists(pkgs_path) and os.path.isdir(pkgs_path) :
res = True
bin_path = get_vle_bin_path(usr_path)
lib_path = get_vle_lib_path(usr_path)
pkgconfig_path = get_vle_pkgconfig_path(usr_path)
if os.path.exists(bin_path) and os.path.isdir(bin_path) :
if os.path.exists(lib_path) and os.path.isdir(lib_path) :
if os.path.exists(pkgconfig_path) and os.path.isdir(pkgconfig_path) :
res = True
return res
def is_pkg_of_rep(rep_path,pkgname) :
def vle_identify(vlepath):
"""returns (vle_version, vle_usr_path) of a models repository path.
vlepath is a models repository path.
vle_version is the vle version.
vle_usr_path is the path of a vle install directory.
The method relies on the following rule :
- vlepath contains a subdirectory named "erecord_conf" that contains :
- a file named "VLE_VERSION" : containing the vle version name.
- a file named "VLE_USR_PATH" : containing the vle install path name.
Note : not controlled :
- vlepath is structured as a VLE_HOME (see is_structured_as_vle_home).
- vle_version is structured as vle-V.rev
(see is_structured_as_vle_version_name).
- vle_usr_path contains directories : bin, lib, lib/pkgconfig
(see is_structured_as_vle_install).
"""
erecord_conf_path = os.path.join(vlepath, ERECORD_CONF_DIRNAME)
if os.path.exists(erecord_conf_path) and os.path.isdir(erecord_conf_path):
vle_version_filepath = os.path.join(erecord_conf_path,
VLE_VERSION_FILENAME)
vle_usr_path_filepath = os.path.join(erecord_conf_path,
VLE_USR_PATH_FILENAME)
if os.path.exists(vle_version_filepath) and os.path.isfile(vle_version_filepath):
if os.path.exists(vle_usr_path_filepath) and os.path.isfile(vle_usr_path_filepath):
file = open(vle_version_filepath, "r")
vle_version = file.read().replace('\n', '')
file.close()
file = open(vle_usr_path_filepath, "r")
vle_usr_path = file.read().replace('\n', '')
file.close()
return (vle_version, vle_usr_path)
return None
def vle_bash_cmd(usr_path):
"""builds and returns text bash commands to be run before calling vle
vle installed under usr_path
"""
bin_path = get_vle_bin_path(usr_path)
lib_path = get_vle_lib_path(usr_path)
pkgconfig_path = get_vle_pkgconfig_path(usr_path)
cmd = ""
cmd = cmd + "export PATH="+bin_path+":$PATH" + ";"
cmd = cmd + "export LD_LIBRARY_PATH="+lib_path+":$LD_LIBRARY_PATH" + ";"
cmd = cmd + "export PKG_CONFIG_PATH="+pkgconfig_path+":$PKG_CONFIG_PATH" + ";"
cmd = cmd + "export VLE_BASEPATH="+usr_path+";"
return cmd
def vle_version_type(version_name):
"""extracts and returns V from version name such as vle-V.rev"""
v = version_name.split("-")[1]
version_type = v.split(".")[0]
return version_type
def is_structured_as_vle_version_name(version_name):
"""returns True if version_name is structured as vle-V.rev and else False
(for example vle-1.1.3, vle-2.0.0)
"""
version_name_structure_ok = False # default
a = version_name.split("-") # ["vle", V.rev]
if len(a) > 1 :
if a[0] == "vle" :
b = a[1]
c = b.split(".") # [V, rev..]
if len(c) > 1 :
version_name_structure_ok = True
return version_name_structure_ok
def get_pkgs_suffix(version_name):
"""returns suffix of pkgs name according to version_name """
suffix = ""
if is_structured_as_vle_version_name(version_name) :
a = version_name.split("-") # ["vle", V.rev]
b = a[1]
c = b.split(".") # [V, rev..]
suffix = c[0] + '.' + c[1]
return suffix
def is_structured_as_vle_home(path, vle_version=None) :
"""determines if the path directory is structured as a VLE_HOME directory.
Returns True if the path directory contains a pkgs subdirectory.
"""
if vle_version is not None :
pkgs_path = get_rep_pkgs_path(path, vle_version)
if os.path.exists(pkgs_path) and os.path.isdir(pkgs_path) :
return True
return False
else :
elements = os.listdir(path)
for e in elements :
if e.startswith('pkgs-') :
if os.path.isdir(os.path.join(path, e)) :
return True
return False
def is_pkg_of_rep(rep_path, vle_version, pkgname) :
"""determines if a vle package exists (as a vle package) under a models
repository path.
......@@ -39,11 +168,11 @@ def is_pkg_of_rep(rep_path,pkgname) :
Suppose that rep_path is a models repository path.
Suppose that pkgname is a vle package.
"""
if pkgname in get_rep_pkgname_list( rep_path ) :
if pkgname in get_rep_pkgname_list(rep_path, vle_version) :
return True
return False
def is_vpz_of_pkg_of_rep(rep_path, pkgname, vpzname) :
def is_vpz_of_pkg_of_rep(rep_path, vle_version, pkgname, vpzname) :
"""determines if a vpz file exists as a vpz file of vle package under a
models repository path.
......@@ -57,14 +186,13 @@ def is_vpz_of_pkg_of_rep(rep_path, pkgname, vpzname) :
res = False
vpz_path = os.path.join(get_rep_pkg_exp_path(rep_path=rep_path,
pkgname=pkgname), vpzname)
vle_version=vle_version, pkgname=pkgname), vpzname)
if os.path.exists(vpz_path) and os.path.isfile(vpz_path) :
res = True
return res
def get_pkgs_name() :
return 'pkgs-1.1'
def get_pkgs_name(vle_version) :
return 'pkgs-' + get_pkgs_suffix(vle_version)
def get_exp_name() :
return 'exp'
......@@ -75,16 +203,16 @@ def get_data_name() :