Commit a616b2c4 authored by Nathalie Rousse's avatar Nathalie Rousse
Browse files

- using file based sessions

- adding django_admin_log into admin.py
- adding access.log file production and using
- other minor modifications
parent 64453ac7
.. _devel_deployment_install_help:
====
Help
====
.. literalinclude:: ../../../../erecord/install/help.txt
erecordenv virtualenv activation
================================
* in bash or sh case :
source /opt/erecord/factory/install/erecordenv/bin/activate
* in csh or tcsh case :
source /opt/erecord/factory/install/erecordenv/bin/activate.csh
* just to control the activation on the erecordenv virtualenv :
which python
cd /opt/erecord/docs
* clean :
make clean
* clean the autogenerated source documentation :
make cleanref
* generate the autogenerated source documentation :
make ref
* generate :
make html
* The resulting documentation is produced under /opt/erecord/factory/docs
-------------------------------------------------------------------------------
First solution (into erecord/docs) :
The generation must be done somewhere else than on the machine where erecord
webservices run (generation not available in deployment situation) :
Under erecord_docs_path, cd erecord/docs
* clean :
make cleanvpzpages
* generate token :
for a user (username, password) who is authorized for every simulator
in limited access, generate a token :
make username=... password=... token_value
* generate pages :
make genvpzpages
* The resulting documentation is produced under erecord_docs_path,
under erecord/factory/docs/vpzpages (gen and private)
=> In deployment case, copy this generated erecord/factory/docs/vpzpages/gen
as /opt/erecord/factory/docs/vpzpages/gen
-------------------------------------------------------------------------------
Second solution (into erecord/docs/vpzpages/from_anywhere) :
The generation must be done somewhere else than on the machine where erecord
webservices run (generation not available in deployment situation) :
Under erecord_docs_path, cd erecord/docs/vpzpages/from_anywhere
or from a copy of from_anywhere somewhere else :
* clean :
source clean.sh
* generate :
follow the instructions of README.txt file (erecord/docs/vpzpages/from_anywhere/README.txt)
* The resulting documentation is produced
under from_anywhere/gen_vpzpages and from_anywhere/private
=> In deployment case, copy this generated from_anywhere/gen_vpzpages
as /opt/erecord/factory/docs/vpzpages/gen
-------------------------------------------------------------------------------
......@@ -21,3 +21,11 @@ Simulators HTML home pages
- :ref:`devel_deployment_install_simulators_html_home_pages`
Misc
====
- :ref:`devel_deployment_install_log`
- :ref:`devel_deployment_install_help`
.. _devel_deployment_install_log:
=====================================
Installation of log file and using it
=====================================
.. literalinclude:: ../../../../erecord/install/prod/install_log.txt
......@@ -7,12 +7,9 @@ Documentation
Introduction
============
The **main documentation** of erecord project is produced with Sphinx.
The **main documentation** of erecord project is produced with Sphinx *(Sphinx is a documentation generator which converts reStructuredText files into HTML websites and other formats including PDF, EPub and man)*.
Sphinx is a documentation generator which converts reStructuredText files into
HTML websites and other formats including PDF, EPub and man).
There is some **other external documentation**, for example html home pages produced for erecord simulators (with a specific python script).
There is some **other external documentation**, for example html home pages produced for erecord simulators *(with a specific python script)*.
Production and generation
=========================
......@@ -32,43 +29,3 @@ Some other external documentation
See "**Simulators home pages generation**" into
:ref:`devel_deployment_install_erecord`.
OLD A SUPPRIMER
===============
Prerequisite
------------
Activate erecordenv virtualenv (if not yet done) :
:ref:`cmd_activate_erecordenv_virtualenv`
Main documentation
------------------
Generation :
The generation commands are done under :ref:`erecord_docs_path
<deploy_erecord_docs_path>`.
.. literalinclude:: ../deployment/include/cmd_make_doc.rst
Some other external documentation
---------------------------------
Generation :
The generation commands are done under :ref:`erecord_docs_path
<deploy_erecord_docs_path>`.
.. literalinclude:: ../deployment/include/cmd_make_vpzpages.rst
For more
--------
For more, see :ref:`documentation generation and installation <geninstall_documentation>`
.. _cmd_activate_erecordenv_virtualenv:
================================
erecordenv virtualenv activation
================================
.. literalinclude:: ../devel/deployment/include/cmd_activate_erecordenv_virtualenv.rst
......@@ -31,8 +31,9 @@ from erecord_cmn.utils.dir_and_file import create_dir_if_absent
from erecord_cmn.utils.using.send_get_and_receive import send_get_and_receive
from erecord_cmn.utils.using.send_post_and_receive import send_post_and_receive
#base_url = "http://erecord.toulouse.inra.fr:8000/"
base_url = "http://127.0.0.1:8000/"
base_url = "http://erecord.toulouse.inra.fr:8000/"
#base_url = "http://127.0.0.1:8000/"
#base_url = "http://147.100.179.168:8000/"
###############################################################################
# in relation with db/vpz resource
......
......@@ -29,12 +29,17 @@ from erecord_cmn.utils.using.obtain_jwt import obtain_jwt
token_value = "unknown" # default
base_url = "http://erecord.toulouse.inra.fr:8000/"
#base_url = "http://127.0.0.1:8000/"
#base_url = "http://147.100.179.168:8000/"
username = None
password = None
if len(sys.argv) >= 3 :
username = sys.argv[1]
password = sys.argv[2]
token_value = obtain_jwt(username=username, password=password)
token_value = obtain_jwt(username=username, password=password,
base_url=base_url)
else :
token_value = "parameters username, password missing"
......
......@@ -7,18 +7,18 @@ Methods that may be used by a user calling the erecord web services from python
from erecord_cmn.utils.using.send_post_and_receive import send_post_and_receive
def obtain_jwt(username, password):
def obtain_jwt(username, password,
base_url="http://erecord.toulouse.inra.fr:8000/"):
"""Return a JWT value for username"""
#
token_value = "unknown" # default
#
url = "http://erecord.toulouse.inra.fr:8000/acs/jwt/obtain/"
url = base_url + "acs/jwt/obtain/"
inputdata = {"username":username, "password":password}
responsedata = send_post_and_receive(url=url, inputdata=inputdata)
#
if "token" in responsedata.keys() :
token_value = responsedata['token']
#
return token_value
#
......@@ -27,7 +27,8 @@ from django.contrib.auth.models import Group
#admin.site.unregister(User)
#admin.site.unregister(Group)
from django.contrib.admin.models import LogEntry
admin.site.register(LogEntry)
vleversion_general_description = \
"Maybe soon available in the 'Action' menu of the vle versions list page : actions to generate -or update- into the database -or delete from the database- the vle version (its models repositories, and their vle packages and their vpz files) ('generate tree','update tree','delete tree'). DON'T FORGET to attach EACH models repository to ONE vle version."
......
......@@ -285,7 +285,7 @@ function update_ids(vlerep_id, vlepkg_id, vlevpz_id, vlevpz_limited)
if (vlevpz_limited=='yes'){
hometext = hometext+ "<p><i>No web home page available <span style='color:red;'>for the selected simulator</span><br> that is in limited access</i></p>";
} else {
home_url= "http://147.100.164.34/docs/vpzpages/gen/homes/.by_ids/"+vlerep_id+"__"+vlepkg_id+"__"+vlevpz_id+"___home.html";
home_url= "http://147.100.179.168/docs/vpzpages/gen/homes/.by_ids/"+vlerep_id+"__"+vlepkg_id+"__"+vlevpz_id+"___home.html";
hometext = hometext+ "<p>You can use the web home page <span style='color:red;'>of the selected simulator</span> :</p>";
hometext = hometext+ "<p><a href="+home_url+" target='_blank'>home page</a></p>";
}
......
......@@ -231,20 +231,22 @@ class MenuPage(APIView):
if vpz is None and pkg is None and rep is None : # no filter
#vlerepfull_list = VleRep.objects.all()
# ordered by vleversion :
# ordered by vleversion (descendant) then name :
vlerepfull_list = list()
vle_version_list = VleVersion.objects.all()
vle_version_list = VleVersion.objects.all().order_by('-name')
for vle_version in vle_version_list :
vlerepfull_list.extend(vle_version.vlerep_list.all())
#vlerepfull_list.extend(vle_version.vlerep_list.all())
vlerepfull_list.extend(vle_version.vlerep_list.all().order_by('name'))
for vlerepfull in vlerepfull_list :
vlerep = { 'id':vlerepfull.id, 'name':vlerepfull.name,
'vlepkg_list':list() }
vlepkgfull_list = vlerepfull.vlepkg_list.all()
#vlepkgfull_list = vlerepfull.vlepkg_list.all()
vlepkgfull_list = vlerepfull.vlepkg_list.all().order_by('name')
for vlepkgfull in vlepkgfull_list :
vlepkg = { 'id':vlepkgfull.id, 'name':vlepkgfull.name,
'vlevpz_list':list() }
vlevpzfull_list = vlepkgfull.vlevpz_list.all()
vlevpzfull_list = vlepkgfull.vlevpz_list.all().order_by('name')
for vlevpzfull in vlevpzfull_list :
vlevpz = { 'id':vlevpzfull.id,
'name':vlevpzfull.name,
......@@ -270,7 +272,7 @@ class MenuPage(APIView):
vlepkgfull = VlePkg.objects.get(pk=pkg)
vlepkg = {'id':vlepkgfull.id, 'name':vlepkgfull.name,
'vlevpz_list':list() }
vlevpzfull_list = vlepkgfull.vlevpz_list.all()
vlevpzfull_list = vlepkgfull.vlevpz_list.all().order_by('name')
for vlevpzfull in vlevpzfull_list :
vlevpz = { 'id':vlevpzfull.id,
'name':vlevpzfull.name,
......@@ -285,11 +287,11 @@ class MenuPage(APIView):
vlerepfull = VleRep.objects.get(pk=rep)
vlerep = { 'id':vlerepfull.id, 'name':vlerepfull.name,
'vlepkg_list':list() }
vlepkgfull_list = vlerepfull.vlepkg_list.all()
vlepkgfull_list = vlerepfull.vlepkg_list.all().order_by('name')
for vlepkgfull in vlepkgfull_list :
vlepkg = { 'id':vlepkgfull.id, 'name':vlepkgfull.name,
'vlevpz_list':list() }
vlevpzfull_list = vlepkgfull.vlevpz_list.all()
vlevpzfull_list = vlepkgfull.vlevpz_list.all().order_by('name')
for vlevpzfull in vlevpzfull_list :
vlevpz = { 'id':vlevpzfull.id,
'name':vlevpzfull.name,
......
......@@ -160,9 +160,12 @@ Notes
- vle environment (several vle versions, models repositories, models,
simulators) : see /opt/erecord/erecord/install/vle/install_vle.txt
- To generate/use simulators HTML home pages : modify
/opt/erecord/erecord/docs/vpzpages/build_vpzpages.py (127.0.0.1 address)
and see /opt/erecord/erecord/install/prod/install_vpz_html_home_pages.txt
- To generate/use simulators HTML home pages : modify base_url into
/opt/erecord/erecord/docs/vpzpages/build_vpzpages.py,get_token.py
(127.0.0.1 address) and see
/opt/erecord/erecord/install/prod/install_vpz_html_home_pages.txt
- See also /opt/erecord/erecord/install/help.txt.
*******************************************************************************
*******************************************************************************
Some help notes for erecord admin
*******************************************************************************
================
erecord database
================
Cleaning the database (/opt/erecord/databases/erecord_default.sqlite3) :
- delete django_admin_log records, then VACUUM (the VACUUM command rebuilds
the database file, repacking it into a minimal amount of disk space) :
cd /opt/erecord/databases
sqlite3 erecord_default.sqlite3
> delete from django_admin_log;
> VACUUM;
===============
log information
===============
See prod/install_log.txt
*******************************************************************************
......@@ -199,6 +199,13 @@ dedicated to the online documentation :
Listen 443
</IfModule>
/etc/apache2/apache2.conf file modifications
--------------------------------------------
Modify into the apache2.conf file the Timeout value (number of seconds
before receives and sends time out, initial value 300)
Timeout 600
erecord_ws file modifications
-----------------------------
Give/control into the erecord_ws file the appropriate values (ServerName,
......@@ -216,6 +223,16 @@ erecord paths), then install it in the appropriate location :
- install erecord_ws file :
sudo cp /opt/erecord/erecord/projects/ws/ws/erecord_ws /etc/apache2/sites-available/erecord_ws.conf
/etc/apache2/sites-available/000-default.conf modifications
-----------------------------------------------------------
Modify the apache2 000-default.conf file so that it corresponds with the
erecord main documentation installation.
- /etc/apache2/sites-available/000-default.conf file modifications :
#DocumentRoot /var/www/html
DocumentRoot /var/www
===
Run
===
......@@ -227,6 +244,8 @@ Run
- Activate erecord_ws site :
sudo a2ensite erecord_ws.conf
- Activate 000-default.conf site : is supposed to be already done.
- Apache run :
sudo systemctl reload apache2
(or sudo service apache2 reload)
......@@ -235,8 +254,6 @@ Run
Notes
=====
- apache error log file : /log/apache2/error.log
- erecord online at :
- erecord web site : http://erecord.toulouse.inra.fr .
- erecord web services : http://erecord.toulouse.inra.fr:8000 .
......@@ -247,5 +264,13 @@ Notes
- simulators HTML home pages : see
/opt/erecord/erecord/install/prod/install_vpz_html_home_pages.txt
- log files :
- log files are into /opt/erecord/factory/log
- original apache log files : /var/log/apache2/error.log
/var/log/apache2/access.log...
- see more at /opt/erecord/erecord/install/prod/install_log.txt.
- See also /opt/erecord/erecord/install/help.txt.
*******************************************************************************
*******************************************************************************
Install log file and use it
*******************************************************************************
"Rename" erecord path as /opt/erecord, if erecord is installed somewhere else
(ln -s 'erecord path' /opt/erecord).
===============
log information
===============
The log folder is /opt/erecord/factory/log. It contains erecord.log file
(produced by erecord) and access.log file (from apache).
The following is about access.log file production and using.
access.log file
---------------
- The apache source files are :
- apache error log file : /var/log/apache2/error.log
- apache access log file : /var/log/apache2/access.log
access.log.1
access.log.2.gz ...
... access.log.14.gz
- The compil source file /opt/erecord/factory/log/access.log
is built from /var/log/apache2/access.log.1 (daily updated by cron).
=====================================
Install to produce and use access.log
=====================================
Configure CRON
--------------
CRON configuration to update compil source file (daily, as root) :
sudo crontab -e
- file modification :
# at 4 a.m every day
0 4 * * * cat /var/log/apache2/access.log.1 >> /opt/erecord/factory/log/access.log
Install goaccess
----------------
sudo apt-get install goaccess
=> /etc/goaccess.conf
================
Using access.log
================
Use goaccess
-------------
cd /opt/erecord/factory/log/tmp
goaccess -f /opt/erecord/factory/log/access.log
goaccess -f /opt/erecord/factory/log/access.log -a > access_report.html
goaccess -f /opt/erecord/factory/log/access.log -a -o csv > access_report.csv
Build traces file
-----------------
cd /opt/erecord/factory/log/tmp/
echo "***TRACES ***" > traces
echo "------------------------ vpz/input --------------------" >> traces
grep /vpz/input /opt/erecord/factory/log/access.log >> traces
echo "------------------------ vpz/output -------------------" >> traces
grep /vpz/output /opt/erecord/factory/log/access.log >> traces
echo "------------------------ vpz/inout --------------------" >> traces
grep /vpz/inout /opt/erecord/factory/log/access.log >> traces
echo "------------------------ vpz/experiment ---------------" >> traces
grep /vpz/experiment /opt/erecord/factory/log/access.log >> traces
echo "------------------------ vpz/report -------------------" >> traces
grep /vpz/report /opt/erecord/factory/log/access.log >> traces
*******************************************************************************
......@@ -45,22 +45,37 @@ Containing :
unzip packages-1.1.2.zip
cd packages-1.1.2
Attention : some modifications done into CMakeLists.txt of
vle.extension.decision, vle.extension.differential-equation, vle.examples :
OPTION WITH_TEST OFF, ...
vle -P vle.output configure build
vle -P vle.extension.celldevs configure build
vle -P vle.extension.cellqss configure build
Attention :
Some modifications done into vle.extension.decision/test :
- files allenrelation.cpp, ss.cpp :
BOOST_GLOBAL_FIXTURE(F);
vle -P vle.extension.decision configure build
vle -P vle.extension.difference-equation configure build
Attention :
Some modifications done into vle.extension.differential-equation/test :
- files test_common.hpp, test_Euler.cpp :
BOOST_GLOBAL_FIXTURE(F);
vle -P vle.extension.differential-equation configure build
vle -P vle.extension.difference-equation configure build
vle -P vle.extension.dsdevs configure build
vle -P vle.extension.fsa configure build
vle -P vle.extension.petrinet configure build
vle -P ext.muparser configure build
vle -P vle.forrester configure build
Attention :
Some modifications done into vle.examples :
- files testgens.cpp, testfsa.cpp, testdecision.cpp, testequation.cpp,
testsmartgardener.cpp, testpetrinet.cpp, testmanager.cpp :
BOOST_GLOBAL_FIXTURE(F);
vle -P vle.examples configure build
erecord package (delivered into erecord source code)
......@@ -92,7 +107,8 @@ Containing :
vle packages
------------
do the same as for vle packages of 'vle' models repository with
do the same as for "vle packages" of "'vle' models repository"
excepted building vle.examples, with
cd /opt/erecord/factory/install/REPOSITORIES_deliveries/vle-1.1.3/recordb
erecord package
......@@ -113,10 +129,15 @@ Containing :
vle -P meteo configure build
vle -P wwdm configure build
Attention : some modifications done into 2CV/src/CMakeLists.txt (DecAgent
not built)
Attention :
Some modifications done into 2CV (because PredicateParameters not found
in this version of vle.extension.decision) :
- file src/CMakeLists.txt :
#DeclareDecisionDynamics(DecAgent "DecAgent.cpp")
DecAgent not built => 2CV-decision.vpz UNUSABLE
vle -P 2CV configure build
vle -P ext.Eigen configure build
vle -P DEtimeStep configure build
vle -P gluePhysic configure build
......@@ -127,17 +148,17 @@ Containing :
vle -P Multiformalism configure build
vle -P glue configure build
vle -P GenCSVcan configure build
vle -P record.reader configure build
vle -P record.tester configure build
vle -P record.tester_test configure build
vle -P record.reader configure build
vle -P record.reader_test configure build
vle -P record.recursive configure build
vle -P record.recursive_test configure build
vle -P flood_wave configure build
vle -P CaliFloPP configure build
vle -P GenGIScan configure build
Not built :
vle -P GenGIScan configure build
vle -P MilSol configure build
vle -P minicrop configure build
vle -P herbsim configure build
......@@ -167,6 +188,39 @@ Containing :
vle -P wheatpest_V8 configure build
vle -P 2CVdt configure build