Improve user experience
This PR regroup several new improvements regarding user experience, especially if using an interactive environment.
I actually tried to test this as much as I could, but you really need a test module for this ^^
Everything seems ok for App, Input, Operation, in-memory...
Not sure about the Execute*()
stuff though. I've commented 2 lines and it still looks OK with my new execute() method,
but may be you should be careful about that.
Changelog:
Module
- Encoding declaration
- Add numpy requirement
- Improve readability when possible (alias otbApplication to otb, return early to avoid deep loops or condition tree, etc...)
- Fix bug with subprocess call to list apps, raise different Exceptions, do not try to subprocess if not in interactive mode since it will fail
- Catch errors better, avoid bare Exception, warn user and exit if OTB can't be found
- Logger should log any error message, while providing user with information regarding context
- User can set its own
logging.basicConfig
, just need to configure it before importing pyotb then declarelogger
var - Add script to remove fstrings from python code
apps.py
- Edit code apps.py in order to avoid loading every functions and variables into pyotb namespace (use functions and
_variable
) - Apps created in apps.py are classes, not just constructor functions
core.py
- Add App argument to make OTB stdout go silent
- Allow user to set_parameters after object instantiation
- Allow user to set a custom name property that will be displayed in logs instead of appname
bm = BandMath() ; bm.name = "MyBandMath"
- Break large function set_parameters into several, use class private methods
- App set_parameters does not execute every time it is called (only if output parameter keys where passed to
__init__
) - Add App attribute
parameters
to store kwargs - Add App
finished
attribute to control if app has ran with success - Add App class functions to control workflow :
execute
,clear(memory, parameters)
,find_output
tools.py
- To store helpers / general functions like
find_otb
Module init
Import
Before :
In [1]: import pyotb
ModuleNotFoundError: No module named 'otbApplication'
After :
In [1]: import pyotb
2022-02-14 20:59:21 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=None
2022-02-14 20:59:21 (INFO) [pyOTB] : Searching for it...
2022-02-14 20:59:21 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 20:59:21 (INFO) [pyOTB] : Using OTB in /opt/otb
2022-02-14 20:59:22 (INFO) [pyOTB] : Successfully loaded 129 OTB applications
This will actually set OTB_APPLICATION_PATH, GDAL_DATA, PROJ_LIB and PYTHONPATH variables.
Exit if OTB can't be found on system
In [1]: import pyotb
2022-02-14 22:10:28 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 22:10:28 (INFO) [pyOTB] : Searching for it...
An exception has occurred, use %tb to see the full traceback.
SystemExit: Can't run without OTB. Exiting.
Warning to recompile SWIG bindings on fresh install
Before :
In [1]: import pyotb
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
(...)
ImportError: libpython3.8.so.rh-python38-1.0: cannot open shared object file: No such file or directory
After :
In [1]: import pyotb
2022-02-14 21:56:18 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 21:56:18 (INFO) [pyOTB] : Searching for it...
2022-02-14 21:56:18 (INFO) [pyOTB] : Found OTB-7.0.0-Linux64
2022-02-14 21:56:18 (INFO) [pyOTB] : Found OTB-8.0.0-Linux64
2022-02-14 21:56:19 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : An error occured while importing Python API
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : It seems like you need to symlink or recompile OTB SWIG bindings
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : Use 'cd /home/vidlb/Applications/OTB-8.0.0-Linux64 ; source otbenv.profile ; ctest -S share/otb/swig/build_wrapping.cmake -VV'
SystemExit: Can't run without OTB. Exiting.
Switch OTB version using env var OTB_ROOT or OTB_DIR
OTB_ROOT=/opt/otb ipython3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import pyotb
2022-02-14 21:59:38 (INFO) [pyOTB] : Found OTB Python API in /opt/otb
2022-02-14 21:59:38 (DEBUG) [pyOTB] : Using library from /opt/otb/lib
2022-02-14 21:59:38 (DEBUG) [pyOTB] : /usr/bin/python3 -c 'import otbApplication; print(otbApplication.Registry.GetAvailableApplications())'
2022-02-14 21:59:38 (INFO) [pyOTB] : Successfully loaded 129 OTB applications
Find other OTB versions on system
In [1]: import pyotb
2022-02-14 21:52:50 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 21:52:50 (INFO) [pyOTB] : Searching for it...
2022-02-14 21:52:51 (INFO) [pyOTB] : Found Applications/OTB-7.0.0-Linux64
2022-02-14 21:52:51 (INFO) [pyOTB] : Found Applications/OTB-8.0.0-Linux64
2022-02-14 21:52:52 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 21:52:52 (INFO) [pyOTB] : Using OTB in /home/vidlb/Applications/OTB-8.0.0-Linux64
2022-02-14 21:52:52 (DEBUG) [pyOTB] : Using library from Applications/OTB-8.0.0-Linux64/lib
2022-02-14 21:52:52 (DEBUG) [pyOTB] : /usr/bin/python3 -c 'import otbApplication; print(otbApplication.Registry.GetAvailableApplications())'
2022-02-14 21:52:53 (INFO) [pyOTB] : Successfully loaded 117 OTB applications
--> Try to load OTB bindings or scan system, help user in case of failure, return env variables
Precedence : OTB_ROOT > python bindings directory
OR search for releases installations : current directory > home directory
OR (for linux) : /opt/otbtf > /opt/otb > /usr/local > /usr
Application init
App
class children
Every apps are Before :
In [6]: type(pyotb.BandMath)
Out[6]: function
After :
In [3]: type(pyotb.BandMath)
Out[3]: abc.ABCMeta
Init object without parameters
Before :
In [7]: b = pyotb.BandMath()
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
(...)
RuntimeError: Exception thrown in otbApplication Application_Execute: ../Modules/Applications/AppMathParser/app/otbBandMath.cxx:198:
itk::ERROR: BandMath(0x129d360): No input Image set...; please set at least one input image
After :
In [6]: b = pyotb.BandMath()
2022-02-14 22:01:12 (WARNING) [pyOTB] : BandMath: No parameters where provided. Use App.set_parameters() then App.execute()
In [7]: b.set_parameters(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif")
In [8]: b.execute()
2022-02-14 22:01:15 (DEBUG) [pyOTB] : BandMath: run execute() with parameters={'il': ['RGB.tif'], 'exp': 'im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', 'out': 'test.tif'}
Writing test.tif...: 100% [**************************************************] (10s)
2022-02-14 22:01:26 (DEBUG) [pyOTB] : BandMath: execution succeeded
Out[8]: True
Wrong parameter name
Before :
In [11]: b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")
2022-02-14 22:03:16 (INFO): Loading metadata from official product
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
(...)
RuntimeError: Exception thrown in otbApplication Application_GetParameterType: ../Modules/Wrappers/ApplicationEngine/src/otbWrapperParameterGroup.cxx:470:
itk::ERROR: ParameterList(0x1660ca0): Could not find parameter typo
After :
In [15]: b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-15-b9613f4f9166> in <module>
----> 1 b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")
(...)
Exception: BandMath: parameter 'typo' was not recognized
Edited by Cresson Remi