Skip to content

Improve user experience

Cresson Remi requested to merge develop into master

This PR regroup several new improvements regarding user experience, especially if using an interactive environment.
I actually tried to test this as much as I could, but you really need a test module for this ^^
Everything seems ok for App, Input, Operation, in-memory...
Not sure about the Execute*() stuff though. I've commented 2 lines and it still looks OK with my new execute() method,
but may be you should be careful about that.

Changelog:

Module

  • Encoding declaration
  • Add numpy requirement
  • Improve readability when possible (alias otbApplication to otb, return early to avoid deep loops or condition tree, etc...)
  • Fix bug with subprocess call to list apps, raise different Exceptions, do not try to subprocess if not in interactive mode since it will fail
  • Catch errors better, avoid bare Exception, warn user and exit if OTB can't be found
  • Logger should log any error message, while providing user with information regarding context
  • User can set its own logging.basicConfig, just need to configure it before importing pyotb then declare logger var
  • Add script to remove fstrings from python code

apps.py

  • Edit code apps.py in order to avoid loading every functions and variables into pyotb namespace (use functions and _variable)
  • Apps created in apps.py are classes, not just constructor functions

core.py

  • Add App argument to make OTB stdout go silent
  • Allow user to set_parameters after object instantiation
  • Allow user to set a custom name property that will be displayed in logs instead of appname bm = BandMath() ; bm.name = "MyBandMath"
  • Break large function set_parameters into several, use class private methods
  • App set_parameters does not execute every time it is called (only if output parameter keys where passed to __init__)
  • Add App attribute parameters to store kwargs
  • Add App finished attribute to control if app has ran with success
  • Add App class functions to control workflow : execute, clear(memory, parameters), find_output

tools.py

  • To store helpers / general functions like find_otb

Module init

Import

Before :

In [1]: import pyotb
ModuleNotFoundError: No module named 'otbApplication'

After :

In [1]: import pyotb
2022-02-14 20:59:21 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=None
2022-02-14 20:59:21 (INFO) [pyOTB] : Searching for it...
2022-02-14 20:59:21 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 20:59:21 (INFO) [pyOTB] : Using OTB in /opt/otb
2022-02-14 20:59:22 (INFO) [pyOTB] : Successfully loaded 129 OTB applications

This will actually set OTB_APPLICATION_PATH, GDAL_DATA, PROJ_LIB and PYTHONPATH variables.

Exit if OTB can't be found on system

In [1]: import pyotb
2022-02-14 22:10:28 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 22:10:28 (INFO) [pyOTB] : Searching for it...
An exception has occurred, use %tb to see the full traceback.

SystemExit: Can't run without OTB. Exiting.

Warning to recompile SWIG bindings on fresh install

Before :

In [1]: import pyotb                                                                                                                   
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
(...)
ImportError: libpython3.8.so.rh-python38-1.0: cannot open shared object file: No such file or directory

After :

In [1]: import pyotb
2022-02-14 21:56:18 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 21:56:18 (INFO) [pyOTB] : Searching for it...
2022-02-14 21:56:18 (INFO) [pyOTB] : Found OTB-7.0.0-Linux64
2022-02-14 21:56:18 (INFO) [pyOTB] : Found OTB-8.0.0-Linux64
2022-02-14 21:56:19 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : An error occured while importing Python API
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : It seems like you need to symlink or recompile OTB SWIG bindings
2022-02-14 21:56:19 (CRITICAL) [pyOTB] : Use 'cd /home/vidlb/Applications/OTB-8.0.0-Linux64 ; source otbenv.profile ; ctest -S share/otb/swig/build_wrapping.cmake -VV'

SystemExit: Can't run without OTB. Exiting.

Switch OTB version using env var OTB_ROOT or OTB_DIR

OTB_ROOT=/opt/otb ipython3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pyotb
2022-02-14 21:59:38 (INFO) [pyOTB] : Found OTB Python API in /opt/otb
2022-02-14 21:59:38 (DEBUG) [pyOTB] : Using library from /opt/otb/lib
2022-02-14 21:59:38 (DEBUG) [pyOTB] : /usr/bin/python3 -c 'import otbApplication; print(otbApplication.Registry.GetAvailableApplications())'
2022-02-14 21:59:38 (INFO) [pyOTB] : Successfully loaded 129 OTB applications

Find other OTB versions on system

In [1]: import pyotb
2022-02-14 21:52:50 (INFO) [pyOTB] : Failed to import otbApplication with PYTHONPATH=/home/vidlb/Projets/git/pyotb
2022-02-14 21:52:50 (INFO) [pyOTB] : Searching for it...
2022-02-14 21:52:51 (INFO) [pyOTB] : Found Applications/OTB-7.0.0-Linux64
2022-02-14 21:52:51 (INFO) [pyOTB] : Found Applications/OTB-8.0.0-Linux64
2022-02-14 21:52:52 (INFO) [pyOTB] : Found /opt/otb/lib/otb/
2022-02-14 21:52:52 (INFO) [pyOTB] : Using OTB in /home/vidlb/Applications/OTB-8.0.0-Linux64
2022-02-14 21:52:52 (DEBUG) [pyOTB] : Using library from Applications/OTB-8.0.0-Linux64/lib
2022-02-14 21:52:52 (DEBUG) [pyOTB] : /usr/bin/python3 -c 'import otbApplication; print(otbApplication.Registry.GetAvailableApplications())'
2022-02-14 21:52:53 (INFO) [pyOTB] : Successfully loaded 117 OTB applications
--> Try to load OTB bindings or scan system, help user in case of failure, return env variables
    Precedence :                                    OTB_ROOT > python bindings directory
        OR search for releases installations    :    current directory > home directory
        OR (for linux)                          :    /opt/otbtf > /opt/otb > /usr/local > /usr

Application init

Every apps are App class children

Before :

In [6]: type(pyotb.BandMath)
Out[6]: function

After :

In [3]: type(pyotb.BandMath)
Out[3]: abc.ABCMeta

Init object without parameters

Before :

In [7]: b = pyotb.BandMath()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
(...)
RuntimeError: Exception thrown in otbApplication Application_Execute: ../Modules/Applications/AppMathParser/app/otbBandMath.cxx:198:
itk::ERROR: BandMath(0x129d360): No input Image set...; please set at least one input image

After :

In [6]: b = pyotb.BandMath()
2022-02-14 22:01:12 (WARNING) [pyOTB] : BandMath: No parameters where provided. Use App.set_parameters() then App.execute()

In [7]: b.set_parameters(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif")

In [8]: b.execute()
2022-02-14 22:01:15 (DEBUG) [pyOTB] : BandMath: run execute() with parameters={'il': ['RGB.tif'], 'exp': 'im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', 'out': 'test.tif'}
Writing test.tif...: 100% [**************************************************] (10s)
2022-02-14 22:01:26 (DEBUG) [pyOTB] : BandMath: execution succeeded
Out[8]: True

Wrong parameter name

Before :

In [11]: b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")
2022-02-14 22:03:16 (INFO): Loading metadata from official product
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
(...)
RuntimeError: Exception thrown in otbApplication Application_GetParameterType: ../Modules/Wrappers/ApplicationEngine/src/otbWrapperParameterGroup.cxx:470:
itk::ERROR: ParameterList(0x1660ca0): Could not find parameter typo

After :

In [15]: b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")                                                                                                                 
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-15-b9613f4f9166> in <module>
----> 1 b = pyotb.BandMath(il='RGB.tif', exp='im1b1*0.2989 + im1b2*0.587 + im1b3*0.114', out="test.tif", typo="true")

(...)

Exception: BandMath: parameter 'typo' was not recognized
Edited by Cresson Remi

Merge request reports