ut_path

Overview

Path Utilities

Installation

Package ut_path can be installed from PyPI or Anaconda.

To install with pip:

$ python -m pip install ut_path

To install with conda:

$ conda install -c conda-forge ut_path

Package logging

(c.f.: Appendix: Package Logging)

Package files

Classification

The Package ut_path consist of the following file types (c.f.: Appendix):

  1. Special files: (c.f.: Appendix: Special python package files)
  2. Dunder modules: (c.f.: Appendix: Special python package modules)
  3. Modules
    1. Modules for Management of Array of Paths
      1. aododopath.py
      2. aopath.py
    2. Modules for Management of Dictionary of Paths
      1. dodopath.py
      2. dopath.py
    3. Modules for Management of Paths
      1. file.py
      2. path.py
    4. Modules for Management of Path names
      1. pathnm.py

Modules for Management of Array of Paths**

The Module Type Modules for Management of Array of Paths contains the following Modules:

Array of Paths Management Rodules
Name Description
aododopath.py Management of Array of Dictionaries of Dictionaries of Paths.
aopath.py Management of Array of Paths.

Module: aopath.py

The Module aopath.py contains the static Classes AoPath.

Class: AoPath

The static Class AoPath is used to manage Array of Paths; it contains the subsequent methods.

Methods
AoPath Methods
Name Description
join Join array of paths using the os separator
mkdirs Make directories
show functions
sh_a_path Show array of paths for path template.
sh_a_path_by_tpl Convert array of path template keys and kwargs Rto array of paths.
sh_aopath_by_gl-ob  
sh_aopath_by_pac  
sh_aopath_mtime_gt  
sh_path_by_tpl_first_exist  
yield functions
yield_path_kwargs_over_path  
yield_path_kwargs_over_dir_path  
yield_path_item_kwargs_over_path_arr  
AoPath Method: join
  1. Convert array of paths (1.argument) by striping the leading or trailing os separator.
  2. join the converted array of paths.
Parameter
Parameter of: AoPath Method: join
Name Type Default Description
aopath TyAoPath   array of paths
Return Value
Return Value of: AoPath Method: join
Name Type Description
path TyPath Path
AoPath Method: sh_a_path

Convert path template to array of paths using glob function of module glob.py.

Parameter
Parameter of: AoPath Method: sh_a_path
Name Type Default Description
path TyPath   Path
Return Value
Return Value of: AoPath Method: sh_a_path
Name Type Description
a_path TyAoPath Array of paths
AoPath Method: sh_a_path_by_tmpl
  1. Select array of path templates from keyword arguments (1.arguments) using the parameter
    • array of path template keys (1.argument);
  2. join the array of path templates with the os separator
  3. convert the created final path template to an array of paths.
Parameter
Parameter of: AoPath Method: sh_a_path_by_tmpl
Name Type Default Description
a_path_tmpl_key TyAoPath   array of path template keys
kwargs TyDic   keyword arguments
Return Value
Return Value of: AoPath Method: sh_a_path_by_tmpl
Name Type Default Description
a_path TyAoPath   Path
AoPath Method: yield_path_kwargs
  1. Create array of paths by executing the function sh_a_path_by_tmpl with the parameter:
    • array of path template keys (2.argument).
  2. Loop over array of paths to yield:
    1. yield path, kwargs (3. argument)
Parameter
Parameter of: AoPath Method: yield_path_kwargs
Name Type Default Description
cls Tyclass   current class
a_path_tmpl_key TyAoPath   array of path template keys
kwargs TyDic   keyword arguments
Return Value
Return Value of: AoPath Method: yield_path_kwargs
Name Type Description
(path, kwargs) TyAoPath Path
AoPath Method: yield_path_kwargs_new
Synopsis

sh_a_path_by_tmpl(a_path_tmpl_key, kwargs)

Description
  1. Create array of directories by executing the function sh_a_path_by_tmpl with the arguments:
    • array of directory template keys (2.argument).
  2. Loop over array of directories to:
    1. create kwargs_new by executing ths given function sh_kwargs_new (4. argument) with the arguments:
      • directory, given kwargs (5. argument)
    2. create array of paths by executing the function sh_a_oath_by_tmpl with the arguments:
      • given array of path template keys (3. argument), kwargs_new
  3. Loop over array of paths within the outer loop to:
    1. yield path, kwargs_new
Parameter
Parameter of: AoPath Method: yield_path_kwargs_new
Name Type Default Description
cls Tyclass   Current class
a_dir_tmpl_key TyAoPath   Array of path template keys
a_path_tmpl_key TyAoPath   Array of path template keys
sh_kwargs_new TyAoPath   Show new keyword arguments function
kwargs TyDic   Keyword arguments
Return Value
Return Value of: AoPath Method: yield_path_kwargs_new
Name Type Description
(path, kwargs_new) TyAoPath Path, new keyword arguments
AoPath Method: yield_path_item_kwargs
  1. Create array of paths by executing the function sh_a_path_by_tmpl with the arguments:
    • array of path template keys (2.argument).
  2. Create array of items by selecting the value in the directory kwargs (4. argument) for the kwargs key (3. argument)
  3. Loop over array of path and array of items to:
    1. yield path, item, kwargs (4. argument)
Parameter
Parameter of: AoPath Method: yield_path_item_kwargs
Name Type Default Description
cls Tyclass   current class
a_path_tmpl_key TyAoPath   array of path template keys
a_arr_key TyAoPath   array of path template keys
kwargs TyDic   keyword arguments
Return Value
Return Value of: AoPath Method: yield_path_item_kwargs
Name Type Description
(path, item, kwargs) TyAoPath Path, Item, keyword arguments
Method: AoPath.yield_path_item_kwargs_new
  1. Create array of directories by executing the function sh_a_path_by_tmpl with the parameter:
    • a_dir_tmpl_key (2.argument).
  2. Create array of items by selecting the value in the directory kwargs (4. argument) for the key arr_key (3. argument)
  3. Loop over the array of directories to:
    1. create kwargs_new by executing ths function sh_kwargs_new (4. argument) with the arguments:
      • directory, given kwargs (5. argument)
    2. create array of paths by executing the function sh_a_oath_by_tmpl with the arguments:
      • given array of path template keys (3. argument), kwargs_new
    3. Loop over array of path and array of items within the outer loop to:
      1. yield path, item, kwargs_new
Parameter
Parameter of: AoPath Method: yield_path_item_kwargs_new
Name Type Default Description
cls Tyclass   current class
a_dir_tmpl_key TyAoPath   array of path template keys
a_path_tmpl_key TyAoPath   array of path template keys
sh_kwargs_new TyAoPath   show new keyword arguments function
kwargs TyDic   keyword arguments
Return Value
Return Value of: AoPath Method: yield_path_item_kwargs_new
Name Type Description
(path, item, kwargs_new) TyAoPath Path, Item, new keyword arguments

Modules for Management of Dictionary of Paths

The Module Type Modules for Management of Dictionary of Paths contains the following Modules:

Dictionaries of Paths Management Modules
Name Description
dodopath.py Management of Dictionary of Dictionaries of Paths.
dopath.py Management of Dictionary of Paths.

Module: dodopath.py

The Module dodoath.py contains the static Classes DoDoPath.

Class: DoDoPath

The static Class DoDoPath is used to manage Dictionary of Dictionaries of Paths; it contains the subsequent methods.

Methods
DoDoPath Methods
Name Description
sh_path Show Path.

Module: dopath.py ==========-======

The Module doath.py contains the static Classes DoPath.

Class: DoDoPath

The static Class DoPath is used to manage Dictionary of Paths; it contains the subsequent methods.

Methods
DoDoPath Methods
Name Description
sh_path Show Path.

Appendix

Package Logging

Description

Logging use the module log.py of the logging package ut_log. The module supports two Logging types:

  1. Standard Logging (std) or
  2. User Logging (usr).

The Logging type can be defined by one of the values 'std' or 'usr' of the parameter log_type; 'std' is the default. The different Logging types are configured by one of the following configuration files:

  1. log.std.yml or
  2. log.usr.yml

The configuration files can be stored in different configuration directories (ordered by increased priority):

  1. <package directory of the log package ut_log>/cfg,
  2. <package directory of the application package ui_eviq_srr>/cfg,
  3. <application directory of the application eviq>/cfg,

The active configuration file is the configuration file in the directory with the highest priority.

Examples

Site-packages-path = /appl/eviq/.pyenv/versions/3.11.12/lib/python3.11/site-packages Log-package = ut_log Application-package = ui_eviq_srr Application-home-path = /appl/eviq

Examples of log configuration-files
Log Configuration
Type Directory Type File
std Log package <Site-packages-path>/<Log-package>/cfg/log.std.yml
Application package <Site-packages-path>/<application-package>/cfg/log.std.yml
Application <application-home-path>/cfg/log.std.yml
usr Log package <site-packages-path>/ut_log/cfg/log.std.yml
Application package <site-packages-path>/ui_eviq_srr/cfg/log.usr.yml
Application <application-path>/cfg/log.usr.yml

Log message types

Logging defines log file path names for the following log message types: .

  1. debug
  2. info
  3. warning
  4. error
  5. critical

Log types and Log directories

Single or multiple Application log directories can be used for each message type:

Log types and directoriesg
Log type Log directory
long short multiple single
debug dbqs dbqs logs
info infs infs logs
warning wrns wrns logs
error errs errs logs
critical crts crts logs

Application parameter for logging

Application parameter used in log naming
Name Decription Range Default Example
appl_data Application data directory     /data/eviq
tenant Application tenant name     UMH
package Application package name     ui_eviq_srr
cmd Application command     evupreg
pid Process ID     681025
log_type Standard logging std std std
Personal logging usr
log_ts_type Seconds since 1.1.1970|ts ts ts ts
Datetime dt
ts if ts_type == ts     1750096540
if ts_type == dt     20250618.203010
log_sw_single_dir Enable single log directory True True True
Enable multiple log directories False

Log files naming

Naming Conventions (table format)
Naming conventions for logging file paths
Type Directory File
debug /<appl_data>/<tenant>/RUN/<package>/<cmd>/debs debs_<ts>_<pid>.log
critical /<appl_data>/<tenant>/RUN/<package>/<cmd>/logs crts_<ts>_<pid>.log
error /<appl_data>/<tenant>/RUN/<package>/<cmd>/logs errs_<ts>_<pid>.log
info /<appl_data>/<tenant>/RUN/<package>/<cmd>/logs infs_<ts>_<pid>.log
warning /<appl_data>/<tenant>/RUN/<package>/<cmd>/logs rnsg_<ts>_<pid>.log
Naming Conventions (tree format)
<appl_data>   Application data folder
│
└── <tenant>  Application tenant folder
    │
    └── RUN  Applications RUN folder for Application log files
        │
        └── <package>  RUN folder of Application package: <package>
            │
            └── <cmd>  RUN folder of Application command <cmd>
                │
                ├── debs  Application command debug messages folder
                │   │
                │   └── debs_<ts>_<pid>.log  debug messages for
                │                            run of command <cmd>
                │                            with pid <pid> at <ts>
                │
                └── logs  Application command log messages folder
                    │
                    ├── crts_<ts>_<pid>.log  critical messages for
                    │                        run of command <cmd>
                    │                        with pid <pid> at <ts>
                    ├── errs_<ts>_<pid>.log  error messages for
                    │                        run of command <cmd>
                    │                        with pid <pid> at <ts>
                    ├── infs_<ts>_<pid>.log  info messages for
                    │                        run of command <cmd>
                    │                        with pid <pid> at <ts>
                    └── wrns_<ts>_<pid>.log  warning messages for
                                             run of command <cmd>
                                             with pid <pid> at <ts>
Naming Examples (table format)
Naming conventions for logging file paths
Type Directory File
debug /appl/eviq/UMH/RUN/ui_eviq_srr/evdomap/debs/ debs_1750096540_354710.log
critical /appl/eviq/UMH/RUN/ui_eviq_srr/evdomap/logs/ crts_1749971151_240257.log
error /appl/eviq/UMH/RUN/ui_eviq_srr/evdomap/logs/ errs_1749971151_240257.log
info /appl/eviq/UMH/RUN/ui_eviq_srr/evdomap/logs/ infs_1750096540_354710.log
warning /appl/eviq/UMH/RUN/ui_eviq_srr/evdomap/logs/ wrns_1749971151_240257.log
Naming Examples (tree format)
/data/eviq/UMH/RUN/ui_eviq_srr/evdomap  Run folder of
│                                       of function evdomap
│                                       of package ui_eviq_srr
│                                       for teanant UMH
│                                       of application eviq
│
├── debs  debug folder of Application function: evdomap
│   │
│   └── debs_1748609414_314062.log  debug messages for run
│                                   of function evdomap
│                                   using pid: 314062 at: 1748609414
│
└── logs  log folder of Application function: evdomap
    │
    ├── errs_1748609414_314062.log  error messages for run
    │                               of function evdomap
    │                               with pid: 314062 at: 1748609414
    ├── infs_1748609414_314062.log  info messages for run
    │                               of function evdomap
    │                               with pid: 314062 at: 1748609414
    └── wrns_1748609414_314062.log  warning messages for run
                                    of function evdomap
                                    with pid: 314062 at: 1748609414

Configuration files

log.std.yml (jinja2 yml file)

Content
version: 1

disable_existing_loggers: False

loggers:

    # standard logger
    std:
        # level: NOTSET
        level: DEBUG
        handlers:
            - std_debug_console
            - std_debug_file
            - std_info_file
            - std_warning_file
            - std_error_file
            - std_critical_file

handlers:

    std_debug_console:
        class: 'logging.StreamHandler'
        level: DEBUG
        formatter: std_debug
        stream: 'ext://sys.stderr'

    std_debug_file:
        class: 'logging.FileHandler'
        level: DEBUG
        formatter: std_debug
        filename: '{{dir_run_debs}}/debs_{{ts}}_{{pid}}.log'
        mode: 'a'
        delay: true

    std_info_file:
        class: 'logging.FileHandler'
        level: INFO
        formatter: std_info
        filename: '{{dir_run_infs}}/infs_{{ts}}_{{pid}}.log'
        mode: 'a'
        delay: true

    std_warning_file:
        class: 'logging.FileHandler'
        level: WARNING
        formatter: std_warning
        filename: '{{dir_run_wrns}}/wrns_{{ts}}_{{pid}}.log'
        mode: 'a'
        delay: true

    std_error_file:
        class: 'logging.FileHandler'
        level: ERROR
        formatter: std_error
        filename: '{{dir_run_errs}}/errs_{{ts}}_{{pid}}.log'
        mode: 'a'
        delay: true

    std_critical_file:
        class: 'logging.FileHandler'
        level: CRITICAL
        formatter: std_critical
        filename: '{{dir_run_crts}}/crts_{{ts}}_{{pid}}.log'
        mode: 'a'
        delay: true

    std_critical_mail:
        class: 'logging.handlers.SMTPHandler'
        level: CRITICAL
        formatter: std_critical_mail
        mailhost : localhost
        fromaddr: 'monitoring@domain.com'
        toaddrs:
            - 'dev@domain.com'
            - 'qa@domain.com'
        subject: 'Critical error with application name'

formatters:

    std_debug:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
    std_info:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
    std_warning:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
    std_error:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
    std_critical:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
    std_critical_mail:
        format: '%(asctime)-15s %(levelname)s-%(name)s-%(process)d::%(module)s.%(funcName)s|%(lineno)s:: %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'
Jinja2-variables
log.std.yml Jinja2 variables
Name Definition Example
{{dir_run_debs}} debug run directory /data/eviq/UMH/RUN/ui_eviq_srr/evupreg/debs
{{dir_run_infs}} info run directory /data/eviq/UMH/RUN/ui_eviq_srr/evupreg/logs
{{dir_run_wrns}} warning run directory /data/eviq/UMH/RUN/ui_eviq_srr/evupreg/logs
{{dir_run_errs}} error run directory /data/eviq/UMH/RUN/ui_eviq_srr/evupreg/logs
{{dir_run_crts}} critical error run directory /data/eviq/UMH/RUN/ui_eviq_srr/evupreg/logs
{{ts}} if log_ts_type == 'ts' Timestamp since 1970 in [sec] 1749483509
if log_ts_type == 'dt' Datetime in tz 'Europe/Berlin' 20250609 17:38:29 GMT+0200
{{pid}} Process ID 79133

Python Terminology

Python Packages

Overview

Python Packages Overview
Name Definition
Python package Python packages are directories that contains the special module __init__.py and other modules, packages, files or directories.
Python sub-package Python sub-packages are python packages which are contained in another python package.
Python package sub-directory directory contained in a python package.
Python package special sub-directory Python package sub-directories with a special meaning like data or cfg

Special python package sub-directories

Special python package sub-directories
Name Description
bin Directory for package scripts.
cfg Directory for package configuration files.
data Directory for package data files.
service Directory for systemd service scripts.

Python package files

Overview

Python package overview files
Name Definition
Python package files Files within a python package.
Python dunder files Package files which are name with leading and trailing double underscores.
special Python files Package files which are not modules and used as python marker files like py.typed.
Python modules Files with suffix .py; they could be empty or contain python code; other modules can be imported into a module.
special Python modules Modules like __init__.py or main.py with special names and functionality.

Python package special files

Python package special files
Name Type Description
py.typed Type checking marker file The py.typed file is a marker file used in Python packages to indicate that the package supports type checking. This is a part of the PEP 561 standard, which provides a standardized way to package and distribute type information in Python.

Python package special modules

Python package special modules
Name Type Description
__init__.py Package directory marker file The dunder (double underscore) module __init__.py is used to execute initialisation code or mark the directory it contains as a package. The Module enforces explicit imports and thus clear namespace use and call them with the dot notation.
__main__.py entry point for the package The dunder module __main__.py serves as package entry point point. The module is executed when the package is called by the interpreter with the command python -m <package name>.
__version__.py Version file The dunder module __version__.py consist of assignment statements used in Versioning.

Python methods

Overview

Python methods overview
Name Description
Python method Python functions defined in python modules.
special Python method Python functions with special names and functionalities.
Python class Classes defined in python modules.
Python class method Python methods defined in python classes
special Python class method Python class functions with special names and functionalities.

Special python class methods

Python methods examples
Name Type Description
__init__ class object constructor method The special method __init__ is called when an instance (object) of a class is created; instance attributes can be defined and initalized in the method.

Table of Contents

Table of Content